Linux Troubleshooting Guide

Linux Troubleshooting Guide

Last updated 14 Nov 2024

During my learning of DevOps tools, I have encountered numerous problems. These difficulties can range from solving the dependency requirement to installing the tools themselves. The majority of solutions to issues are found on the Internet, but some require more time to trial and error for the right fix. Below are some of the more common issues and solutions that could be useful to every Linux user. They are listed in the troubleshooting guide below.

TopicFind previous server uptime
Source22.04.5 LTS (Jammy Jellyfish)
ErrorVPS went offline e.g., every 3 to 4 weeks
SolutionThe wtmp file logs all logins and system boots. Use the last command to read this file
last -f /var/log/wtmp reboot
RemarkSometimes a VPS may go offline often and you don’t know why. The cron job can be used to schedule a periodic reboot, but the frequency of the reboot depends on the historical uptimes.
Topic Generate an SSH Key Pair
SourceUNIX
Errorn/a
SolutionTo SSH from A to B without password
A: ssh-keygen -t rsa
A: cp $HOME/.ssh/id_rsa.pub B
RemarkGenerate an SSH key pair on local and copy the public key (id_rsa.pub) to hosts that require remote connection in from the local.
TopicYum install ius-release-el7.rpm
SourceCentOS 7
ErrorCannot open: https://centos7.iuscommunity.org/ius-release.rpm. Skipping.
Solution#yum -y install https://centos7.iuscommunity.org/ius-release.rpm
yum -y install https://repo.ius.io/ius-release-el7.rpm
RemarkIUS got new URLs and check the ius site for RHEL/CentOS 7 setup
TopicYum -y install PHP packages for LEMP stack
SourceCentOS 7
ErrorERROR: Service ‘web’ failed to build: The command ‘/bin/sh -c yum -y install nginx-1.12.2 openssl –enablerepo=nginx && yum -y install https://centos7.iuscommunity.org/ius-release.rpm && yum -y install php71u-fpm php71u-cli php71u-mysqlnd php71u-soap php71u-xml php71u-zip php71u-json php71u-mcrypt php71u-mbstring php71u-zip php71u-gd –enablerepo=ius && yum clean all’ returned a non-zero code: 1
Solutionyum -y install
#php71u-fpm \
#php71u-cli \
#php71u-mysqlnd \
#php71u-soap \
#php71u-xml \
#php71u-zip \
#php71u-json \
#php71u-mcrypt \
#php71u-mbstring \
#php71u-zip \
#php71u-gd \
php74-fpm \
php74-cli \
php74-mysqlnd \
php74-soap \
php74-xml \
php74-zip \
php74-json \
php74-mcrypt \
php74-mbstring \
php74-zip \
php74-gd
\
RemarkCheck available packages at https://repo.ius.io/7/x86_64/packages/p/
Topicsudo yum update – Error: docker-ce requires containerd.io >= 1.4.1
SourceCentOS 7
Error$ sudo yum update
Error:
Problem:
package docker-ce-3:20.10.16-3.el7.x86_64 requires containerd.io >= 1.4.1, but none
of the providers can be installed
Solution$ sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
RemarkThe server had docker installed beforehand.
TopicError: Failed to download metadata for repo ‘appstream’
SourceCentOS 7 / CentOS 8
Error$ sudo yum install gitlab-runner
CentOS Linux 8 - AppStream 101 B/s | 38 B 00:00
Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
Solution$ sudo yum install dnf -y
$ sudo dnf update -y

$ sudo dnf upgrade
$ sudo dnf --disablerepo '*' --enablerepo=extras swap centos-linux-repos centos-stream-repos
RemarkThe upgrade path is CentOS 7 -> CentOS 8 -> CentOS 8 Stream
Topic500 Internal Server Error when loading web page using .htaccess .htpasswd
SourceApache/2.4.29 (Ubuntu)
Error# /var/log/apache2/error.log
.htaccess: Invalid command 'AuthGroupFile', perhaps misspelled or defined by a module not included in the server configuration
Solution$ a2enmod authz_groupfile
$ systemctl restart apache
Remark$ sudo a2enmod rewrite is commonly mentioned on the Internet but $ sudo a2enmod authz_groupfile is also required in this case.
TopicChange static IP address on Ubuntu 20.04.4 LTS
SourceVNC Viewer > bash
Error[Set 123.123.123.123 as new IP with subnet 255.255.255.0]
$ ifconfig ens3 123.123.123.123/24
$ ifconfig ens3
$ cat /etc/network/interfaces

iface eth0 inet static
address 123.123.123.123
netmask 255.255.255.0
gateway 123.123.123.1
dns-nameservers 8.8.8.8 8.8.4.4
[if systemctl restart networking.service does not work]
$ systemctl list-unit-files | grep -i network
systemd-networkd.service enabled enable
$ systemctl restart systemd-networkd
Solution$ vi /etc/netplan/00-installer-config.yaml
(update new IP addresses, gateway4 and nameservers)
[Use try to check for yaml errors before apply]
$ netplan try
$ netplan apply
RemarkA KVM was migrated and reassigned a new IP. Using ifconfig to update the static IP does not work because on network restart will reinstate the old IP. In this case, updating the 00-installer-config.yaml and using command netplan worked.
TopicFailed to synchronize cache for repo ‘AppStream’ on CentOS 8
SourceCentOS 8
ErrorError: Failed to synchronize cache for repo ‘AppStream’
Solution$ sed -i 's/mirrorlist=/#mirrorlist=/g' /etc/yum.repos.d/CentOS-*
$ sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
RemarkThere are two lines in some CentOS-* files. The solution is to comment out the deprecated mirrorlist.centos.org and uncomment the vault.centos.org
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=AppStream&infra=$infra
baseurl=http://vault.centos.org/$contentdir/$releasever/AppStream/$basearch/os/
gpgcheck=1
TopicError: Unable to find a match: certbot on AlmaLinux 8.6
Source AlmaLinux 8.x | Let’s Encrypt cerbot
Error$ sudo dnf -y install certbot
Last metadata expiration check: 0:00:02 ago on Tue 23 Aug 2022 04:55:26 AM EDT.
No match for argument: certbot
Error: Unable to find a match: certbot
Solutionsudo dnf makecache --refresh
sudo dnf update
sudo dnf install epel-release
sudo dnf -y install certbot
sudo dnf install certbot python3-certbot-nginx
Remarkdnf update is optional but dnf install epel-release probably need to be installed.
TopicMongoDB: v5.x and v6.x installed but mongod service failed to start
SourceDebian 10 (buster) | Official MongoDB Manual > Install on Debian
Error$ sudo systemctl status mongod
mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: failed (Result: signal) since Fri 2022-09-02 17:40:03 +08; 4s ago
Docs: https://docs.mongodb.org/manual
Process: 3780 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=killed, signal=ILL)
Main PID: 3780 (code=killed, signal=ILL)
Solution$sudo service mongod stop
$sudo apt-get purge mongodb-org*
$ sudo rm -r /var/log/mongodb
$ sudo rm -r /var/lib/mongodb
$sudo apt-get autoremove


$ cd /etc/apt/sources.list.d/
$ rm -Rf mongodb-org-5.0.list mongodb-org-6.0.list


$ wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install -y mongodb-org
RemarkThe common solution found on the Internet is chown -R mongodb:mongodb /var/lib/mongodb and chown mongodb:mongodb /tmp/mongodb-27017.sock but this does not work for this case. The reason was MongoDB v5 and v6 cannot run on QEMU VM which is mentioned on here. To check VM CPU, run cat /proc/cpuinfo. Above solution is to remove files from v5 or v6 installation attempts, remove them from souces.list.d and repeat installation step with v4.4 which is the latest patch releases for MongoDB 4.
TopicUpgrade from system Python 3.6 to 3.8
Source MindsDB in Docker on AlmaLinux 8.x
Error2022-09-05 08:55:44,919 - ERROR - /opt/conda/lib/python3.7/site-packages/snowflake/connector/options.py:99: UserWarning: You have an incompatible version of 'pyarrow' installed (6.0.1), please install a version that adheres to: 'pyarrow<8.1.0,>=8.0.0; extra == "pandas"'
_expected_pyarrow_version,
Solution$ sudo dnf groupinstall 'development tools'
$ sudo dnf install bzip2-devel expat-devel gdbm-devel \
ncurses-devel openssl-devel readline-devel wget \
sqlite-devel tk-devel xz-devel zlib-devel libffi-devel
$ wget https://www.python.org/ftp/python/${VERSION}/Python-${VERSION}.tgz
$ tar -xf Python-${VERSION}.tgz
$ cd Python-${VERSION}
$ ./configure --enable-optimizations
$ make -j 4
$ sudo make altinstall
$ python3.8 --version
RemarkLinuxize: How to Install Python 3.8 on CentOS 8
TopicChange Python and Pip version for User in Linux
Source AlmaLinux 8.x
Errorn.a
Solution$ vi ~/.bashrc
User specific aliases and functions
alias python='/usr/local/bin/python3.8'

alias pip='/usr/local/bin/pip3.8'

$ . ~/.bashrc
$ python --version
Python 3.8.13

$ pip --version
pip 22.0.4 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)
Remarklinuxconfig.org: How to change from default to alternative Python version on Debian Linux
TopicOrigin is unreachable | Error code 523
Source AlmaLinux 8.x
ErrorOrigin is unreachable | Error code 523
Visit cloudflare.com for more information.
Browser Working
Cloudflare Working
Host Error
Solution$ sudo systemctl stop firewalld
To permanently enable or disable firewall on AlmaLinux
$ sudo systemctl disable firewalld
RemarkIf it is not due to Nginx conf, Let’s Encrypt SSL and DNS setup, then it is probably the firewall!
TopicFind Installed Google Chrome Version
SourceDebian / AlmaLinux
ErrorNone
Solution$ google-chrome --version
Google Chrome 107.0.5304.121
RemarkChromeDriver (WebDriver for Chrome) must be compatible with Google Chrome browser version. Use case: Selenium setup
TopicMake Python modules (selenium/ webdriver-manager) available to all users
SourceDebian
Erroruser2 has access to Selenium but not user1 hence Python-selenium scripts will fail when executed by user1.
user1@~$ pip3 freeze | grep selenium
su - user2
Password:
user2@~$ pip3 freeze | grep selenium
selenium==4.6.0
Solution$ sudo pip3 install --system selenium==4.6.0
RemarkUse --system to install using the system scheme (overrides
--user on Debian systems) so that all users will have access to Selenium libraries.
TopicInstall mysqlclient database connector for MySQL/MariaDB and Python 3.x failed
Source AlmaLinux
Error$ pip install mysqlclient
Using cached mysqlclient-2.1.1.tar.gz (88 kB)
× python setup.py bdist_wheel did not run successfully.
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for mysqlclient
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
Solution$ ls /usr/bin/python*
$ python -V
Python 3.8.13
$ python3 -V
Python 3.8.13
$ python3.8 -m pip install mysqlclient
RemarkThere are many errors. Tried to update setuptools, wheel, gcc and reinstall mariadb-server etc. but none of these helped. The correct solution (at least in this case) is due to Multiple Python versions as mentioned here. Note that these errors occurred even after setting python and python3 to /usr/bin/python3.8 3 in ~/.bashrc Optional: $ sudo dnf remove python3 to remove old version 3.6
TopicPostfix – SASL authentication failure: No worthy mechs found
Source Debian
Error$ tail -f mail.log
postfix/smtp[538803]: warning: SASL authentication failure: No worthy mechs found
postfix/smtp[538803]: CC87B806E7: to=test-devops@example.com, relay=smtp.mail.com[xx.xx.xx.xx]:587, delay=570, delays=569/0.01/0.92/0, dsn=4.7.0, status=deferred (SASL authentication failed; cannot authenticate to server smtp.mail.com[xx.xx.xx.xx]: no mechanism available)
Solution$ apt-get install libsasl2-modules
RemarkAn error occurs during the sending of an email to a relayhost (main.cf) using Postfix.
TopicSelenium – RequestsDependencyWarning: urllib3 (1.25.2) or chardet (3.0.4) doesn’t match a supported version!
Source Ubuntu
Error$ python /home/user/selenium/script.py
/home/user/venv/lib/python3.8/site-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.1) or chardet (3.0.4) doesn’t match a supported version!
warnings.warn(“urllib3 ({}) or chardet ({}) doesn’t match a supported “
Solution$ pip3 install --upgrade requests
Remarkdriver = webdriver.Chrome("/usr/bin/chromedriver", options=options)
driver.get("https://google.com/")

This is due to different requests module installed by the OS and the python dependencies in the local installation.
TopicGoogle Chrome and chromedriver –
/bin/sh: 1: google-chrome-dev: not found
Exception: No such driver version 115.0.5790.110 for linux64
Source Ubuntu
Error$ python /home/user/selenium2022/script.py
/bin/sh: 1: google-chrome: not found
/bin/sh: 1: google-chrome-stable: not found
/bin/sh: 1: google-chrome-beta: not found
/bin/sh: 1: google-chrome-dev: not found
/bin/sh: 1: google-chrome: not found
/bin/sh: 1: google-chrome-stable: not found
/bin/sh: 1: google-chrome-beta: not found
/bin/sh: 1: google-chrome-dev: not found
/bin/sh: 1: google-chrome: not found
/bin/sh: 1: google-chrome-stable: not found
/bin/sh: 1: google-chrome-beta: not found
/bin/sh: 1: google-chrome-dev: not found


File "/home/user/.local/lib/python3.8/site-packages/webdriver_manager/drivers/chrome.py"
raise Exception(f"No such driver version {browser_version} for {platform}")
Exception: No such driver version 115.0.5790.110 for linux64
$ chromedriver --version
ChromeDriver 114.0.5735.90 (386bc09e8f4f2e025eddae123f36f6263096ae49-refs/branch-heads/5735@
SolutionMUST install BOTH chrome and chromedriver – Guide: https://skolo.online/documents/webscrapping/#step-1-download-chrome
$ wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
E.g., Above will install Google Chrome 115.0.5790.110
Go https://chromedriver.chromium.org/downloads/version-selection leads to https://googlechromelabs.github.io/chrome-for-testing/
On download page, one will see chrome or chromedriver in table column 1 – Get linux64
RemarkBoth Google Chrome and chromedriver are required to work together. The browser and driver must be same version. You may download current stable (google-chrome-stable_current_amd64.deb), install, check version and download matching chromedriver.
TopicSelenium + Chrome + chromedriver – Exception: No such driver version 115.0.5790.110 for linux64
Source Ubuntu
ErrorException: No such driver version 115.0.5790.110 for linux64
$ chromedriver --version
ChromeDriver 115.0.5790.110

Followed all steps to install selenium (via pip), matching chrome and chromedriver but still does not work when run selenium python script.
Solution$ pip uninstall selenium
$ pip uninstall webdriver-manager
$ pip install selenium
$ pip install webdriver-manager

The correct version webdriver-manager will be installed.
RemarkIf chrome browser and chromedriver are installed properly and matching version, just need to sort out Selenium and webdriver-manager via pip
TopicPHP mail() fails to send email via script but success with command-line | Postfix Ignores PHP mail() function
Source Ubuntu
Error[Success – from=mail@myserver.com]
$ echo "This is a test email body." | mail -s "Subject" -a "From: mail@myserver.com" recipient@gmail.com
Aug 17 14:52:21 devops1 postfix/qmgr[558698]: 630A9136DDE: removed
Aug 17 14:54:16 devops1 postfix/pickup[558697]: 65482136DDE: uid=1000 from=mail@myserver.com
Aug 17 14:54:16 devops1 postfix/cleanup[558933]: 65482136DDE: message-id=<20230817065416.65482136DDE@devops1>


[Fail – from=<www-data>]
Aug 17 15:01:46 devops1 postfix/qmgr[558698]: B4D8F136DE1: removed
Aug 17 15:03:13 devops1 postfix/pickup[558697]: D4269136DE1: uid=33 from=<www-data>
Aug 17 15:03:13 devops1 postfix/cleanup[559113]: D4269136DE1: message-id=<20230817070313.D4269136DE1@devops1>

Solution$ sudo vi /etc/postfix/canonical
www-data mail@myserver.com
$ sudo postmap /etc/postfix/canonical
$ sudo postconf -e sender_canonical_maps=hash:/etc/postfix/canonical
$ sudo chown root:root /etc/postfix/canonical
RemarkAfter install and config Postfix, the mail in command line works via relayhost. However, PHP mail() does not work even if /var/log/mail.log: status=sent (250 OK… This is because the from=www-data instead of relayhost sender email address. Solution here
TopicDeprecated ‘network-scripts’ package breaks yum and dnf update in RHEL9 based OS templates
SourceRHEL9 based OS (e.g., CentOS Stream 9, AlmaLinux 9 and Rocky 9)
Error$ sudo yum update
Error:
Problem: package network-scripts-10.11.5-1.el9.x86_64 from @System requires initscripts(x86-64) = 10.11.5-1.el9, but none of the providers can be installed
cannot install both initscripts-10.11.6-1.el9.x86_64 from baseos and initscripts-10.11.5-1.el9.x86_64 from @System
cannot install both initscripts-10.11.6-1.el9.x86_64 from baseos and initscripts-10.11.5-1.el9.x86_64 from baseos
cannot install the best update candidate for package initscripts-10.11.5-1.el9.x86_64
problem with installed package network-scripts-10.11.5-1.el9.x86_64
(try to add ‘--allowerasing‘ to command line to replace conflicting packages or ‘--skip-broken‘ to skip uninstallable packages or ‘--nobest‘ to use not only best candidate packages)
Solution$ sudo yum update --skip-broken
$ sudo dnf update --skip-broken
However, if the above still does not work, try
$ dnf upgrade --refresh --allowerasing
$ systemctl enable NetworkManager
RemarkThe deprecated ‘network-scripts’ package, still required by SolusVM 1, is the cause of this error. If you erase the network scripts package --allowerasing within your VPS and configure NetworkManager instead, please keep in mind that by doing this, the ‘reconfigure network’ button in SolusVM will not work.
Leave a Comment

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply

Your email address will not be published. Required fields are marked *