Continuing the “quick&dirty” series, here’s a solution I implemented on a server where sometimes (once a week or so)
mySQL just dies.
This has unknown (=still to investigate) reasons. The problems appears to happen when there is heavy load on the
Apache server: spikes are unusual and they’re made by malicious scanner probably. Still, the mySQL server shouldn’t
die but just respond slowly, as it’s under heavy load because it powers some web sites on that server.
This has to be investigated, of course.
I the meanwhile, mySQL server has to be up and Apache has to be responsive. Without implementing a “professional”
solution with all the bells and whistles, a thing that can be done is to create a simple watchdog script in
The following script is made for Gentoo Linux, but it should trivial to adapt it to any
The script is designed to be run as a cronjob, because it’s the safest thing: there is no need to ensure the script
itself is always running, as it’s called every few minutes. The definition “few minutes” depend on how long you
want it to check the health of the system.
The locking system, basically just a simple lockfile, is needed to prevent the script being executed twice at the
same time. It’s actually an extremely unlikely event, which could happen only if you schedule the script to be run
very often, like every minute.
Here’s an example of the notification e-mails that are being sent in case of a problem.
MYSQL server not active, restarting...
* Checking mysqld configuration for mysql ... [ ok ]
* Stopping mysql ... [ ok ]
* Starting mysql ... [ ok ]
Linux cattlegrid 5.14.17-x86_64-linode150 #1 SMP Thu Nov 11 13:17:05 EST 2021 x86_64 AMD EPYC 7601 32-Core Processor AuthenticAMD GNU/Linux
17:47:28 up 11 days, 18:06, 2 users, load average: 0,09, 0,19, 0,23
The rest of the script is explained in the comments. It’s actually very simple, maybe even silly, but works like a