Hello
I have Magento 2.3.4
Site Ground Cloud Server: 5 CPUs, 4 GB RAM
Every time run the cron, the server kills it after several seconds.
I made a script that kills all the pending crons from the table before the cron runs
but i still face the same issue while running the cron and this is why only several cron jobs complete every time and not all of them (server kills the process).
I have attache the answer from the server support.
I will be very happy if someone can help me here to solve this issue.
---------------------------------------------------------------------------------------------
The statistics in your User Area are not in real time. The RAM is being exhausted only for a few seconds which is why you cannot see the spikes in the statistics inside the User Area.
Here is what happens:
1) The following cron job runs at the beginning of every hour.
0 * * * * /usr/local/php73/bin/php-cli -f /home/customer/www/arabinstruments.com/web/bin/magento cron:run
2) The queries spawned by the Magento cron are causing a very high number of read requests to the actual hard disk.
2020-08-03 01:03:24.638695-05 | IOPs-Read: 12411 | IOPs-Write: 2248
2020-08-03 01:02:22.869841-05 | IOPs-Read: 24143 | IOPs-Write: 3033
2020-08-03 01:01:21.723683-05 | IOPs-Read: 35169 | IOPs-Write: 1687
2020-08-02 03:01:23.304137-05 | IOPs-Read: 39000 | IOPs-Write: 2086
2020-08-02 03:00:21.864408-05 | IOPs-Read: 13095 | IOPs-Write: 1458
3) Because of that the server's RAM depletes for a few seconds and this causes a high load on the server.
4) During the high server load our monitoring system starts killing the PHP processes in order to stabilize back the server.
Aug 03 01:00:28 c16608: Critical load (101.80) reached! Killing all php,ftp,smtp,mailnull,archivers procs!
Aug 03 02:00:23 c16608: Critical load (43.42) reached! Killing all php,ftp,smtp,mailnull,archivers procs!
A well-optimized application should not generate more than 4000 read and write request in total. As you can see from the above logs, the queries spawned by the Magento cron are generating between 10000-35000 requests per second.
If the number of the input/output operations is too high, the corresponding processes become very slow and RAM consumable. Adding more RAM to your server may solve the issue, but this is not the correct approach in this case. The correct approach would be to check why the queries spawned by the Magento cron are generating that much read requests and to optimize them.
