FreePBX - High Load Averages
-
I failed to run
sarin a timely fashion. When this spike occurs again, I'll know to run it. -
@eddiejennings said in FreePBX - High Load Averages:
I failed to run
sarin a timely fashion. When this spike occurs again, I'll know to run it.If you don't have sar installed already, go ahead and install it so you know what "good" looks like too.
-
@dafyre said in FreePBX - High Load Averages:
@eddiejennings said in FreePBX - High Load Averages:
I failed to run
sarin a timely fashion. When this spike occurs again, I'll know to run it.If you don't have sar installed already, go ahead and install it so you know what "good" looks like too.
Yeah, I ran it and got a copy of the results. That how I knew I missed my opportunity.
-
@bigbear said in FreePBX - High Load Averages:
What size instance did you go for on Vultr? Not relevant, but curious.
The 1 CPU, 1 GB RAM, 25 GB SSD instance ($5 / month).
-
@scottalanmiller said in FreePBX - High Load Averages:
By default you will have about 8-12 httpd processes. How many did you have?
19 httpd processes during the problem time. 11 when the problem went away.
-
@eddiejennings said in FreePBX - High Load Averages:
@dafyre said in FreePBX - High Load Averages:
@eddiejennings said in FreePBX - High Load Averages:
I failed to run
sarin a timely fashion. When this spike occurs again, I'll know to run it.If you don't have sar installed already, go ahead and install it so you know what "good" looks like too.
Yeah, I ran it and got a copy of the results. That how I knew I missed my opportunity.
Did you check the logs (usually stored in /var/log/sysstat/)?
Try
sar -f /var/log/sysstat/sa02to see the stats from yesterday. -
In the middle of an episode now. I'm collecting data and will post in a moment.
-
Here are the
sar -dresults.For ease of reading, I put the window where the load average spike was at the top. Next is
topfrom about 2/3 of the way through. Since this seems to solve itself, I'm more curious as to figure out what's causing tasks to be queued.Linux 3.10.0-514.26.2.el7.x86_64 (blah.domain.com) 10/10/2017 _x86_64_ (1 CPU) DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util 01:30:02 PM dev252-0 9.94 0.74 385.64 38.87 0.01 0.86 0.18 0.18 01:30:02 PM dev253-0 9.81 0.66 385.64 39.40 0.01 0.90 0.19 0.18 01:30:02 PM dev253-1 0.01 0.08 0.00 8.00 0.00 1.00 0.17 0.00 01:40:02 PM dev252-0 9.34 6.86 354.74 38.70 0.01 0.81 0.17 0.16 01:40:02 PM dev253-0 9.18 5.87 354.74 39.28 0.01 0.84 0.18 0.16 01:40:02 PM dev253-1 0.12 0.99 0.00 8.00 0.00 0.56 0.10 0.00 01:50:01 PM dev252-0 12.80 251.00 333.08 45.64 0.01 1.16 0.30 0.38 01:50:01 PM dev253-0 12.25 238.55 313.44 45.07 0.01 1.16 0.28 0.34 01:50:01 PM dev253-1 4.01 12.45 19.65 8.00 0.07 18.65 0.12 0.05 Linux 3.10.0-514.26.2.el7.x86_64 (blah.domain.com) 10/10/2017 _x86_64_ (1 CPU) 12:00:02 AM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util 12:10:01 AM dev252-0 12.24 233.79 399.69 51.75 0.02 1.27 0.31 0.38 12:10:01 AM dev253-0 12.07 233.47 379.27 50.76 0.01 1.14 0.27 0.33 12:10:01 AM dev253-1 2.59 0.33 20.42 8.00 0.13 49.31 0.26 0.07 12:20:01 AM dev252-0 11.28 129.02 334.39 41.08 0.01 1.19 0.30 0.33 12:20:01 AM dev253-0 9.72 80.71 328.20 42.05 0.01 1.27 0.26 0.26 12:20:01 AM dev253-1 6.81 48.31 6.19 8.00 0.03 4.17 0.12 0.08 12:30:01 AM dev252-0 8.89 0.37 339.30 38.20 0.01 1.01 0.20 0.18 12:30:01 AM dev253-0 8.75 0.00 339.30 38.77 0.01 1.04 0.20 0.18 12:30:01 AM dev253-1 0.05 0.37 0.00 8.00 0.00 0.67 0.11 0.00 12:40:01 AM dev252-0 7.81 0.45 297.80 38.19 0.01 0.83 0.19 0.15 12:40:01 AM dev253-0 7.66 0.03 294.60 38.46 0.01 0.87 0.19 0.15 12:40:01 AM dev253-1 0.45 0.42 3.20 8.00 0.00 1.93 0.04 0.00 12:50:02 AM dev252-0 9.27 0.34 362.79 39.18 0.01 0.69 0.16 0.15 12:50:02 AM dev253-0 9.13 0.09 361.99 39.66 0.01 0.72 0.16 0.15 12:50:02 AM dev253-1 0.13 0.24 0.80 8.00 0.00 1.35 0.12 0.00 01:00:02 AM dev252-0 8.93 21.75 326.97 39.07 0.01 0.77 0.16 0.15 01:00:02 AM dev253-0 8.78 21.45 326.97 39.69 0.01 0.80 0.17 0.15 01:00:02 AM dev253-1 0.04 0.30 0.00 8.00 0.00 0.41 0.09 0.00 01:10:02 AM dev252-0 12.50 256.42 408.88 53.23 0.01 1.09 0.27 0.33 01:10:02 AM dev253-0 12.31 256.17 378.13 51.53 0.01 0.95 0.20 0.25 01:10:02 AM dev253-1 3.88 0.26 30.74 8.00 0.20 51.04 0.24 0.09 01:20:02 AM dev252-0 9.64 6.37 345.35 36.49 0.01 0.79 0.18 0.18 01:20:02 AM dev253-0 9.47 6.15 336.18 36.14 0.01 0.82 0.19 0.18 01:20:02 AM dev253-1 1.17 0.22 9.16 8.00 0.00 2.71 0.02 0.00 01:30:01 AM dev252-0 9.63 10.99 366.75 39.22 0.01 0.89 0.17 0.16 01:30:01 AM dev253-0 9.49 10.78 366.75 39.79 0.01 0.92 0.17 0.16 01:30:01 AM dev253-1 0.03 0.20 0.00 8.00 0.00 0.87 0.20 0.00 01:40:01 AM dev252-0 9.94 40.40 369.56 41.23 0.01 0.88 0.18 0.18 01:40:01 AM dev253-0 9.78 40.21 363.37 41.28 0.01 0.90 0.18 0.18 01:40:01 AM dev253-1 0.80 0.19 6.18 8.00 0.01 7.92 0.08 0.01 01:50:01 AM dev252-0 9.05 39.06 333.04 41.10 0.01 0.85 0.17 0.15 01:50:01 AM dev253-0 8.88 38.42 327.97 41.26 0.01 0.88 0.17 0.15 01:50:01 AM dev253-1 0.72 0.64 5.09 8.00 0.00 6.84 0.06 0.00 02:00:01 AM dev252-0 8.61 0.20 330.00 38.33 0.01 0.84 0.17 0.15 02:00:01 AM dev253-0 8.42 0.01 329.98 39.17 0.01 0.87 0.18 0.15 02:00:01 AM dev253-1 0.02 0.19 0.00 8.00 0.00 0.57 0.14 0.00 02:10:01 AM dev252-0 10.85 178.08 373.68 50.83 0.01 1.00 0.24 0.26 02:10:01 AM dev253-0 10.66 177.83 364.17 50.85 0.01 0.95 0.19 0.20 02:10:01 AM dev253-1 1.22 0.24 9.51 8.00 0.05 42.90 0.51 0.06 02:20:01 AM dev252-0 8.84 5.29 330.72 38.02 0.01 0.80 0.15 0.14 02:20:01 AM dev253-0 8.68 4.91 330.72 38.68 0.01 0.83 0.16 0.14 02:20:01 AM dev253-1 0.05 0.38 0.00 8.00 0.00 0.71 0.14 0.00 02:30:01 AM dev252-0 8.63 0.92 329.72 38.31 0.01 0.78 0.16 0.14 02:30:01 AM dev253-0 8.48 0.00 329.72 38.89 0.01 0.81 0.17 0.14 02:30:01 AM dev253-1 0.12 0.92 0.00 8.00 0.00 0.50 0.09 0.00 02:40:01 AM dev252-0 10.39 18.80 401.01 40.39 0.01 0.86 0.17 0.18 02:40:01 AM dev253-0 10.24 18.65 401.01 40.98 0.01 0.88 0.18 0.18 02:40:01 AM dev253-1 0.02 0.15 0.00 8.00 0.00 1.00 0.18 0.00 02:50:01 AM dev252-0 9.64 79.81 346.50 44.24 0.01 0.90 0.19 0.19 02:50:01 AM dev253-0 9.32 75.35 328.13 43.29 0.01 0.89 0.18 0.17 02:50:01 AM dev253-1 2.85 4.46 18.38 8.00 0.06 19.69 0.07 0.02 03:00:02 AM dev252-0 8.91 16.85 323.36 38.17 0.01 0.77 0.17 0.15 03:00:02 AM dev253-0 8.74 16.46 323.36 38.89 0.01 0.81 0.18 0.15 03:00:02 AM dev253-1 0.05 0.39 0.00 8.00 0.00 0.28 0.03 0.00 03:10:01 AM dev252-0 11.32 259.35 366.04 55.27 0.02 1.95 0.30 0.34 03:10:01 AM dev253-0 11.09 258.06 337.96 53.72 0.02 1.96 0.30 0.33 03:10:01 AM dev253-1 3.67 1.29 28.08 8.00 0.04 10.70 0.05 0.02 03:20:01 AM dev252-0 8.81 8.00 322.37 37.49 0.01 0.98 0.19 0.17 03:20:01 AM dev253-0 8.64 5.87 322.37 37.98 0.01 1.02 0.20 0.17 03:20:01 AM dev253-1 0.27 2.13 0.00 8.00 0.00 0.55 0.08 0.00 03:30:01 AM dev252-0 8.83 25.89 326.49 39.90 0.01 0.80 0.18 0.16 03:30:01 AM dev253-0 8.69 22.91 326.49 40.23 0.01 0.83 0.19 0.16 03:30:01 AM dev253-1 0.37 2.98 0.00 8.00 0.00 0.52 0.08 0.00 03:40:01 AM dev252-0 8.53 0.14 322.66 37.84 0.01 0.80 0.17 0.15 03:40:01 AM dev253-0 8.40 0.00 322.66 38.43 0.01 0.83 0.18 0.15 03:40:01 AM dev253-1 0.02 0.14 0.00 8.00 0.00 0.80 0.10 0.00 03:50:01 AM dev252-0 9.30 0.00 359.01 38.61 0.01 0.77 0.17 0.15 03:50:01 AM dev253-0 9.17 0.00 359.01 39.15 0.01 0.80 0.17 0.16 03:50:01 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:00:01 AM dev252-0 9.12 0.88 354.46 38.98 0.01 0.80 0.17 0.16 04:00:01 AM dev253-0 8.97 0.01 354.46 39.52 0.01 0.83 0.18 0.16 04:00:01 AM dev253-1 0.11 0.87 0.00 8.00 0.00 0.39 0.06 0.00 04:10:01 AM dev252-0 18.91 237.02 391.35 33.23 0.02 1.23 0.51 0.96 04:10:01 AM dev253-0 18.68 236.53 364.37 32.17 0.02 1.26 0.50 0.93 04:10:01 AM dev253-1 3.43 0.49 26.98 8.00 0.09 26.85 0.11 0.04 04:20:01 AM dev252-0 8.17 16.84 309.93 39.98 0.01 1.18 0.23 0.19 04:20:01 AM dev253-0 8.06 16.75 309.93 40.52 0.01 1.23 0.24 0.19 04:20:01 AM dev253-1 0.01 0.10 0.00 8.00 0.00 1.00 0.14 0.00 04:30:02 AM dev252-0 8.84 0.00 341.94 38.69 0.01 0.78 0.18 0.16 04:30:02 AM dev253-0 8.72 0.00 341.94 39.22 0.01 0.81 0.18 0.16 04:30:02 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:40:02 AM dev252-0 8.16 0.11 306.14 37.52 0.01 0.86 0.17 0.14 04:40:02 AM dev253-0 8.02 0.00 306.14 38.19 0.01 0.90 0.17 0.14 04:40:02 AM dev253-1 0.01 0.11 0.00 8.00 0.00 0.12 0.12 0.00 04:50:01 AM dev252-0 7.09 9.28 251.23 36.76 0.01 0.82 0.16 0.11 04:50:01 AM dev253-0 7.00 9.28 251.23 37.24 0.01 0.86 0.17 0.12 04:50:01 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:00:02 AM dev252-0 8.72 0.07 333.19 38.23 0.01 0.80 0.18 0.16 05:00:02 AM dev253-0 8.58 0.00 333.19 38.86 0.01 0.83 0.18 0.16 05:00:02 AM dev253-1 0.01 0.07 0.00 8.00 0.00 0.00 0.00 0.00 05:10:02 AM dev252-0 10.97 163.28 358.36 47.57 0.01 1.13 0.23 0.26 05:10:02 AM dev253-0 10.78 163.06 337.65 46.46 0.01 0.96 0.19 0.20 05:10:02 AM dev253-1 2.62 0.22 20.71 8.00 0.21 79.52 0.23 0.06 05:20:01 AM dev252-0 8.23 7.02 303.95 37.79 0.01 0.90 0.18 0.15 05:20:01 AM dev253-0 8.06 6.67 303.95 38.53 0.01 0.93 0.18 0.14 05:20:01 AM dev253-1 0.04 0.35 0.00 8.00 0.00 1.00 0.85 0.00 05:30:01 AM dev252-0 9.80 0.52 388.72 39.72 0.01 0.82 0.17 0.16 05:30:01 AM dev253-0 9.63 0.41 388.72 40.42 0.01 0.85 0.17 0.17 05:30:01 AM dev253-1 0.01 0.11 0.00 8.00 0.00 1.00 0.13 0.00 05:40:01 AM dev252-0 9.62 0.01 380.32 39.53 0.01 0.84 0.17 0.16 05:40:01 AM dev253-0 9.47 0.01 380.32 40.16 0.01 0.87 0.17 0.16 05:40:01 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:50:01 AM dev252-0 7.96 0.11 309.45 38.89 0.01 0.87 0.17 0.14 05:50:01 AM dev253-0 7.84 0.00 309.45 39.49 0.01 0.90 0.18 0.14 05:50:01 AM dev253-1 0.01 0.11 0.00 8.00 0.00 1.00 0.12 0.00 06:00:01 AM dev252-0 8.85 15.84 336.92 39.88 0.01 0.86 0.16 0.15 06:00:01 AM dev253-0 8.71 15.84 336.92 40.48 0.01 0.90 0.17 0.15 06:00:01 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 06:10:02 AM dev252-0 11.35 142.47 406.56 48.35 0.01 1.13 0.22 0.25 06:10:02 AM dev253-0 11.17 142.33 368.50 45.73 0.01 0.91 0.19 0.21 06:10:02 AM dev253-1 4.77 0.14 38.06 8.00 0.30 63.66 0.09 0.04 06:20:01 AM dev252-0 9.11 0.01 348.27 38.25 0.01 0.87 0.16 0.15 06:20:01 AM dev253-0 8.97 0.01 348.27 38.82 0.01 0.90 0.17 0.15 06:20:01 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 06:30:01 AM dev252-0 9.57 0.00 370.63 38.74 0.01 0.79 0.17 0.16 06:30:01 AM dev253-0 9.46 0.00 370.63 39.19 0.01 0.82 0.17 0.16 06:30:01 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 06:40:01 AM dev252-0 8.84 0.11 339.69 38.46 0.01 0.85 0.18 0.16 06:40:01 AM dev253-0 8.70 0.00 339.69 39.05 0.01 0.88 0.19 0.16 06:40:01 AM dev253-1 0.01 0.11 0.00 8.00 0.00 1.00 0.13 0.00 06:50:02 AM dev252-0 8.75 0.12 336.83 38.53 0.01 0.91 0.18 0.16 06:50:02 AM dev253-0 8.65 0.01 336.83 38.94 0.01 0.95 0.19 0.16 06:50:02 AM dev253-1 0.01 0.11 0.00 8.00 0.00 1.00 0.12 0.00 07:00:01 AM dev252-0 6.96 0.11 252.95 36.35 0.01 0.72 0.17 0.12 07:00:01 AM dev253-0 6.83 0.00 252.95 37.01 0.01 0.76 0.17 0.12 07:00:01 AM dev253-1 0.01 0.11 0.00 8.00 0.00 1.00 0.12 0.00 07:10:01 AM dev252-0 9.22 70.69 311.74 41.48 0.01 0.89 0.21 0.19 07:10:01 AM dev253-0 9.12 70.58 306.22 41.30 0.01 0.86 0.18 0.16 07:10:01 AM dev253-1 0.70 0.11 5.53 8.00 0.04 61.35 0.49 0.03 07:20:01 AM dev252-0 7.77 0.07 295.80 38.06 0.01 0.78 0.18 0.14 07:20:01 AM dev253-0 7.66 0.00 295.80 38.64 0.01 0.82 0.18 0.14 07:20:01 AM dev253-1 0.01 0.07 0.00 8.00 0.00 0.00 0.00 0.00 07:30:01 AM dev252-0 9.43 0.00 379.00 40.21 0.01 0.87 0.17 0.16 07:30:01 AM dev253-0 9.31 0.00 379.00 40.71 0.01 0.90 0.18 0.17 07:30:01 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:40:02 AM dev252-0 9.80 0.50 384.10 39.24 0.01 0.88 0.17 0.17 07:40:02 AM dev253-0 9.67 0.50 384.10 39.78 0.01 0.91 0.17 0.17 07:40:02 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:50:01 AM dev252-0 10.26 0.23 403.08 39.32 0.01 0.84 0.16 0.16 07:50:01 AM dev253-0 10.13 0.00 403.08 39.78 0.01 0.87 0.16 0.16 07:50:01 AM dev253-1 0.03 0.23 0.00 8.00 0.00 1.41 0.24 0.00 08:00:01 AM dev252-0 9.67 0.00 380.94 39.38 0.01 0.95 0.17 0.17 08:00:01 AM dev253-0 9.55 0.00 380.94 39.91 0.01 0.98 0.18 0.17 08:00:01 AM dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 08:10:01 AM dev252-0 8.58 33.98 309.32 40.01 0.01 1.03 0.20 0.17 08:10:01 AM dev253-0 8.46 33.71 295.24 38.88 0.01 0.94 0.17 0.14 08:10:01 AM dev253-1 1.79 0.27 14.08 8.00 0.08 42.57 0.19 0.03 08:20:01 AM dev252-0 9.54 0.10 363.35 38.11 0.01 0.82 0.17 0.16 08:20:01 AM dev253-0 9.37 0.00 363.35 38.77 0.01 0.85 0.17 0.16 08:20:01 AM dev253-1 0.01 0.10 0.00 8.00 0.00 0.00 0.00 0.00 08:30:01 AM dev252-0 7.87 0.26 292.76 37.23 0.01 0.81 0.17 0.13 08:30:01 AM dev253-0 7.74 0.00 292.76 37.85 0.01 0.84 0.17 0.13 08:30:01 AM dev253-1 0.03 0.26 0.00 8.00 0.00 0.84 0.16 0.00 08:40:01 AM dev252-0 12.61 295.61 340.57 50.43 0.01 1.09 0.24 0.31 08:40:01 AM dev253-0 11.40 233.28 334.94 49.83 0.01 1.16 0.22 0.25 08:40:01 AM dev253-1 8.49 62.32 5.63 8.00 0.02 2.13 0.07 0.06 08:50:01 AM dev252-0 10.26 70.09 337.74 39.74 0.01 0.94 0.27 0.28 08:50:01 AM dev253-0 9.42 31.42 325.61 37.89 0.01 0.98 0.25 0.24 08:50:01 AM dev253-1 6.35 38.67 12.13 8.00 0.01 1.39 0.08 0.05 09:00:01 AM dev252-0 9.18 3.81 328.34 36.18 0.01 1.02 0.30 0.28 09:00:01 AM dev253-0 8.99 2.81 328.34 36.85 0.01 1.06 0.31 0.28 09:00:01 AM dev253-1 0.13 1.00 0.00 8.00 0.00 0.68 0.09 0.00 09:10:02 AM dev252-0 11.57 210.57 356.26 49.00 0.01 1.10 0.35 0.41 09:10:02 AM dev253-0 11.32 209.14 335.07 48.07 0.01 1.04 0.31 0.36 09:10:02 AM dev253-1 2.83 1.43 21.19 8.00 0.08 29.33 0.20 0.06 09:20:01 AM dev252-0 9.21 19.13 335.23 38.50 0.01 1.09 0.30 0.28 09:20:01 AM dev253-0 9.02 18.70 329.72 38.62 0.01 1.13 0.31 0.28 09:20:01 AM dev253-1 0.74 0.43 5.51 8.00 0.00 5.74 0.05 0.00 09:30:02 AM dev252-0 11.11 113.42 325.10 39.48 0.01 1.22 0.28 0.31 09:30:02 AM dev253-0 10.52 100.73 325.10 40.46 0.01 1.29 0.29 0.31 09:30:02 AM dev253-1 1.59 12.69 0.00 8.00 0.00 0.62 0.12 0.02 09:40:01 AM dev252-0 7.39 5.41 247.88 34.26 0.01 1.07 0.31 0.23 09:40:01 AM dev253-0 7.24 4.83 246.45 34.71 0.01 1.10 0.30 0.22 09:40:01 AM dev253-1 0.25 0.58 1.43 8.00 0.01 53.57 0.53 0.01 09:50:02 AM dev252-0 9.69 5.50 346.63 36.33 0.01 1.21 0.36 0.34 09:50:02 AM dev253-0 9.53 5.18 343.15 36.56 0.01 1.20 0.32 0.31 09:50:02 AM dev253-1 0.47 0.32 3.47 8.00 0.03 63.39 0.88 0.04 10:00:02 AM dev252-0 9.39 20.13 324.59 36.72 0.01 1.19 0.29 0.27 10:00:02 AM dev253-0 9.12 13.65 324.59 37.09 0.01 1.23 0.30 0.27 10:00:02 AM dev253-1 0.81 6.48 0.00 8.00 0.00 0.44 0.06 0.01 10:10:01 AM dev252-0 13.53 309.25 322.27 46.67 0.02 1.25 0.32 0.43 10:10:01 AM dev253-0 13.34 308.34 317.79 46.92 0.02 1.25 0.29 0.39 10:10:01 AM dev253-1 0.67 0.91 4.49 8.00 0.00 4.88 0.59 0.04 10:20:02 AM dev252-0 9.76 20.04 352.26 38.16 0.01 1.27 0.36 0.35 10:20:02 AM dev253-0 9.61 19.88 325.23 35.93 0.01 1.14 0.31 0.29 10:20:02 AM dev253-1 3.40 0.16 27.03 8.00 0.16 45.86 0.18 0.06 10:30:01 AM dev252-0 9.12 43.14 291.33 36.68 0.01 1.18 0.29 0.27 10:30:01 AM dev253-0 8.97 41.37 291.33 37.08 0.01 1.22 0.30 0.27 10:30:01 AM dev253-1 0.22 1.78 0.00 8.00 0.00 0.55 0.08 0.00 10:40:01 AM dev252-0 11.45 79.54 372.85 39.50 0.01 1.10 0.30 0.35 10:40:01 AM dev253-0 11.07 67.40 359.68 38.57 0.01 1.08 0.28 0.31 10:40:01 AM dev253-1 3.16 12.14 13.16 8.00 0.07 23.53 0.14 0.04 10:50:01 AM dev252-0 7.92 6.46 287.52 37.11 0.01 0.80 0.16 0.13 10:50:01 AM dev253-0 7.78 5.96 287.52 37.72 0.01 0.83 0.17 0.13 10:50:01 AM dev253-1 0.06 0.50 0.00 8.00 0.00 0.46 0.08 0.00 11:00:01 AM dev252-0 10.37 9.63 404.71 39.95 0.01 0.85 0.17 0.17 11:00:01 AM dev253-0 10.20 9.02 404.71 40.58 0.01 0.88 0.17 0.18 11:00:01 AM dev253-1 0.08 0.61 0.00 8.00 0.00 0.73 0.20 0.00 11:10:01 AM dev252-0 8.47 122.05 299.60 49.77 0.01 0.91 0.19 0.16 11:10:01 AM dev253-0 8.37 121.98 299.60 50.36 0.01 0.94 0.19 0.16 11:10:01 AM dev253-1 0.01 0.07 0.00 8.00 0.00 0.60 0.20 0.00 11:20:01 AM dev252-0 8.09 0.70 296.62 36.76 0.01 0.76 0.16 0.13 11:20:01 AM dev253-0 7.95 0.60 296.62 37.39 0.01 0.80 0.17 0.13 11:20:01 AM dev253-1 0.01 0.11 0.00 8.00 0.00 1.00 0.12 0.00 11:30:01 AM dev252-0 7.87 0.20 298.17 37.93 0.01 0.75 0.16 0.13 11:30:01 AM dev253-0 7.71 0.00 298.17 38.67 0.01 0.79 0.17 0.13 11:30:01 AM dev253-1 0.03 0.20 0.00 8.00 0.00 0.53 0.07 0.00 11:40:01 AM dev252-0 7.91 0.20 299.06 37.82 0.01 0.86 0.17 0.14 11:40:01 AM dev253-0 7.78 0.09 299.06 38.47 0.01 0.89 0.18 0.14 11:40:01 AM dev253-1 0.01 0.11 0.00 8.00 0.00 0.38 0.12 0.00 11:50:01 AM dev252-0 9.73 1.34 373.22 38.49 0.01 0.82 0.17 0.16 11:50:01 AM dev253-0 9.61 1.27 373.22 38.97 0.01 0.86 0.17 0.16 11:50:01 AM dev253-1 0.01 0.07 0.00 8.00 0.00 0.00 0.00 0.00 12:00:01 PM dev252-0 9.38 9.96 348.15 38.18 0.01 0.80 0.16 0.15 12:00:01 PM dev253-0 9.18 7.72 348.15 38.76 0.01 0.83 0.17 0.15 12:00:01 PM dev253-1 0.28 2.24 0.00 8.00 0.00 0.61 0.09 0.00 12:10:01 PM dev252-0 11.24 68.67 377.66 39.71 0.01 0.77 0.18 0.20 12:10:01 PM dev253-0 11.16 68.10 377.66 39.96 0.01 0.81 0.18 0.21 12:10:01 PM dev253-1 0.07 0.57 0.00 8.00 0.00 0.38 0.05 0.00 12:20:01 PM dev252-0 10.17 67.09 348.83 40.91 0.01 0.79 0.18 0.18 12:20:01 PM dev253-0 10.01 63.91 348.83 41.23 0.01 0.83 0.18 0.18 12:20:01 PM dev253-1 0.40 3.19 0.00 8.00 0.00 0.63 0.08 0.00 12:30:01 PM dev252-0 11.32 73.12 387.48 40.70 0.01 0.98 0.17 0.19 12:30:01 PM dev253-0 11.14 72.04 383.56 40.89 0.01 1.01 0.17 0.19 12:30:01 PM dev253-1 0.63 1.09 3.92 8.00 0.00 3.98 0.05 0.00 12:40:01 PM dev252-0 9.54 27.83 359.99 40.64 0.01 0.82 0.19 0.18 12:40:01 PM dev253-0 9.24 22.79 349.00 40.24 0.01 0.82 0.18 0.16 12:40:01 PM dev253-1 2.00 5.04 10.98 8.00 0.03 16.11 0.10 0.02 12:50:01 PM dev252-0 13.21 168.62 459.35 47.55 0.01 1.04 0.23 0.30 12:50:01 PM dev253-0 12.65 147.03 436.96 46.15 0.01 1.04 0.19 0.25 12:50:01 PM dev253-1 5.50 21.58 22.40 8.00 0.05 9.44 0.10 0.06 01:00:01 PM dev252-0 9.47 20.41 330.61 37.05 0.01 0.81 0.17 0.16 01:00:01 PM dev253-0 9.30 19.75 329.37 37.54 0.01 0.84 0.17 0.16 01:00:01 PM dev253-1 0.24 0.67 1.24 8.00 0.00 1.63 0.11 0.00 01:10:02 PM dev252-0 11.36 128.90 373.16 44.20 0.01 0.99 0.22 0.25 01:10:02 PM dev253-0 11.15 127.88 334.19 41.45 0.01 0.96 0.19 0.21 01:10:02 PM dev253-1 5.00 1.01 38.96 8.00 0.06 11.88 0.10 0.05 01:20:01 PM dev252-0 11.28 12.59 425.62 38.84 0.01 0.97 0.19 0.21 01:20:01 PM dev253-0 11.07 10.27 425.62 39.36 0.01 1.00 0.19 0.21 01:20:01 PM dev253-1 0.29 2.32 0.00 8.00 0.00 0.61 0.10 0.00topoutputtop - 13:47:33 up 6 days, 2:12, 1 user, load average: 6.47, 5.67, 2.87 Tasks: 111 total, 1 running, 110 sleeping, 0 stopped, 0 zombie %Cpu(s): 5.2 us, 3.7 sy, 0.0 ni, 90.8 id, 0.1 wa, 0.0 hi, 0.0 si, 0.2 st KiB Mem : 1016380 total, 243200 free, 527096 used, 246084 buff/cache KiB Swap: 2097148 total, 1688092 free, 409056 used. 253448 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 9871 root 20 0 157568 2072 1500 R 6.7 0.2 0:00.01 top 1 root 20 0 43380 2452 1428 S 0.0 0.2 3:22.61 systemd 2 root 20 0 0 0 0 S 0.0 0.0 0:00.10 kthreadd 3 root 20 0 0 0 0 S 0.0 0.0 0:04.54 ksoftirqd/0 7 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0 8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh 9 root 20 0 0 0 0 S 0.0 0.0 1:10.13 rcu_sched 10 root rt 0 0 0 0 S 0.0 0.0 0:03.73 watchdog/0 12 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kdevtmpfs 13 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 netns 14 root 20 0 0 0 0 S 0.0 0.0 0:02.51 khungtaskd 15 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 writeback 16 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kintegrityd 17 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset 18 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kblockd 19 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 md 25 root 20 0 0 0 0 S 0.0 0.0 0:26.06 kswapd0 26 root 25 5 0 0 0 S 0.0 0.0 0:00.00 ksmd 27 root 39 19 0 0 0 S 0.0 0.0 0:09.18 khugepaged 28 root 20 0 0 0 0 S 0.0 0.0 0:00.00 fsnotify_mark 29 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 crypto 37 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kthrotld 39 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kmpath_rdacd 40 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kpsmoused 41 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ipv6_addrconf 60 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 deferwq 92 root 20 0 0 0 0 S 0.0 0.0 0:03.85 kauditd 264 root 0 -20 0 0 0 S 0.0 0.0 0:00.05 ata_sff 280 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_0 281 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 scsi_tmf_0 282 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_1 283 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 scsi_tmf_1 285 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ttm_swap 359 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kdmflush 360 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset 371 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kdmflush 372 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset 385 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfsalloc 386 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfs_mru_cache 387 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfs-buf/dm-0 388 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfs-data/dm-0 389 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfs-conv/dm-0 390 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfs-cil/dm-0 391 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfs-reclaim/dm- 392 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfs-log/dm-0 393 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfs-eofblocks/d 394 root 20 0 0 0 0 S 0.0 0.0 2:56.36 xfsaild/dm-0 469 root 20 0 36120 1340 1200 S 0.0 0.1 2:01.94 systemd-journal 484 root 20 0 127084 0 0 S 0.0 0.0 0:00.02 lvmetad 498 root 20 0 43632 292 288 S 0.0 0.0 0:00.07 systemd-udevd 516 root 20 0 0 0 0 S 0.0 0.0 0:00.00 vballoon 517 root 20 0 0 0 0 S 0.0 0.0 0:06.08 hwrng 581 root 20 0 0 0 0 S 0.0 0.0 0:00.00 jbd2/vda1-8 582 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ext4-rsv-conver 597 root 16 -4 55412 432 312 S 0.0 0.0 0:19.40 auditd 619 avahi 20 0 27976 428 300 S 0.0 0.0 0:13.94 avahi-daemon 620 dbus 20 0 24536 788 456 S 0.0 0.1 2:42.62 dbus-daemon 623 avahi 20 0 27976 24 0 S 0.0 0.0 0:00.00 avahi-daemon 627 root 20 0 105476 160 92 S 0.0 0.0 0:00.04 sshd 629 root 20 0 553148 552 268 S 0.0 0.1 1:26.93 tuned 630 root 20 0 24308 984 768 S 0.0 0.1 1:27.94 systemd-logind 636 root 20 0 6964 180 140 S 0.0 0.0 0:00.20 mdadm 637 root 20 0 262428 1908 1608 S 0.0 0.2 0:59.10 rsyslogd 639 root 20 0 541036 12456 9172 S 0.0 1.2 0:38.44 httpd 645 polkitd 20 0 527508 912 608 S 0.0 0.1 0:59.68 polkitd 653 root 20 0 14988 340 232 S 0.0 0.0 0:00.04 incrond 661 chrony 20 0 115844 728 624 S 0.0 0.1 0:01.74 chronyd 665 nobody 20 0 15540 96 92 S 0.0 0.0 0:00.00 dnsmasq 697 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 cfg80211 708 root 20 0 25840 0 0 S 0.0 0.0 0:00.02 atd 721 root 20 0 126248 808 400 S 0.0 0.1 0:34.60 crond 733 root 20 0 27100 620 616 S 0.0 0.1 0:00.00 xinetd 738 root 20 0 110032 64 60 S 0.0 0.0 0:00.00 agetty 746 asterisk 20 0 749892 49024 27388 S 0.0 4.8 1:35.82 httpd 752 mysql 20 0 113252 52 52 S 0.0 0.0 0:00.01 mysqld_safe 822 asterisk 20 0 748824 47468 27728 S 0.0 4.7 4:24.32 httpd 828 mongodb 20 0 483944 9532 588 S 0.0 0.9 28:42.09 mongod 850 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 dio/dm-0 1034 mysql 20 0 1177564 165648 2656 S 0.0 16.3 59:20.55 mysqld 1148 root 20 0 89032 600 516 S 0.0 0.1 0:04.09 master 1207 root 20 0 112876 468 236 S 0.0 0.0 0:00.06 dhclient 3395 asterisk 20 0 650608 20176 9308 S 0.0 2.0 0:00.04 httpd 3758 asterisk 20 0 657232 38972 20556 S 0.0 3.8 0:45.70 httpd 3843 root 0 -20 0 0 0 S 0.0 0.0 0:00.74 kworker/0:2H 5837 root 20 0 115240 44 40 S 0.0 0.0 0:00.00 safe_asterisk 5844 asterisk 20 0 1645160 41756 5708 S 0.0 4.1 76:50.41 asterisk 6477 root 20 0 0 0 0 S 0.0 0.0 0:00.05 kworker/0:0 7038 asterisk 20 0 746004 43712 23384 S 0.0 4.3 0:35.92 httpd 10255 root 20 0 0 0 0 S 0.0 0.0 0:00.63 kworker/0:1 10422 root 20 0 148384 5936 4588 S 0.0 0.6 0:00.08 sshd 11069 root 0 -20 0 0 0 S 0.0 0.0 0:03.32 kworker/0:1H 14137 asterisk 20 0 747104 47268 28128 S 0.0 4.7 0:10.80 httpd 14138 asterisk 20 0 650384 20712 9536 S 0.0 2.0 0:00.10 httpd 15724 root 20 0 115380 2076 1676 S 0.0 0.2 0:00.02 bash 16487 asterisk 20 0 1005508 19880 4076 S 0.0 2.0 7:23.89 PM2 v2.7.1: God 16503 asterisk 20 0 1214940 18216 1884 S 0.0 1.8 7:22.57 letschat 16833 asterisk 20 0 195328 668 248 S 0.0 0.1 0:00.23 pnp_server 17498 asterisk 20 0 1087448 37676 7188 S 0.0 3.7 0:09.16 node /var/www/h 17615 root 20 0 0 0 0 S 0.0 0.0 0:00.19 kworker/u2:1 19180 asterisk 20 0 741236 28020 13168 S 0.0 2.8 1:13.27 httpd 19600 asterisk 20 0 659456 45256 26600 S 0.0 4.5 0:27.39 httpd 20008 asterisk 20 0 650888 20976 9400 S 0.0 2.1 0:01.55 httpd 20137 asterisk 20 0 390124 16112 9436 S 0.0 1.6 0:00.19 php 20168 asterisk 20 0 650860 20816 9260 S 0.0 2.0 0:01.34 httpd 27522 root 20 0 0 0 0 S 0.0 0.0 0:00.52 kworker/0:2 30030 asterisk 20 0 417620 26224 10636 S 0.0 2.6 0:23.46 php 30330 root 20 0 0 0 0 S 0.0 0.0 0:00.04 kworker/u2:0 30730 postfix 20 0 89136 3572 2572 S 0.0 0.4 0:00.03 pickup 30793 postfix 20 0 89312 2804 2660 S 0.0 0.3 0:00.15 qmgr 31123 root 20 0 398792 11796 4960 S 0.0 1.2 14:21.45 php 31176 root 20 0 185668 1504 1144 S 0.0 0.1 0:09.04 fail2ban-server -
Ah. Reading through the man pages for
sar. Fromsar -qlooks like there were other spikes. Looks like I have some data collection and cross-referencing in my future.
-
One more little tidbit of information. I can replicate the spike by viewing the Connectivity > Firewall page(s).
