How to investigate load issue
Get your current sysload:-
18:54:44 up 22 days, 22:47, 1 user, load average: 2.54, 2.69, 2.40
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
karlo pts/0 5ed07f32.cm-7-1b 18:49 0.00s 0.09s 0.00s w
cat /proc/loadavg
2.49 2.68 2.39 1/149 2085
The load on server is around 2.5. This means before an interrupt (program call) is done, it has to wait for 2.5 tasks. It’s like a queue. On a 20 core system, a system load of 2 would be no problem. If it is a 2 core system then we need to investigate the issue as below:-
1. Check for CPU usage using top or htop.
2 High IO
High input/output on your disks can lead to a high sysload. You can use iostat and iotop to investigate this. You might have to install them first. On RHEL based systems, iostat is in the package sysstat.
To get the current IO, try “iostat 1?:
iostat 1:- Below is the output:-
Linux 3.2.2 (srv02.leanback.eu) 06/26/2013 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.53 0.14 1.19 2.73 0.25 95.16
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
xvdap1 0.84 2.63 2.59 5222348 5128012
xvdap2 5.87 33.14 44.00 65726181 87260968
avg-cpu: %user %nice %system %iowait %steal %idle
0.50 0.00 0.50 75.38 0.00 23.62
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
xvdap1 0.00 0.00 0.00 0 0
xvdap2 46.00 0.00 788.00 0 788
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 0.00 49.49 0.00 50.51
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
xvdap1 0.00 0.00 0.00 0 0
xvdap2 7.00 0.00 32.00 0 32
If you have high read/write numbers, this is the problem. Investigate what process is causing it with iotop.
iotop
The process at the top is the one which uses most IO. There might be something wrong there. Kill it.
Still if not fixed, then check if the system is swapping
If your system is using swap, this can lead to a high system load. Check out your memory usage with ‘free’:
$ free -m
total used free shared buffers cached
Mem: 990 958 32 0 27 276
-/+ buffers/cache: 653 336
Swap: 1023 73 950
As can be seen above, total of about 1GB RAM . There is only 32MB left. And it has used Swap (see the Swap row, and then under the ‘free’ column). This is not a good sign. Scroll back to the part about htop, and sort on memory. This will tell you what is using your precious RAM.
If that is done, you can:
1) Clear your SWAP. Only so this if you have enough free memory. As root, do: “swapoff -a; swapon -a”.
2) Disable swap as much as possible. As root, do “echo vm.swappiness=0 >> /etc/sysctl.conf ” and reboot. This will make sure your system only swapps if it is out of memory.
If still you are having problems then check for hardware. For example, if a HDD is broken, or your RAID array is rebuilding, this will impact performance (and system load). You might want to check your dmesg output ” dmesg | less ” for errors. There might be something in /var/log/messages .
If your HDDs and bios support SMART, you might get info from there. Install smartmontools , find out what devices your file systems are on ( cat /etc/fstab | cut -d’ ‘ -f1 | grep -v none ) and scan them:
$ sudo smartctl -a /dev/sda
by-
We Specialize In cPanel Server Management
Affordable 24/7 Technical Support Company
