If your server crashes frequently and if your log throws out the following error message, then you are probably running with buggy kernel version “2.6.18-194.32.1.el5“. This needs to be upgraded to “kernel-2.6.18-238” atleast.
Mar 6 00:36:06 mytv kernel: “echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message.
Mar 6 00:36:06 mytv kernel: mysqld D 0000000000000014 0 2193 2166 2194 (NOTLB)
Mar 6 00:36:06 mytv kernel: ffff8104329ffe18 0000000000000082 ffff810363481558 ffff810432a92a78
Mar 6 00:36:06 mytv kernel: 0000000300000001 0000000000000007 ffff8103317fd7a0 ffff81041d767040
Mar 6 00:36:06 mytv kernel: 000145a166ba6621 0000000001178623 ffff8103317fd988 0000000280008d94
Mar 6 00:36:06 mytv kernel: Call Trace:
Mar 6 00:36:06 mytv kernel: [] __down_read+0x7a/0×92
Mar 6 00:36:06 mytv kernel: [] do_page_fault+0×446/0×874
Mar 6 00:36:06 mytv kernel: [] thread_return+0×62/0xfe
Mar 6 00:36:06 mytv kernel: [] sys_getsockname+0x9c/0xb2
Mar 6 00:36:06 mytv kernel: [] error_exit+0×0/0×84
Mar 6 00:36:06 mytv kernel:
Mar 6 00:36:06 mytv kernel: INFO: task mysqld:27457 blocked for more than 120 seconds.
Mar 6 00:36:06 mytv kernel: “echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message.
Mar 6 00:36:06 mytv kernel: mysqld D 0000000000000014 0 27457 2166 27463 27441 (NOTLB)
Mar 6 00:36:06 mytv kernel: ffff8101db559e18 0000000000000082 0000000000000000 0000000000000001
Mar 6 00:36:06 mytv kernel: ffff8101db559e88 0000000000000001 ffff81029e23f040 ffff810171c3d080
Mar 6 00:36:06 mytv kernel: 000145a4237abea6 000000000000bde6 ffff81029e23f228 0000000200000000
Mar 6 00:36:06 mytv kernel: Call Trace:
Mar 6 00:36:06 mytv kernel: [] generic_file_aio_read+0×34/0×39
Mar 6 00:36:06 mytv kernel: [] __down_read+0x7a/0×92
Mar 6 00:36:06 mytv kernel: [] do_page_fault+0×446/0×874
Mar 6 00:36:06 mytv kernel: [] autoremove_wake_function+0×0/0x2e
Mar 6 00:36:06 mytv kernel: [] thread_return+0×62/0xfe
Mar 6 00:36:06 mytv kernel: [] sys_faccessat+0×148/0x18d
Mar 6 00:36:06 mytv kernel: [] error_exit+0×0/0×84
Mar 6 00:36:06 mytv kernel:
Mar 6 00:36:06 mytv kernel: INFO: task mysqld:27464 blocked for more than 120 seconds.
Mar 6 00:36:06 mytv kernel: “echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message.
Mar 6 00:36:06 mytv kernel: mysqld D 0000000000000014 0 27464 2166 27466 27463 (NOTLB)
Mar 6 00:36:06 mytv kernel: ffff81023fe05e18 0000000000000082 0000000000000000 0000000000000001
Mar 6 00:36:06 mytv kernel: ffff81023fe05e88 0000000000000007 ffff8102151677a0 ffff81043e0c0040
Mar 6 00:36:06 mytv kernel: 000145ae237fd7de 0000000003d504c4 ffff810215167988 0000000200000000
Mar 6 00:36:06 mytv kernel: Call Trace:
Mar 6 00:36:06 mytv kernel: [] generic_file_aio_read+0×34/0×39
Mar 6 00:36:06 mytv kernel: [] __down_read+0x7a/0×92
Mar 6 00:36:06 mytv kernel: [] do_page_fault+0×446/0×874
Mar 6 00:36:06 mytv kernel: [] autoremove_wake_function+0×0/0x2e
Mar 6 00:36:06 mytv kernel: [] sys_faccessat+0×148/0x18d
Mar 6 00:36:06 mytv kernel: [] error_exit+0×0/0×84
Mar 6 00:36:06 mytv kernel:
Steps to upgrade kernel
======================
Create a file “/etc/yum.repo.d/centalt.repo” and add the following entry in to that file..
[CentALT] name=CentALT Packages for Enterprise Linux 5 - $basearch baseurl= enabled=1 gpgcheck=0
then give the following command
#yum upgrade kernel*
This will upgrade the kernel version to”kernel-2.6.18-238.5.1.1.el5″ please make sure that you have default=0 in /etc/grub.conf
Now reboot the server so that the server will have the newer kernel version loaded.
If you don’t trust this repository you can download it from CentOS official site by changing the CentosBase repo file “/etc/yum.repos.d/CentOS-Base.repo” from
[base] name=CentOS-$releasever - Base mirrorlist= =os #baseurl= gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5to
[base] name=CentOS-$releasever - Base mirrorlist= =os #baseurl= gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 And try by using the following command#yum upgrade kernel*
Then reboot the server which will upgrade the kernel version
http://jiemelody.blog.51cto.com/405459/590528