Hunting down elusive sources of iowait

A question I'm asked daily is "How can I find out what is generating iowait on my server?" Sure, you can dig through pages of lsof output, restart services, or run strace, but it can be a frustrating process. I saw a process on this blog post, and I changed the regexes to fit Red Hat and CentOS systems a bit better:

# /etc/init.d/syslog stop
# echo 1 > /proc/sys/vm/block_dump
# dmesg | egrep "READ|WRITE|dirtied" | egrep -o '([a-zA-Z]*)' | sort | uniq -c | sort -rn | head
1526 mysqld
819 httpd
429 kjournald
35 qmail
27 in
7 imapd
6 irqbalance
5 pop
4 pdflush
3 spamc

In my specific situation, it looks like MySQL is the biggest abuser of my disk, followed by Apache and the filesystem journaling. As expected, qmail is a large contender, too.

Don't forget to set things back to their normal state when you're done!

# echo 0 > /proc/sys/vm/block_dump
# /etc/init.d/syslog start

注：

我的执行有问题，，大家可以用试试

#dmesg|awk '{print $2}'|sort|uniq -c|sort -rn|head
   1270 kjournald(1930):
    164 pdflush(23280):
    156 mysqld(10011):
     20 mysqld(9950):
     18 kjournald(1934):
     15 mysqld(9605):
     15 mysqld(10054):
     13 mysqld(9838):
     12 mysqld(9752):
     10 mysqld(10109):