一.前言
linux下有大量的系统命令vmstat、iostat等可以反映系统的总体io情况,但是不能监测具体进程的io情况,本文将介绍两种方法:
1.如果内核版本大于2.6.20,通过 /proc/pid/io 便可以获取进程的io信息。
2.通过echo 1 > /proc/sys/vm/block_dump ,来把 block 读写(WRITE/READ/DIRTY)状况 dump 到日志里,通过 dmesg 命令来查看。
二. /proc/pid/io(详细解释:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/proc.txt;hb=HEAD)
示例:
test:/tmp# dd if=/dev/zero of=/tmp/test.dat &
[1] 3828
test:/tmp # cat /proc/3828/io
rchar: 323934931 // 读出的总字节数,read或者pread()中的长度参数总和(pagecache中统计而来,不代表实际磁盘的读入)
wchar: 323929600 // 写入的总字节数,write或者pwrite中的长度参数总和
syscr: 632687 // read()或者pread()总的调用次数
syscw: 632675 // write()或者pwrite()总的调用次数
read_bytes: 0 // 实际从磁盘中读取的字节总数 (这里if=/dev/zero 所以没有实际的读入字节数)
write_bytes: 323932160 // 实际写入到磁盘中的字节总数
cancelled_write_bytes: 0 // 由于截断pagecache导致应该发生而没有发生的写入字节数(可能为负数)
检测工具dstat ( ) 的top_io 模块就是利用此功能实现。dstat的详细说明可以到这里
三.block_dump具体操作步骤是:
#sysctl vm.block_dump=1
or
# echo 1 > /proc/sys/vm/block_dump
然后就可以通过 dmesg 就可以观察到各个进程 IO 活动的状况了:
#dmesg -c
kjournald(542): WRITE block 222528 on dm-0
kjournald(542): WRITE block 222552 on dm-0
bash(18498): dirtied inode 5892488 (ld-linux-x86-64.so.2) on dm-0
bash(18498): dirtied inode 5892482 (ld-2.5.so) on dm-0
dmesg(18498): dirtied inode 11262038 (ld.so.cache) on dm-0
dmesg(18498): dirtied inode 5892496 (libc.so.6) on dm-0
dmesg(18498): dirtied inode 5892489 (libc-2.5.so) on dm-0
通过一些脚本分析这些数据就可以得到各个进程的IO情况。下文便描述了这种方法(不同版本的linux下这里dmesg日志格式可能不同)
How to find per-process I/O statistics on Linux
Newer Linux kernels have per-process I/O accounting and you can use the iotop tool to find out what’s performing I/O, but in many cases I’m trying to find the source of an I/O problem in an older kernel. I found sort of a hack-ish way to do that today, while trying to figure out why a system was basically unresponsive.
I founda post on Stack Overflowthat showed a way you can get per process I/O statistics from the kernel even in older kernels. I adapted this to my needs, and wrote a little script.
Here’s how you use it. First, get it:
wget
Then turn on kernel messages about I/O:
echo 1 > /proc/sys/vm/block_dump
This makes the kernel start writing messages about every I/O operation that takes place. Now all you have to do is get those messages and feed them into my script:
while true; do sleep 1; dmesg -c; done | perl iodump
Wait a little while, then cancel the script. The results should look something like the following:
:~# while true; do sleep 1; dmesg -c; done | perl iodump
^C# Caught SIGINT.
TASK PID TOTAL READ WRITE DIRTY DEVICES
firefox 4450 4538 251 4287 0 sda4, sda3
kjournald 2100 551 0 551 0 sda4
firefox 28452 185 185 0 0 sda4
kjournald 782 59 0 59 0 sda3
pdflush 31 30 0 30 0 sda4, sda3
syslogd 2485 2 0 2 0 sda3
firefox 28414 2 2 0 0 sda4, sda3
firefox 28413 1 1 0 0 sda4
firefox 28410 1 1 0 0 sda4
firefox 28307 1 1 0 0 sda4
firefox 28451 1 1 0 0 sda4
I deliberately generated a bunch
of I/O by deleting my Firefox history and cache.
这里还有个网上找的脚本分析dmesg 的输出的:
#!/bin/bash
#/etc/init.d/rsyslog stop
/etc/init.d/syslog stop
echo 1 > /proc/sys/vm/block_dump
dmesg -c >/dev/null
sleep 60
#dmesg | awk '/(READ|WRITE|dirtied)/ {process[$2]++} END {for (x in process) {print process[x], x;} }' |sort -nr |awk '{print $2" "$1}' | head -n 10
dmesg | awk '/WRITE/ {process[$2]++} END {for (x in process) {print process[x], x;} }' |sort -nr |awk '{print $2" "$1}' | head -n 10
echo 0 > /proc/sys/vm/block_dump
/etc/init.d/syslog start
#/etc/init.d/rsyslog start
阅读(1899) | 评论(0) | 转发(0) |