Chinaunix首页 | 论坛 | 博客
  • 博客访问: 842933
  • 博文数量: 185
  • 博客积分: 495
  • 博客等级: 下士
  • 技术积分: 1417
  • 用 户 组: 普通用户
  • 注册时间: 2012-09-02 15:12
个人简介

治肾虚不含糖,专注内核性能优化二十年。 https://github.com/KnightKu

文章分类

全部博文(185)

文章存档

2019年(1)

2018年(12)

2017年(5)

2016年(23)

2015年(1)

2014年(22)

2013年(82)

2012年(39)

分类: 高性能计算

2019-05-29 17:42:49

# time lfs find /mnt/lustre/ --name *99968*
/mnt/lustre//mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968
/mnt/lustre//mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968

real 1m15.127s
user 0m0.768s
sys 0m31.878s

# time find /mnt/lustre/ -name *99968*
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968

real 1m9.172s
user 0m1.697s
sys 0m31.739s

# time ./build/src/dfind/dfind /mnt/lustre/mdtest.out/ --name *99968* --print
[2019-05-28T17:51:06] Walking /mnt/lustre/mdtest.out
[2019-05-28T17:51:16] Items walked 14105
[2019-05-28T17:51:26] Items walked 30185
[2019-05-28T17:51:36] Items walked 46531
[2019-05-28T17:51:46] Items walked 62787
[2019-05-28T17:51:56] Items walked 78672
[2019-05-28T17:52:06] Items walked 94231
[2019-05-28T17:52:16] Items walked 110084
[2019-05-28T17:52:26] Items walked 126021
[2019-05-28T17:52:36] Items walked 141879
[2019-05-28T17:52:46] Items walked 157511
[2019-05-28T17:52:56] Items walked 173821
[2019-05-28T17:53:06] Items walked 189565
[2019-05-28T17:53:13] Walked 200003 items in 126.816778 seconds (1577.102043 files/sec)
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968

real 2m7.214s
user 0m2.493s
sys 0m55.490s

# time mpirun -np 4 ./build/src/dfind/dfind /mnt/lustre/mdtest.out/ --name *99968* --print
[2019-05-28T17:55:10] Walking /mnt/lustre/mdtest.out
[2019-05-28T17:55:20] Items walked 58428
[2019-05-28T17:55:30] Items walked 126146
[2019-05-28T17:55:40] Items walked 194686
[2019-05-28T17:55:41] Walked 200003 items in 30.882637 seconds (6476.227995 files/sec)
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968

real 0m31.384s
user 0m2.638s
sys 0m45.172s

# time mpirun -np 8 ./build/src/dfind/dfind /mnt/lustre/mdtest.out/ --name *99968* --print
[2019-05-28T17:56:01] Walking /mnt/lustre/mdtest.out
[2019-05-28T17:56:11] Items walked 60890
[2019-05-28T17:56:21] Items walked 130253
[2019-05-28T17:56:31] Items walked 198819
[2019-05-28T17:56:31] Walked 200003 items in 30.248788 seconds (6611.934258 files/sec)
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968

real 0m30.662s
user 0m3.725s
sys 0m50.439s

# time ./pfind /mnt/lustre/mdtest.out/ -name *99968*
--------------------------------------------------------------------------
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968
[0] WARNING, dropped processing of the directory /#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.91256 as the queue is full
/mnt/lustre/mdtest.out/
[DONE] found: 2 (scanned 100001 files, scanned dirents: 200002, unknown dirents: 0)
MATCHED 2/100001

real 1m9.247s
user 0m1.249s
sys 0m29.351s

# time mpirun --allow-run-as-root -n 4 ./pfind /mnt/lustre/mdtest.out/ -name *99968*
/mnt/lustre/mdtest.out/
/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968
[DONE] found: 2 (scanned 100001 files, scanned dirents: 200002, unknown dirents: 0)
MATCHED 2/100001

real 0m19.732s
user 0m1.614s
sys 0m29.576s



10万个文件,单客户端

线程数 find
lfs find dfind pfind
1 1m9.172s
1m15.127s
2m7.214s
1m9.247s
4 N/A
N/A
0m31.384s
0m19.732s
8 N/A
N/A
0m30.662s
0m19.732s


性能方面pfind > dfind >> find > lfs find

后续几篇详细讲这个几个查找工具,以及性能优劣的原因

阅读(145) | 评论(0) | 转发(0) |
0

上一篇:Oprofile/operf使用小结

下一篇:没有了

给主人留下些什么吧!~~
评论热议
请登录后评论。

登录 注册