find lfs-find dfind pfind性能对比

# time lfs find /mnt/lustre/ --name *99968*

/mnt/lustre//mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968

/mnt/lustre//mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968

real 1m15.127s

user 0m0.768s

sys 0m31.878s

# time find /mnt/lustre/ -name *99968*

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968

real 1m9.172s

user 0m1.697s

sys 0m31.739s

# time ./build/src/dfind/dfind /mnt/lustre/mdtest.out/ --name *99968* --print

[2019-05-28T17:51:06] Walking /mnt/lustre/mdtest.out

[2019-05-28T17:51:16] Items walked 14105

[2019-05-28T17:51:26] Items walked 30185

[2019-05-28T17:51:36] Items walked 46531

[2019-05-28T17:51:46] Items walked 62787

[2019-05-28T17:51:56] Items walked 78672

[2019-05-28T17:52:06] Items walked 94231

[2019-05-28T17:52:16] Items walked 110084

[2019-05-28T17:52:26] Items walked 126021

[2019-05-28T17:52:36] Items walked 141879

[2019-05-28T17:52:46] Items walked 157511

[2019-05-28T17:52:56] Items walked 173821

[2019-05-28T17:53:06] Items walked 189565

[2019-05-28T17:53:13] Walked 200003 items in 126.816778 seconds (1577.102043 files/sec)

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968

real 2m7.214s

user 0m2.493s

sys 0m55.490s

# time mpirun -np 4 ./build/src/dfind/dfind /mnt/lustre/mdtest.out/ --name *99968* --print

[2019-05-28T17:55:10] Walking /mnt/lustre/mdtest.out

[2019-05-28T17:55:20] Items walked 58428

[2019-05-28T17:55:30] Items walked 126146

[2019-05-28T17:55:40] Items walked 194686

[2019-05-28T17:55:41] Walked 200003 items in 30.882637 seconds (6476.227995 files/sec)

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968

real 0m31.384s

user 0m2.638s

sys 0m45.172s

# time mpirun -np 8 ./build/src/dfind/dfind /mnt/lustre/mdtest.out/ --name *99968* --print

[2019-05-28T17:56:01] Walking /mnt/lustre/mdtest.out

[2019-05-28T17:56:11] Items walked 60890

[2019-05-28T17:56:21] Items walked 130253

[2019-05-28T17:56:31] Items walked 198819

[2019-05-28T17:56:31] Walked 200003 items in 30.248788 seconds (6611.934258 files/sec)

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.99968

real 0m30.662s

user 0m3.725s

sys 0m50.439s

# time ./pfind /mnt/lustre/mdtest.out/ -name *99968*

--------------------------------------------------------------------------

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968

[0] WARNING, dropped processing of the directory /#test-dir.0-0/mdtest_tree.0/dir.mdtest.0.91256 as the queue is full

/mnt/lustre/mdtest.out/

[DONE] found: 2 (scanned 100001 files, scanned dirents: 200002, unknown dirents: 0)

MATCHED 2/100001

real 1m9.247s

user 0m1.249s

sys 0m29.351s

# time mpirun --allow-run-as-root -n 4 ./pfind /mnt/lustre/mdtest.out/ -name *99968*

/mnt/lustre/mdtest.out/

/mnt/lustre/mdtest.out/#test-dir.0-0/mdtest_tree.0/file.mdtest.0.99968

[DONE] found: 2 (scanned 100001 files, scanned dirents: 200002, unknown dirents: 0)

MATCHED 2/100001

real 0m19.732s

user 0m1.614s

sys 0m29.576s

10万个文件,单客户端

线程数	find	lfs find	dfind	pfind
1	1m9.172s	1m15.127s	2m7.214s	1m9.247s
4	N/A	N/A	0m31.384s	0m19.732s
8	N/A	N/A	0m30.662s	0m19.732s

性能方面pfind > dfind >> find > lfs find

后续几篇详细讲这个几个查找工具,以及性能优劣的原因