python共享内存测试

python的multiprocessing模块提供两种共享内存，sharedctypes与Manager，

Manager效率较低，但支持远程共享内存。

sharedctypes效率较高，快Manager两个数量级，在多进程访问时与普通内存访问相当

结果比较如下：

test array

elapsed 0:00:00.119707

test dict

elapsed 0:00:00.152856

test shared manager list

elapsed 0:00:37.87666

test sharedctypes list in main process

elapsed 0:00:00.154170

test sharedctypes list in subprocess

elapsed 0:00:00.303328

elapsed 0:00:00.327261

elapsed 0:00:00.349312

elapsed 0:00:00.318946

elapsed 0:00:00.364301

elapsed 0:00:00.370176

elapsed 0:00:00.343588

elapsed 0:00:00.348737

代码如下：

import array
from datetime import datetime, timedelta

size = 1000000
def tranverse(a):
    t = datetime.now()
    for i in range(size):
        a[i]
    print 'elapsed %s'% (datetime.now()- t)

a = array.array('i', [i for i in range(size)])
print 'test array'
tranverse(a)


a = {}
for i in range(size):
    a[i] = i
print 'test dict'
tranverse(a)


from multiprocessing import Manager
manager = Manager()
a = manager.list([i for i in range(size)])
print 'test shared manager list'
tranverse(a)


from multiprocessing.sharedctypes import RawArray
a = RawArray( 'i', [i for i in range(size)] )
print 'test sharedctypes list in main process'
tranverse(a)

from multiprocessing import Process
ps = [Process(target=tranverse, args=(a, )) for i in range(8)]
print 'test sharedctypes list in subprocess'
for p in ps:
    p.start()
for p in ps:
    p.join()