雪峰磁针石 测试 linux pythontesting.blog.chinaunix.net

深圳自动化测试python:113938272 武冈深圳高级群:66250781

  • 博客访问: 18392700
  • 博文数量: 822
  • 博客积分: 10495
  • 博客等级: 上将
  • 技术积分: 9513
  • 用 户 组: 普通用户
  • 注册时间: 2006-07-18 10:51
文章分类

全部博文(822)

文章存档

2012年(8)

2011年(90)

2010年(97)

2009年(166)

2008年(208)

2007年(179)

2006年(93)

微信关注

IT168企业级官微



微信号:IT168qiye



系统架构师大会



微信号:SACC2013

订阅
热词专题

分类: Python/Ruby

python snmp 自动化2-在python中使用snmp


#2012-02-23 磁针石

#承接软件自动化实施与培训 验证码破解 软件破解 脚本开发 测试和python培训等

#gtalk ouyangchongwu#gmail.com qq 37391319 博客:testing.blog.chinaunix.net

#版权所有,转载刊登请来函联系

#自动化测试和python群组: http://groups.google.com/group/automation_testing_python

#python qq group: 深圳自动化测试python群:113938272

#武冈深圳qq群:66250781 都梁深圳湖南户外群:49494279

#参考资料

为什么不使用pysnmp

         Pysnmp是一个纯pythonsnmp模块,不过我们的mib文件,大概有20%的,Pysnmp不能成功读取,报错如下:

# build-pysnmp-mib -o fsp150cm-sa.mib.txt fsp150cm-sa.mib

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

Traceback (most recent call last):

  File "/usr/bin/libsmi2pysnmp", line 5, in <module>

    pkg_resources.run_script('pysnmp==4.2.1', 'libsmi2pysnmp')

  File "/usr/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 489, in run_script

  File "/usr/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 1214, in run_script

  File "/usr/lib/python2.7/site-packages/pysnmp-4.2.1-py2.7.egg/EGG-INFO/scripts/libsmi2pysnmp", line 435, in <module>

   

  File "/usr/lib/python2.7/site-packages/pysnmp-4.2.1-py2.7.egg/EGG-INFO/scripts/libsmi2pysnmp", line 147, in __genTypeDef

   

KeyError: 'syntax'

smidump -k -f python fsp150cm-sa.mib | /usr/bin/libsmi2pysnmp fails

make sure you are using libsmi version > 0.4.5 (or svn)

 

         网上有篇netsnmppysnmp比较的评论:

This isn't so much an answer as much as me sharing my experience testing these libraries.

 

I ran some snmpwalk benchmarks on PySNMP 4.x and net-snmp using python's multiprocessing.Pool. Benchmarks were pretty dirty (basically scaled up the thread pool until I didn't get any performance gains), so reader be-ware. Here's some observations:

 

netsnmp's python bindings:

 

    * Could pull down 11958 oids/sec on the quad core test box.

    * Didn't consume a lot of CPU. Seemed to be waiting on network I/O most of the time (keep reading).

    * Didn't support snmpbulkwalk, unfortunately. So this generates more network traffic and a single thread of execution tends to be slow.

    * It has some annoying MIB look up behavior. I had to specify 'ifName' instead of 'IF-MIB::ifName', which could lead to some ambiguity. I also couldn't seem to find a way to control MIB lookups very well.

    * Threading will not work well. Even if you want one thread of execution, run it in a separate process so you don't starve other threads. This library is thread safe, but not thread friendly.

 

PySNMP4:

 

    * Came in at 5560 oids/sec on the same box.

    * Very CPU intensive. I attribute this to the packet parsing being done in python.

    * MIB lookups I thought were really nice.

    * snmpwalks would leak some unrelated OIDs. For example, I'd walk IF-MIB::ifXTable and at the end I'd get IF-MIB::ifStackTable. IF-MIB dump.

    * I'd almost certainly tailor a wrapper for my application instead of using this library directly. Specifically I'd wrap all the error handling to use Exceptions.

    * I'm not a big fan of writing/reading asynchronous code, so I'd just ignore all the async bits and run big SNMP operations in a separate process.

 

Overall, I'm really kind of disappointed. There's really not a "best overall" library. Apart from the API and performance, PySNMP4 is great. Apart from having some strange MIB/oid lookup handling behavior and not supporting many bulk operations, NetSNMP's python bindings were great.

         可见pysnmp强在mib解析等方面,性能方面不能和netsnmp媲美。既然我们的mib pysnmp无法解析,只好放弃,期待pysnmp尽快开发出自己的好的mib解析器。只有选择netsnmp

        

为什么不使用subprocess

         Subprocess会经常出现超时。

 

Python Get示例:

单个get

print "Get ntpClientEnabled"

oid = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0')

oidList = netsnmp.VarList(oid)

resultList = netsnmp.snmpget(oid ,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

 

以下是通过类的方法进行Get

 

session = netsnmp.Session(Version=2,DestHost='172.23.192.44',Community='private')

oid = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0',2,'INTEGER')

oidList = netsnmp.VarList(oid)

resultList = session.get(oidList)

print resultList         

 

多个get

print "Get ntpClientEnabled"

oid = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0')

oid1 =  netsnmp.Varbind('1.3.6.1.4.1.2544.1.12.8.1.1.1.5','1.1.1.1','15','INTEGER')

resultList = netsnmp.snmpget(oid ,oid1,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

 

以下是通过类的方法进行Get

session = netsnmp.Session(Version=2,DestHost='172.23.192.44',Community='private')

oid2 = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0',2,'INTEGER')

oid3 = netsnmp.Varbind('.1.3.6.1.4.1.2544.1.12.8.1.1.1.5','1.1.1.1',20,'INTEGER')

oidList = netsnmp.VarList(oid3,oid2)

resultList = session.get(oidList)

print resultList

 

注意类和函数的方式的区别。类方法始终要求传入的参数为oid列表,函数的方式则可以把单个或者多个oid作为参数传入,不需要oid列表。通过代码可以看到函数的方式其实调用类方法,为此,可能效率要比类方法低。

Python Set示例

 

调用netsnmp.snmpset一次set多个值:

print "Set ntpClientEnabled"

oid = netsnmp.Varbind('1.3.6.1.4.1.2544.1.12.2.1.10.1','0','2','INTEGER')

oid1 =  netsnmp.Varbind('1.3.6.1.4.1.2544.1.12.8.1.1.1.5','1.1.1.1','15','INTEGER')

resultList = netsnmp.snmpset(oid, oid1,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

         上面的1.3.6.1.4.1.前面是否加点号没有关系。Varbindinit方法如下,tag可以用tid,比如ntpClientEnablediid表示后面带的序号,比如0,1.1.1.1val表示具体的值,type表示类型,更详细的参见python netsnmp库的READMEVal'2''15'可以不加引号,以为这里是整型,建议还是全部加上引号。

         通过tcpdump抓包分析,snmpset多个值也发送了一次请求,因此多个set集合在一起发送,会明显提高效率。

class Varbind(object):

    def __init__(self, tag=None, iid=None, val=None, type=None):

        

         以下是通过类的方法进行set

session = netsnmp.Session(Version=2,DestHost='172.23.192.44',Community='private')

oid2 = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0',2,'INTEGER')

oid3 = netsnmp.Varbind('.1.3.6.1.4.1.2544.1.12.8.1.1.1.5','1.1.1.1',20,'INTEGER')

oidList = netsnmp.VarList(oid3,oid2)

resultList = session.set(oidList)

print resultList         

 

Mib翻译

通过snmptranslate -Tso >toid.txt可以导出tidoid的对应表。

通过snmptranslate –Td 可以获取oid的详细定义。

# snmptranslate -Td .1.3.6.1.4.1.2544.1.12.2.1.10.1

CM-SYSTEM-MIB::ntpClientEnabled

ntpClientEnabled OBJECT-TYPE

  -- FROM       CM-SYSTEM-MIB

  -- TEXTUAL CONVENTION TruthValue

  SYNTAX        INTEGER {true(1), false(2)}

  MAX-ACCESS    read-write

  STATUS        current

  DESCRIPTION   "This allows to enable/disable the NTP client."

::= { iso(1) org(3) dod(6) internet(1) private(4) enterprises(1) advaMIB(2544) products(1) fsp150cm(12) cmSystemMIB(2) cmSystemObjects(1) cmTimeObjects(10) 1 }

为了提高可读性,我们需要在测试中使用true来表示1false来表示2,并用ntpClientEnabled来表示.1.3.6.1.4.1.2544.1.12.2.1.10.1,遗憾的是,netsnmppython库有如下限制“Access to the parsed MIB database is not yet implemented.”为此MIB的转换需要自行完成。为此拟定如下数据结构:

{ntpClientEnabled.1.3.6.1.4.1.2544.1.12.2.1.10.1}

{ ntpClientEnabled:{ true:1, false:2}}

2个字典只针对有别名的部分oid

如果mib库不大,可以通过如下的脚本获取:

#!/usr/bin/python

# -*- coding: utf-8 -*-

# Function: Ssh to remote server

# Author:         Andrew Xu

# CreateDate: 2012/02/28

 

import subprocess

import re

import multiprocessing

import pickle

import time

import random

 

def producer(sequence, input_p):

    for item in sequence:

        # Put the item on the queue

        input_p.send(item)

 

# 消费者

def consumer(lock,pipe):

    output_p, input_p = pipe

    input_p.close() # 关闭管道输入口

    while True:

             lock.acquire()

             try:

                   time.sleep(1)

                   item = output_p.recv()

                   # 管道读取失败的预防,重试一次

             except Exception:

                   item = output_p.recv()

             lock.release()

             if item == None:

                   break

        

             tid = item

             oid = snmpDict[item]

        

             # 处理部分

             try:

                   result = subprocess.check_output(['snmptranslate', '-Td',oid])

                   # subprocess失败的预防,重试一次

             except Exception:

                   time.sleep(random.randint(1,8))

                   result = subprocess.check_output(['snmptranslate', '-Td',oid])

            

             syntaxLine = syntax.search(result)

             if  syntaxLine:               

                   syntaxLine = syntaxLine.group(1)       

                   valueitems = items.findall(syntaxLine)

                   snmpValueDict[tid] = dict(valueitems)         

             else:

                   lock.acquire()

                   print(tid,"None")

                   lock.release()            

                   continue

            

             lock.acquire()

             print(tid,dict(valueitems))

             lock.release()

       

   

 

if __name__ == '__main__':

   

    snmpDict = dict()

    snmpValueDict = dict()

    # 读取语法部分

    syntax = re.compile(r'^  SYNTAX.*?\{(.*?)\}',re.MULTILINE)

    # 读取字符串 和对应的值

    items = re.compile('(\w*)\((\w*)\)')

    f = open ("snmpValueDict.txt",'w')

   

    # 生成tidoid对应的字典

    for line in open("tid_oid.txt"):

         tid, oid = line.strip().split(',')

         snmpDict[tid] = oid   

 

    pickle.dump(snmpValueDict, open("snmpTidOid.txt", "w"))     

        

    print "length of snmpDict:" + str(len(snmpDict))

   

    # 进程数、创建管道,锁等

    p_num = 10

    process = []   

    (output_p, input_p) = multiprocessing.Pipe()

    lock = multiprocessing.Lock()

   

    # 定义消费进程

    for i in range(p_num):

         t =multiprocessing.Process(target=consumer,args=(lock,(output_p, input_p),))     

         t.daemon=True

         process.append(t)   

 

    # 启动消费进程

    for i in range(p_num):

         process[i].start()

        

    # 关闭输出管道,以往管道填充数据

    output_p.close()

    sequence = snmpDict.values() + [None]*p_num

    producer(snmpDict.keys(), input_p)   

    # 数据填充完毕,打开输入管道

    input_p.close()

   

    # 等待结束

    for i in range(p_num):

         process[i].join()

   

    pickle.dump(snmpValueDict, open("snmpValueDict.txt", "w"))   

    print "length of snmpDict:" + str(len(snmpDict))   

    print "length of snmpValueDict:" + str(len(snmpValueDict))     

 

以上也是一个多进程程序的演示,如果miboid超过1000条,建议采用如下脚本来提取:

#!/usr/bin/python

# -*- coding: utf-8 -*-

# Function: Ssh to remote server

# Author:         Andrew Xu

# CreateDate: 2012/02/28

 

import re

import glob

import pickle

 

typeDict = dict()

snmpDict = dict()

snmpValueDict = dict()

 

# tid and oid dict

for line in open("tid_oid.txt"):

    tid, oid = line.strip().split(',')

    snmpDict[tid] = oid   

   

pickle.dump(snmpDict, open("snmpTidOid.txt", "w"))   

 

# type Dict

for mibFile in glob.glob(r"/home/share/andrew/soft/mib/110/MIBs/*.mib"):

    files = open(mibFile).read()

    text =re.sub("\s+"," ",files)

    result = re.findall(r'(\w*)\s+OBJECT-TYPE SYNTAX\s+\w+\s\{(.*?\(\w\).*?)\}',text)

 

    for dictName,values in result:

        splitValues = re.findall('(\w+)\s*?\((\w+)\)',values)

        typeDict[dictName] = dict(splitValues)

 

pickle.dump(snmpDict, open("typeDict.txt", "w"))               

 

# snmp Value Dict

for mibFile in glob.glob(r"D:\soft\mib\110\MIBs\*.mib"):

    print mibFile

    files = open(mibFile).read()

    text =re.sub("\s+"," ",files)

    result = re.findall(r'\s(\w*?) ::= TEXTUAL-CONVENTION STATUS current DESCRIPTION.*?SYNTAX\s+\w+\s\{(.*?\(\w\).*?)\}',text)

    print result   

    for dictName,values in result:

        splitValues = re.findall('(\w+)\s*?\((\w+)\)',values)

        snmpValueDict[dictName] = dict(splitValues)

       

    result = re.findall(r'\s(\w*?) ::= TEXTUAL-CONVENTION STATUS current DESCRIPTION.*?SYNTAX\s+\w+\s\{(.*?\(\w\).*?)\}',text)

 

pickle.dump(snmpValueDict, open("snmpValueDict.txt", "w"))   

 

上面的脚本需要根据mib文件格式的不同进行相应的调整。

现在就可以加载存储在文件中的字典,利用tid代替oid进行测试了。

 

#!/usr/bin/python

# -*- coding: utf-8 -*-

# Function: Ssh to remote server

# Author:         Andrew Xu

# CreateDate: 2012/02/28

 

 

import netsnmp

import re

import pickle

 

snmpTOidDict = pickle.load(open(r"snmpTidOid.txt"))

snmpValueDict = pickle.load(open(r"snmpValueDict.txt"))

 

print "Set ntpClientEnabled"

print snmpValueDict['ntpClientEnabled']

oid = netsnmp.Varbind(snmpTOidDict['ntpClientEnabled'], '0', snmpValueDict['ntpClientEnabled']['true'],'INTEGER')

oid1 =  netsnmp.Varbind(snmpTOidDict['ecpaControlDuration'],'1.1.1.1','15','INTEGER')

resultList = netsnmp.snmpset(oid, oid1,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

 

下面搞个复杂点的用例,在某产品中创建和删除保护组:

命令行执行如下:

# snmpset -v2c -c private   172.23.192.44 cmFacProtGroupSwitchMode.1.1.1.1 i 1 cmFacProtGroupWorkPort.1.1.1.1 o 1.3.6.1.4.1.2544.1.12.4.1.7.1.1.1.1.1.1 cmFacProtGroupProtPort.1.1.1.1 o 1.3.6.1.4.1.2544.1.12.4.1.7.1.1.1.1.1.2 cmFacProtGroupRowStatus.1.1.1.1 i 4

 

CM-PROTECTION-MIB::cmFacProtGroupSwitchMode.1.1.1.1 = INTEGER: oneplusone(1)

CM-PROTECTION-MIB::cmFacProtGroupWorkPort.1.1.1.1 = OID: CM-FACILITY-MIB::cmEthernetNetPortIndex.1.1.1.1

CM-PROTECTION-MIB::cmFacProtGroupProtPort.1.1.1.1 = OID: CM-FACILITY-MIB::cmEthernetNetPortIndex.1.1.1.2

CM-PROTECTION-MIB::cmFacProtGroupRowStatus.1.1.1.1 = INTEGER: createAndGo(4)

 

# snmpset -v2c -c private  172.23.192.44 cmFacProtGroupRowStatus.1.1.1.1 i 6 CM-PROTECTION-MIB::cmFacProtGroupRowStatus.1.1.1.1 = INTEGER: destroy(6)

 

通过python实现如下

#!/usr/bin/python

# -*- coding: utf-8 -*-

# Function: Ssh to remote server

# Author:         Andrew Xu

# CreateDate: 2012/02/28

 

import netsnmp

import re

import pickle

 

snmpTOidDict = pickle.load(open(r"snmpTidOid.txt"))

snmpValueDict = pickle.load(open(r"snmpValueDict.txt"))

 

print  snmpValueDict['cmFacProtGroupRowStatus']

 

oid = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupSwitchMode'], '1.1.1.1',

                      snmpValueDict['cmFacProtGroupSwitchMode']['oneplusone'],'INTEGER')

oid1 = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupWorkPort'], '1.1.1.1',

                      snmpTOidDict['cmEthernetNetPortIndex'] + '.1.1.1.1','OBJECTID')

oid2 = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupProtPort'], '1.1.1.1',

                      snmpTOidDict['cmEthernetNetPortIndex'] + '.1.1.1.2','OBJECTID')

oid3 = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupRowStatus'], '1.1.1.1',

                      snmpValueDict['cmFacProtGroupRowStatus']['createAndGo'],'INTEGER')

 

resultList = netsnmp.snmpset(oid, oid1,oid2, oid3,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

 

oid3 = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupRowStatus'], '1.1.1.1',

                      snmpValueDict['cmFacProtGroupRowStatus']['destroy'],'INTEGER')

resultList = netsnmp.snmpset( oid3,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

Normal 0 7.8 pt 0 2 false false false EN-US ZH-CN X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.5pt; mso-bidi-font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi; mso-font-kerning:1.0pt;}

当然上面这种调用方式,参数显得有点多,正式形成库时会再次封装。

 

性能

session = netsnmp.Session(Version=2,DestHost='172.23.192.44',Community='private')

oid = netsnmp.Varbind('1.3.6.1.4.1.2544.1.12','',2,'INTEGER')

oidList = netsnmp.VarList(oid)

resultList = session.walk(oidList)

print resultList         

print len(resultList)

 

通过上述代码,执行time ./test.py walk的效率:耗时:5m38.919s oid个数:13385

平均秒只能获取约40oid。和参考资料中测试得5560 oids/sec有百倍左右的差距。不过我们的设备已经吃到近90%cpu,这部分有待以后继续研究优化。

 

 

 


阅读(453752) | 评论(0) | 转发(1) |
给主人留下些什么吧!~~
评论热议
请登录后评论。

登录 注册