Plumber分布式数据采集系统（二）Manager API

一,API

1.根目录: /api/$API_VERSION
example

2.注册
URI:POST /register
PARAMETER:
ip 注册agent所用IP
port 注册agent所用端口
type 注册agent的类型,可选值为source或sink
topic agent负责的topic,如果负责多个topic则出现多次
RESPONSE:
success 注册成功
failed:$REASON 注册失败
example

DELETE 

success

3.处理结果上报
URI: POST /report/
PARAMETER: N/A
CONTENT:

{
    "topic":"topic_name" ,//名称
    "time":123456 , //上报的数据所在时间戳，精确到毫秒
    "input": 12345 , //输入的record数量
    "output": 12345 , //输出的record数量
    "error": 0 , //处理中出错的record数量
    "desc" : "description" //本次上报的描述信息
}

4.Overview
概览
URI: GET /overview
PARAMETER:N/A
RESPONSE:

{
    "errcode":0, //出错代码,0表示没有错误
    "errmsg":"", //出错信息,errorCode=0的时候会省略
    "data":{
        "topics": 10 , //topic数量
        "agents": 100 , //agent数量
        "machines": 23 , //机器数量(通过ip区分)
        "alerts": 3 //告警数量
    } //数据
}

errorcode非0的时候表示出现了错误
example: N/A
5.Topic
Topic列表
URI:GET /topic/list
PARAMETER: N/A
RESPONSE:

{
    "errcode":0, //出错代码,0表示没有错误
    "errmsg":"", //出错信息,errorCode=0的时候会省略
    "data":[
        "topic1" , //topic id
        "topic2"
    ]
}

6.Topic状态信息
URI: GET /topic/state
PARAMETER:
topic 要获取信息的topic名称,为空的话则返回全部topic信息
RESPONSE:

{
    "errcode":0, //出错代码,0表示没有错误
    "errmsg":"", //出错信息,errorCode=0的时候会省略
    "data":[
        {
        "topic": "topic1" , //topic id
        "sourceNumber": 10 , //source agent数量
        "sinkNumber": 3 , //sink agent数量
        "alertNumber": 20 , //告警数量
        "monitor": "" //监控地址
        } //第一个topic的信息
    ]
}

example:
GET
{"errcode":0,"errmsg":"","data":[{"topic":"topic1","sourceNumber":10,"sinkNumber":3,"alertNumber":20,"monitor":""},{"topic":"topic2","sourceNumber":8,"sinkNumber":2,"alertNumber":20,"monitor":""}]}
7.Topic详情
URI: GET /topic/detail/$TOPIC_ID
PARAMETER: N/A
RESPONSE:

{
    "errcode":0, //出错代码,0表示没有错误
    "errmsg":"", //出错信息,errorCode=0的时候会省略
    "data":{
        "topic": "topic1" , //topic id
        "agentList": [
            {
            "agent" : "127.0.0.1:10086" , //agent id
            "ip": "127.0.0.1" , //agent所在节点ip
            "port": "10086" , //agnet所用端口
            "type": "source" //agent类型
            }
        ] ,
        "sourceNumber": 10 , //source agent数量
        "sinkNumber": 3 , //sink agent数量
        "alertNumber": 2 , //告警数量
        "monitor": "" //监控地址
    }
}

example:

GET 

{"errcode":0,"errmsg":"","data":{"topic":"topic1","agentList":[{"agent":"127.0.0.1:10086","ip":"127.0.0.1","port":"10086","type":"source"},{"agent":"127.0.0.2:10010","ip":"127.0.0.2","port":"10010","type":"sink"}],"sourceNumber":10,"sinkNumber":3,"alertNumber":2,"monitor":"http:123.com"}}

8.Agent
Agent列表
URI:GET /agent/list
PARAMETER: N/A
RESPONSE:

{
    "errcode":0, //出错代码,0表示没有错误
    "errmsg":"", //出错信息,errorCode=0的时候会省略
    "data":[
        "127.0.0.1:10086" //agentID
    ]
}

example:

GET 

{"errcode":0,"errmsg":"","data":["127.0.0.1:10086","127.0.0.1:10010","127.0.0.2:10010"]}

9.Agent状态信息
URI: GET /agent/state
PARAMETER:
location 要获取的agent的过滤条件,格式为ip:port,ip或port可以省略，表示获取所有
RESPONSE:

{
    "errcode":0, //出错代码,0表示没有错误
    "errmsg":"", //出错信息,errorCode=0的时候会省略
    "data":[
        {
        "agent" : "127.0.0.1:10086" , //agent id
        "ip": "127.0.0.1" , //agent所在节点ip
        "port": "10086" , //agnet所用端口
        "type": "source" , //agent类型
        "topicNumber": 10, //agent负责的topic数量
        "tags":{
            "key":"value",
            "key2":"v2"
        }// agent的一些标签,key/value格式与数量不固定
    ]
}

example:

GET 

{"errcode":0,"errmsg":"","data":[{"agent":"10.10.64.141:34545","ip":"10.10.64.141","port":"34545","type":"source","topicNumber":1,"tags":{"version":"0.0.8"}},{"agent":"10.16.43.131:9934","ip":"10.16.43.131","port":"9934","type":"sink","topicNumber":1,"tags":{"name":"kafka2hdfs_zhiqu"}}]}

10.Agent详情
URI: GET /agent/detail/$AGENT_ID
PARAMETER: N/A
RESPONSE:

{
    "errcode":0, //出错代码,0表示没有错误
    "errmsg":"", //出错信息,errorCode=0的时候会省略
    "data":{
        "agent": "127.0.0.1:10086" , //agent id
        "ip": "127.0.0.1" , //agent所在节点ip
        "port": "10086" , //agent所用端口
        "type": "source" , //agent类型
        "topicList": [
           {
               "topic": "topic1" , //topic id
               "sourceNumber": 10 , //source agent数量
               "sinkNumber": 3 , //sink agent数量
               "alertNumber": 20 , //告警数量
               "monitor": "" //监控地址
           }
        ] ,//topic列表
        "topicNumber":10,
        "tags":{
            "key":"value",
            "key2":"v2"
        }// agent的一些标签,key/value格式与数量不固定
    }
}

example:

GET 

{"errcode":0,"errmsg":"","data":{"agent":"10.11.161.62:34545","ip":"10.11.161.62","port":"34545","type":"source","topicNumber":1,"topicList":[{"topic":"zhiqu","sourceNumber":14,"sinkNumber":1,"agentNumber":15,"monitor":"%2Fd&to=now","alertNumber":0}],"tags":{"version":"1.0.1"}}}

11.告警查询
URI: GET /alert/query
PARAMETER:
topic 要查询的告警的topic名称，如果不指定此参数则返回所有topic的告警。可以指定多个topic
level 要查询的告警的等级，如果不指定此参数则返回所有级别的告警。可选级别为 FATAL , WARN , NOTICE , IGNORE
state 。可以指定多个级别进行查询 要查询的告警的状态，如果不指定此参数则返回所有级别的告警。可选状态为 on , fixed , ignored 。可以指定多个状态进行查询
RESPONSE:

{
    "errcode":0, //出错代码,0表示没有错误
    "errmsg":"", //出错信息,errorCode=0的时候会省略
    "data": [
        {
            "key": { // 告警的meta信息
                "id": "2017-02-03 07:00:00|zhiqu|10.11.161.62:34545|NO_DATA", // 本条告警的id
                "topic": "zhiqu", // 本条告警的topic
                "agent": "10.11.161.62:34545", // 本条告警的agent
                "time": 1486076400000, // 本条告警的目标时间
                "type": "NO_DATA" // 本条告警的类型
            },
            "value": { // 告警内容
                "level": "NOTICE", // 告警级别
                "state": "on", // 告警状态
                "message": "no data found", // 告警信息
                "lastSend": 0 //告警最后一次发送时间
            }
        }, // 第一条告警
        {
            "key": {
                "id": "2017-02-03 07:00:00|zhiqu|10.10.64.156:34545|NO_DATA",
                "topic": "zhiqu",
                "agent": "10.10.64.156:34545",
                "time": 1486076400000,
                "type": "NO_DATA"
            },
            "value": {
                "level": "NOTICE",
                "state": "on",
                "message": "no data found",
                "lastSend": 0
            }
        } // 第二条告警
    ]
}

example:

GET 

{"errcode": 0,"errmsg": "","data": [{"key": {"id": "2017-02-03 07:00:00|zhiqu|10.11.161.62:34545|NO_DATA", "topic": "zhiqu","agent": "10.11.161.62:34545","time": 1486076400000,"type": "NO_DATA" },"value": {"level": "NOTICE","state": "on","message": "no data found","lastSend": 0}}]}

12.忽略告警
URI: GET /alert/ignore
PARAMETER:
id 要忽略的告警的id，取告警查询返回的key中的的id的值。每次只能指定唯一id
RESPONSE:

{
    "errcode":0, //出错代码,0表示没有错误
    "errmsg":"", //出错信息,errorCode=0的时候会省略
    "data": "success" // success表示成功，failed表示失败
}

example:

GET %2013:00:00|app-test1|10.16.39.198:34545|NO_DATA

{"errcode": 0,"errmsg": "","data": "success"}

归档表 archive

将time截取到小时，记录每个时段的采集状况。从详表汇总到Topic级别而来