Chinaunix首页 | 论坛 | 博客
  • 博客访问: 1210754
  • 博文数量: 259
  • 博客积分: 10
  • 博客等级: 民兵
  • 技术积分: 2518
  • 用 户 组: 普通用户
  • 注册时间: 2012-10-13 16:12
个人简介

科技改变世界,技术改变人生。

文章分类

全部博文(259)

分类: 大数据

2015-09-27 11:19:08

告警发现一个supervisor停止,重启启动服务,但无法正常运行,查看日志报错如下:

2015-09-27 10:21:53 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2015-09-27 10:21:53 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=dn11.21.abc:2181,dn12.21.abc:2181,dn13.21.abc:2181,dn14.21.abc:2181,dn15.21.abc:2181 sessionTimeout=20000 watcher=com.netflix.curator.ConnectionState@6ffc7f8
2015-09-27 10:21:53 o.a.z.ClientCnxn [WARN] SASL configuration failed: javax.security.auth.login.LoginException: Zookeeper client cannot authenticate using the 'Client' section of the supplied JAAS configuration: '/etc/storm/conf/storm_jaas.conf' because of a RuntimeException: java.lang.SecurityException: /etc/storm/conf/storm_jaas.conf (No such file or directory) Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
2015-09-27 10:21:53 o.a.z.ClientCnxn [INFO] Opening socket connection to server dn18.21.abc/192.168.12.119:2181
2015-09-27 10:21:53 b.s.zookeeper [INFO] Zookeeper state update: :auth-failed:none
2015-09-27 10:21:53 c.n.c.ConnectionState [ERROR] Authentication failed
2015-09-27 10:21:53 o.a.z.ClientCnxn [INFO] Socket connection established to dn18.21.abc/192.168.12.119:2181, initiating session
2015-09-27 10:21:53 o.a.z.ClientCnxn [INFO] Session establishment complete on server dn18.21.abc/192.168.12.119:2181, sessionid = 0x74f284f080e3366, negotiated timeout = 20000
2015-09-27 10:21:53 c.n.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2015-09-27 10:21:53 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
2015-09-27 10:21:53 c.n.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2015-09-27 10:21:53 o.a.z.ZooKeeper [INFO] Session: 0x74f284f080e3366 closed
2015-09-27 10:21:53 o.a.z.ClientCnxn [INFO] EventThread shut down
2015-09-27 10:21:53 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2015-09-27 10:21:53 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=dn11.21.abc:2181,dn12.21.abc:2181,dn13.21.abc:2181,dn14.21.abc:2181,dn15.21.abc:2181/storm sessionTimeout=20000 watcher=com.netflix.curator.ConnectionState@60a46327
2015-09-27 10:21:53 o.a.z.ClientCnxn [WARN] SASL configuration failed: javax.security.auth.login.LoginException: Zookeeper client cannot authenticate using the 'Client' section of the supplied JAAS configuration: '/etc/storm/conf/storm_jaas.conf' because of a RuntimeException: java.lang.SecurityException: /etc/storm/conf/storm_jaas.conf (No such file or directory) Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
2015-09-27 10:21:53 o.a.z.ClientCnxn [INFO] Opening socket connection to server kafka-3/192.168.12.82:2181
2015-09-27 10:21:53 b.s.cluster [WARN] Received event :auth-failed::none: with disconnected Zookeeper.
2015-09-27 10:21:53 o.a.z.ClientCnxn [INFO] Socket connection established to kafka-3/192.168.12.82:2181, initiating session
2015-09-27 10:21:53 c.n.c.ConnectionState [ERROR] Authentication failed
2015-09-27 10:21:53 o.a.z.ClientCnxn [INFO] Session establishment complete on server kafka-3/192.168.12.82:2181, sessionid = 0x94f882b7e5d2e38, negotiated timeout = 20000
2015-09-27 10:21:53 c.n.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2015-09-27 10:21:53 c.n.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2015-09-27 10:21:54 b.s.d.supervisor [INFO] Starting supervisor with id 59cf6aaf-72b6-4878-88d6-fbdc409f7ed4 at host dn23.21.abc
2015-09-27 10:21:54 b.s.event [ERROR] Error when processing event
java.lang.RuntimeException: java.io.EOFException
        at backtype.storm.utils.Utils.deserialize(Utils.java:86) ~[storm-core-0.9.1.2.1.5.0-695.jar:0.9.1.2.1.5.0-695]
        at backtype.storm.utils.LocalState.snapshot(LocalState.java:45) ~[storm-core-0.9.1.2.1.5.0-695.jar:0.9.1.2.1.5.0-695]
        at backtype.storm.utils.LocalState.get(LocalState.java:56) ~[storm-core-0.9.1.2.1.5.0-695.jar:0.9.1.2.1.5.0-695]
        at backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:207) ~[storm-core-0.9.1.2.1.5.0-695.jar:0.9.1.2.1.5.0-695]
        at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.4.0.jar:na]
        at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na]
        at clojure.core$apply.invoke(core.clj:603) ~[clojure-1.4.0.jar:na]
        at clojure.core$partial$fn__4070.doInvoke(core.clj:2343) ~[clojure-1.4.0.jar:na]
        at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.4.0.jar:na]
        at backtype.storm.event$event_manager$fn__2647.invoke(event.clj:39) ~[storm-core-0.9.1.2.1.5.0-695.jar:0.9.1.2.1.5.0-695]
        at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.io.EOFException: null
        at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325) ~[na:1.7.0_45]
        at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794) ~[na:1.7.0_45]
        at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801) ~[na:1.7.0_45]
        at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299) ~[na:1.7.0_45]
        at backtype.storm.utils.Utils.deserialize(Utils.java:81) ~[storm-core-0.9.1.2.1.5.0-695.jar:0.9.1.2.1.5.0-695]
        ... 11 common frames omitted
2015-09-27 10:21:54 b.s.util [INFO] Halting process: ("Error when processing an event")


提示什么null值之类的,没有特别明确的内容,后来在网上搜索了下,解决方法:
删除storm.yaml中配置的storm.local.dir指向的目录中的supervisor和workers两个目录,再次启动即可。
找不到的可以使用命令搜索下:
find / -name supervisor
find / -name workers

mv supervisor supervisor.bak
mv workers supervisor.bak

rm -rf  supervisor
rm -rf workers

重命名是为了以防万一,如果supervisor运行正常了,在删除.bak文件目录
阅读(5183) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~