苦逼房奴一枚
全部博文(65)
分类: 大数据
2013-08-07 10:11:53
Installation Hadoop-cacti-jtg gathers data by accessing JMX information. JMX support is fairly easy to enable with hadoop. Hadoop conf/hadoop-metrics.properties Setup all metrics objects to use NullContextWithUpdateThread. The period must match the period of your polling. For example my monitoring station polls at 5 minute intervals. Thus I have set period to 300(seconds). dfs.class=org.apache.hadoop.metrics.spi.NullContextWithUpdateThread dfs.period=300 jvm.class=org.apache.hadoop.metrics.spi.NullContextWithUpdateThread jvm.period=300 HBase HBase provides detailed instructions for JMX: http://hadoop.apache.org/hbase/docs/current/metrics.html conf/hadoop-metrics.properties hbase.class=org.apache.hadoop.metrics.spi.NullContextWithUpdateThread hbase.period=300 jvm.class=org.apache.hadoop.metrics.spi.NullContextWithUpdateThread jvm.period=300 rpc.class=org.apache.hadoop.metrics.spi.NullContextWithUpdateThread rpc.period=300 Enable JMX on Each Hadoop Node cd/conf vi jmxremote.access monitorRole readonly controlRole readwrite vi jmxremote.password monitorRole controlRole chmod 600 jmxremote.password chown $HADOOPUSER jxmremote.password vi hadoop.env ... export HADOOP_JMX_BASE="-Dcom.sun.management.jmxremote.ssl=false" export HADOOP_JMX_BASE="$HADOOP_JMX_BASE -Dcom.sun.management.jmxremote.password.file=/opt/hadoop-0.19.1/conf/jmxremote.password" export HADOOP_JMX_BASE="$HADOOP_JMX_BASE -Dcom.sun.management.jmxremote.access.file=/opt/hadoop-0.19.1/conf/jmxremote.access" # Command specific options appended to HADOOP_OPTS when specified export HADOOP_NAMENODE_OPTS="$HADOOP_JMX_BASE -Dcom.sun.management.jmxremote.port=10001" export HADOOP_SECONDARYNAMENODE_OPTS="$HADOOP_JMX_BASE -Dcom.sun.management.jmxremote.port=10002" export HADOOP_DATANODE_OPTS="$HADOOP_JMX_BASE -Dcom.sun.management.jmxremote.port=10003" export HADOOP_BALANCER_OPTS="$HADOOP_JMX_BASE -Dcom.sun.management.jmxremote.port=10004" export HADOOP_JOBTRACKER_OPTS="$HADOOP_JMX_BASE -Dcom.sun.management.jmxremote.port=10005" ... Restart Hadoop. If you have done this correctly you should notice your system is now listening on ports 10001-10005. At this point you should be able to use jconsole (found in your JDK bin) to connect to these ports to test. HBase The process above is similar for HBase. Each HBase component has its own variable like HBASE_REGIONSERVER_OPS inside hbase-env.sh. Change them and restart. Again look here: http://hadoop.apache.org/hbase/docs/current/metrics.html $HBASE_HOME/conf/hbase-env.sh Add the lines: JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false" JMX_OPTS="$JMX_OPTS -Dcom.sun.management.jmxremote.password.file=$HBASE_HOME/conf/jmxremote.passwd" JMX_OPTS="$JMX_OPTS -Dcom.sun.management.jmxremote.access.file=$HBASE_HOME/conf/jmxremote.access" export HBASE_MASTER_OPTS="$JMX_OPTS -Dcom.sun.management.jmxremote.port=10101" export HBASE_REGIONSERVER_OPTS="$JMX_OPTS -Dcom.sun.management.jmxremote.port=10102" 以上配置好之后重启hadoop,就可以利用jconsole远程连接查看相关数据