Chinaunix首页 | 论坛 | 博客
  • 博客访问: 1185185
  • 博文数量: 259
  • 博客积分: 10
  • 博客等级: 民兵
  • 技术积分: 2518
  • 用 户 组: 普通用户
  • 注册时间: 2012-10-13 16:12
个人简介

科技改变世界,技术改变人生。

文章分类

全部博文(259)

分类: HADOOP

2016-03-17 11:57:58

ambari默认3个journalnode节点,但是如果一个节点出现问题,需要增加补充,ambari界面没有操作的选项,所以只能通过其他命令方式操作,
看到之前有个文章是将HA降级,之后重新做HA,这样的风险太高了,操作负载,从网上找到了其他方式,分享给需要的朋友,也希望ambari新版本可以将这个增加journalnode功能,添加进去。

操作前提示:如果你对ambari这些操作一点都不熟悉,建议不要进行操作,以免ambari管理界面异常,导致无法管理。
可以先在测试环境操作练习,确认无误后,再进行正式环境操作。


增加journalnode

1、分配角色:
curl -u admin:admin -H 'X-Requested-By: Ambari' -X POST

我的环境:
curl -u admin:admin -H "X-Requested-By: Ambari" -X POST

查看下内容:
curl -u admin:admin -H "X-Requested-By: ambari" -X GET


2、安装Journalnode:
curl -u admin:admin -H 'X-Requested-By: Ambari' -X PUT -d ‘{“RequestInfo”:{“context”:”Install JournalNode”},”Body”:{“HostRoles”:{“state”:”INSTALLED”}}}’



我的环境:
curl -u admin:admin -H 'X-Requested-By: Ambari' -X PUT -d '{"RequestInfo":{"context":"Install JournalNode"},"Body":{"HostRoles":{"state":"INSTALLED"}}}'




3、更新HDFS配置
Login to Ambari Web UI and modify the HDFS Configuration. Search for dfs.namenode.shared.edits.dir and add the new JournalNode. Make sure you don’t mess up the format

for the journalnode list provided. The following is a format of a typical 3 JournalNode shared edits definition.


qjournal://my-jn-node-1.host.com:8485;my-jn-node-1.host.com:8485;my-jn-node-1.host.com:8485/MyLAB

我的环境配置:
qjournal://testserver4.bj:8485;testserver1.bj:8485;testserver2.bj:8485;testserver3.bj:8485/testcluster


4、创建journalnode目录

Time to create the required directory structure on the new Journalnode. You have to create this directory structure based on your cluster installation. If unsure, you

can find this value from $HADOOP_CONF/hdfs-site.xml file. Look for the parameter value for dfs.journalnode.edits.dir. In my case, it happens to be

/hadoop/qjournal/namenode/.

我的环境:
dfs.journalnode.edits.dir /hadoop/hdfs/journal
ll -d /hadoop/hdfs/journal
drwxr-xr-x 3 hdfs hadoop 4096 Feb  2 10:56 /hadoop/hdfs/journal

Make sure you add the HDFS Nameservice directory. You can find this value from $HADOOP_CONF/hdfs-site.xml file. The value can be found for parameter dfs.nameservices.

In my example, I have “MyLab”. So I will create the directory structure as /hadoop/qjournal/namenode/MyLab.

我的环境:
dfs.nameservices testcluster

ll -d /hadoop/hdfs/journal/testcluster/
drwxr-xr-x 3 hdfs hadoop 4096 Mar 16 18:40 /hadoop/hdfs/journal/testcluster/

mkdir -p /hadoop/hdfs/journal/testcluster/
chown hdfs:hadoop -R /hadoop/hdfs/journal/


5、同步数据
Copy or Sync the directory ‘current’ under the ‘shared edits’ location from an existing JournalNode. Make sure that the ownership for all these newly created

directories and sync’ed files is right.

我的环境:
scp -r /hadoop/hdfs/journal/testcluster/* 10.11.32.51:/hadoop/hdfs/journal/testcluster/
chown hdfs:hadoop -R /hadoop/hdfs/journal/


6、启动服务查看日志

从ambari界面启动journalnode服务

2016-03-17 11:48:27,644 INFO  server.Journal (Journal.java:scanStorageForLatestEdits(187)) - Scanning storage FileJournalManager(root=/hadoop/hdfs/journal/testcluster)
2016-03-17 11:48:27,791 INFO  server.Journal (Journal.java:scanStorageForLatestEdits(193)) - Latest log is EditLogFile
(file=/hadoop/hdfs/journal/testcluster/current/edits_inprogress_0000000000000010224,first=0000000000000010224,last=0000000000000010232,inProgress=true,hasCorruptHeader=
false)
2016-03-17 11:49:37,304 INFO  namenode.FileJournalManager (FileJournalManager.java:finalizeLogSegment(133)) - Finalizing edits file
/hadoop/hdfs/journal/testcluster/current/edits_inprogress_0000000000000010238 -> /hadoop/hdfs/journal/testcluster/current/edits_0000000000000010238-0000000000000010251


参考:
阅读(2665) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~