东南西北风andyhuang.blog.chinaunix.net

脚踏实地,一步一个脚印!

  • 博客访问: 2441066
  • 博文数量: 638
  • 博客积分: 8199
  • 博客等级: 中将
  • 技术积分: 8983
  • 用 户 组: 普通用户
  • 注册时间: 2010-05-25 20:19
个人简介

脚踏实地、勇往直前!

文章分类

全部博文(638)

文章存档

2017年(19)

2016年(48)

2015年(50)

2014年(169)

2013年(52)

2012年(177)

2011年(93)

2010年(30)

微信关注

IT168企业级官微



微信号:IT168qiye



系统架构师大会



微信号:SACC2013

订阅
热词专题

分类: 大数据

原文地址:ambari本地源自动化安装hortonworks hadoop 作者:GangLin_Lan

环境配置

Hostname

IP

OS

Ambari

172.16.235.164

Centos X64 base install

Maste.hadoop

172.16.235.165

Centos X64 base install

Datanode1.hadoop

172.16.235.166

Centos X64 base install

操作系统建议以software server 方式安装,base 方式安装后,需要安装Development tools,本文以base举例。

 

一:制作ambarihadoop本地源

首先需要有一台能够联网的linux机器

1:制作ambari1.4.1.25)本地源

[root@controller ~]# wget http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.4.1.25/ambari.repo

[root@controller ~]# cp ambari.repo /etc/yum.repos.d/

将网络源中的包同步到本地:

reposync  -r HDP-UTILS-1.1.0.16

reposync  -r Updates-ambari-1.4.1.25

reposync  -r ambari-1.x

会得到相应的三个目录,每个目录中有相应的rpm

准备一台http服务器,本文以172.16.235.164作为http服务器发布安装源

mkdir /var/www/html/ambari

HDP-UTILS-1.1.0.16 Updates-ambari-1.4.1.25 ambari-1.x复制到 /var/www/html/ambari

cd /var/www/html/ambari

createrepo ./    #没有createrepo命令自己去安装

这样ambari安装源就做好了,下载好jdk-6u31-linux-x64.bin备用

 

2:准备hortonworks hadoop HDP2.0.6本地源

准备HDP网络源

vim /etc/yum.repos.d/HDP.repo,添加以下内容

[HDP-2.0.6]

name=HDP

baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.0.6.0

enabled=1

gpgcheck=0


同步网络源到本地:

reposync  -r HDP-2.0.6  #将近1G

HDP-2.0.6 复制到http server/var/www/html/hdp

cd /var/www/html/hdp

createrepo ./  

这样hadoop的本地源就做好了

 

3:制作系统本地源(安装ambarihadoop时,可能会依赖些系统包,这时需要有系统的本地源来安装)

http server 挂载系统光盘

mount /dev/cdrom  /var/www/html/centos/

 

二:安装ambari

系统准备:关闭iptablesselinux,启动ntpd服务

[root@ambari ]vim /etc/hosts

172.16.235.164  ambari

172.16.235.165  master.hadoop

172.16.235.166  datanode1.hadoop

配置与所有主机的无密码登入

[root@ambari ~]# ssh-keygen  #一路回车(所有主机都要执行)

[root@ambari ~]# cd .ssh/

[root@ambari .ssh]# ls

id_rsa  id_rsa.pub

[root@ambari .ssh]# scp id_rsa.pub master.hadoop:/root/.ssh/1

The authenticity of host 'master.hadoop (172.16.235.165)' can't be established.

RSA key fingerprint is fc:33:53:5f:f2:20:86:92:ac:96:09:e2:dc:e7:33:ea.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'master.hadoop,172.16.235.165' (RSA) to the list of known hosts.

root@master.hadoop's password:

id_rsa.pub                                                                                        100%  393     0.4KB/s   00:00   

[root@ambari .ssh]# scp id_rsa.pub datanode1.hadoop:/root/.ssh/1

The authenticity of host 'datanode1.hadoop (172.16.235.166)' can't be established.

RSA key fingerprint is fc:33:53:5f:f2:20:86:92:ac:96:09:e2:dc:e7:33:ea.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'datanode1.hadoop,172.16.235.166' (RSA) to the list of known hosts.

root@datanode1.hadoop's password:

id_rsa.pub                                                                                        100%  393     0.4KB/s   00:00  

登入master.hadoop

[root@master ~]# cd .ssh/

[root@master .ssh]# cat 1 >authorized_keys

登入datanode1.hadoop

[root@datanode1 ~]# cd .ssh/

[root@datanode1 .ssh]# cat 1 >authorized_keys

回到ambari 测试无密码登入

[root@ambari .ssh]# ssh master.hadoop

Last login: Wed Dec  4 11:11:15 2013 from 172.16.38.48

[root@master ~]# exit

logout

Connection to master.hadoop closed.

[root@ambari .ssh]# ssh datanode1.hadoop

Last login: Wed Dec  4 10:38:06 2013 from 172.16.38.48

[root@datanode1 ~]# exit

logout

Connection to datanode1.hadoop closed.

[root@ambari .ssh]#

 

安装与配置ambari服务:

[root@ambari ]vim /etc/yum.repos.d/ambari.repo  添加以下内容

[ambari-1.x]

name=Ambari 1.x

baseurl=http://172.16.235.164/ambari/

gpgcheck=0

enabled=1

 

[HDP-UTILS-1.1.0.16]

name=Hortonworks Data Platform Utils Version - HDP-UTILS-1.1.0.16

baseurl=http://172.16.235.164/ambari/

gpgcheck=0

enabled=1

 

[Updates-ambari-1.4.1.25]

name=ambari-1.4.1.25 - Updates

baseurl=http://172.16.235.164/ambari/

gpgcheck=

enabled=1

[root@ambari ]vim /etc/yum.repos.d/centos.repo  添加以下内容

[centos]

baseurl=http://172.16.235.164/centos/

enabled=1

gpgcheck=0

执行安装ambari

[root@ambari yum.repos.d]# yum -y install ambari-server

Loaded plugins: fastestmirror, security

Repository 'Updates-ambari-1.4.1.25': Error parsing config: Error parsing "gpgcheck = ''": invalid boolean value

Repository 'centos' is missing name in configuration, using id

Loading mirror speeds from cached hostfile

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package ambari-server.noarch 0:1.4.1.25-1 will be installed

--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-1.4.1.25-1.noarch

--> Running transaction check

---> Package postgresql-server.x86_64 0:8.4.13-1.el6_3 will be installed

--> Processing Dependency: postgresql-libs(x86-64) = 8.4.13-1.el6_3 for package: postgresql-server-8.4.13-1.el6_3.x86_64

--> Processing Dependency: postgresql(x86-64) = 8.4.13-1.el6_3 for package: postgresql-server-8.4.13-1.el6_3.x86_64

--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-8.4.13-1.el6_3.x86_64

--> Running transaction check

---> Package postgresql.x86_64 0:8.4.13-1.el6_3 will be installed

---> Package postgresql-libs.x86_64 0:8.4.13-1.el6_3 will be installed

--> Finished Dependency Resolution

 

Dependencies Resolved

 

====================================================================================================================================

 Package                           Arch                   Version                          Repository                          Size

====================================================================================================================================

Installing:

 ambari-server                     noarch                 1.4.1.25-1                       HDP-UTILS-1.1.0.16                  37 M

Installing for dependencies:

 postgresql                        x86_64                 8.4.13-1.el6_3                   centos                             2.8 M

 postgresql-libs                   x86_64                 8.4.13-1.el6_3                   centos                             200 k

 postgresql-server                 x86_64                 8.4.13-1.el6_3                   centos                             3.4 M

 

Transaction Summary

====================================================================================================================================

Install       4 Package(s)

 

Total download size: 43 M

Installed size: 88 M

Downloading Packages:

(1/4): ambari-server-1.4.1.25-1.noarch.rpm                                                                   |  37 MB     00:00    

(2/4): postgresql-server-8.4.13-1.el6_3.x86_64.rpm                                                           | 3.4 MB     00:00    

(3/4): postgresql-libs-8.4.13-1.el6_3.x86_64.rpm                                                             | 200 kB     00:00    

(4/4): postgresql-8.4.13-1.el6_3.x86_64.rpm                                                                  | 2.8 MB     00:00    

------------------------------------------------------------------------------------------------------------------------------------

Total                                                                                                43 MB/s |  43 MB     00:00    

Running rpm_check_debug

Running Transaction Test

Transaction Test Succeeded

Running Transaction

  Installing : postgresql-libs-8.4.13-1.el6_3.x86_64                                                                            1/4

  Installing : postgresql-8.4.13-1.el6_3.x86_64                                                                                 2/4

  Installing : postgresql-server-8.4.13-1.el6_3.x86_64                                                                          3/4

  Installing : ambari-server-1.4.1.25-1.noarch                                                                                  4/4

  Verifying  : postgresql-server-8.4.13-1.el6_3.x86_64                                                                          1/4

  Verifying  : postgresql-libs-8.4.13-1.el6_3.x86_64                                                                            2/4

  Verifying  : ambari-server-1.4.1.25-1.noarch                                                                                  3/4

  Verifying  : postgresql-8.4.13-1.el6_3.x86_64                                                                                 4/4

 

Installed:

  ambari-server.noarch 0:1.4.1.25-1                                                                                                 

 

Dependency Installed:

  postgresql.x86_64 0:8.4.13-1.el6_3     postgresql-libs.x86_64 0:8.4.13-1.el6_3     postgresql-server.x86_64 0:8.4.13-1.el6_3   

 

Complete!

 

开始配置ambari 服务:

首先上传jdk文件

[root@ambari ]cd /var/lib/ambari-server/resources  jdk-6u31-linux-x64.bin上传到该目录

[root@ambari resources]# chmod +x jdk-6u31-linux-x64.bin

[root@ambari yum.repos.d]# ambari-server setup

Using python  /usr/bin/python2.6

Initializing...

Setup ambari-server

Checking SELinux...

SELinux status is 'enabled'

SELinux mode is 'permissive'

WARNING: SELinux is set to 'permissive' mode and temporarily disabled.

OK to continue [y/n] (y)?

Customize user account for ambari-server daemon [y/n] (n)?

Adjusting ambari-server permissions and ownership...

Checking iptables...

WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.

OK to continue [y/n] (y)?

Checking JDK...

JDK already exists, using /var/lib/ambari-server/resources/jdk-6u31-linux-x64.bin

Installing JDK to /usr/jdk64

Successfully installed JDK to /usr/jdk64/jdk1.6.0_31

Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-6.zip to /var/lib/ambari-server/resources/jce_policy-6.zip

To download the JCE Policy files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.htmlNot accepting will result in errors when configuring Kerberos security.

Do you accept the Oracle Binary Code License Agreement [y/n] (y)?

Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-6.zip

Completing setup...

Configuring database...

Enter advanced database configuration [y/n] (n)?

Default properties detected. Using built-in database.

Checking PostgreSQL...

Running initdb: This may take upto a minute.

About to start PostgreSQL

Configuring local database...

Connecting to the database. Attempt 1...

Configuring PostgreSQL...

Restarting PostgreSQL

Ambari Server 'setup' completed successfully.

[root@ambari ~]# ambari-server start

Using python  /usr/bin/python2.6

Starting ambari-server

Ambari Server running with 'root' privileges.

Server PID at: /var/run/ambari-server/ambari-server.pid

Server out at: /var/log/ambari-server/ambari-server.out

Server log at: /var/log/ambari-server/ambari-server.log

Ambari Server 'start' completed successfully.

到此ambari安装好

 

三:本地安装hadoop

ambari配置本地hadoo源地址:

[root@ambari ~]# cd /var/lib/ambari-server/resources/stacks/HDPLocal/2.0.6/repos/

[root@ambari repos]# vim repoinfo.xml  修改红色部分为本地源地址

 

   

      http://172.16.235.164/hdp/

      HDP-2.0.6

      HDP

   

 

这样在ambari执行hadoop安装时,会将本地源地址配置到所有主机上,在为所有主机安装ambari-agent时,会将ambari server下的/etc/yum.repo/ambari.repo文件复制到所有主机

 

主机准备(master.hadoopdatanode1.hadoop)配置相同,只举例master.hadoop

关闭iptablesselinux、启动ntpd服务(略)

[root@master ~]# vim /etc/hosts 添加以下

172.16.235.164  ambari

172.16.235.165  master.hadoop

172.16.235.166  datanode1.hadoop

[root@master ~]# vim /etc/yum.repos.d/centos.repo 添加以下内容

[root

[centos]

baseurl=http://172.16.235.164/centos/

enabled=1

gpgcheck=0

@master ~]# yum -y groupinstall "Development tools"

 

登入ambari管理页面

http://172.16.235.164:8080/ 账户:admin 密码:admin


配置集群名称



选择HDP版本



选择目标主机,一行一个主机。要让ambar server自动在所以主机安装ambari-agent,需要提供ambari server root账户的私钥,即/root/.ssh/ id_rsa文件,选中使用本地源安装hadoop,然后下一步


确认并且选择主机



确认OK



下一步选择要安装的组件,选好后下一步




选择组件分配在哪些主机,都是master 的服务



选择安装client,主机有限,master同时安装client



配置服务,其中nagios中要填写密码及报警接受邮箱必填,其他选择性配置



显示配置信息



开始安装,过程稍微有点长



安装过程




Google浏览器反应比较慢,用另一某浏览器已经显示succes







Ganglia



nagios:


阅读(78) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~
评论热议
请登录后评论。

登录 注册