impala环境搭建

(1)禁用hugepage
使用命令cat /sys/kernel/mm/transparent_hugepage/defrag查看是不是打开，如果不是nerver，需要配置。
执行命令：runlevel，确定运行级别，ubuntu默认为2
添加文件：/etc/init.d/dishuge

#!/bin/sh -e

echo "[dancylog]enable transparent_hugepage"
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
    echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
    echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

设置可执行权限：chmod 777 dishuge

进入/etc/rc2.d目录（rc加运行级别），添加一个软连接：
ln -s ../init.d/dishuge S10dishuge
重启就可以了，可以使用命令：
cat  /sys/kernel/mm/transparent_hugepage/defrag确定是否设置成功了。

(2)禁用swap
echo "vm.swappiness=0" >> /etc/sysctl.conf
sysctl -p
重启使用命令：
cat /proc/sys/vm/swappines
查看是否设置成功。

(3)安装ssh，mysql，ntp

sudo apt-get install openssh-client=1:6.6p1-2ubuntu1
sudo apt-get install openssh-server
sudo apt-get install mysql-server mysql-client libmysql-java ntp

(4)卸载iptables
sudo apt-get remove iptables

(5)安装jdk
sudo mkdir /usr/lib/jvm
进入jdk所在的目录
sudo tar zxvf jdk-7u79-linux-x64.tar.gz -C /usr/lib/jvm
sudo gedit /etc/profile

export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_79
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

/etc/environment

export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_79
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:$JAVA_HOME/bin"

配置jdk默认版本

sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.7.0_79/bin/java 300
sudo update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/jdk1.7.0_79/bin/javac 300
sudo update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/jdk1.7.0_79/bin/jar 300
sudo update-alternatives --install /usr/bin/javah javah /usr/lib/jvm/jdk1.7.0_79/bin/javah 300
sudo update-alternatives --install /usr/bin/javap javap /usr/lib/jvm/jdk1.7.0_79/bin/javap 300

sudo update-alternatives --config java

java -version
echo $JAVA_HOME
(6)修改hostname
需要修改2个配置文件
/etc/hosts
/etc/hostname
根据需要更改就行了。
(7)ssh免交互
ssh-keygen -t rsa
cp id_rsa.pub authorized_keys
不用更改权限，生成的文件每个电脑需要拷贝一份。
(8)mysql 配置
需要修改配置文件：/etc/mysql/my.cnf

bind-address = 0.0.0.0
//////////////////////下面的复制。

transaction-isolation = READ-COMMITTED

key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550

log_bin=/var/lib/mysql/mysql_binary_log

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 52M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES

重启mysql
service mysql restart
添加必要的数据库：

create database hive DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database am DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
grant all on hive.* to 'hl'@'%' identified by 'hl123';
grant all on hue.* to 'hl'@'%' identified by 'hl123';
grant all on am.* to 'hl'@'%' identified by 'hl123';
grant all on oozie.* to 'hl'@'%' identified by 'hl123';
grant all on scm.* to 'hl'@'%' identified by 'hl123';
FLUSH PRIVILEGES;
exit;

如果想删除对某个数据库的权限：revoke all on wifi.* from 'hl'@'%';
(9)添加用户
useradd --system --home=/opt/cm-5.4.0/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
(10) cm-5.4.0修改配置
更改：/opt/cm-5.4.0/etc/cloudera-scm-agent/config.ini
server_host改为hostname中的名字。
如果是agent，只需要更改config.ini，下面的不需要
配置数据库
/opt/cm-5.4.0/share/cmf/schema/scm_prepare_database.sh mysql scm -hlocalhost -uroot -p --scm-host localhost scm scm scm
查看/opt/cm-5.4.0/etc/cloudera-scm-server/db.properties是否修改正确，
如果不正确安装下面更改：

com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=localhost
com.cloudera.cmf.db.name=cloudera
com.cloudera.cmf.db.user=root
com.cloudera.cmf.db.password=hl123

(11)拷贝CDH5
mkdir -p /opt/cloudera/parcel-repo/
拷贝CDH5到这个目录下。
(12)添加必要的目录
mkdir -p /opt/cm-5.4.0/run/cloudera-scm-agent;mkdir -p /var/lib/cloudera-scm-server
(13)启动server和agent
/opt/cm-5.4.0/etc/init.d/cloudera-scm-server start
/opt/cm-5.4.0/etc/init.d/cloudera-scm-agent start
(14)卸载CDH5和重装
删除数据库：

drop database scm;
drop database hive;
drop database hue;
drop database oozie;
drop database am;

删除用户：

userdel -r hdfs
userdel -r solr
userdel -r zookeeper
userdel -r llama
userdel -r httpfs
userdel -r mapred
userdel -r sqoop
userdel -r yarn
userdel -r kms
userdel -r hive
userdel -r sqoop2
userdel -r oozie
userdel -r hbase
userdel -r sentry
userdel -r impala
userdel -r spark
userdel -r hue
userdel -r cloudera-scm
groupdel hadoop
groupdel flume
groupdel hdfs
groupdel sqoop
groupdel hive

删除文件：

rm /impala/ /dfs/ /yarn/ -rf
rm /var/log/hive/ /var/log/hue/ /var/log/impalad/ /var/log/oozie/ /var/log/sqoop2/ /var/log/zookeeper/ -rf
rm /var/log/cloudera-scm-* /var/log/hadoop-* -rf
rm /var/tmp -rf
rm /var/lib/cloudera-* /var/lib/hadoop-* /var/lib/hue/ /var/lib/impala/ /var/lib/oozie/ /var/lib/sqoop2/ /var/lib/zookeeper/ /var/lib/spark/ -rf