一、Hive下载安装
先从网址下载一个最新版本,我下载的是hive-0.12.0.tar.gz,推荐将此版本解压到Hadoop安装目录下:
tar -zxvf hive-0.12.0.tar.gz
cd /home/hadoop/hive-0.12.0
首先仍然是配置一下环境变量,打开/etc/profile文件,加入如下配置:
HIVE_HOME=/home/hadoop/hive-0.12.0
PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$PATH
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$HADOOP_HOME/lib:$HBASE_HOME/lib:$HIVE_HOME/lib:$CLASSPATH
export HIVE_HOME
export PATH
export CLASSPATH
使配置生效后,进入$HIVE_HOME/conf目录,会发现有四个模板文件:
hive-default.xml.template
hive-env.sh.template
hive-exec-log4j.properties.template
hive-log4j.properties.template
将其复制成正式配置文件:
cp hive-default.xml.template hive-site.xml
cp hive-env.sh.template hive-env.sh
cp hive-exec-log4j.properties.template hive-exec-log4j.properties
cp hive-log4j.properties.template hive-log4j.properties
官方0.12.0的发布版本中的 hive-default.xml.template 中有 bug,在 2000行:
auth 修改为:auth
hive-site.xml 文件:
首先修改配置项
hive.metastore.schema.verification
false(这个值改成false)
文件最后加上几行:
hive.aux.jars.path
file:///home/hadoop/hive-0.12.0/lib/hive-hbase-handler-0.12.0.jar,file:///home/hadoop/hive-0.12.0/lib/hbase-0.94.18.jar,file:///home/hadoop/hive-0.12.0/lib/zookeeper-3.4.5.jar
注意写法,目录形式是file:///,一定要一字不差,然后将对应的jar包从hbase目录下拷贝到相应位置,如果hive下已有对应jar包,建议删除后使用hbase目录下的。
二、启动并测试hive(以下内容来自http://blog.csdn.net/luckymelina/article/details/16707801)
进入hive目录下,敲命令:
bin/hive -hiveconf hbase.master=master:60000
a) 创建hbase识别的表
CREATE TABLE hbase_table_1(key int, value string) STORED
BY'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
SERDEPROPERTIES("hbase.columns.mapping" = ":key,cf1:val")TBLPROPERTIES ("hbase.table.name" = "xyz");
b) 新建hive的数据表
create table pokes(foo int,bar string)row format delimited fields terminated by ',';
c) 批量导入数据
load data local inpath '/home/hadoop/1.txt' overwrite into table pokes;
1.txt文件的内容为
1,hello
2,pear
3,world
d) 使用sql导入hbase_table_1
insert overwrite table hbase_table_1 select * from pokes;
这一步如果正常说明hbase与hive已做到了正常连接,输出日志:
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201404191501_0011, Tracking URL =
Kill Command = /home/hadoop/hadoop/libexec/../bin/hadoop job -kill job_201404191501_0011
Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
2014-04-19 21:14:21,307 Stage-0 map = 0%, reduce = 0%
2014-04-19 21:14:35,384 Stage-0 map = 100%, reduce = 0%, Cumulative CPU 2.39 sec
2014-04-19 21:14:36,391 Stage-0 map = 100%, reduce = 0%, Cumulative CPU 2.39 sec
2014-04-19 21:14:37,398 Stage-0 map = 100%, reduce = 0%, Cumulative CPU 2.39 sec
2014-04-19 21:14:38,406 Stage-0 map = 100%, reduce = 0%, Cumulative CPU 2.39 sec
2014-04-19 21:14:39,416 Stage-0 map = 100%, reduce = 100%, Cumulative CPU 2.39 sec
MapReduce Total cumulative CPU time: 2 seconds 390 msec
Ended Job = job_201404191501_0011
MapReduce Jobs Launched:
Job 0: Map: 1 Cumulative CPU: 2.39 sec HDFS Read: 232 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 390 msec
OK
Time taken: 57.471 seconds
e) 查看数据
hive>select * from hbase_table_1;
OK
1 hello
2 pear
3 world
说明相应数据已成功的从Hive导入到HBase中
阅读(3567) | 评论(0) | 转发(1) |