这个yaml文件包含一个ConfigMap，5个Service，8个pod，这里需要注意的是ConfigMap中HDFS_MASTER_SERVICE和HDOOP_YARN_MASTER不要使用IP，使用HDFS service的名称，否则datanode将会连接不上namenode，出现错误【ipc.Client: Retrying connect to server: xxx:9000.】
-
执行创建命令
kubectl create -f hadoop.yaml
-
检查是否创建成功
kubectl get configmap -o wide
kubectl get svc -o wide
kubectl get po -o wide
全部正常，搭建成功！
但是上面的搭建方式是以单个POD声明的，这种方式不稳定，如果不小心删除后就没有了，其实我们可以使用Replication Controller方式进行搭建，这样的话始终可以确保保留相应数量的POD，具体的yaml文件如下：
apiVersion: v1
kind: ConfigMap
metadata:
name: kube-hadoop-conf
namespace: default
data:
HDFS_MASTER_SERVICE: hadoop-hdfs-master
HDOOP_YARN_MASTER: hadoop-yarn-master
---
apiVersion: v1
kind: Service
metadata:
name: hadoop-hdfs-master
spec:
type: NodePort
selector:
name: hdfs-master
ports:
- name: rpc
port: 9000
targetPort: 9000
- name: http
port: 50070
targetPort: 50070
nodePort: 32007
---
apiVersion: v1
kind: ReplicationController
metadata:
name: hdfs-master
labels:
name: hdfs-master
spec:
replicas: 1
selector:
name: hdfs-master
template:
metadata:
labels:
name: hdfs-master
spec:
containers:
- name: hdfs-master
image: 10.3.13.184/library/kubernetes-hadoop:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9000
- containerPort: 50070
env:
- name: HADOOP_NODE_TYPE
value: namenode
- name: HDFS_MASTER_SERVICE
valueFrom:
configMapKeyRef:
name: kube-hadoop-conf
key: HDFS_MASTER_SERVICE
- name: HDOOP_YARN_MASTER
valueFrom:
configMapKeyRef:
name: kube-hadoop-conf
key: HDOOP_YARN_MASTER
restartPolicy: Always
---
apiVersion: v1
kind: ReplicationController
metadata:
name: hadoop-datanode
labels:
app: hadoop-datanode
spec:
replicas: 3
selector:
name: hadoop-datanode
template:
metadata:
labels:
name: hadoop-datanode
spec:
containers:
- name: hadoop-datanode
image: 10.3.13.184/library/kubernetes-hadoop:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9000
- containerPort: 50070
env:
- name: HADOOP_NODE_TYPE
value: datanode
- name: HDFS_MASTER_SERVICE
valueFrom:
configMapKeyRef:
name: kube-hadoop-conf
key: HDFS_MASTER_SERVICE
- name: HDOOP_YARN_MASTER
valueFrom:
configMapKeyRef:
name: kube-hadoop-conf
key: HDOOP_YARN_MASTER
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: hadoop-yarn-master
spec:
type: NodePort
selector:
name: yarn-master
ports:
- name: "8030"
port: 8030
- name: "8031"
port: 8031
- name: "8032"
port: 8032
- name: http
port: 8088
targetPort: 8088
nodePort: 32088
---
apiVersion: v1
kind: ReplicationController
metadata:
name: yarn-master
labels:
name: yarn-master
spec:
replicas: 1
selector:
name: yarn-master
template:
metadata:
labels:
name: yarn-master
spec:
containers:
- name: yarn-master
image: 10.3.13.184/library/kubernetes-hadoop:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9000
- containerPort: 50070
env:
- name: HADOOP_NODE_TYPE
value: resourceman
- name: HDFS_MASTER_SERVICE
valueFrom:
configMapKeyRef:
name: kube-hadoop-conf
key: HDFS_MASTER_SERVICE
- name: HDOOP_YARN_MASTER
valueFrom:
configMapKeyRef:
name: kube-hadoop-conf
key: HDOOP_YARN_MASTER
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: yarn-node
spec:
clusterIP: None
selector:
name: yarn-node
ports:
- port: 8040
---
apiVersion: v1
kind: ReplicationController
metadata:
name: yarn-node
labels:
name: yarn-node
spec:
replicas: 3
selector:
name: yarn-node
template:
metadata:
labels:
name: yarn-node
spec:
containers:
- name: yarn-node
image: 10.3.13.184/library/kubernetes-hadoop:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8040
- containerPort: 8041
- containerPort: 8042
env:
- name: HADOOP_NODE_TYPE
value: yarnnode
- name: HDFS_MASTER_SERVICE
valueFrom:
configMapKeyRef:
name: kube-hadoop-conf
key: HDFS_MASTER_SERVICE
- name: HDOOP_YARN_MASTER
valueFrom:
configMapKeyRef:
name: kube-hadoop-conf
key: HDOOP_YARN_MASTER
restartPolicy: Always
