HBase 0.94.21+ZooKeeper-3.4.6 分布式安装

178次阅读

没有评论

共计 22223 个字符，预计需要花费 56 分钟才能阅读完成。

HBase 0.94.21+ZooKeeper-3.4.6 分布式安装

1：配置 hosts 和 ssh 无密码登陆

Hadoop+HBase 搭建云存储总结 PDF http://www.linuxidc.com/Linux/2013-05/83844.htm

HBase 结点之间时间不一致造成 regionserver 启动失败 http://www.linuxidc.com/Linux/2013-06/86655.htm

Hadoop+ZooKeeper+HBase 集群配置 http://www.linuxidc.com/Linux/2013-06/86347.htm

Hadoop 集群安装 &HBase 实验环境搭建 http://www.linuxidc.com/Linux/2013-04/83560.htm

基于 Hadoop 集群的 HBase 集群的配置 http://www.linuxidc.com/Linux/2013-03/80815.htm‘

Hadoop 安装部署笔记之 -HBase 完全分布模式安装 http://www.linuxidc.com/Linux/2012-12/76947.htm

单机版搭建 HBase 环境图文教程详解 http://www.linuxidc.com/Linux/2012-10/72959.htm

2：配置 hbase

解压 tar zxf hbase-0.94.21.tar.gz

配置 hbase-env.sh

[linuxidc@linuxidc02 hbase-0.94.21]$ cd conf
[linuxidc@linuxidc02 conf]$ cat hbase-env.sh
#
#/**
# * Copyright 2007 The Apache Software Foundation
# *
# * Licensed to the Apache Software Foundation (ASF) under one
# * or more contributor license agreements. See the NOTICE file
# * distributed with this work for additional information
# * regarding copyright ownership. The ASF licenses this file
# * to you under the Apache License, Version 2.0 (the
# * “License”); you may not use this file except in compliance
# * with the License. You may obtain a copy of the License at
# *
# * http://www.apache.org/licenses/LICENSE-2.0
# *
# * Unless required by applicable law or agreed to in writing, software
# * distributed under the License is distributed on an “AS IS” BASIS,
# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# * See the License for the specific language governing permissions and
# * limitations under the License.
# */

# Set environment variables here.

# This script sets variables multiple times over the course of starting an hbase process,
# so try to keep things idempotent unless you want to take an even deeper look
# into the startup scripts (bin/hbase, etc.)

# The Java implementation to use. Java 1.6 required.
export JAVA_HOME=$HOME/jdk1.7.0_45

# Extra Java CLASSPATH elements. Optional.
export HBASE_CLASSPATH=$HOME/hadoop/hadoop-1.2.1/conf

# The maximum amount of heap to use, in MB. Default is 1000.
# export HBASE_HEAPSIZE=1000

# Extra Java runtime options.
# Below are what we set by default. May only work with SUN JVM.
# For more on why as well as other possible settings,
# see http://wiki.apache.org/hadoop/PerformanceTuning
export HBASE_OPTS=”-XX:+UseConcMarkSweepGC”

# Uncomment one of the below three options to enable java garbage collection logging for the server-side processes.

# This enables basic gc logging to the .out file.
# export SERVER_GC_OPTS=”-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps”

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export SERVER_GC_OPTS=”-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH>”

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0_34+ and 1.7.0_2+.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export SERVER_GC_OPTS=”-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M”

# Uncomment one of the below three options to enable java garbage collection logging for the client processes.

# This enables basic gc logging to the .out file.
# export CLIENT_GC_OPTS=”-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps”

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export CLIENT_GC_OPTS=”-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH>”

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0_34+ and 1.7.0_2+.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export CLIENT_GC_OPTS=”-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M”

# Uncomment below if you intend to use the EXPERIMENTAL off heap cache.
# export HBASE_OPTS=”$HBASE_OPTS -XX:MaxDirectMemorySize=”
# Set hbase.offheapcache.percentage in hbase-site.xml to a nonzero value.

# Uncomment and adjust to enable JMX exporting
# See jmxremote.password and jmxremote.access in $JRE_HOME/lib/management to configure remote password access.
# More details at: http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html
#
# export HBASE_JMX_BASE=”-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false”
# export HBASE_MASTER_OPTS=”$HBASE_MASTER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10101″
# export HBASE_REGIONSERVER_OPTS=”$HBASE_REGIONSERVER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10102″
# export HBASE_THRIFT_OPTS=”$HBASE_THRIFT_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10103″
# export HBASE_ZOOKEEPER_OPTS=”$HBASE_ZOOKEEPER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10104″

# File naming hosts on which HRegionServers will run. $HBASE_HOME/conf/regionservers by default.
# export HBASE_REGIONSERVERS=${HBASE_HOME}/conf/regionservers

# File naming hosts on which backup HMaster will run. $HBASE_HOME/conf/backup-masters by default.
# export HBASE_BACKUP_MASTERS=${HBASE_HOME}/conf/backup-masters

# Extra ssh options. Empty by default.
# export HBASE_SSH_OPTS=”-o ConnectTimeout=1 -o SendEnv=HBASE_CONF_DIR”

# Where log files are stored. $HBASE_HOME/logs by default.
# export HBASE_LOG_DIR=${HBASE_HOME}/logs

# Enable remote JDWP debugging of major HBase processes. Meant for Core Developers
# export HBASE_MASTER_OPTS=”$HBASE_MASTER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8070″
# export HBASE_REGIONSERVER_OPTS=”$HBASE_REGIONSERVER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071″
# export HBASE_THRIFT_OPTS=”$HBASE_THRIFT_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072″
# export HBASE_ZOOKEEPER_OPTS=”$HBASE_ZOOKEEPER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073″

# A string representing this instance of hbase. $USER by default.
# export HBASE_IDENT_STRING=$USER

# The scheduling priority for daemon processes. See ‘man nice’.
# export HBASE_NICENESS=10

# The directory where pid files are stored. /tmp by default.
export HBASE_PID_DIR=/home/linuxidc/pids

# Seconds to sleep between slave commands. Unset by default. This
# can be useful in large clusters, where, e.g., slave rsyncs can
# otherwise arrive faster than the master can service them.
# export HBASE_SLAVE_SLEEP=0.1

# Tell HBase whether it should manage it’s own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=true

修改的地方
#JDK 的安装目录
export JAVA_HOME=$HOME/jdk1.7.0_45
#hadoop 的配置目录
export HBASE_CLASSPATH=$HOME/hadoop/hadoop-1.2.1/conf

和最后一行

export HBASE_MANAGES_ZK=true

#true：表示 zookeeper 交给 hbase 管理，启动 hbase 时，会自动启动 hbase-site.xml 里的 hbase.zookeeper.quorum 属性中的所有 zookeeper 实例 #false：表示启动 hbase 时，要手动启动所有 zookeeper 实例 
 
 配置 hbase-site.xml

[linuxidc@linuxidc02 conf]$ cat hbase-site.xml
<?xml version=”1.0″?>
<?xml-stylesheet type=”text/xsl” href=”https://www.linuxidc.com/Linux/2014-09/configuration.xsl”?>
<!–
/**
* Copyright 2010 The Apache Software Foundation
*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* “License”); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an “AS IS” BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
–>
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://linuxidc01:9000/user/linuxidc/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.master</name>
<value>linuxidc01:60000</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>linuxidc01,linuxidc02</value>
</property>

</configuration>

配置 regionservers

[linuxidc@linuxidc02 conf]$ cat regionservers
linuxidc01
linuxidc02

更多详情见请继续阅读下一页的精彩内容 ：http://www.linuxidc.com/Linux/2014-09/106534p2.htm

3：配置 zookeeper

————————————– 分割线 ————————————–

ZooKeeper 集群配置 http://www.linuxidc.com/Linux/2013-06/86348.htm

使用 ZooKeeper 实现分布式共享锁 http://www.linuxidc.com/Linux/2013-06/85550.htm

分布式服务框架 ZooKeeper — 管理分布式环境中的数据 http://www.linuxidc.com/Linux/2013-06/85549.htm

ZooKeeper 集群环境搭建实践 http://www.linuxidc.com/Linux/2013-04/83562.htm

ZooKeeper 服务器集群环境配置实测 http://www.linuxidc.com/Linux/2013-04/83559.htm

ZooKeeper 集群安装 http://www.linuxidc.com/Linux/2012-10/72906.htm

————————————– 分割线 ————————————–

解压 tar zxf zookeeper-3.4.6.tar.gz

[linuxidc@linuxidc02 conf]# cp zoo_sample.cfg zoo.cfg
[linuxidc@linuxidc02 conf]$ vi zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/home/linuxidc/zookeeper-3.4.6/data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to “0” to disable auto purge feature
#autopurge.purgeInterval=1
server.1=linuxidc01:2888:3888
server.2=linuxidc02:2888:3888

4:Hadoop-core-1.2.1.jar 替换 hbase-0.94.21/lib 目录下的 hadoop-core-1.0.4.jar

[linuxidc@linuxidc02 hadoop-1.2.1]$ cp ./hadoop-core-1.2.1.jar ../../hbase-0.94.21/lib

[linuxidc@linuxidc02 hadoop-1.2.1]$ rm hadoop-core-1.0.4.jar

5: 配置环境变量和复制文件到 linuxidc01

[linuxidc@linuxidc01 ~]$ cat .bash_profile
# .bash_profile

# Get the aliases and functions
if [-f ~/.bashrc]; then
. ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH
export Java_HOME=$HOME/jdk1.7.0_45
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export HADOOP_HOME=$HOME/hadoop/hadoop-1.2.1
export ANT_HOME=$HOME/apache-ant-1.9.4
export HBASE_HOME=$HOME/hbase-0.94.21

expor PATH=$PATH:$ANT_HOME/bin:$HBASE_HOME/bin

scp -r hbase-0.94.21 linuxidc@linuxidc01:/home/linuxidc

scp -r zookeeper-3.4.6 linuxidc@linuxidc01:/home/linuxidc

6: 启动 HBase

[linuxidc@linuxidc01 ~]$ cd hbase-0.94.21
[linuxidc@linuxidc01 hbase-0.94.21]$ bin/start-hbase.sh
localhost: starting zookeeper, logging to /home/linuxidc/hbase-0.94.21/bin/../logs/hbase-linuxidc-zookeeper-linuxidc01.out
starting master, logging to /home/linuxidc/hbase-0.94.21/logs/hbase-linuxidc-master-linuxidc01.out
linuxidc01: starting regionserver, logging to /home/linuxidc/hbase-0.94.21/bin/../logs/hbase-linuxidc-regionserver-linuxidc01.out
linuxidc02: starting regionserver, logging to /home/linuxidc/hbase-0.94.21/bin/../logs/hbase-linuxidc-regionserver-linuxidc02.out
[linuxidc@linuxidc01 hbase-0.94.21]$ jps
15627 HQuorumPeer
15705 HMaster
15880 HRegionServer
4529 NameNode
16040 Jps
4822 JobTracker
[linuxidc@linuxidc01 hbase-0.94.21]$ bin/hbase shell
HBase Shell; enter ‘help<RETURN>’ for list of supported commands.
Type “exit<RETURN>” to leave the HBase Shell
Version 0.94.21, r83b4a1ee9b9a2fa4c7ae1739259e041cabe8edc2, Fri Jun 27 16:14:16 UTC 2014

hbase(main):001:0> status
1 servers, 0 dead, 2.0000 average load

hbase(main):002:0> exit

只启动了一台服务器

7：查看日志

[linuxidc@linuxidc02 logs]$ cat hbase-linuxidc-regionserver-linuxidc02.log

2014-08-16 15:37:57,847 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at linuxidc01,60000,1408174832154 that we are up with port=60020, startcode=1408174674748
2014-08-16 15:37:57,854 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: Master rejected startup because clock is out of sync
org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server linuxidc02,60020,1408174674748 has been rejected;Reported time is too far out of sync with master. Time difference of 158488ms > max allowed of 30000ms
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2116)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server linuxidc02,60020,1408174674748 has been rejected; Reported time is too far out of sync with master. Time difference of 158488ms > max allowed of 30000ms
at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:235)
at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:169)
at org.apache.hadoop.hbase.master.HMaster.regionServerStartup(HMaster.java:1084)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1434)

at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1012)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87)
at com.sun.proxy.$Proxy9.regionServerStartup(Unknown Source)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2114)
… 2 more

时间不同步的问题，

解决方法：

[root@linuxidc02 ~]# date -s “2015-08-16 16:10:00”; clock -w

8：重新启动 hbase

停止 [linuxidc@linuxidc01 hbase-0.94.21]$ bin/stop-hbase.sh

如果 linuxidc02 的服务没全部关闭，先用

[linuxidc@linuxidc02 ~]$ jps
19487 HQuorumPeer
19577 HRegionServer
19839 Jps
3982 TaskTracker
3888 DataNode
[linuxidc@linuxidc02 ~]$

然后 kill -s 9 19487
然后 kill -s 9 19577

[linuxidc@linuxidc01 hbase-0.94.21]$ bin/hbase shell
HBase Shell; enter ‘help<RETURN>’ for list of supported commands.
Type “exit<RETURN>” to leave the HBase Shell
Version 0.94.21, r83b4a1ee9b9a2fa4c7ae1739259e041cabe8edc2, Fri Jun 27 16:14:16 UTC 2014

hbase(main):001:0> status
2 servers, 0 dead, 1.0000 average load

hbase(main):002:0>

HBase 0.94.21+ZooKeeper-3.4.6 分布式安装