安装 HBase 0.20.x

我把HBase官方的文档译了一下，没完成，方便对比着看。

Requirements

• Java 1.6.x, preferably from Sun. Use the latest version available.
• Java 1.6.x ,sun ，最好是最新版
• This version of HBase will only run on Hadoop 0.20.x.
• 指定的Hadoop版 本
• ssh must be installed and sshd must be running to use Hadoop’s scripts to manage remote Hadoop daemons. You must be able to ssh to all nodes, including your local node, using passwordless login (Google “ssh passwordless login”).
• ssh 和sshd 必须有，机器之间无须输入密码登录
• HBase depends on ZooKeeper as of release 0.20.0. HBase keeps the location of its root table, who the current master is, and what regions are currently participating in the cluster in ZooKeeper. Clients and Servers now must know theirZooKeeper Quorum locations before they can do anything else (Usually they pick up this information from configuration supplied on their CLASSPATH). By default, HBase will manage a single ZooKeeper instance for you. In standalone andpseudo-distributed modes this is usually enough, but for fully-distributed mode you should configure a ZooKeeper quorum (more info below).
• 需要指定版本的 ZooKeeper ，客户和服务在工作之前，需要在他们的CLASSPATH提供的配置中获得 他们的 ZookKeeper Quorum locations 信息。 缺省情况下，HBase管理一个Zookeeper实例。在standalone和 pseudo-distibuted 模式下是足够的，但是在 fully-distributed 模式，你需要配置一个 ZooKeeper quorum。（下面有更详细的信息）
• Hosts must be able to resolve the fully-qualified domain name of the master.
• 所有主机必须能解释 master 的 fully-qualified domain name 。
• HBase currently is a file handle hog. The usual default of 1024 on *nix systems is insufficient if you are loading any significant amount of data into regionservers. See the FAQ: Why do I see “java.io.IOException…(Too many open files)” in my logs? for how to up the limit. Also, as of 0.18.x Hadoop DataNodes have an upper-bound on the number of threads they will support (dfs.datanode.max.xcievers). The default is 256 threads. Up this limit on your hadoop cluster.
• 需要调整系统相关的文件句柄上限和线程上限。/etc/security /limits.conf 和 dfs.datanode.max.xcievers。
• The clocks on cluster members should be in basic alignments. Some skew is tolerable but wild skew could generate odd behaviors. Run NTP on your cluster, or an equivalent.
• 集群内所有节点时间同步，使用NTP。
• HBase servers put up 10 listeners for incoming connections by default. Up this number if you have a dataset of any substance by setting hbase.regionserver.handler.count in your hbase-site.xml.
• 缺省10个 listerners 处理进入到连接，可以通过配置 hbase-site.xml 的 hbase.regionserver.handler.count 来修改。
• This is the current list of patches we recommend you apply to your running Hadoop cluster:
• 下面是两个推荐使用的Hadoop集群的补丁：
• HADOOP-4681/HDFS-127 “DFSClient block read failures cause open DFSInputStream to become unusable”. This patch will help with the ever-popular, “No live nodes contain current block”. The hadoop version bundled with hbase has this patch applied. Its an HDFS client fix so this should do for usual usage but if your cluster is missing the patch, and in particular if calling hbase from a mapreduce job, you may run into this issue. If running with hadoop 0.20.2, it has this patch in place already.
• HDFS-630: “In DFSOutputStream.nextBlockOutputStream(), the client can exclude specific datanodes when locating the next block”. Dead DataNodes take ten minutes to timeout at NameNode. In the meantime the NameNode can still send DFSClients to the dead DataNode as host for a replicated block. DFSClient can get stuck on trying to get block from a dead node. This patch allows DFSClients pass NameNode lists of known dead DataNodes.

Windows

需要安装Cygwin

Getting Started

What follows presumes you have obtained a copy of HBase, see Releases, and are installing for the first time. If upgrading your HBase instance, see Upgrading

假定已经获得Hbase的拷贝， 升级请看这里 Upgrading

Three modes are described: standalone, pseudo-distributed (where all servers are run on a single host), and fully-distributed. If new to HBase start by following the standalone instructions.

三 种模式：standalone , preudo-distributed, fully-distributed.

Begin by reading Requirements.

Whatever your mode, define ${HBASE_HOME} to be the location of the root of your HBase installation, e.g. /user/local/hbase. Edit ${HBASE_HOME}/conf/hbase-env.sh. In this file you can set the heapsize for HBase, etc. At a minimum, set JAVA_HOME to point at the root of your Java installation.

无论使用 什么模式，都要定义 ${HBASE_HOME}到安装目录，编辑 ${HBASE_HOME}/conf/hbase-env.sh ，设置 HBase的 heapsize 等等，至少要设置 JAVA_HOME.

Standalone mode

If you are running a standalone operation, there should be nothing further to configure; proceed to Running and Confirming Your Installation. If you are running a distributed operation, continue reading.

standalone模式不需要配置，直接运行就可以。

Distributed Operation: Pseudo- and Fully-distributed modes

Distributed modes require an instance of the Hadoop Distributed File System (DFS). See the Hadoop requirements and instructions for how to set up a DFS.

分布式模式下需要一个HDFS实例。看这里Hadoop requirements and instructions如何安装一个DFS

Pseudo-distributed mode

A pseudo-distributed mode is simply a distributed mode run on a single host. Once you have confirmed your DFS setup, configuring HBase for use on one host requires modification of ${HBASE_HOME}/conf/hbase-site.xml, which needs to be pointed at the running Hadoop DFS instance. Use hbase-site.xml to override the properties defined in ${HBASE_HOME}/conf/hbase-default.xml(hbase-default.xml itself should never be modified). At a minimum the hbase.rootdir property should be redefined in hbase-site.xml to point HBase at the Hadoop filesystem to use. For example, adding the property below to your hbase-site.xml says that HBase should use the /hbase directory in the HDFS whose namenode is at port 9000 on your local machine:

pseudo-distributed模式，仅仅在一台机器上运行一个 distributed模式。DFS setup 后，配置 HBase 在一台机器上运行需要修改 ${HBASE_HOOME}/conf/hbase-site.xml 文件， 至少要 重新定义 hbase.rootdir 属性，指向要使用的 HDFS 。例如增加下面的属性到 hbase-site.xml , 说明 HBase 应该使用HDFS的 /hbase 目录，HDFS的名字节点在 localhost ，端口号为 9000

<configuration>

…

<property>

<name>hbase.rootdir</name>

<value>hdfs://localhost:9000/hbase</value>

<description>The directory shared by region servers.

</description>

</property>

…

</configuration>

Note: Let HBase create the directory. If you don’t, you’ll get warning saying HBase needs a migration run because the directory is missing files expected by HBase (it’ll create them if you let it).

Also Note: Above we bind to localhost. This means that a remote client cannot connect. Amend accordingly, if you want to connect from a remote location.

注意：要让 HBase自己创建这个目录。否则你会得到一个警告，HBase needs a migration run because the directory is missing files expected by HBase (it’ll create them if you let it).

同时要注意：上面绑定的是 localhost . 这表示 远程的客户无法连接。如果需要远程了解需要做相应的修改。

Fully-Distributed Operation

For running a fully-distributed operation on more than one host, the following configurations must be made in addition to those described in the pseudo-distributed operation section above.

fully-distributed 模式比较复杂一点，下面的配置内容需要额外增加到上面描述的 pseudo-distributed 操作段落上面。

In hbase-site.xml, set hbase.cluster.distributed to true.

在 hbase-site.xml ，设置 hbase.cluster.distributed 为true.

<configuration>

…

<property>

<name>hbase.cluster.distributed</name>

<value>true</value>

<description>The mode the cluster will be in. Possible values are

false: standalone and pseudo-distributed setups with managed Zookeeper

true: fully-distributed with unmanaged Zookeeper Quorum (see hbase-env.sh)

</description>

</property>

…

</configuration>

In fully-distributed mode, you probably want to change your hbase.rootdir from localhost to the name of the node running the HDFS NameNode. In addition to hbase-site.xml changes, a fully-distributed mode requires that you modify${HBASE_HOME}/conf/regionservers. The regionserver file lists all hosts running HRegionServers, one host per line (This file in HBase is like the Hadoop slaves file at ${HADOOP_HOME}/conf/slaves).

hbase.rootdir 改为 HDFS 的名字节点。修改 conf/regionservers 文件。一行一个HRegionServer 的主机。就如 Hadoop 的 slaves 文件一样。

A distributed HBase depends on a running ZooKeeper cluster. All participating nodes and clients need to be able to get to the running ZooKeeper cluster. HBase by default manages a ZooKeeper cluster for you, or you can manage it on your own and point HBase to it. To toggle HBase management of ZooKeeper, use the HBASE_MANAGES_ZK variable in ${HBASE_HOME}/conf/hbase-env.sh. This variable, which defaults to true, tells HBase whether to start/stop the ZooKeeper quorum servers alongside the rest of the servers.

需要 Zookeeper 集群。 所有节点和客户都需要能够到达所有的 Zookeeper集群。 HBase缺省管理一个Zookeeper，或者你自己管理，然后让HBase指向它。让 HBase 管理的，修改 conf/hbase-env.sh 文件，让其中的 HBASE_MANAGERS_ZK 为 true ， 告诉 HBase 是否随 ZooKeeper quorum servers 的启动/关闭而进行启动/关闭。

When HBase manages the ZooKeeper cluster, you can specify ZooKeeper configuration using its canonical zoo.cfg file (see below), or just specify ZookKeeper options directly in the ${HBASE_HOME}/conf/hbase-site.xml (If new to ZooKeeper, go the path of specifying your configuration in HBase’s hbase-site.xml). Every ZooKeeper configuration option has a corresponding property in the HBase hbase-site.xml XML configuration file named hbase.zookeeper.property.OPTION. For example, the clientPortsetting in ZooKeeper can be changed by setting the hbase.zookeeper.property.clientPort property. For the full list of available properties, see ZooKeeper’s zoo.cfg. For the default values used by HBase, see ${HBASE_HOME}/conf/hbase-default.xml.

当 HBase 管理 ZooKeeper 集群， 你可以用规定的 zoo.cfg 文件来描述 Zookeeper ， 或者直接在 hbase-site.xml 中描述 Zookeeper 的选项。如果是新手，直接在 hbase-site.xml 中配置比较好。每一个ZooKeeper 配置选项 在 HBase 的 hbase-site.xml 文件中都有 对应的属性，名字为 hbase.zookeeper.property.OPTION，其中 OPTION 是变化的参数，如 hbase.zookeeper.property.clientPort 属性。从 ZooKeeper 的 zoo.cfg 可以获得所有的属性。从 hbase-defalut.xml 可以获得缺省的属性。

At minimum, you should set the list of servers that you want ZooKeeper to run on using the hbase.zookeeper.quorum property. This property defaults to localhost which is not suitable for a fully distributed HBase (it binds to the local machine only and remote clients will not be able to connect). It is recommended to run a ZooKeeper quorum of 3, 5 or 7 machines, and give each ZooKeeper server around 1GB of RAM, and if possible, its own dedicated disk. For very heavily loaded clusters, run ZooKeeper servers on separate machines from the Region Servers (DataNodes and TaskTrackers).

至少，使用 hbase.zookeeper.quorum 属性设置 Zookeeper 运行的服务器。 缺省 为localhost，这和 fully distributed 模式不合，仅绑定本地机器，远程机器无法连接。建议运行 3,5或者7台机器的 Zookeeper quorum ， 给每个ZooKeeper服务器1G的内存, 如果可以的话，拥有专属的磁盘 。对于负载非常重的集群，在其他的机器上运行ZooKeeper服务，不要和 Region Servers（DataNodes 和 TaskTrackers） 放在一起。

To point HBase at an existing ZooKeeper cluster, add a suitably configured zoo.cfg to the CLASSPATH. HBase will see this file and use it to figure out where ZooKeeper is. Additionally set HBASE_MANAGES_ZK in ${HBASE_HOME}/conf/hbase-env.sh to false so that HBase doesn’t mess with your ZooKeeper setup:

要把HBase指向已经存在的 Zookeeper 集群，增加一个合适的配置文件 zoo.cfg 到 CLASSPATH . HBase 将会使用这个文件来找到 ZooKeeper 的位置。 另外设置 hbase-env.sh 的 HBASE_MANAGERS_ZK 为 false ，以免HBase将 Zookeeper的设置搞乱。

…

# Tell HBase whether it should manage it’s own instance of Zookeeper or not.

export HBASE_MANAGES_ZK=false

As an example, to have HBase manage a ZooKeeper quorum on nodes rs{1,2,3,4,5}.example.com, bound to port 2222 (the default is 2181), use:

作为一个例子， HBase管理在节点 rs[1,2,3,4,5].example.com 上的 ZooKeeper quorum ，绑定 端口 2222 （缺省是2181）：

${HBASE_HOME}/conf/hbase-env.sh:

…

# Tell HBase whether it should manage it’s own instance of Zookeeper or not.

export HBASE_MANAGES_ZK=true

${HBASE_HOME}/conf/hbase-site.xml:

<configuration>

…

<property>

<name>hbase.zookeeper.property.clientPort</name>

<value>2222</value>

<description>Property from ZooKeeper’s config zoo.cfg.

The port at which the clients will connect.

</description>

</property>

…

<property>

<name>hbase.zookeeper.quorum</name>

<value>rs1.example.com,rs2.example.com,rs3.example.com,rs4.example.com,rs5.example.com</value>

<description>Comma separated list of servers in the ZooKeeper Quorum.

For example, “host1.mydomain.com,host2.mydomain.com,host3.mydomain.com“.

By default this is set to localhost for local and pseudo-distributed modes

of operation. For a fully-distributed setup, this should be set to a full

list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh

this is the list of servers which we will start/stop ZooKeeper on.

</description>

</property>

…

</configuration>

When HBase manages ZooKeeper, it will start/stop the ZooKeeper servers as a part of the regular start/stop scripts. If you would like to run it yourself, you can do:

当 HBase来管理Zookeeper，它会将启动/关闭Zookeeper servers作为正常启动/关闭脚本的一部分。如果你想自己来运行，可以：

${HBASE_HOME}/bin/hbase-daemons.sh {start,stop} zookeeper

Note that you can use HBase in this manner to spin up a ZooKeeper cluster, unrelated to HBase. Just make sure to set HBASE_MANAGES_ZK to false if you want it to stay up so that when HBase shuts down it doesn’t take ZooKeeper with it.

For more information about setting up a ZooKeeper cluster on your own, see the ZooKeeper Getting Started Guide. HBase currently uses ZooKeeper version 3.2.0, so any cluster setup with a 3.x.x version of ZooKeeper should work.

要得到更多的 ZooKeeper 集群的配置信息，请看 Getting Started Guide 。 HBase 当前使用 3.2.0 的版本，因此 3.x.x 版本的资料可以参考。

Of note, if you have made HDFS client configuration on your Hadoop cluster, HBase will not see this configuration unless you do one of the following:

注意，如果你 在 Hadoop集群中进行 HDFS client 的配置，HBase无法看到这些配置，除非你做下面的任何一项工作：

• Add a pointer to your HADOOP_CONF_DIR to CLASSPATH in hbase-env.sh.
• 在 hbase-env.sh 中增加一个路径指向 HADOOP_CONF_DIR
• Add a copy of hdfs-site.xml (or hadoop-site.xml) to ${HBASE_HOME}/conf, or
• 增加一个 hdfs-site.xml 或者 hadoop-site.xml 的拷贝到 ${HBASE_HOME}/conf 目录下，或者
• if only a small set of HDFS client configurations, add them to hbase-site.xml.
• 如果 HBase的有关配置很少，增加配置内容到 hbase-site.xml 中。

An example of such an HDFS client configuration is dfs.replication. If for example, you want to run with a replication factor of 5, hbase will create files with the default of 3 unless you do the above to make the configuration available to HBase.

例如，需要调整 dfs 的复制因子。 dfs.replication ,hbase缺省是 3 ,你想修改的话就在 HBase的配置文件中修改。

Running and Confirming Your Installation

If you are running in standalone, non-distributed mode, HBase by default uses the local filesystem.

非分布式，将使用本地文件系统

If you are running a distributed cluster you will need to start the Hadoop DFS daemons and ZooKeeper Quorum before starting HBase and stop the daemons after HBase has shut down.

分布式需要 先启动/后关闭 HDFS daemons 和 ZooKeeper Quorum 。

Start and stop the Hadoop DFS daemons by running ${HADOOP_HOME}/bin/start-dfs.sh. You can ensure it started properly by testing the put and get of files into the Hadoop filesystem. HBase does not normally use the mapreduce daemons. These do not need to be started.

启动 HDFS。

关闭 HDFS 之前必须先确保 HBase已经关闭成功。

Start up your ZooKeeper cluster.

启动 ZooKeeper 集群。

Start HBase with the following command:

使用下面的命令：

${HBASE_HOME}/bin/start-hbase.sh

Once HBase has started, enter ${HBASE_HOME}/bin/hbase shell to obtain a shell against HBase from which you can execute commands. Type ‘help’ at the shells’ prompt to get a list of commands. Test your running install by creating tables, inserting content, viewing content, and then dropping your tables. For example:

使用 HBase shell 来测试。

hbase> # Type “help” to see shell help screen

hbase> help

hbase> # To create a table named “mylittletable” with a column family of “mylittlecolumnfamily”, type

hbase> create “mylittletable”, “mylittlecolumnfamily”

hbase> # To see the schema for you just created “mylittletable” table and its single “mylittlecolumnfamily”, type

hbase> describe “mylittletable”

hbase> # To add a row whose id is “myrow”, to the column “mylittlecolumnfamily:x” with a value of ‘v’, do

hbase> put “mylittletable”, “myrow”, “mylittlecolumnfamily:x”, “v”

hbase> # To get the cell just added, do

hbase> get “mylittletable”, “myrow”

hbase> # To scan you new table, do

hbase> scan “mylittletable”

To stop HBase, exit the HBase shell and enter:

下面的命令关闭 HBase .

${HBASE_HOME}/bin/stop-hbase.sh

If you are running a distributed operation, be sure to wait until HBase has shut down completely before stopping the Hadoop daemons.

The default location for logs is ${HBASE_HOME}/logs.

HBase also puts up a UI listing vital attributes. By default its deployed on the master host at port 60010 (HBase RegionServers listen on port 60020 by default and put up an informational http server at 60030).

端口 : 60010(master) 60020(region servers)