-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.\
internal.jaxp.DocumentBuilderFactoryImpl

使 用上面指定的分析器就不再抛出异常

Requirements

• Java 1.6.x, preferably from Sun. Use the latest version available.
• Java 1.6.x ,sun ，最好是最新版
• This version of HBase will only run on Hadoop 0.20.x.
• 指定的Hadoop版 本
• ssh must be installed and sshd must be running to use Hadoop’s scripts to manage remote Hadoop daemons. You must be able to ssh to all nodes, including your local node, using passwordless login (Google “ssh passwordless login”).
• ssh 和sshd 必须有，机器之间无须输入密码登录
• HBase depends on ZooKeeper as of release 0.20.0. HBase keeps the location of its root table, who the current master is, and what regions are currently participating in the cluster in ZooKeeper. Clients and Servers now must know theirZooKeeper Quorum locations before they can do anything else (Usually they pick up this information from configuration supplied on their CLASSPATH). By default, HBase will manage a single ZooKeeper instance for you. In standalone andpseudo-distributed modes this is usually enough, but for fully-distributed mode you should configure a ZooKeeper quorum (more info below).
• 需要指定版本的 ZooKeeper ，客户和服务在工作之前，需要在他们的CLASSPATH提供的配置中获得 他们的 ZookKeeper Quorum locations 信息。 缺省情况下，HBase管理一个Zookeeper实例。在standalone和 pseudo-distibuted 模式下是足够的，但是在 fully-distributed 模式，你需要配置一个 ZooKeeper quorum。（下面有更详细的信息）
• Hosts must be able to resolve the fully-qualified domain name of the master.
• 所有主机必须能解释 master 的 fully-qualified domain name 。
• HBase currently is a file handle hog. The usual default of 1024 on *nix systems is insufficient if you are loading any significant amount of data into regionservers. See the FAQ: Why do I see “java.io.IOException…(Too many open files)” in my logs? for how to up the limit. Also, as of 0.18.x Hadoop DataNodes have an upper-bound on the number of threads they will support (dfs.datanode.max.xcievers). The default is 256 threads. Up this limit on your hadoop cluster.
• 需要调整系统相关的文件句柄上限和线程上限。/etc/security /limits.conf 和 dfs.datanode.max.xcievers。
• The clocks on cluster members should be in basic alignments. Some skew is tolerable but wild skew could generate odd behaviors. Run NTP on your cluster, or an equivalent.
• 集群内所有节点时间同步，使用NTP。
• HBase servers put up 10 listeners for incoming connections by default. Up this number if you have a dataset of any substance by setting hbase.regionserver.handler.count in your hbase-site.xml.
• 缺省10个 listerners 处理进入到连接，可以通过配置 hbase-site.xml 的 hbase.regionserver.handler.count 来修改。
• This is the current list of patches we recommend you apply to your running Hadoop cluster:
• 下面是两个推荐使用的Hadoop集群的补丁：
• HADOOP-4681/HDFS-127 “DFSClient block read failures cause open DFSInputStream to become unusable”. This patch will help with the ever-popular, “No live nodes contain current block”. The hadoop version bundled with hbase has this patch applied. Its an HDFS client fix so this should do for usual usage but if your cluster is missing the patch, and in particular if calling hbase from a mapreduce job, you may run into this issue. If running with hadoop 0.20.2, it has this patch in place already.
• HDFS-630: “In DFSOutputStream.nextBlockOutputStream(), the client can exclude specific datanodes when locating the next block”. Dead DataNodes take ten minutes to timeout at NameNode. In the meantime the NameNode can still send DFSClients to the dead DataNode as host for a replicated block. DFSClient can get stuck on trying to get block from a dead node. This patch allows DFSClients pass NameNode lists of known dead DataNodes.

Windows

需要安装Cygwin

Getting Started

What follows presumes you have obtained a copy of HBase, see Releases, and are installing for the first time. If upgrading your HBase instance, see Upgrading

假定已经获得Hbase的拷贝， 升级请看这里 Upgrading

Three modes are described: standalone, pseudo-distributed (where all servers are run on a single host), and fully-distributed. If new to HBase start by following the standalone instructions.

三 种模式：standalone , preudo-distributed, fully-distributed.

Begin by reading Requirements.

Whatever your mode, define ${HBASE_HOME} to be the location of the root of your HBase installation, e.g. /user/local/hbase. Edit ${HBASE_HOME}/conf/hbase-env.sh. In this file you can set the heapsize for HBase, etc. At a minimum, set JAVA_HOME to point at the root of your Java installation.

无论使用 什么模式，都要定义 ${HBASE_HOME}到安装目录，编辑 ${HBASE_HOME}/conf/hbase-env.sh ，设置 HBase的 heapsize 等等，至少要设置 JAVA_HOME.

Standalone mode

If you are running a standalone operation, there should be nothing further to configure; proceed to Running and Confirming Your Installation. If you are running a distributed operation, continue reading.

standalone模式不需要配置，直接运行就可以。

Distributed Operation: Pseudo- and Fully-distributed modes

Distributed modes require an instance of the Hadoop Distributed File System (DFS). See the Hadoop requirements and instructions for how to set up a DFS.

分布式模式下需要一个HDFS实例。看这里Hadoop requirements and instructions如何安装一个DFS

Pseudo-distributed mode

A pseudo-distributed mode is simply a distributed mode run on a single host. Once you have confirmed your DFS setup, configuring HBase for use on one host requires modification of ${HBASE_HOME}/conf/hbase-site.xml, which needs to be pointed at the running Hadoop DFS instance. Use hbase-site.xml to override the properties defined in ${HBASE_HOME}/conf/hbase-default.xml(hbase-default.xml itself should never be modified). At a minimum the hbase.rootdir property should be redefined in hbase-site.xml to point HBase at the Hadoop filesystem to use. For example, adding the property below to your hbase-site.xml says that HBase should use the /hbase directory in the HDFS whose namenode is at port 9000 on your local machine:

pseudo-distributed模式，仅仅在一台机器上运行一个 distributed模式。DFS setup 后，配置 HBase 在一台机器上运行需要修改 ${HBASE_HOOME}/conf/hbase-site.xml 文件， 至少要 重新定义 hbase.rootdir 属性，指向要使用的 HDFS 。例如增加下面的属性到 hbase-site.xml , 说明 HBase 应该使用HDFS的 /hbase 目录，HDFS的名字节点在 localhost ，端口号为 9000

<configuration>

…

<property>

<name>hbase.rootdir</name>

<value>hdfs://localhost:9000/hbase</value>

<description>The directory shared by region servers.

</description>

</property>

…

</configuration>

Note: Let HBase create the directory. If you don’t, you’ll get warning saying HBase needs a migration run because the directory is missing files expected by HBase (it’ll create them if you let it).

Also Note: Above we bind to localhost. This means that a remote client cannot connect. Amend accordingly, if you want to connect from a remote location.

注意：要让 HBase自己创建这个目录。否则你会得到一个警告，HBase needs a migration run because the directory is missing files expected by HBase (it’ll create them if you let it).

同时要注意：上面绑定的是 localhost . 这表示 远程的客户无法连接。如果需要远程了解需要做相应的修改。

Fully-Distributed Operation

For running a fully-distributed operation on more than one host, the following configurations must be made in addition to those described in the pseudo-distributed operation section above.

fully-distributed 模式比较复杂一点，下面的配置内容需要额外增加到上面描述的 pseudo-distributed 操作段落上面。

In hbase-site.xml, set hbase.cluster.distributed to true.

在 hbase-site.xml ，设置 hbase.cluster.distributed 为true.

<configuration>

…

<property>

<name>hbase.cluster.distributed</name>

<value>true</value>

<description>The mode the cluster will be in. Possible values are

false: standalone and pseudo-distributed setups with managed Zookeeper

true: fully-distributed with unmanaged Zookeeper Quorum (see hbase-env.sh)

</description>

</property>

…

</configuration>

In fully-distributed mode, you probably want to change your hbase.rootdir from localhost to the name of the node running the HDFS NameNode. In addition to hbase-site.xml changes, a fully-distributed mode requires that you modify${HBASE_HOME}/conf/regionservers. The regionserver file lists all hosts running HRegionServers, one host per line (This file in HBase is like the Hadoop slaves file at ${HADOOP_HOME}/conf/slaves).

hbase.rootdir 改为 HDFS 的名字节点。修改 conf/regionservers 文件。一行一个HRegionServer 的主机。就如 Hadoop 的 slaves 文件一样。

A distributed HBase depends on a running ZooKeeper cluster. All participating nodes and clients need to be able to get to the running ZooKeeper cluster. HBase by default manages a ZooKeeper cluster for you, or you can manage it on your own and point HBase to it. To toggle HBase management of ZooKeeper, use the HBASE_MANAGES_ZK variable in ${HBASE_HOME}/conf/hbase-env.sh. This variable, which defaults to true, tells HBase whether to start/stop the ZooKeeper quorum servers alongside the rest of the servers.

需要 Zookeeper 集群。 所有节点和客户都需要能够到达所有的 Zookeeper集群。 HBase缺省管理一个Zookeeper，或者你自己管理，然后让HBase指向它。让 HBase 管理的，修改 conf/hbase-env.sh 文件，让其中的 HBASE_MANAGERS_ZK 为 true ， 告诉 HBase 是否随 ZooKeeper quorum servers 的启动/关闭而进行启动/关闭。

When HBase manages the ZooKeeper cluster, you can specify ZooKeeper configuration using its canonical zoo.cfg file (see below), or just specify ZookKeeper options directly in the ${HBASE_HOME}/conf/hbase-site.xml (If new to ZooKeeper, go the path of specifying your configuration in HBase’s hbase-site.xml). Every ZooKeeper configuration option has a corresponding property in the HBase hbase-site.xml XML configuration file named hbase.zookeeper.property.OPTION. For example, the clientPortsetting in ZooKeeper can be changed by setting the hbase.zookeeper.property.clientPort property. For the full list of available properties, see ZooKeeper’s zoo.cfg. For the default values used by HBase, see ${HBASE_HOME}/conf/hbase-default.xml.

当 HBase 管理 ZooKeeper 集群， 你可以用规定的 zoo.cfg 文件来描述 Zookeeper ， 或者直接在 hbase-site.xml 中描述 Zookeeper 的选项。如果是新手，直接在 hbase-site.xml 中配置比较好。每一个ZooKeeper 配置选项 在 HBase 的 hbase-site.xml 文件中都有 对应的属性，名字为 hbase.zookeeper.property.OPTION，其中 OPTION 是变化的参数，如 hbase.zookeeper.property.clientPort 属性。从 ZooKeeper 的 zoo.cfg 可以获得所有的属性。从 hbase-defalut.xml 可以获得缺省的属性。

At minimum, you should set the list of servers that you want ZooKeeper to run on using the hbase.zookeeper.quorum property. This property defaults to localhost which is not suitable for a fully distributed HBase (it binds to the local machine only and remote clients will not be able to connect). It is recommended to run a ZooKeeper quorum of 3, 5 or 7 machines, and give each ZooKeeper server around 1GB of RAM, and if possible, its own dedicated disk. For very heavily loaded clusters, run ZooKeeper servers on separate machines from the Region Servers (DataNodes and TaskTrackers).

至少，使用 hbase.zookeeper.quorum 属性设置 Zookeeper 运行的服务器。 缺省 为localhost，这和 fully distributed 模式不合，仅绑定本地机器，远程机器无法连接。建议运行 3,5或者7台机器的 Zookeeper quorum ， 给每个ZooKeeper服务器1G的内存, 如果可以的话，拥有专属的磁盘 。对于负载非常重的集群，在其他的机器上运行ZooKeeper服务，不要和 Region Servers（DataNodes 和 TaskTrackers） 放在一起。

To point HBase at an existing ZooKeeper cluster, add a suitably configured zoo.cfg to the CLASSPATH. HBase will see this file and use it to figure out where ZooKeeper is. Additionally set HBASE_MANAGES_ZK in ${HBASE_HOME}/conf/hbase-env.sh to false so that HBase doesn’t mess with your ZooKeeper setup:

要把HBase指向已经存在的 Zookeeper 集群，增加一个合适的配置文件 zoo.cfg 到 CLASSPATH . HBase 将会使用这个文件来找到 ZooKeeper 的位置。 另外设置 hbase-env.sh 的 HBASE_MANAGERS_ZK 为 false ，以免HBase将 Zookeeper的设置搞乱。

…

# Tell HBase whether it should manage it’s own instance of Zookeeper or not.

export HBASE_MANAGES_ZK=false

As an example, to have HBase manage a ZooKeeper quorum on nodes rs{1,2,3,4,5}.example.com, bound to port 2222 (the default is 2181), use:

作为一个例子， HBase管理在节点 rs[1,2,3,4,5].example.com 上的 ZooKeeper quorum ，绑定 端口 2222 （缺省是2181）：

${HBASE_HOME}/conf/hbase-env.sh:

…

# Tell HBase whether it should manage it’s own instance of Zookeeper or not.

export HBASE_MANAGES_ZK=true

${HBASE_HOME}/conf/hbase-site.xml:

<configuration>

…

<property>

<name>hbase.zookeeper.property.clientPort</name>

<value>2222</value>

<description>Property from ZooKeeper’s config zoo.cfg.

The port at which the clients will connect.

</description>

</property>

…

<property>

<name>hbase.zookeeper.quorum</name>

<value>rs1.example.com,rs2.example.com,rs3.example.com,rs4.example.com,rs5.example.com</value>

<description>Comma separated list of servers in the ZooKeeper Quorum.

For example, “host1.mydomain.com,host2.mydomain.com,host3.mydomain.com“.

By default this is set to localhost for local and pseudo-distributed modes

of operation. For a fully-distributed setup, this should be set to a full

list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh

this is the list of servers which we will start/stop ZooKeeper on.

</description>

</property>

…

</configuration>

When HBase manages ZooKeeper, it will start/stop the ZooKeeper servers as a part of the regular start/stop scripts. If you would like to run it yourself, you can do:

当 HBase来管理Zookeeper，它会将启动/关闭Zookeeper servers作为正常启动/关闭脚本的一部分。如果你想自己来运行，可以：

${HBASE_HOME}/bin/hbase-daemons.sh {start,stop} zookeeper

Note that you can use HBase in this manner to spin up a ZooKeeper cluster, unrelated to HBase. Just make sure to set HBASE_MANAGES_ZK to false if you want it to stay up so that when HBase shuts down it doesn’t take ZooKeeper with it.

For more information about setting up a ZooKeeper cluster on your own, see the ZooKeeper Getting Started Guide. HBase currently uses ZooKeeper version 3.2.0, so any cluster setup with a 3.x.x version of ZooKeeper should work.

要得到更多的 ZooKeeper 集群的配置信息，请看 Getting Started Guide 。 HBase 当前使用 3.2.0 的版本，因此 3.x.x 版本的资料可以参考。

Of note, if you have made HDFS client configuration on your Hadoop cluster, HBase will not see this configuration unless you do one of the following:

注意，如果你 在 Hadoop集群中进行 HDFS client 的配置，HBase无法看到这些配置，除非你做下面的任何一项工作：

• Add a pointer to your HADOOP_CONF_DIR to CLASSPATH in hbase-env.sh.
• 在 hbase-env.sh 中增加一个路径指向 HADOOP_CONF_DIR
• Add a copy of hdfs-site.xml (or hadoop-site.xml) to ${HBASE_HOME}/conf, or
• 增加一个 hdfs-site.xml 或者 hadoop-site.xml 的拷贝到 ${HBASE_HOME}/conf 目录下，或者
• if only a small set of HDFS client configurations, add them to hbase-site.xml.
• 如果 HBase的有关配置很少，增加配置内容到 hbase-site.xml 中。

An example of such an HDFS client configuration is dfs.replication. If for example, you want to run with a replication factor of 5, hbase will create files with the default of 3 unless you do the above to make the configuration available to HBase.

例如，需要调整 dfs 的复制因子。 dfs.replication ,hbase缺省是 3 ,你想修改的话就在 HBase的配置文件中修改。

Running and Confirming Your Installation

If you are running in standalone, non-distributed mode, HBase by default uses the local filesystem.

非分布式，将使用本地文件系统

If you are running a distributed cluster you will need to start the Hadoop DFS daemons and ZooKeeper Quorum before starting HBase and stop the daemons after HBase has shut down.

分布式需要 先启动/后关闭 HDFS daemons 和 ZooKeeper Quorum 。

Start and stop the Hadoop DFS daemons by running ${HADOOP_HOME}/bin/start-dfs.sh. You can ensure it started properly by testing the put and get of files into the Hadoop filesystem. HBase does not normally use the mapreduce daemons. These do not need to be started.

启动 HDFS。

关闭 HDFS 之前必须先确保 HBase已经关闭成功。

Start up your ZooKeeper cluster.

启动 ZooKeeper 集群。

Start HBase with the following command:

使用下面的命令：

${HBASE_HOME}/bin/start-hbase.sh

Once HBase has started, enter ${HBASE_HOME}/bin/hbase shell to obtain a shell against HBase from which you can execute commands. Type ‘help’ at the shells’ prompt to get a list of commands. Test your running install by creating tables, inserting content, viewing content, and then dropping your tables. For example:

使用 HBase shell 来测试。

hbase> # Type “help” to see shell help screen

hbase> help

hbase> # To create a table named “mylittletable” with a column family of “mylittlecolumnfamily”, type

hbase> create “mylittletable”, “mylittlecolumnfamily”

hbase> # To see the schema for you just created “mylittletable” table and its single “mylittlecolumnfamily”, type

hbase> describe “mylittletable”

hbase> # To add a row whose id is “myrow”, to the column “mylittlecolumnfamily:x” with a value of ‘v’, do

hbase> put “mylittletable”, “myrow”, “mylittlecolumnfamily:x”, “v”

hbase> # To get the cell just added, do

hbase> get “mylittletable”, “myrow”

hbase> # To scan you new table, do

hbase> scan “mylittletable”

To stop HBase, exit the HBase shell and enter:

下面的命令关闭 HBase .

${HBASE_HOME}/bin/stop-hbase.sh

If you are running a distributed operation, be sure to wait until HBase has shut down completely before stopping the Hadoop daemons.

The default location for logs is ${HBASE_HOME}/logs.

HBase also puts up a UI listing vital attributes. By default its deployed on the master host at port 60010 (HBase RegionServers listen on port 60020 by default and put up an informational http server at 60030).

端口 : 60010(master) 60020(region servers)

介绍

HBase 的数据模型是继 Bigtable 数据模型的之后的克隆版，特别适用于密集的数据系统。由于关系型数据库的架构是基于单台机器的，无法获得高的伸缩性。例如，连接两个表的JOIN操作，在内存中完成，这就无法跨越网络。关系分布式数据库的公司，需要做很多重新设计的工作，这就是他们的许可证这个贵的原因。其他的可选方法是复制，而当从机写超载时，最后的选择是分表。在这个时候，数据的正则化是你在类中唯一记住要看的，这也是为什么数据模型要在纸上写得明明白白，还不是仅仅给你看的。

概览

看简单点，HBase可以概括成一个 Map<byte[], Map<byte[], Map<byte[], Map<Long, byte[]>>>。第一个Map是映射从 row keys 到 column families。第二个Map是映射从 column families 到他们的 column keys。第三个Map是映射从 column keys 到他们的 timestamps 。 最后，最后的Map映射 timestamps 到一个单一的值。keys 一般为 字符串 strings， timestamps 是一个长整型 longs。而值则为一个不解释的字节数组。 列的 keys 总在其 families 后面，表现如：family:key。因为一个 family 映射到另一个其他的map，这在理论上允许一个 family 包含无限个 column keys。因此，为了获取一个值，用户需要使用三个 keys 来 get ：

row key+column key+timestamp -> value

行键+列键+时间戳 -> 值

行

HBase 以数组格式来处理 row key，但 row key 本身是有字符串的形式表现。row key Map 一个特性就是以一个词典顺序来保存。例如，从1到100的数字，就是按照 1,10,100,11,12,13,14,15,16,17,18,19,2,20,21,…,9,91,92,93,94,95,96,97,98,99 这样的方式来保存。

要想以自然顺序来保存整型数，row keys 必须在左边以0填充。利用这一点，row key Map 的功能可以通过提供一个 scaner 来增强， scaner 带有一个 start row key 和 一个 stop row key。例如，如果 row keys 是日期格式 YYYYMMDD，获取 2008年7月整个月的内容，就是打开一个 scaner （20080700到20080800）。它并不关心指定的 row keys 存在与否，唯一要关心的，就是这个调用不会返回 stop row key，因此，stop row key 必须给 scaner 指定好。

列簇

在 Hbase中，列成员重组具有同一性质的数据，并不限制数据类型。簇是表模式的一部分，为每行保存同种数据。与 froms rows to forws 不同的是 column keys 可以是稀少的。例如，row “20080702” 可以拥有自己的“info:”成员，该成员下有如下几个 column keys：
info:aaa
info:bbb
info:ccc

同样，row “20080703”仅有：
info:12342
在使用 column keys 的时候，开发者必须要非常小心。因为长度为0 的 key是允许的，这说明，在前面的例子中，数据可以被插入到 column key “info:” 中。我们强烈推荐，仅仅在没有其他的 keys 指定时使用空的 key。同样，由于一个成员中的数据是同一种类的，参考性能与时间戳 ，很多属性可以指定。

时间戳

根据成员的配置情况，HBase的值可以是以多版本的方式保存。缺省情况下，HBase将每个新值的时间戳设置为当前时间 milliseconds，并且当一个 cell 被请求时，返回最新的版本。开发者可以在插入数据时自定义时间戳，然后再通过指定这个时间戳来重新获取该值。

Family Attributes成员属性

可以为每一个簇指定下面的属性：

已实现的

*压缩
o record：准确到由rowkey+columnkey+timestamp 获得的每一个值都会在单独压缩。

o block：HDFS中的 block 被压缩。一个block可能包含多条record（block size > record size），或者一条记录的一部分（block size < record size）。
*时间戳
o 最大编号：一个值拥有不同版本的最大编号。
o 生存期： 比指定版本旧的内容将被当做垃圾被收集。

* 块缓存：在HDFS中，通过LRU排序算法获得缓存块。当等待全内存的存储时，改良随机读的性能，是一个很好的特性。

还没有实现的

* 内存保存：成员的所有值都保存在内存中。
* 长度：写入的值的长度不能超过指定的字节数。参考 [WWW] HBASE-742

实例

一个好的例子，演示Hbase的数据模型，就是 一个 blog。因为它具有简单和范围小的特点。

* blog条目，由标题，副标题，日期，作者，类型（或者标签），文本和评论组成，可以被登录后的用户进行创建和更新。
* 用户，由用户名，密码，和一个名称，可以登录或者登出。
* 评论，由标题，作者，文本组成，可以被游客匿名写入。

原ERD

HBase 目标模式

优先的解决方案 :

blogtable 的 row key 是连续的，由type(两个缩写字母)和时间戳组成。按照这样的方式，行首先是根据 type 来集成，然后时间日期。这表示能有更多的机会在同一个 region 内获得所需的数据。如你所见，在BLOGENTITY和COMMENT之间的1:*的关系，将 comments 的每一个属性放在 blogentery 中，作为的一个 family 。用日期作为column key，所有的 comments 已经被排序过。

这样设计的好处就是当你显示blog的“首页”，你仅仅需要获取从 blogtable 中 family “info:” 。当你展示一个实际的blog条目，你可以获取整行。另外一个好处就是用时间戳作为 row key，需要显示时，scanner可以获取连续的 rows ，例如， 最近你一个月的 条目。