本文属于机器翻译版本。若本译文内容与英语原文存在差异,则一律以英文原文为准。
使用 HBase 快照
HBase 使用内置的快照command-runner.jar
与 AWS CLI 或 AWS SDK for Java 结合使用的步骤以交互方式运行这些命令。有关其他类型的 HBase 备份的更多信息,请参阅 HBase 文档中的 HBase 备份
使用表创建快照
hbase snapshot create -n
snapshotName
-ttableName
从 AWS CLI 使用 command-runner.jar:
aws emr add-steps --cluster-id j-2AXXXXXXGAPLF \ --steps Name="HBase Shell Step",Jar="command-runner.jar",\ Args=[ "hbase", "snapshot", "create","-n","
snapshotName
","-t","tableName
"]
AWS SDK for Java
HadoopJarStepConfig hbaseSnapshotConf = new HadoopJarStepConfig() .withJar("command-runner.jar") .withArgs("hbase","snapshot","create","-n","snapshotName","-t","tableName");
如果您的快照名称不唯一,则创建操作将失败,返回代码为 -1
或 255
,但您可能不会看到一条错误消息,指出出现了什么问题。要使用同一快照名称,请将其删除,然后重新创建。
删除快照
hbase shell >> delete_snapshot '
snapshotName
'
查看快照信息
hbase snapshot info -snapshot
snapshotName
将 快照导出至 Amazon S3
如果在导出快照时未指定-mappers
值HBase 将使用任意计算来确定映射器的数量。此值可能非常大,具体取决于您的表大小,这将对导出过程中正在运行的作业产生负面影响。因此,我们建议您指定 -mappers
参数和/或 -bandwidth
参数(指定每秒使用的带宽 (MB)),以限制导出操作使用的集群资源。或者,您可以在低使用率期间运行导出快照操作。
hbase snapshot export -snapshot
snapshotName
\ -copy-to s3://bucketName
/folder
-mappers 2
command-runner.jar
从 使用 AWS CLI:
aws emr add-steps --cluster-id j-2AXXXXXXGAPLF \ --steps Name="HBase Shell Step",Jar="command-runner.jar",\ Args=[ "hbase", "snapshot", "export","-snapshot","
snapshotName
","-copy-to","s3://bucketName
/folder
","-mappers","2","-bandwidth","50"]
AWS SDK for Java:
HadoopJarStepConfig hbaseImportSnapshotConf = new HadoopJarStepConfig() .withJar("command-runner.jar") .withArgs("hbase","snapshot","export", "-snapshot","snapshotName","-copy-to", "s3://bucketName/folder", "-mappers","2","-bandwidth","50");
从 Amazon S3 导入快照
尽管这是导入,但此处使用的 HBase 选项仍为 export
。
sudo -u hbase hbase snapshot export \ -D hbase.rootdir=s3://
bucketName
/folder
\ -snapshotsnapshotName
\ -copy-to hdfs://masterPublicDNSName
:8020/user/hbase \ -mappers 2
command-runner.jar
从 使用 AWS CLI:
aws emr add-steps --cluster-id j-2AXXXXXXGAPLF \ --steps Name="HBase Shell Step",Jar="command-runner.jar", \ Args=["sudo","-u","hbase","hbase snapshot export","-snapshot","
snapshotName
", \ "-D","hbase.rootdir=s3://bucketName
/folder
", \ "-copy-to","hdfs://masterPublicDNSName
:8020/user/hbase","-mappers","2","-chmod","700"]
AWS SDK for Java:
HadoopJarStepConfig hbaseImportSnapshotConf = new HadoopJarStepConfig() .withJar("command-runner.jar") .withArgs("sudo","-u","hbase","hbase","snapshot","export", "-D","hbase.rootdir=s3://path/to/snapshot", "-snapshot","snapshotName","-copy-to", "hdfs://masterPublicDNSName:8020/user/hbase", "-mappers","2","-chuser","hbase");
从 HBase shell 内的快照还原表
hbase shell >> disable
tableName
>> restore_snapshotsnapshotName
>> enabletableName
HBase 当前不支持在 HBase shell 中找到的所有快照命令。例如,没有用于还原快照的 HBase 命令行选项,因此您必须在 Shell 中还原快照。这意味着
command-runner.jar
必须运行 Bash 命令。
由于此处使用的命令为 echo
,因此您的 Shell 命令可能仍将失败,即使 Amazon EMR 运行的命令返回退出代码 0
也是如此。如果您选择以步骤形式运行 shell 命令,请检查步骤日志。
echo 'disable
tableName
; \ restore_snapshotsnapshotName
; \ enabletableName
' | hbase shell
以下是使用 AWS CLI 的步骤。首先,创建以下snapshot.json
文件:
[ { "Name": "restore", "Args": ["bash", "-c", "echo $'disable \"
tableName
\"; restore_snapshot \"snapshotName
\"; enable \"tableName
\"' | hbase shell"], "Jar": "command-runner.jar", "ActionOnFailure": "CONTINUE", "Type": "CUSTOM_JAR" } ]
aws emr add-steps --cluster-id j-2AXXXXXXGAPLF \ --steps file://./snapshot.json
AWS SDK for Java:
HadoopJarStepConfig hbaseRestoreSnapshotConf = new HadoopJarStepConfig() .withJar("command-runner.jar") .withArgs("bash","-c","echo $'disable \"tableName\"; restore_snapshot \"snapshotName\"; enable \"snapshotName\"' | hbase shell");