diff --git a/docs/engine-usage/hbase.md b/docs/engine-usage/hbase.md new file mode 100644 index 00000000000..9284db1f478 --- /dev/null +++ b/docs/engine-usage/hbase.md @@ -0,0 +1,104 @@ +--- +title: HBase Engine +sidebar_position: 15 +--- + +# HBase engine usage documentation + +This article mainly introduces the installation, use and configuration of the `hbase` engine plugin in `Linkis`. + +## 1. Preliminary work + +### 1.1 Engine Verification + +It is strongly recommended that you check these environment variables for the executing user before executing `hbase` tasks. The specific way is + +``` +sudo su - ${username} +echo ${JAVA_HOME} +``` + +## 2. Engine plugin installation + +### 2.1 Engine plugin preparation [non-default engine](./overview.md) + +Method 2: Compile the engine plug-in separately (requires a `maven` environment) + +``` +# compile +cd ${linkis_code_dir}/linkis-engineconn-plugins/hbase/ +mvn clean install -DskipTests -Dhbase.profile=1.2|1.4|2.2|2.5 + +# hbase.profile 1.2|1.4|2.2|2.5, The corresponding HBase version is: 1.2.0, 1.4.3, 2.2.6, 2.5.3 +# default hbase.profile=2.5, Compile plug-ins for different HBase versions on demand +# The compiled engine plug-in package is located in the following directory +${linkis_code_dir}/linkis-engineconn-plugins/hbase/target/out/ +``` + +[EngineConnPlugin engine plugin installation](../deployment/install-engineconn.md) + +### 2.2 Upload and load engine plugins + +Upload the engine plug-in package in 2.1 to the engine directory of the server + +```bash +${LINKIS_HOME}/lib/linkis-engineplugins +``` +The directory structure after uploading is as follows +``` +linkis-engineconn-plugins/ +├── hbase +│   ├── dist +│   │   └── 2.5.3 +│   │   ├── conf +│   │   └── lib +│   └── plugin +│   └── 2.5.3 +``` +### 2.3 Engine refresh + +#### 2.3.1 Restart and refresh +Refresh the engine by restarting the `linkis-cg-linkismanager` service +```bash +cd ${LINKIS_HOME}/sbin +sh linkis-daemon.sh restart cg-linkismanager +``` + +### 2.3.2 Check if the engine is refreshed successfully +You can check whether the `last_update_time` of this table in the `linkis_engine_conn_plugin_bml_resources` in the database is the time when the refresh is triggered. + +```sql +#Login to the linkis database +select * from linkis_cg_engine_conn_plugin_bml_resources; +``` + + +## 3. Use of HBase engine + +The `HBase` engine of `Linkis` runs a Jruby environment in the background to execute instructions in hbase-shell, +and is fully compatible with hbase-shell instructions. + + +### 3.1 Submit tasks through `Linkis-cli` + +```shell + sh bin/linkis-cli -engineType hbase-2.5.3 -code "list" \ + -codeType shell -runtimeMap linkis.hbase.zookeeper.quorum=zk1,zk2,zk3 +``` + +More `Linkis-Cli` command parameter reference: [`Linkis-Cli` usage](../user-guide/linkiscli-manual.md) + +HBase engine supports connection parameters: + +| Configuration name | Remarks and default value | Is it necessary | +| --- | --- | --- | +| linkis.hbase.zookeeper.quorum | ZooKeeper address for connecting to HBase, e.g. zk1,zk2,zk3, default value: localhost | Required | +| linkis.hbase.zookeeper.property.clientPort | ZooKeeper port for connecting to HBase, default value: 2181 | Optional, usually the default value | +| linkis.zookeeper.znode.parent | ZooKeeper znode path for connecting to HBase, default value: /hbase | Optional, usually the default value | +| linkis.hbase.rootdir | HDFS root directory for the HBase cluster, default value: /hbase | Optional, usually the default value | +| linkis.hbase.security.authentication | Authentication method for connecting to the HBase cluster: simple or kerberos, default value: simple | Optional, depending on the cluster configuration | +| linkis.hbase.kerberos.principal | HBase Kerberos authentication principal, e.g. hbase@HADOOP.LINKIS.COM | Required for Kerberos authentication | +| linkis.hbase.keytab.file | Path to the keytab file required for HBase Kerberos authentication, e.g. /tmp/hbase.keytab | Required for Kerberos authentication | +| linkis.hbase.kerberos.proxy.user | Kerberos proxy user, ensure that the Kerberos authenticated user has the privilege to proxy as a regular user | Optional, can be omitted | +| linkis.hbase.regionserver.kerberos.principal | HBase RegionServer's Kerberos authentication principal, e.g. hbase/_HOST@HADOOP.LINKIS.COM (obtain from the cluster configuration file) | Required for Kerberos authentication | +| linkis.hbase.master.kerberos.principal | HBase Master's Kerberos authentication principal, e.g. hbase/_HOST@HADOOP.LINKIS.COM (obtain from the cluster configuration file) | Required for Kerberos authentication | diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/engine-usage/hbase.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/engine-usage/hbase.md new file mode 100644 index 00000000000..cba264831c3 --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/engine-usage/hbase.md @@ -0,0 +1,101 @@ +--- +title: HBase +sidebar_position: 15 +--- + +# HBase 引擎使用文档 + +本文主要介绍在 `Linkis` 中,`HBase` 引擎插件的安装、使用和配置。 + +## 1. 前置工作 + +### 1.1 引擎验证 + +强烈建议您在执行 `HBase` 任务之前,检查下执行用户的这些环境变量。具体方式是 +``` +sudo su - ${username} +echo ${JAVA_HOME} +``` + +## 2. 引擎插件安装 + +### 2.1 引擎插件准备(二选一)[非默认引擎](./overview.md) + +方式一:单独编译引擎插件(需要有 `maven` 环境) + +``` +# 编译 +cd ${linkis_code_dir}/linkis-engineconn-plugins/hbase/ +mvn clean install -DskipTests -Dhbase.profile=1.2|1.4|2.2|2.5 + +# hbase.profile 支持1.2|1.4|2.2|2.5,对应的HBase版本是1.2.0,1.4.3,2.2.6,2.5.3 +# 默认hbase.profile=2.5,按需编译不同HBase版本的插件 +# 编译出来的引擎插件包,位于如下目录中 +${linkis_code_dir}/linkis-engineconn-plugins/hbase/target/out/ +``` + +[EngineConnPlugin 引擎插件安装](../deployment/install-engineconn.md) + +### 2.2 引擎插件的上传和加载 + +将 2.1 中的引擎插件包上传到服务器的引擎目录下 +```bash +${LINKIS_HOME}/lib/linkis-engineplugins +``` +上传后目录结构如下所示 +``` +linkis-engineconn-plugins/ +├── hbase +│   ├── dist +│   │   └── 2.5.3 +│   │   ├── conf +│   │   └── lib +│   └── plugin +│   └── 2.5.3 +``` +### 2.3 引擎刷新 + +#### 2.3.1 重启刷新 +通过重启 `linkis-cg-linkismanager` 服务刷新引擎 +```bash +cd ${LINKIS_HOME}/sbin +sh linkis-daemon.sh restart cg-linkismanager +``` + +### 2.3.2 检查引擎是否刷新成功 +可以查看数据库中的 `linkis_engine_conn_plugin_bml_resources` 这张表的 `last_update_time` 是否为触发刷新的时间。 + +```sql +#登陆到linkis的数据库 +select * from linkis_cg_engine_conn_plugin_bml_resources; +``` + + +## 3. HBase引擎的使用 + +`Linkis` 的 `HBase` 引擎是通过后台运行一个Jruby环境来执行hbase-shell中的指令,完全兼容hbase-shell的指令。 + + +### 3.1 通过 `Linkis-cli` 提交任务 + +```shell + sh bin/linkis-cli -engineType hbase-2.5.3 -code "list" \ + -codeType shell -runtimeMap linkis.hbase.zookeeper.quorum=zk1,zk2,zk3 +``` + +更多 `Linkis-Cli` 命令参数参考: [`Linkis-Cli` 使用](../user-guide/linkiscli-manual.md) + +HBase引擎,支持的连接参数有: + +| 配置名称 | 备注及默认值信息 | 是否必须 | +|-----------------|----------------|----------------------------------------| +| linkis.hbase.zookeeper.quorum | 连接hbase的zk地址,如:zk1,zk2,zk3, 默认值:localhost | 必须 | +| linkis.hbase.zookeeper.property.clientPort | 连接hbase的zk端口,默认值:2181 | 非必须,一般就是默认值 | +| linkis.zookeeper.znode.parent | 连接HBase的zk znode path,默认值:/hbase | 非必须,一般就是默认值 | +| linkis.hbase.rootdir | HBase集群的hdfs根目录,默认值:/hbase | 非必须,一般就是默认值 | +| linkis.hbase.security.authentication | HBase集群连接的认证方式:simple|kerberos,默认值:simple | 非必须,视自己集群情况定 | +| linkis.hbase.kerberos.principal | HBase kerberos认证principal,如:hbase@HADOOP.LINKIS.COM | kerberos认证方式下必须 | +| linkis.hbase.keytab.file | HBase kerberos认证所需keytab文件地址,如:/tmp/hbase.keytab | kerberos认证方式下必须 | +| linkis.hbase.kerberos.proxy.user | kerberos代理用户,需保证kerberos认证用户有代理普通用户的权限 | 非必须,可以不用代理用户 | +| linkis.hbase.regionserver.kerberos.principal | 如:hbase/_HOST@HADOOP.LINKIS.COM,自行从集群配置文件中获取 | kerberos认证方式下必须 | +| linkis.hbase.master.kerberos.principal | 如:hbase/_HOST@HADOOP.LINKIS.COM,自行从集群配置文件中获取 | kerberos认证方式下必须 |