diff --git a/docs/feature/ecm-takes-over-ec.md b/docs/feature/ecm-takes-over-ec.md new file mode 100644 index 00000000000..d2714e94882 --- /dev/null +++ b/docs/feature/ecm-takes-over-ec.md @@ -0,0 +1,12 @@ +--- +title: ECM takes over EC +sidebar_position: 0.2 +--- + +## 1. Requirement Background +When the ECM restarts, can choose not to kill the engine, and it can take over the existing surviving engine. +Make engine conn manager(ECM) service stateless . + + +## 2. Instructions for use +this feature is enabled by default \ No newline at end of file diff --git a/docs/feature/hive-engine-support-concurrent.md b/docs/feature/hive-engine-support-concurrent.md new file mode 100644 index 00000000000..53b0e7c0e3d --- /dev/null +++ b/docs/feature/hive-engine-support-concurrent.md @@ -0,0 +1,21 @@ +--- +title: hive engine support concurrent +sidebar_position: 0.2 +--- + +## 1. Requirement Background +hiveEngineConn supports concurrency, reducing the resource consumption of starting hive engine. + +## 2. Instructions for use +First, modify linkis-engineconn.properties file in linkis-engineconn-plugins/hive/src/main/resources directory, +and set linkis.hive.engineconn.concurrent.support to true. +``` +# support parallelism execution +linkis.hive.engineconn.concurrent.support=true +``` + +Second, submit a hive task ,when first task finished ,submit another task. Your could see hive engine has been reused. + + +## 3. Precautions +1, Submit second hive task when first task has been finished. diff --git a/docs/feature/storage-add-support-oss.md b/docs/feature/storage-add-support-oss.md new file mode 100644 index 00000000000..83e3c8912e6 --- /dev/null +++ b/docs/feature/storage-add-support-oss.md @@ -0,0 +1,36 @@ +--- +title: Extend linkis-storage add support OSS filesystem +sidebar_position: 0.2 +--- + +## 1. Requirement Background +Extend linkis-storage add support OSS filesystem + +## 2. Instructions for use +To store log and resultSet in OSS, add the following configs in conf/linkis-cg-entrance.properties. +``` +#eg: +wds.linkis.entrance.config.log.path=oss://linkis/tmp/ +wds.linkis.resultSet.store.path=oss://linkis/tmp/ +wds.linkis.filesystem.hdfs.root.path=oss://taihao-linkis/tmp/ +wds.linkis.fs.oss.endpoint=https://oss-cn-hangzhou.aliyuncs.com +wds.linkis.fs.oss.bucket.name=linkis +wds.linkis.fs.oss.accessKeyId=your accessKeyId +wds.linkis.fs.oss.accessKeySecret=your accessKeySecret +``` + +Add the following configs in engine engineconn plugins conf. Let me use hive conf for example: +modify linkis-engineconn-plugins/hive/src/main/resources/linkis-engineconn.properties and +add the following configs. +``` +#eg: +wds.linkis.fs.oss.endpoint=https://oss-cn-hangzhou.aliyuncs.com +wds.linkis.fs.oss.bucket.name=linkis +wds.linkis.fs.oss.accessKeyId=your accessKeyId +wds.linkis.fs.oss.accessKeySecret=your accessKeySecret +``` + + +## 3. Precautions +1, you have an OSS bucket. +2, you have accessKeyId, accessKeySecret to access the above OSS bucket. \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/feature/ecm-takes-over-ec.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/feature/ecm-takes-over-ec.md new file mode 100644 index 00000000000..8ee92a55071 --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/feature/ecm-takes-over-ec.md @@ -0,0 +1,10 @@ +--- +title: 当ECM重新启动时，可以选择不杀死引擎，而是可以接管现有的存活引擎 +sidebar_position: 0.2 +--- + +## 需求背景 +当ECM重新启动时，可以选择不杀死引擎，而是可以接管现有的存活引擎。使引擎连接管理器(ECM)服务无状态。 + +## 使用说明 +此功能默认已启用。 \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/feature/hive-engine-support-concurrent.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/feature/hive-engine-support-concurrent.md new file mode 100644 index 00000000000..086a2187df0 --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/feature/hive-engine-support-concurrent.md @@ -0,0 +1,20 @@ +--- +title: hive engine支持并发，支持复用 +sidebar_position: 0.2 +--- + +## 1. 需求背景 +hiveEngineConn支持并发，减少启动hive引擎的资源消耗。 + +## 2. 使用说明 +首先，在linkis-engineconn-plugins/hive/src/main/resources目录下修改linkis-engineconn.properties文件， +并将linkis.hive.engineconn.concurrent.support设置为true。 +``` +# 支持并行执行 +linkis.hive.engineconn.concurrent.support=true +``` + +第二，提交一个hive任务，当第一个任务完成后，再提交另一个任务。您可以看到hive引擎已被重用。 + +## 3. 注意事项 +1、等第一个hive任务执行成功后，再提交第二个hive任务。 \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/feature/storage-add-support-oss.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/feature/storage-add-support-oss.md new file mode 100644 index 00000000000..a279521c98c --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/feature/storage-add-support-oss.md @@ -0,0 +1,32 @@ +--- +title: 扩展linkis-storage以支持OSS文件系统 +sidebar_position: 0.2 +--- + +## 1. 需求背景 +扩展linkis-storage以支持OSS文件系统。 + +## 2. 使用说明 +为了在OSS中存储日志和resultSet，请在conf/linkis-cg-entrance.properties中添加以下配置。示例： +``` +wds.linkis.entrance.config.log.path=oss://linkis/tmp/ +wds.linkis.resultSet.store.path=oss://linkis/tmp/ +wds.linkis.filesystem.hdfs.root.path=oss://taihao-linkis/tmp/ +wds.linkis.fs.oss.endpoint=https://oss-cn-hangzhou.aliyuncs.com +wds.linkis.fs.oss.bucket.name=linkis +wds.linkis.fs.oss.accessKeyId=your accessKeyId +wds.linkis.fs.oss.accessKeySecret=your accessKeySecret +``` + +在engine engineconn插件conf中添加以下配置。以hive conf为例：修改linkis-engineconn-plugins/hive/src/main/resources/linkis-engineconn.properties， +并添加以下配置。示例： +``` +wds.linkis.fs.oss.endpoint=https://oss-cn-hangzhou.aliyuncs.com +wds.linkis.fs.oss.bucket.name=linkis +wds.linkis.fs.oss.accessKeyId=your accessKeyId +wds.linkis.fs.oss.accessKeySecret=your accessKeySecret +``` + +## 3. 注意事项 +1、您需要拥有一个OSS存储桶。 +2、您需要accessKeyId和accessKeySecret以访问上述OSS存储桶。 \ No newline at end of file