Skip to content

Commit

Permalink
fix: add 1.4.0 feature document apache#712
Browse files Browse the repository at this point in the history
  • Loading branch information
CharlieYan24 committed Jun 5, 2023
1 parent 990c7fc commit f69cbd2
Show file tree
Hide file tree
Showing 6 changed files with 131 additions and 0 deletions.
12 changes: 12 additions & 0 deletions docs/feature/ecm-takes-over-ec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
title: ECM takes over EC
sidebar_position: 0.2
---

## 1. Requirement Background
When the ECM restarts, can choose not to kill the engine, and it can take over the existing surviving engine.
Make engine conn manager(ECM) service stateless .


## 2. Instructions for use
this feature is enabled by default
21 changes: 21 additions & 0 deletions docs/feature/hive-engine-support-concurrent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
title: hive engine support concurrent
sidebar_position: 0.2
---

## 1. Requirement Background
hiveEngineConn supports concurrency, reducing the resource consumption of starting hive engine.

## 2. Instructions for use
First, modify linkis-engineconn.properties file in linkis-engineconn-plugins/hive/src/main/resources directory,
and set linkis.hive.engineconn.concurrent.support to true.
```
# support parallelism execution
linkis.hive.engineconn.concurrent.support=true
```

Second, submit a hive task ,when first task finished ,submit another task. Your could see hive engine has been reused.


## 3. Precautions
1, Submit second hive task when first task has been finished.
36 changes: 36 additions & 0 deletions docs/feature/storage-add-support-oss.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
title: Extend linkis-storage add support OSS filesystem
sidebar_position: 0.2
---

## 1. Requirement Background
Extend linkis-storage add support OSS filesystem

## 2. Instructions for use
To store log and resultSet in OSS, add the following configs in conf/linkis-cg-entrance.properties.
```
#eg:
wds.linkis.entrance.config.log.path=oss://linkis/tmp/
wds.linkis.resultSet.store.path=oss://linkis/tmp/
wds.linkis.filesystem.hdfs.root.path=oss://taihao-linkis/tmp/
wds.linkis.fs.oss.endpoint=https://oss-cn-hangzhou.aliyuncs.com
wds.linkis.fs.oss.bucket.name=linkis
wds.linkis.fs.oss.accessKeyId=your accessKeyId
wds.linkis.fs.oss.accessKeySecret=your accessKeySecret
```

Add the following configs in engine engineconn plugins conf. Let me use hive conf for example:
modify linkis-engineconn-plugins/hive/src/main/resources/linkis-engineconn.properties and
add the following configs.
```
#eg:
wds.linkis.fs.oss.endpoint=https://oss-cn-hangzhou.aliyuncs.com
wds.linkis.fs.oss.bucket.name=linkis
wds.linkis.fs.oss.accessKeyId=your accessKeyId
wds.linkis.fs.oss.accessKeySecret=your accessKeySecret
```


## 3. Precautions
1, you have an OSS bucket.
2, you have accessKeyId, accessKeySecret to access the above OSS bucket.
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: 当ECM重新启动时,可以选择不杀死引擎,而是可以接管现有的存活引擎
sidebar_position: 0.2
---

## 需求背景
当ECM重新启动时,可以选择不杀死引擎,而是可以接管现有的存活引擎。使引擎连接管理器(ECM)服务无状态。

## 使用说明
此功能默认已启用。
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: hive engine支持并发,支持复用
sidebar_position: 0.2
---

## 1. 需求背景
hiveEngineConn支持并发,减少启动hive引擎的资源消耗。

## 2. 使用说明
首先,在linkis-engineconn-plugins/hive/src/main/resources目录下修改linkis-engineconn.properties文件,
并将linkis.hive.engineconn.concurrent.support设置为true。
```
# 支持并行执行
linkis.hive.engineconn.concurrent.support=true
```

第二,提交一个hive任务,当第一个任务完成后,再提交另一个任务。您可以看到hive引擎已被重用。

## 3. 注意事项
1、等第一个hive任务执行成功后,再提交第二个hive任务。
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
title: 扩展linkis-storage以支持OSS文件系统
sidebar_position: 0.2
---

## 1. 需求背景
扩展linkis-storage以支持OSS文件系统。

## 2. 使用说明
为了在OSS中存储日志和resultSet,请在conf/linkis-cg-entrance.properties中添加以下配置。示例:
```
wds.linkis.entrance.config.log.path=oss://linkis/tmp/
wds.linkis.resultSet.store.path=oss://linkis/tmp/
wds.linkis.filesystem.hdfs.root.path=oss://taihao-linkis/tmp/
wds.linkis.fs.oss.endpoint=https://oss-cn-hangzhou.aliyuncs.com
wds.linkis.fs.oss.bucket.name=linkis
wds.linkis.fs.oss.accessKeyId=your accessKeyId
wds.linkis.fs.oss.accessKeySecret=your accessKeySecret
```

在engine engineconn插件conf中添加以下配置。以hive conf为例:修改linkis-engineconn-plugins/hive/src/main/resources/linkis-engineconn.properties,
并添加以下配置。示例:
```
wds.linkis.fs.oss.endpoint=https://oss-cn-hangzhou.aliyuncs.com
wds.linkis.fs.oss.bucket.name=linkis
wds.linkis.fs.oss.accessKeyId=your accessKeyId
wds.linkis.fs.oss.accessKeySecret=your accessKeySecret
```

## 3. 注意事项
1、您需要拥有一个OSS存储桶。
2、您需要accessKeyId和accessKeySecret以访问上述OSS存储桶。

0 comments on commit f69cbd2

Please sign in to comment.