Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev 1.1.13 webank streamis flink log4j #5011

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
299 commits
Select commit Hold shift + click to select a range
0391c86
Merge branch 'dev-1.4.0-part1-mdc-log' into dev-1.1.10-webank
peacewong Mar 28, 2023
5ca292c
Merge remote-tracking branch 'origin/dev-1.1.10-webank' into dev-1.1.…
peacewong Mar 28, 2023
219271f
optimize about submit empty code after code preprocessing (#113)
casionone Mar 29, 2023
f430358
Fix resultset (#115)
casionone Mar 29, 2023
6970d35
Dev 1.3.2 fix config cache bug (#117)
v-kkhuang Mar 29, 2023
167c212
Revert "support spark.external.default.jars (#101)"
casionone Mar 29, 2023
d4649ce
Dev 1.3.2 update tenant (#114)
v-kkhuang Mar 30, 2023
83579db
scala support jar (#119)
casionone Mar 30, 2023
69acd10
fix(common util):fix getStringToMap function bug about ECResourceInfo…
ahaoyao Apr 3, 2023
3a25e2b
Dev 1.3.2 fix stddev samp bug (#122)
v-kkhuang Apr 3, 2023
0521a21
revert DoubleType NaN
casionone Apr 3, 2023
041125e
fix spark resource log NPE
casionone Apr 3, 2023
c602c8d
Revert "【1.1.10】Fix the garbled code problem of spark function (#93)"
casionone Apr 4, 2023
9e5c0a9
Code optimization (#125)
v-kkhuang Apr 5, 2023
2a5d724
bug fix and feature optimize (#124)
mayinrain Apr 5, 2023
141babf
add 1.1.10 webank web (#127)
mayinrain Apr 6, 2023
07f17c5
feat:Add single test to ByteTimeUtils and fix code that triggers null…
ahaoyao Apr 11, 2023
c47f786
fix:Fix spelling errors in words (#129)
ahaoyao Apr 11, 2023
0d57c9b
feat:add ldap-userNameFormat example value (#128)
ahaoyao Apr 11, 2023
4b266bf
Revert "Fix custom variable settings (#107)"
casionone Apr 11, 2023
3bf1d8e
fix JobHistoryQueryServiceImpl cache bug (#133)
v-kkhuang Apr 13, 2023
5ce48b2
update spark datatype
v-kkhuang Apr 18, 2023
a488132
Compatible with trino version 407.
HmhWz Apr 20, 2023
5ff9f98
add dss gateway support parser. (#135)
HmhWz Apr 21, 2023
1344067
Supports the conversion of positive and negative byte numbers to gb
Apr 11, 2023
f89902f
Merge pull request #136 from WeDataSphere/dev-1.1.10-webank
casionone Apr 23, 2023
bbc7065
Optimization of token exception scenario description
Apr 13, 2023
cc354b3
Merge pull request #134 from WeDataSphere/dev-1.1.11-webank-token-des…
casionone Apr 23, 2023
6b168a4
Merge pull request #130 from WeDataSphere/dev-1.1.11-webank-support-p…
casionone Apr 23, 2023
462eadf
Basic Information Management Delete Interface Add Administrator Authe…
Apr 11, 2023
57f5446
Merge pull request #132 from WeDataSphere/dev-1.1.11-webank-delete-in…
casionone Apr 24, 2023
8634d6b
1. Add the eclist interface input parameter: queueName
v-kkhuang Apr 24, 2023
d0c3655
adjust sqoop ec
casionone Apr 24, 2023
a9f4cd4
add datasource dependency
casionone Apr 24, 2023
347892c
add missing import
casionone Apr 25, 2023
516ae46
fix spark garbled code bug
v-kkhuang Apr 25, 2023
b963e0a
fix spark garbled code bug
v-kkhuang Apr 25, 2023
2704776
Merge pull request #137 from WeDataSphere/dev-1.4.0-xc-sqoop
casionone Apr 25, 2023
4c5ff8b
add instanceName parameters
v-kkhuang Apr 25, 2023
f9b06fe
Code Formatting
v-kkhuang Apr 25, 2023
a2f118d
adjust dss gateway support
casionone Apr 25, 2023
035dc65
Code Formatting
v-kkhuang Apr 26, 2023
2a22230
add doubleToStringTest
v-kkhuang Apr 26, 2023
937ae5e
fix Fixed session issue
peacewong Apr 26, 2023
2e3ff39
update to ms
peacewong Apr 26, 2023
db5026f
format code
v-kkhuang Apr 26, 2023
1325cc0
format code
v-kkhuang Apr 26, 2023
bef7b5c
Merge pull request #139 from WeDataSphere/dev-1.1.11-ec-list-add-unlo…
casionone Apr 26, 2023
12b1b73
Merge pull request #138 from WeDataSphere/dev-1.1.11-fix-stddev-samp-bug
casionone Apr 26, 2023
ff982b2
[1.1.11] Optimize UDF function logic (#131)
v-kkhuang Apr 26, 2023
407a800
update get configuration logic (#140)
Apr 26, 2023
ab57098
fix class no found (#144)
casionone Apr 26, 2023
6ef7ae8
Merge branch 'dev-1.1.10-fix-fixedsession' into dev-1.1.11-webank
peacewong Apr 27, 2023
9e02cee
update test
peacewong Apr 27, 2023
658c85e
add log
peacewong Apr 28, 2023
e01e464
Merge branch 'dev-1.1.10-fix-fixedsession' into dev-1.1.11-webank
peacewong Apr 28, 2023
f3476ab
Parse to specific instance for flow/entrance/kill in DSSGatewayParser.
HmhWz Apr 28, 2023
0b8a8fd
version num to 1.1.11-wedatasphere
casionone May 3, 2023
474c3ff
fix:select exclude route label is empty (#147)
casionone May 3, 2023
886daa7
added askEngineHook (#143)
casionone May 5, 2023
76b20b5
1.1.11 webank bdpclient (#142)
casionone May 5, 2023
0257e34
Optimize jdbc driver
peacewong May 6, 2023
d9b1772
fix Fixed session issue
peacewong Apr 26, 2023
90a3556
update to ms
peacewong Apr 26, 2023
f80d82a
add log
peacewong Apr 28, 2023
25887d7
Merge remote-tracking branch 'origin/dev-1.1.11-webank' into dev-1.4.…
peacewong May 7, 2023
f31de8e
add license
peacewong May 8, 2023
e5221dc
Dev 1.1.11 ec list add unlock time (#148)
v-kkhuang May 8, 2023
642f341
update var name
peacewong May 9, 2023
2142e8a
Fix error Status call back
peacewong May 9, 2023
d7566aa
Dev 1.4.0 jdbc optimize (#149)
peacewong May 9, 2023
d2bffc2
fix eclist api:filter after group by (#151)
casionone May 9, 2023
b9c2fd8
Optimization of token exception scenario description (#152)
ahaoyao May 10, 2023
97a38bb
ec list add unlock time---code_view (#153)
v-kkhuang May 10, 2023
141a04c
optimize udf permissions (#150)
v-kkhuang May 10, 2023
4428f71
Fix duplicate error codes
ahaoyao May 11, 2023
e76abd1
remove duplicate dependency of disruptor
casionone May 12, 2023
59ff4cc
JDBC Optimize
peacewong May 15, 2023
8ad47b7
Fix build error
peacewong May 15, 2023
30abd1b
Dev 1.1.11 optimize udf permissions (#154)
v-kkhuang May 15, 2023
2cf371c
Code Format
peacewong May 16, 2023
b64c1a3
fix web bugs and optimization (#156)
mayinrain May 17, 2023
f4cda50
Dev 1.1.11 webank fixoom (#157)
peacewong May 17, 2023
9885996
Fix issue with annotation task hang at the end of Scala code
peacewong May 18, 2023
597d07a
add runScheduleIntercept
peacewong May 18, 2023
4919bfb
Fix storage error
peacewong May 18, 2023
71fdbe6
Fix build error
peacewong May 18, 2023
6fd06d3
Fix build error
peacewong May 19, 2023
0790658
Merge branch 'dev-1.4.0-jdbc-optimize' into dev-1.1.11-webank
peacewong May 22, 2023
7a744d1
Add creator level job limit
peacewong May 22, 2023
77f0614
remove CacheOutputExecuteResponse
peacewong May 23, 2023
5d2ce94
Fix build error
peacewong May 23, 2023
937f722
The dss prod label interface forwarding problem was fixed.
HmhWz May 23, 2023
c51249a
add scheduler conf
peacewong May 29, 2023
11ca3e6
feat: do not kill ec when ecm restart part-1 (#4184) (#4452)
CharlieYan24 Apr 25, 2023
418b806
fix: do not clean ec resource and service info when handler ecm close…
CharlieYan24 Apr 30, 2023
e1bfa07
update to 1.1.13-wedatasphere
casionone May 30, 2023
0876f74
add sql connection
peacewong May 30, 2023
aa19c42
Merge remote-tracking branch 'origin/dev-1.1.11-webank' into dev-1.1.…
peacewong May 30, 2023
acb79ab
add sql connection (#168)
casionone May 30, 2023
2052d08
Add TicketID to Node
peacewong Jun 6, 2023
ad2fa2a
Fixed ECM not cleaning up when starting EC exception
peacewong Jun 6, 2023
d618f58
move File to java dir
peacewong Jun 6, 2023
01d30c9
Because the information needs to be pushed to createEngine, there is …
peacewong Jun 6, 2023
3c4a098
Fix build error
peacewong Jun 6, 2023
4aaf412
Fix build error
peacewong Jun 7, 2023
01f51a2
Fix build error
peacewong Jun 7, 2023
e91658a
Merge branch 'dev-1.1.12-fix-scala' into dev-1.1.13-webank
peacewong Jun 7, 2023
dea9bd2
Merge branch 'dev-1.1.13-webank' into dev-1.1.13-webank-ecm-ha
peacewong Jun 7, 2023
f4f5748
Fix Integration Test
peacewong Jun 9, 2023
9d6d0d0
code optimize
peacewong Jun 10, 2023
1627579
Fix NPE bug #4621
peacewong Jun 13, 2023
c6f685d
Merge pull request #175 from WeDataSphere/dev-1.1.11-webank
casionone Jun 14, 2023
ba4e082
itsm json fix
v-kkhuang Jun 7, 2023
3b8322f
print real ip
ahaoyao Jun 1, 2023
8ab9742
Add exception logs when kill job interface
ahaoyao May 31, 2023
747bdcc
delete setStatus
ahaoyao Jun 11, 2023
f9537ab
properties values support the Chinese
Apr 23, 2023
a991760
Merge pull request #155 from WeDataSphere/dev-1.1.11-webank-error-cod…
casionone Jun 14, 2023
afc2999
update error code
v-kkhuang May 30, 2023
c53a066
Optimization of incorrect 500 scenario return results
ahaoyao Jun 6, 2023
e38de08
Modify comments
ahaoyao Jun 7, 2023
5779f11
remove author info
ahaoyao Jun 11, 2023
ff9d153
Implement custom DefaultErrorAttributes
ahaoyao Jun 14, 2023
e3d0496
add linkis_ps_udf_user_load index
v-kkhuang Jun 13, 2023
083e131
sync 1.4.0 code
v-kkhuang May 18, 2023
4bb615c
fix spark memory count error
v-kkhuang Jun 14, 2023
9f8c44e
Security Work Order Upgrade
ahaoyao May 26, 2023
9f6594b
udf all add description
v-kkhuang May 18, 2023
e43df35
Code optimization
v-kkhuang May 29, 2023
be19cfd
Code optimization
v-kkhuang Jun 12, 2023
9b73ea5
Code optimization
v-kkhuang Jun 15, 2023
50905fe
add timo out task scan
peacewong Jun 15, 2023
3b7843f
Merge branch 'dev-1.1.12-fix-scala' into dev-1.1.13-webank-ecm-ha
peacewong Jun 15, 2023
c817cfc
remove node scorer service
peacewong Jun 17, 2023
8b3acfb
Optimization of log issues, adding queue names
ahaoyao Jun 15, 2023
e934b9c
【1.1.13】fix queueresources sql bug (#166)
v-kkhuang Jun 18, 2023
de66da9
【1.1.13】interface fileinfo add totalline (#165)
v-kkhuang Jun 18, 2023
1f6caf4
【1.1.13】remove tenant for entrance (#163)
v-kkhuang Jun 18, 2023
424ee85
【1.1.13】Adding specified parameter validation when modifying configur…
v-kkhuang Jun 18, 2023
b49e25f
【1.1.13】 Operation and maintenance optimization: Service version info…
v-kkhuang Jun 18, 2023
8d3cb47
Service support merge ps-data-source-manager ps-metadataquery into ps…
casionone Jun 18, 2023
b584d0d
remove node scorer service invoke
peacewong Jun 18, 2023
07b9ad0
Merge branch 'dev-1.1.13-webank-ecm-ha' into dev-1.1.13-webank
peacewong Jun 19, 2023
35bdb39
FIX conflict
peacewong Jun 19, 2023
af30f6c
Fix build issue
peacewong Jun 19, 2023
0d93f08
add mapper
casionone Jun 19, 2023
3eb9041
feature add config template
casionone Jun 19, 2023
abba3d2
remove try and warn
peacewong Jun 21, 2023
77eba01
add test for TemplateConfigKey dao
casionone Jun 21, 2023
2f0ccd8
code optimize
casionone Jun 25, 2023
834d06d
fix bug and code optimize
casionone Jun 25, 2023
821535b
code optimize
casionone Jun 25, 2023
4260dca
queue resources and add parameter maxapps
v-kkhuang Jun 16, 2023
10c263d
fix oom & shell error optimize
casionone Jun 25, 2023
5c10dc9
Merge branch 'dev-1.1.13-webank-xc-conf' into dev-1.1.13-webank-xc
casionone Jun 25, 2023
b36d6ff
Merge remote-tracking branch 'wds/dev-1.1.13-webank' into dev-1.1.13-…
casionone Jun 25, 2023
71f2e7d
【1.1.13】queue resources and add parameter maxapps (#190)
v-kkhuang Jun 26, 2023
109da2e
Adapt data source script fallback
ahaoyao Jun 26, 2023
3264a22
【1.1.13】fix licensed check error (#192)
v-kkhuang Jun 28, 2023
f79b57a
spotless:apply
casionone Jun 28, 2023
a144e9d
add sys funtion script (#187)
ahaoyao Jul 2, 2023
828a6ba
A single executor engine type should not create multiple executors #4713
peacewong Jul 3, 2023
cca270e
Merge branch 'dev-1.1.13-webank-ecm-ha' into dev-1.1.13-webank
peacewong Jul 3, 2023
2a5836b
set default value to false
peacewong Jul 3, 2023
e0e816e
code optimization
v-kkhuang Jul 4, 2023
8f0a9ba
optimize shell error msg
casionone Jul 4, 2023
6390aa7
fix shell file path bug
v-kkhuang May 29, 2023
ea38692
Support configuration template (#195)
casionone Jul 4, 2023
6eb8112
Fix scala dont exit issue
peacewong Jul 4, 2023
f060aeb
openEngineLog check with permission
casionone Jul 5, 2023
89b85ff
flink-log4j
yangwenzea Jul 6, 2023
308ceb8
【1.1.13】 Operation and maintenance optimization: Service version info…
v-kkhuang Jul 7, 2023
3b12068
【1.1.13】Adding specified parameter validation when modifying configur…
v-kkhuang Jul 7, 2023
50bb11d
openEngineLog check with historyAdmin permission
casionone Jul 7, 2023
29c78f6
revert sql result deal with \n
casionone Jul 7, 2023
6dfd4b4
rollback
ahaoyao Jul 7, 2023
dbcc74c
code optimization
v-kkhuang Jul 6, 2023
147c710
code optimize after code review (#209)
v-kkhuang Jul 10, 2023
bf25933
code optimize after code review (#206)
casionone Jul 11, 2023
8e33da3
fix NPE for YarnResourceRequester (#212)
v-kkhuang Jul 11, 2023
282bb29
Optimization for linkis web and new features (#211)
mayinrain Jul 11, 2023
686ab57
fix bug get user special limit config setting
casionone Jul 11, 2023
a58a718
upgrade commons-fileupload (#214)
casionone Jul 12, 2023
9ab3670
fixed 'job is killed by connection lost'
casionone Jul 13, 2023
8b0f6b6
code opmitization
v-kkhuang Jul 13, 2023
5bfbb0a
fix eclist heartbeat_msg
casionone Jul 13, 2023
78d0eeb
optimize cli
peacewong Jul 14, 2023
cb6d871
add hive current feature
peacewong Jul 14, 2023
a557aae
code format
peacewong Jul 14, 2023
3c4989c
Dev 1.1.13 webank web (#217)
mayinrain Jul 16, 2023
8e1f6a8
[Bugfix] Task state changes should before engine state changes (#4775)
guoshupei Jul 15, 2023
c690905
add ex exit send task log
peacewong Jul 17, 2023
809d7b5
add appconn plugin
htyredc Jul 20, 2023
ba6c36d
Revert "add appconn plugin"
htyredc Jul 20, 2023
2e71e2a
set timeout (#4794)
guoshupei Jul 21, 2023
be782c0
update null logical
peacewong Jul 25, 2023
eef7e45
Merge remote-tracking branch 'origin/dev-1.1.13-webank' into dev-1.1.…
peacewong Jul 25, 2023
f33b659
Fix Build error
peacewong Jul 27, 2023
85f354c
remove useless method
peacewong Jul 27, 2023
75adb91
add ecm instance log
v-kkhuang Jul 6, 2023
00137b1
When requesting EC, print linkismanager instance information (#219)
v-kkhuang Jul 31, 2023
4164e7a
Optimization of token exception description (#220)
ahaoyao Jul 31, 2023
573a57d
EC start with app version parameter (#224)
v-kkhuang Jul 31, 2023
48694ee
Spark & Hive Engine Log Ooptimization (#221)
v-kkhuang Jul 31, 2023
225f230
hive ec supports concurrency and when ec exit push task failed log (…
peacewong Aug 1, 2023
6211612
support cross cluster label task (#222)
lemonjuicelove Aug 1, 2023
d52d6bc
upgrade to 1.1.14-wedatasphere
casionone Aug 1, 2023
9f909f9
【1.1.14】New User Configuration and Configuration Management for Linki…
v-kkhuang Aug 1, 2023
a3d546b
fix update rule mapper
lemonjuicelove Aug 1, 2023
4f31b71
fix line break bug
casionone Aug 1, 2023
1ef7a8d
code format
casionone Aug 1, 2023
895a607
fix cache log write bug (#229)
casionone Aug 1, 2023
4abc447
fix null value bug
peacewong Aug 2, 2023
c13f82f
fix resultset
ahaoyao Aug 2, 2023
602fa95
fix Duplicate resource display
ahaoyao Aug 2, 2023
469c44a
Fix npe
peacewong Aug 2, 2023
4bae00d
Merge branch 'dev-1.1.14-hive-cur-ec-error' into dev-1.1.14-webank
peacewong Aug 2, 2023
46bcc39
Merge remote-tracking branch 'origin/dev-1.1.14-webank' into dev-1.1.…
peacewong Aug 2, 2023
05efb18
merge dev-1.1.13-webank-merge-1.12-webank-flink into dev-1.1.14-weban…
Alexkun Aug 3, 2023
47643e4
1. linkis-engineplugin-hive - fix compile error
Alexkun Aug 3, 2023
d5f7462
1. linkis-engineplugin-hive - fix compile error
Alexkun Aug 3, 2023
b0710fc
edit public
yangwenzea Aug 3, 2023
91a65a3
fix ClassCastException Ljava.lang.Object; cannot be cast to scala.run…
casionone Aug 2, 2023
2d571c5
【1.1.14】fix user conf error (#237)
v-kkhuang Aug 3, 2023
45722b7
【1.1.14】update log (#238)
v-kkhuang Aug 3, 2023
6b8e74f
Dev 1.1.14 webank jayceyang (#239)
lemonjuicelove Aug 3, 2023
feb4c66
support ec startup params with templateid (#234)
casionone Aug 3, 2023
588fcba
Fix JDBC Fixed session issue
peacewong Aug 4, 2023
1580bd9
Merge branch 'dev-1.1.14-hive-cur-ec-error' into dev-1.1.14-webank
peacewong Aug 4, 2023
b9184b6
Merge remote-tracking branch 'origin/dev-1.1.14-webank' into dev-1.1.…
peacewong Aug 4, 2023
3fdcdc5
add template rpc response entity
casionone Aug 4, 2023
3a7e847
remove hooks
peacewong Aug 4, 2023
aeaefd0
Merge remote-tracking branch 'origin/dev-1.1.14-webank' into dev-1.1.…
peacewong Aug 4, 2023
37719c0
Fix queue not exit exception
peacewong Aug 4, 2023
a976c9d
fix Extra invisible characters
ahaoyao Aug 4, 2023
7163148
Optimize EC request linkismanager and print logs (#242)
v-kkhuang Aug 7, 2023
2a20bfb
Fix configuration management and user configuration bugs (#241)
v-kkhuang Aug 7, 2023
4e523dc
fix conf template id apply api NPE
casionone Aug 7, 2023
2ba8823
switch on
peacewong Aug 9, 2023
37becf1
【1.1.14】code view fix (#247)
v-kkhuang Aug 10, 2023
904baae
Remove cross cluster (#246)
lemonjuicelove Aug 10, 2023
700b6a8
【1.1.14】base conf data sql syc (#244)
v-kkhuang Aug 10, 2023
5222be9
optimize token desc
ahaoyao Aug 10, 2023
618c659
Fix default null value issue
peacewong Aug 14, 2023
06d455a
Fix default null value issue
peacewong Aug 14, 2023
7f49115
add configKey and userConfig page; support download all result set wh…
mayinrain Aug 15, 2023
3d05d79
add tryCatch to log and progress method
peacewong Aug 15, 2023
3e8611f
Merge remote-tracking branch 'origin/dev-1.1.14-webank' into dev-1.1.…
peacewong Aug 15, 2023
e16399e
Merge branch 'dev-1.1.14-webank' into dev-1.1.13-webank-streamis-flin…
Alexkun Aug 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion docs/errorcode/linkis-engineconn-plugin-core-errorcode.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
| module name(模块名) | error code(错误码) | describe(描述) |enumeration name(枚举)| Exception Class(类名)|
| -------- | -------- | ----- |-----|-----|
|linkis-engineconn-plugin-core |10001|Failed to createEngineConnLaunchRequest(创建 EngineConnLaunchRequest失败)|FAILED_CREATE_ELR|EngineconnCoreErrorCodeSummary|
|linkis-engineconn-plugin-core |10001|The engine plug-in material is abnormal, please check whether the material is uploaded successfully(引擎插件物料异常,请检查物料是否上传成功)|EN_PLUGIN_MATERIAL_SOURCE_EXCEPTION|EngineconnCoreErrorCodeSummary|
|linkis-engineconn-plugin-core |10001|The engine plugin material is abnormal, please check whether the material is uploaded successfully(引擎插件物料异常,请检查物料是否上传成功)|EN_PLUGIN_MATERIAL_SOURCE_EXCEPTION|EngineconnCoreErrorCodeSummary|
|linkis-engineconn-plugin-core |10001|EngineTypeLabel are requested(需要参数 EngineTypeLabel)|ETL_REQUESTED|EngineconnCoreErrorCodeSummary|
|linkis-engineconn-plugin-core |20000|Cannot instance EngineConnExecution(无法实例化 EngineConnExecution)|CANNOT_INSTANCE_ECE|EngineconnCoreErrorCodeSummary|
|linkis-engineconn-plugin-core |20000|Cannot find default ExecutorFactory(找不到默认的 ExecutorFactory)|CANNOT_DEFAULT_EF|EngineconnCoreErrorCodeSummary|
Expand Down
10 changes: 5 additions & 5 deletions docs/errorcode/python-errorcode.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
## python errorcode

| 模块名(服务名) | 错误码 | 描述 |enumeration name| Exception Class|
| -------- | -------- | ----- |-----|-----|
|python|41001| |PYTHON_EXECUTE_ERROR|LinkisPythonErrorCodeSummary|
|python|60003|Pyspark process has stopped, query failed!(Pyspark 进程已停止,查询失败!)|PYSPARK_PROCESSS_STOPPED|LinkisPythonErrorCodeSummary|
|python|400201|Invalid python session.(无效的 python 会话.)|INVALID_PYTHON_SESSION|LinkisPythonErrorCodeSummary|
| 模块名(服务名) | 错误码 | 描述 |enumeration name| Exception Class|
| -------- |--------| ----- |-----|-----|
|python| 60002 | |PYTHON_EXECUTE_ERROR|LinkisPythonErrorCodeSummary|
|python| 60003 |Pyspark process has stopped, query failed!(Pyspark 进程已停止,查询失败!)|PYSPARK_PROCESSS_STOPPED|LinkisPythonErrorCodeSummary|
|python| 400201 |Invalid python session.(无效的 python 会话.)|INVALID_PYTHON_SESSION|LinkisPythonErrorCodeSummary|
1 change: 1 addition & 0 deletions docs/info-1.3.2.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,5 @@
|------------------| ----- |----------------------------------------------------------------------|------| ------------------------------------------------------- |
| linkis-jobhistory | 新增 | wds.linkis.jobhistory.admin | hadoop |可以查看所有历史任务的用户 注意:wds.linkis.governance.station.admin 为管理用户(也具有可以查看所有历史任务的权限)|
| linkis | 新增 | wds.linkis.governance.station.admin.token | /具有管理员权限的特殊token|
| linkis | 新增 | linkis.configuration.remove.application.cache | IDE |清除该应用的配置缓存|
| cg-entrance | 新增 | linkis.entrance.auto.clean.dirty.data.enable | true |entrance重启调用ps-jobhistory接口是否开启,ture为开启,取值范围:true或false|
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,6 @@ private static long parseByteString(String str, ByteUnit unit) {
} else {
throw new NumberFormatException("Failed to parse byte string: " + str);
}
suffix = suffix.toLowerCase();
// Check for invalid suffixes
if (suffix != null && !byteSuffixes.containsKey(suffix)) {
throw new NumberFormatException("Invalid suffix: \"" + suffix + "\"");
Expand Down Expand Up @@ -297,6 +296,18 @@ public static long byteStringAsGb(String str) {
return parseByteString(str, ByteUnit.GiB);
}

/**
* Convert a passed byte string (e.g. -50b, -100k, or -250m) to gibibytes for internal use.
*
* <p>If no suffix is provided, the passed number is assumed to be in gibibytes.
*/
public static long negativeByteStringAsGb(String str) {
if (str.startsWith("-")) {
return Math.negateExact(parseByteString(str.substring(1), ByteUnit.GiB));
}
return parseByteString(str, ByteUnit.GiB);
}

/**
* Returns a byte array with the buffer's contents, trying to avoid copying the data if possible.
*/
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.linkis.common.utils;

import org.apache.linkis.common.io.FsPath;

import java.io.File;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class ResultSetUtils {

// Sort in ASC order by numx in the result set _numx.dolphin file name
public static Comparator<FsPath> getResultSetFileComparatorOrderByNameNum() {

Comparator<FsPath> comparator =
(o1, o2) -> {
// get the num of file name
String regx = "\\d+";

String[] res1 = o1.getPath().split(File.separator);
String fileName1 = res1[res1.length - 1];
Matcher matcher1 = Pattern.compile(regx).matcher(fileName1);
int num1 = matcher1.find() ? Integer.parseInt(matcher1.group()) : Integer.MAX_VALUE;

String[] res2 = o2.getPath().split(File.separator);
String fileName2 = res2[res2.length - 1];
Matcher matcher2 = Pattern.compile(regx).matcher(fileName2);
int num2 = matcher2.find() ? Integer.parseInt(matcher2.group()) : Integer.MAX_VALUE;

return num1 - num2;
};
return comparator;
}

public static void sortByNameNum(List<FsPath> fsPathList) {
Collections.sort(fsPathList, getResultSetFileComparatorOrderByNameNum());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,10 @@
import java.util.Iterator;
import java.util.Map;

import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;

Expand All @@ -49,6 +52,9 @@ public class VariableOperationUtils {
private static final String[] CYCLES =
new String[] {CYCLE_YEAR, CYCLE_MONTH, CYCLE_DAY, CYCLE_HOUR, CYCLE_MINUTE, CYCLE_SECOND};

private static final ObjectMapper mapper =
JsonMapper.builder().enable(DeserializationFeature.FAIL_ON_TRAILING_TOKENS).build();

/**
* yyyy-MM-dd HH:mm:ss
*
Expand Down Expand Up @@ -78,30 +84,44 @@ public static ZonedDateTime toZonedDateTime(Date date) {
* @param str
* @return
*/
@Deprecated
public static String replaces(ZonedDateTime dateTime, String str)
throws VariableOperationFailedException {
return replaces(dateTime, str, true);
try {
JsonNode rootNode = mapper.readTree(str);
if (rootNode.isArray() || rootNode.isObject()) {
replaceJson(dateTime, rootNode);
return rootNode.toString();
}
} catch (Exception e) {
return replace(dateTime, str);
}
return replace(dateTime, str);
}

/**
* json support variable operation
*
* @param codeType
* @param dateTime
* @param str
* @param format
* @return
*/
public static String replaces(ZonedDateTime dateTime, String str, boolean format)
public static String replaces(String codeType, ZonedDateTime dateTime, String str)
throws VariableOperationFailedException {
try {
JsonNode rootNode = JsonUtils.jackson().readTree(str);
if (rootNode.isArray() || rootNode.isObject()) {
replaceJson(dateTime, rootNode);
return rootNode.toString();
String languageType = CodeAndRunTypeUtils.getLanguageTypeByCodeType(codeType, "");
if (languageType.equals(CodeAndRunTypeUtils.LANGUAGE_TYPE_JSON())) {
try {
JsonNode rootNode = mapper.readTree(str);
if (rootNode.isArray() || rootNode.isObject()) {
replaceJson(dateTime, rootNode);
return rootNode.toString();
}
} catch (Exception e) {
return replace(dateTime, str);
}
} catch (Exception e) {
return replace(dateTime, str);
}

return replace(dateTime, str);
}

Expand Down Expand Up @@ -197,7 +217,7 @@ private static void replaceJson(ZonedDateTime dateTime, JsonNode object)
} else if (temp.isObject()) {
replaceJson(dateTime, temp);
} else {
arrayNode.insert(i, replace(dateTime, temp.toString()));
arrayNode.set(i, replace(dateTime, temp.toString()));
}
}
} else if (object.isObject()) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import org.apache.linkis.common.utils.{Logging, Utils}
import org.apache.commons.io.IOUtils
import org.apache.commons.lang3.StringUtils

import java.io.{File, FileInputStream, InputStream, IOException}
import java.io._
import java.util.Properties
import java.util.concurrent.TimeUnit
import java.util.concurrent.locks.ReentrantReadWriteLock
Expand Down Expand Up @@ -140,15 +140,20 @@ private[conf] object BDPConfiguration extends Logging {

private def initConfig(config: Properties, filePath: String) {
var inputStream: InputStream = null

var reader: InputStreamReader = null
var buff: BufferedReader = null
Utils.tryFinally {
Utils.tryCatch {
inputStream = new FileInputStream(filePath)
config.load(inputStream)
reader = new InputStreamReader(inputStream, "UTF-8")
buff = new BufferedReader(reader)
config.load(buff)
} { case e: IOException =>
logger.error("Can't load " + filePath, e)
}
} {
IOUtils.closeQuietly(buff)
IOUtils.closeQuietly(reader)
IOUtils.closeQuietly(inputStream)
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -143,9 +143,10 @@ object VariableUtils extends Logging {
}
initAllDateVars(run_date, nameAndType)
val codeOperation = parserVar(code, nameAndType)
parserDate(codeOperation, run_date)
parserDate(codeType, codeOperation, run_date)
}

@deprecated
private def parserDate(code: String, run_date: CustomDateType): String = {
if (Configuration.VARIABLE_OPERATION) {
val zonedDateTime: ZonedDateTime = VariableOperationUtils.toZonedDateTime(run_date.getDate)
Expand All @@ -155,6 +156,15 @@ object VariableUtils extends Logging {
}
}

private def parserDate(codeType: String, code: String, run_date: CustomDateType): String = {
if (Configuration.VARIABLE_OPERATION) {
val zonedDateTime: ZonedDateTime = VariableOperationUtils.toZonedDateTime(run_date.getDate)
VariableOperationUtils.replaces(codeType, zonedDateTime, code)
} else {
code
}
}

private def initAllDateVars(
run_date: CustomDateType,
nameAndType: mutable.Map[String, variable.VariableType]
Expand Down Expand Up @@ -337,7 +347,7 @@ object VariableUtils extends Logging {
*
* @param code
* :code
* @param codeType
* @param languageType
* :SQL,PYTHON
* @return
*/
Expand All @@ -346,27 +356,37 @@ object VariableUtils extends Logging {

var varString: String = null
var errString: String = null
var rightVarString: String = null

languageType match {
case CodeAndRunTypeUtils.LANGUAGE_TYPE_SQL =>
varString = """\s*--@set\s*.+\s*"""
rightVarString = """^\s*--@set\s*.+\s*"""
errString = """\s*--@.*"""
case CodeAndRunTypeUtils.LANGUAGE_TYPE_PYTHON | CodeAndRunTypeUtils.LANGUAGE_TYPE_SHELL =>
varString = """\s*#@set\s*.+\s*"""
rightVarString = """^\s*#@set\s*.+\s*"""
errString = """\s*#@"""
case CodeAndRunTypeUtils.LANGUAGE_TYPE_SCALA =>
varString = """\s*//@set\s*.+\s*"""
rightVarString = """^\s*//@set\s*.+\s*"""
errString = """\s*//@.+"""
case CodeAndRunTypeUtils.LANGUAGE_TYPE_JAVA =>
varString = """\s*!!@set\s*.+\s*"""
rightVarString = """^\s*!!@set\s*.+\s*"""
case _ =>
return nameAndValue
}

val customRegex = varString.r.unanchored
val customRightRegex = rightVarString.r.unanchored
val errRegex = errString.r.unanchored
code.split("\n").foreach { str =>
{

if (customRightRegex.unapplySeq(str).size < customRegex.unapplySeq(str).size) {
logger.warn(s"code:$str is wrong custom variable format!!!")
}
str match {
case customRegex() =>
val clearStr = if (str.endsWith(";")) str.substring(0, str.length - 1) else str
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,27 @@
* limitations under the License.
*/

package org.apache.linkis.cli.core.present;
package org.apache.linkis.common.conf;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

public class PresentModeImplTest {
/** BDPConfiguration Tester */
public class BDPConfigurationTest {

@Test
@DisplayName("enumTest")
public void enumTest() {
public void testGetOption() {

String stdoutName = PresentModeImpl.STDOUT.getName();
String textFileName = PresentModeImpl.TEXT_FILE.getName();
Assertions.assertEquals(
"properties支持中文",
BDPConfiguration.getOption(
CommonVars.apply("linkis.jobhistory.error.msg.tip", "properties支持中文"))
.get());

Assertions.assertEquals("STDOUT", stdoutName);
Assertions.assertEquals("TEXT_FILE", textFileName);
Assertions.assertEquals(
"properties支持中文(默认)",
BDPConfiguration.getOption(
CommonVars.apply("linkis.jobhistory.error.msg.tip1", "properties支持中文(默认)"))
.get());
}
}
Loading