Skip to content

Commit

Permalink
QA 1.16.0 (#273)
Browse files Browse the repository at this point in the history 
* ATLP-446
added kerberos in image
added validation check for krb user

* New version: 1.14.0

* ATLP-446
fixed user checking

* ATLP-446
add ability to append realms to krb5.conf

* ATLP-446. Fixed notes

* ATLP-446. Moved kinit to jail

* enable kerberos for chroot'ed env

* enable kerberos for chroot'ed env

* ATLP-446. Fixed notes

* ARACHNE-2359. Back-merge of multi-statement SQL files execution fix

* ATLP-446. Made kinit in chroot mode

* ATLP-446. Changed krb auth enum class

* ATLP-446. added spaces to template

* ATLP-446. removed extra lines

* ATLP-446. turned off cleanup environment for debug

* ATLP-446. reverted debug notes back

* ATLP-446. set krb admin server properly

* ATLP-446. Fixed notes

* ATLP-446. Removed unnecessary dependency

* ATLP-446. Added condition for kerberos init, fixed notes

* Refactoring

* ATLP-446. made some refactoring

* ATLP-446. made some refactoring

* ATLP-446. Added [domain-realm] section for correct login to impala (#145)

* ATLP-446. Fixed cp command in jail.sh (#147)

* ATLP-446. Fixed execution of kinit on host (#150)

* ATLP-446. Extended krb5.conf both for host and for jail areas (#151)

* ATLP-524. Replaced kerberos login logic with module from ArachneCommons (#159)

* Added login to kerberos by password (#160)

* ARACHNE-2555. Added connection pooling for DN health check (#154)

(cherry picked from commit 2bceaad)

* ARACHNE-2556 EE: set default timezone in docker container to UTC (#155)

(cherry picked from commit 4de1a00)

* Fixed dockerImagePrefix

* Switched container timezone from UTC to GMT (#158)
(cherry picked from commit e4187dd)

* SqlRender version

(cherry picked from commit 10c9884)

* Reverted dockerImagePrefix

* ARACHNE-2575. Changed key used in DataSource-s cache to get a new connection when ds username/password changed (#169)

* Changed ojdbc6 and impala driver versions (#170)

* ARACHNE-2627. Added logging and moved metadata extraction into async mode [back_merge]  (#172)

* Fix R libraries versions (#178)

* Fix R libraries versions for Atlas 2.7 (#179)

* Adds Support of BQ keyfile json for authentication of EE jobs (#180)

* Updates DatabaseConnector version (#184)

* Upgrades R packages versions (#186)

* Adds Netezza support (#185)

* Adds cron scheduled job to refresh Kerberos ticket (#187)

* Added parameter to define whether execution results should be deleted after callback (#190)

* Prevents R packages updates (#191)

* Exposes data source name as Environment variable (#194)

* Excluded extra folders from analyses results (#196)

* added default exclusion for analysis

* Added R packages for new CC analysis. Adds ANALYSIS_ID env param (#195)

* added DATA_SOURCE_NAME to params (#201)

* Added synchronous mode of analysis execution (#202)

* Added synchronous mode of analysis execution

* Cleanup

* Cleanup

* Forgotten diff for sync execution mode (#204)

* Added synchronous mode of analysis execution

* Cleanup

* Cleanup

* Forgotten diff for sync mode

* Updated version of r-env. Removed unused docker files (#206) (#208) (#209)

* Updated version of r-env. Removed unused docker files (#206)

* rearranged libraries (#206)

(cherry picked from commit 1783b94)

* Back merge from 1.14 & version increment (#205)

* Build JAR that can be used as a dependency in other projects additionally to executable one (#211)

* Use R libs file from DockerEnv repo to avoid duplication (#210)

* Fixes build process conforming PLP deps (#216)

* Fixes docker build after classifier has been set (#217)

* Attempt to fix docker build

* Fixes docker build

* run apt-get update in non-interactive mode

* Remove redundant dependencies (#218)

* Adds geospatial libs (#219)

* Fixes user home dir (#222)

* improve swagger documentation (#220)

* Re-uses DataSourceAuthResolver-s from arachne-commons (#230)

* Fixes NPE during non-Kerberos auth (#231)

* Updates R-env image version (#232)

* Develop backmerge from qa (#237)

* change r-env version (#240)

* Upgrade to zip4j 2.1.1 (#242)

* Upgrade to zip4j 2.1.1

* Removed unused ZipException declarations

* Fixes removal of Keyfile from result zip (#246)

* Fixes R dist build. Fixes jail mode for BQ (#248)

* Back merge release 1.15.0 (#266)

* ARACHNE-2359. Back-merge of multi-statement SQL files execution fix (#138)

* ARACHNE-2359. Merge fix (#140)

* ATLP-446. Added keytab propagation for Impala (#137)

* ATLP-446. Added [domain-realm] section for correct login to impala (#144)

* ATLP-446. Fixed cp command in jail.sh (#146)

*  ATLP-446. Fixed kinit command for user-password mode (#148)

* ATLP-446. Fixed execution of kinit on host (#149)

* ATLP-446. Extended krb5.conf both for host and for jail areas (#152)

* ARACHNE-2556 EE: set default timezone in docker container to UTC (#155)

* ARACHNE-2555. Added connection pooling for DN health check (#154)

(cherry picked from commit 2bceaad)

* Fixed

* Fixed

* Fixed

* Switched container timezone from UTC to GMT (#158)

(cherry picked from commit e4187dd)

* SqlRender version

* Reverted dockerImagePrefix (#165)

* ARACHNE-2575. Changed key used in DataSource-s cache to get a new connection when ds username/password changed [back-merge] (#167)

* ojdbc version fix

* ARACHNE-2627. Added logging and moved metadata extraction into async mode (#171)

* fix DatabaseConnector & FeatureExtraction versions (#174)

* ARACHNE-2648 Datanode health check does not work as expected

* update version to 1.13.1-QA (#188)

* 1.14.0-QA (#197)

* Merge fix

* Merge fix #2

* Removed redundant metadata extraction (merge issue) (#199)

* Added DATA_SOURCE_NAME to chroot params (#201)
(cherry picked from commit d0c4814)

* Start R scripts in isolated PID namespace when chroot mode is enabled (#224)

* Start R scripts in isolated PID namespace when chroot mode is enabled

* Do not show env variables in ps

* Attempt to fix script

* fixes issue with non-root user

* removes redundant param

* Qa backmerge from master (#235)

* ARACHNE-1717 1.10.0

* ARACHNE-1854 Updated version

* ARACHNE-2005 Release 1.11.2

* ARACHNE-2106 Release 1.11.3

* ARACHNE-2357 Release 1.12.1

* ARACHNE-1668. Updated drivers to support Postgres 10

(cherry picked from commit 41fb866)

* Changed version 1.12.2-qa -> 1.12.2

* ARACHNE-2521. Fixed return of execution status

*  ARACHNE-2539. Fixed execution of SQL containing multiple queries per file in Oracle

* Release 1.12.3

* ARACHNE-2555. Added connection pooling for DN health check (#154)

* ARACHNE-2556 EE: set default timezone in docker container to UTC (#157)

* Switched container timezone from UTC to GMT (#158)

* Release 1.14.0 (#203)

* Updated version of r-env. Removed unused docker files (#206) (#208)

* Updated version of r-env. Removed unused docker files (#206)

* rearranged libraries (#206)

* change version to qa

* change version to qa (#236)

* change impala jdbc driver version (#238)

* change impala jdbc driver version

* change impala jdbc driver version

* fix readme

* fixes download link

* change r-env version (#241)

* QA 1.15.0 (#244)

* Fixes removal of Keyfile from result zip (#247)

* Fixes R dist build. Fixes jail mode for BQ (#249)

* Removes extra installs. Updates to reflect changes from R-env (#251)

* Adds CDM v5.3.1 detection (#253)

* ARACHNE-2722 set right version for sql-render (#261)

* Updates python to v3.6 making PLP models run succesfully (#257)

* Fixes issue with ojdbc and timezone region (#262)

* ARACHNE-2722 - Cohort (simple counts) fails on BigQuery (#264)

- use SingleStatementSqlExecutor for BigQuery request

* Optimize cdm detect (#267)

* optimization for CDM version detection
* Hello Spring boot 2!

* Fix logger configuration - failing after spring boot 2 migration (#268)

* Migrate deprecated properties to spring-boot-2 schema (#269)

* Migrate deprecated properties to spring-boot-2 schema

Fix tomcat 9.0.26 packed into spring boot 2.1.9 is have a bug, rolling back to 2.1.8 to stable tomcat 9.0.25
 - failing after spring boot 2 migration

* downgrade embedded tomcat to 9.0.24

* upgrade embedded tomcat to 9.0.27

* Adds support for Apache Hive LLAP (#272)

Fixes: OHDSI/WebAPI#1168 Adds support for Apache Hive LLAP

Co-authored-by: anastasiia klochkova <[email protected]>
Co-authored-by: Anton Gackovka <[email protected]>
Co-authored-by: Vitaly Koulakov <[email protected]>
Co-authored-by: Pavel Grafkin <[email protected]>
Co-authored-by: mpozhidaeva <[email protected]>
Co-authored-by: OdysseusBot <[email protected]>
Co-authored-by: ssuvorov-fls <[email protected]>
Co-authored-by: AntonStepanof <[email protected]>
Co-authored-by: Alex Cumarav <[email protected]>
  • Loading branch information
@aklochkova @agackovka
@wivern @pavgra @marpozh @OdysseusBot @ssuvorov-fls @AntonStepanof @acumarav
10 people authored Feb 11, 2020
1 parent f39def5 commit c99a0be
Show file tree
Hide file tree
Showing 31 changed files with 1,055 additions and 636 deletions.
253 changes: 123 additions & 130 deletions pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@

<artifactId>execution_engine</artifactId>
<groupId>com.odysseusinc.arachne</groupId>
<version>1.15.1-QA</version>
<version>1.16.0-SNAPSHOT</version>
<packaging>jar</packaging>

<properties>
<java.version>1.8</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<spring.boot.version>1.5.4.RELEASE</spring.boot.version>
<spring.boot.version>2.1.9.RELEASE</spring.boot.version>

<start-class>com.odysseusinc.arachne.executionengine.ExecutionEngineStarter</start-class>

Expand All @@ -23,12 +23,13 @@
<postgresql.version>42.2.1</postgresql.version>
<jtds.version>1.3.1</jtds.version>
<mysql.version>5.1.30</mysql.version>
<tomcat.version>9.0.27</tomcat.version>
</properties>

<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.4.RELEASE</version>
<version>2.1.9.RELEASE</version>
<relativePath/>
</parent>

Expand Down Expand Up @@ -147,7 +148,7 @@
<dependency>
<groupId>org.ohdsi.sql</groupId>
<artifactId>SqlRender</artifactId>
<version>1.6.3</version>
<version>1.6.3-SNAPSHOT</version>
</dependency>

<dependency>
Expand All @@ -167,6 +168,18 @@
<version>2.7.2</version>
</dependency>

<dependency>
<groupId>com.odysseusinc</groupId>
<artifactId>data-source-manager</artifactId>
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>

</dependencies>

<profiles>
Expand Down Expand Up @@ -205,49 +218,14 @@
<id>impala</id>
<properties>
<impala.classpath>/impala/</impala.classpath>
<impala.driver.version>2.6.15</impala.driver.version>
</properties>
<dependencies>
<!-- Impala driver -->
<dependency>
<groupId>hive</groupId>
<artifactId>hive_metastore</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>hive</groupId>
<artifactId>hive_service</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>com.cloudera</groupId>
<artifactId>ImpalaJDBC4</artifactId>
<version>2.5.42</version>
</dependency>
<dependency>
<groupId>libfb</groupId>
<artifactId>libfb303</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>libthrift</groupId>
<artifactId>libthrift</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>ql</groupId>
<artifactId>ql</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>TCLI</groupId>
<artifactId>TCLI</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.4.6</version>
<type>pom</type>
<artifactId>ImpalaJDBC42</artifactId>
<version>${impala.driver.version}</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
Expand All @@ -272,101 +250,110 @@
</goals>
<configuration>
<groupId>com.cloudera</groupId>
<artifactId>ImpalaJDBC4</artifactId>
<version>2.5.42</version>
<packaging>jar</packaging>
<file>${impala.classpath}/ImpalaJDBC41.jar</file>
</configuration>
</execution>
<execution>
<id>hive_metastore</id>
<phase>initialize</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<groupId>hive</groupId>
<artifactId>hive_metastore</artifactId>
<version>1.0.0</version>
<artifactId>ImpalaJDBC42</artifactId>
<version>${impala.driver.version}</version>
<packaging>jar</packaging>
<file>${impala.classpath}/hive_metastore.jar</file>
</configuration>
</execution>
<execution>
<id>hive_service</id>
<phase>initialize</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<groupId>hive</groupId>
<artifactId>hive_service</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<file>${impala.classpath}/hive_service.jar</file>
</configuration>
</execution>
<execution>
<id>libfb</id>
<phase>initialize</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<groupId>libfb</groupId>
<artifactId>libfb303</artifactId>
<version>0.9.0</version>
<packaging>jar</packaging>
<file>${impala.classpath}/libfb303-0.9.0.jar</file>
</configuration>
</execution>
<execution>
<id>libthrift</id>
<phase>initialize</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<groupId>libthrift</groupId>
<artifactId>libthrift</artifactId>
<version>0.9.0</version>
<packaging>jar</packaging>
<file>${impala.classpath}/libthrift-0.9.0.jar</file>
</configuration>
</execution>
<execution>
<id>ql</id>
<phase>initialize</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<groupId>ql</groupId>
<artifactId>ql</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<file>${impala.classpath}/ql.jar</file>
</configuration>
</execution>
<execution>
<id>TCLI</id>
<phase>initialize</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<groupId>TCLI</groupId>
<artifactId>TCLI</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<file>${impala.classpath}/TCLIServiceClient.jar</file>
<file>${impala.classpath}/ImpalaJDBC42.jar</file>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>hive</id>
<properties>
<hive-jdbc.version>3.1.2</hive-jdbc.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
<version>${hive-jdbc.version}</version>
<exclusions>
<exclusion>
<groupId>org.apache.tomcat</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>org.eclipse.jetty</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
<exclusion>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
</exclusion>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
<exclusion>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
</exclusion>
<exclusion>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP-java7</artifactId>
</exclusion>
<exclusion>
<groupId>javax.ws.rs</groupId>
<artifactId>javax.ws.rs-api</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-server</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-core</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-client</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-json</artifactId>
</exclusion>
<exclusion>
<groupId>com.tdunning</groupId>
<artifactId>json</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-server</artifactId>
<version>1.19.4</version>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-core</artifactId>
<version>1.19.4</version>
<exclusions>
<exclusion>
<groupId>javax.ws.rs</groupId>
<artifactId>jsr311-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-client</artifactId>
<version>1.19.4</version>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-json</artifactId>
<version>1.19.4</version>
</dependency>
</dependencies>
</profile>
<profile>
<id>bigquery</id>
<properties>
Expand Down Expand Up @@ -562,14 +549,20 @@
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>1.5.0.RELEASE</version>
<version>${spring.boot.version}</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
<configuration>
<classifier>exec</classifier>
<requiresUnpack>
<dependency>
<groupId>com.cloudera</groupId>
<artifactId>ImpalaJDBC42</artifactId>
</dependency>
</requiresUnpack>
</configuration>
</execution>
</executions>
Expand Down
Loading

0 comments on commit c99a0be

Please sign in to comment.