Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate Strongbox from OrientDB to JanusGraph #19

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Migrate Strongbox from OrientDB to JanusGraph #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2f42188
issues/1649: Migrate Strongbox from OrientDB to JanusGraph
sbespalov Jan 27, 2020
e08b500
issues/1775: Upgrade to Cassandra to 4.0-beta2
steve-todorov May 18, 2020
652738d
issues/1968: Implement JanusGraph id block queues
sbespalov Dec 12, 2020
4d372f4
issues/1968: switched POM version
sbespalov Jan 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ def isMasterBranch = 'master'.equals(env.BRANCH_NAME);
pipeline {
agent {
node {
label 'alpine-jdk8-mvn-3.5'
label 'alpine-jdk8-mvn3.6'
}
}
parameters {
Expand Down
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# strongbox-db

## The goal of this project
This project has been created in order speed up the building of the OrientDB database snapshots during the initial [Strongbox](https://github.com/strongbox/strongbox) startup. Before this project was born, it took some time to do all of the database changes from the beginning of time up until the most recent version, so that the OrientDB database structure would be populated properly.
This project has been created in order speed up the building of the JanusGraph database snapshots during the initial [Strongbox](https://github.com/strongbox/strongbox) startup. Before this project was born, it took some time to do all of the database changes from the beginning of time up until the most recent version, so that the JanusGraph database structure would be populated properly.

## Architecture
This application consists of the following modules:
* `strongbox-db-liquibase` which creates a `jar` file consiting of the liquibase changesets that construct the [Strongbox](https://github.com/strongbox/strongbox) OrientDB database schema
* `strongbox-db-schema` which creates a `jar` file consiting of the changesets that construct the [Strongbox](https://github.com/strongbox/strongbox) JanusGraph database schema
* `strongbox-db-import` which creates a `zip` file with packed built database snapshot built from the liquibase changesets

## How is this project used in [strongbox](https://github.com/strongbox/strongbox) ?
[strongbox](https://github.com/strongbox/strongbox) uses this project submodules as required dependencies. During the startup of [Strongbox](https://github.com/strongbox/strongbox), the application detects if an OrientDB database already exists. If not, then the application uses an extracted empty OrientDB database snapshot from the `strongbox-db-import` artifact. If the OrientDB database already exists, then the application applies all missing liquibase changesets from the `strongbox-db-liquibase` artifact.
[strongbox](https://github.com/strongbox/strongbox) uses this project submodules as required dependencies. During the startup of [Strongbox](https://github.com/strongbox/strongbox), the application detects if a JanusGraph database already exists. If one doesn't exist, it uses an extracted empty JanusGraph database snapshot from the `strongbox-db-import` artifact; if a database exists, then the application applies all missing changesets from the `strongbox-db-schema` artifact.

## How to build this project ?
To build the code, simply execute:
`mvn clean install`

## What's the result of the build process ?
This project produces the following artifacts:
* `strongbox-db-liquibase-${version}.jar` : Located in `strongbox-db-liquibase/target`, which contains current database liquibase changesets
* `strongbox-db-schema-${version}.jar` : Located in `strongbox-db-schema/target`, which contains current database changesets
* `strongbox-db-import-${version}.zip` : Located in `strongbox-db-import/target`, which contains zipped fresh database snapshot built from the above changesets
12 changes: 6 additions & 6 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>org.carlspring.strongbox</groupId>
<artifactId>strongbox-parent</artifactId>
<version>1.0-SNAPSHOT</version>
<version>1.0-PR-62-SNAPSHOT</version>
</parent>

<artifactId>strongbox-db</artifactId>
<packaging>pom</packaging>
<version>1.0-SNAPSHOT</version>
<version>1.0-PR-19-SNAPSHOT</version>

<properties>
<serverId />
<deployUrl />
Expand Down Expand Up @@ -48,7 +48,7 @@
</dependencies>

<modules>
<module>strongbox-db-liquibase</module>
<module>strongbox-db-schema</module>
<module>strongbox-db-import</module>
<module>strongbox-db-server</module>
</modules>
Expand Down
182 changes: 126 additions & 56 deletions strongbox-db-import/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,87 +8,38 @@
<parent>
<groupId>org.carlspring.strongbox</groupId>
<artifactId>strongbox-db</artifactId>
<version>1.0-SNAPSHOT</version>
<version>1.0-PR-19-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>strongbox-db-import</artifactId>
<inceptionYear>2019</inceptionYear>

<properties>
<strongbox.orientdb.path>${project.build.directory}/db</strongbox.orientdb.path>
</properties>

<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>strongbox-db-liquibase</artifactId>
<artifactId>strongbox-db-server</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>strongbox-db-server</artifactId>
<artifactId>strongbox-db-schema</artifactId>
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>

<dependency>
<groupId>com.orientechnologies</groupId>
<artifactId>orientdb-jdbc</artifactId>
<artifactId>spring-boot-starter</artifactId>
</dependency>

<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
</dependency>

<!-- liquibase -->
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant</artifactId>
<version>1.10.5</version>
</dependency>
<dependency>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-core</artifactId>
<exclusions>
<exclusion>
<groupId>org.apache.ant</groupId>
<artifactId>ant</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.unbroken-dome.liquibase-orientdb</groupId>
<artifactId>liquibase-orientdb</artifactId>
</dependency>
<dependency>
<groupId>com.mattbertolini</groupId>
<artifactId>liquibase-slf4j</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
</dependency>

<!-- helpers -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
</dependencies>

<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
Expand All @@ -101,7 +52,30 @@
<goal>run</goal>
</goals>
<configuration>
<jvmArguments>-Xmx1024m</jvmArguments>
<jvmArguments>
-Xmx1024m
-Djdk.attach.allowAttachSelf=true
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED
--add-exports java.base/jdk.internal.ref=ALL-UNNAMED
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
--add-exports java.management.rmi/com.sun.jmx.remote.internal.rmi=ALL-UNNAMED
--add-exports java.rmi/sun.rmi.registry=ALL-UNNAMED
--add-exports java.rmi/sun.rmi.server=ALL-UNNAMED
--add-exports java.sql/java.sql=ALL-UNNAMED

--add-opens java.base/java.lang.module=ALL-UNNAMED
--add-opens java.base/jdk.internal.loader=ALL-UNNAMED
--add-opens java.base/jdk.internal.ref=ALL-UNNAMED
--add-opens java.base/jdk.internal.reflect=ALL-UNNAMED
--add-opens java.base/jdk.internal.math=ALL-UNNAMED
--add-opens java.base/jdk.internal.module=ALL-UNNAMED
--add-opens java.base/jdk.internal.util.jar=ALL-UNNAMED
--add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED
</jvmArguments>
<arguments>
<argument>--strongbox.db.janus-graph.storage-root=${project.build.directory}/db</argument>
<argument>--strongbox.dbimport.root=${project.build.directory}</argument>
</arguments>
</configuration>
</execution>
</executions>
Expand All @@ -120,11 +94,107 @@
<descriptors>
<descriptor>src/main/assembly/db-schema.xml</descriptor>
</descriptors>
</configuration>
</configuration>
</execution>
</executions>
</plugin>
</plugin>
</plugins>
</build>

<profiles>
<profile>
<id>jdk-default</id>
<activation>
<jdk>!11</jdk>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<id>strongbox-export-db</id>
<phase>prepare-package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>run</goal>
</goals>
<configuration>
<jvmArguments>
-Xmx1024m
</jvmArguments>
<arguments>
<argument>--strongbox.db.janus-graph.storage-root=${project.build.directory}/db</argument>
<argument>--strongbox.dbimport.root=${project.build.directory}</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>jdk11</id>
<activation>
<jdk>11</jdk>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<id>strongbox-export-db</id>
<phase>prepare-package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>run</goal>
</goals>
<configuration>
<jvmArguments>
-Xmx1024m
-Djdk.attach.allowAttachSelf=true
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED
--add-exports java.base/jdk.internal.ref=ALL-UNNAMED
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
--add-exports
java.management.rmi/com.sun.jmx.remote.internal.rmi=ALL-UNNAMED
--add-exports
java.rmi/sun.rmi.registry=ALL-UNNAMED
--add-exports
java.rmi/sun.rmi.server=ALL-UNNAMED
--add-exports
java.sql/java.sql=ALL-UNNAMED

--add-opens
java.base/java.lang.module=ALL-UNNAMED
--add-opens
java.base/jdk.internal.loader=ALL-UNNAMED
--add-opens
java.base/jdk.internal.ref=ALL-UNNAMED
--add-opens
java.base/jdk.internal.reflect=ALL-UNNAMED
--add-opens
java.base/jdk.internal.math=ALL-UNNAMED
--add-opens
java.base/jdk.internal.module=ALL-UNNAMED
--add-opens
java.base/jdk.internal.util.jar=ALL-UNNAMED
--add-opens
jdk.management/com.sun.management.internal=ALL-UNNAMED
</jvmArguments>
<arguments>
<argument>--strongbox.db.janus-graph.storage-root=${project.build.directory}/db</argument>
<argument>--strongbox.dbimport.root=${project.build.directory}</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

</project>
4 changes: 2 additions & 2 deletions strongbox-db-import/src/main/assembly/db-schema.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
</formats>
<fileSets>
<fileSet>
<directory>${project.build.directory}/db/strongbox</directory>
<outputDirectory>META-INF/resources/strongbox/db/strongbox</outputDirectory>
<directory>${project.build.directory}/db</directory>
<outputDirectory>META-INF/org/carlsparing/strongbox/db</outputDirectory>
</fileSet>
</fileSets>
</assembly>
Loading