Skip to content

Commit

Permalink
Merge pull request #11 from manasa-kandimalla/spark-2.4
Browse files Browse the repository at this point in the history
Added githubactions.yml file and updated ReadMe.md file
  • Loading branch information
badrinathpatchikolla authored Nov 1, 2022
2 parents e3de4b3 + 6af2a29 commit d7bf61a
Show file tree
Hide file tree
Showing 5 changed files with 78 additions and 51 deletions.
51 changes: 51 additions & 0 deletions .github/workflows/neo4j-almaren-githubactions.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Neo4j.Almaren
on: [push, pull_request]

jobs:
Build:
runs-on: ubuntu-20.04
services:
neo4j:
image: neo4j:3.5
ports:
- 7474:7474
- 7687:7687
env:
neo4j-user: neo4j
neo4j-password: neo4j
postgres:
image: postgres:13.4
env:
POSTGRES_PASSWORD: postgres
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Update NEO4j password
run: |
curl -H "Content-Type: application/json" -XPOST -d '{"password":"neo4j1234"}' -u neo4j:neo4j http://localhost:7474/user/neo4j/password
- name : Check out repository code
uses: actions/checkout@v2
- name: Setup JDK
uses: actions/setup-java@v3
with:
distribution: temurin
java-version: 8
cache: sbt
- name: Build and test scala version
run: |
PGPASSWORD="postgres" psql -c 'create database almaren;' -U postgres -h localhost
PGPASSWORD="postgres" psql -c "ALTER USER postgres PASSWORD 'postgres' ;" -U postgres -h localhost
PGPASSWORD="postgres" psql -c 'create role runner;' -U postgres -h localhost
PGPASSWORD="postgres" psql -c 'ALTER ROLE "runner" WITH LOGIN SUPERUSER INHERIT CREATEDB CREATEROLE REPLICATION;' -U postgres -h localhost
sbt ++2.12.15 test
rm -rf "$HOME/.ivy2/local" || true
find $HOME/Library/Caches/Coursier/v1 -name "ivydata-*.properties" -delete || true
find $HOME/.ivy2/cache -name "ivydata-*.properties" -delete || true
find $HOME/.cache/coursier/v1 -name "ivydata-*.properties" -delete || true
find $HOME/.sbt -name "*.lock" -delete || true
42 changes: 0 additions & 42 deletions .travis.yml

This file was deleted.

27 changes: 22 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,35 @@
# Neo4j Connector

[![Build Status](https://travis-ci.com/modakanalytics/neo4j.almaren.svg?branch=master)](https://travis-ci.com/modakanalytics/neo4j.almaren)
[![Build Status](https://github.com/modakanalytics/neo4j.almaren/actions/workflows/neo4j-almaren-githubactions.yml/badge.svg)](https://github.com/modakanalytics/neo4j.almaren/actions/workflows/neo4j-almaren-githubactions.yml)

Neo4j Connector was implemented using [https://github.com/neo4j-contrib/neo4j-spark-connector](https://github.com/neo4j-contrib/neo4j-spark-connector).
For more details check the following [link](https://github.com/neo4j-contrib/neo4j-spark-connector).

To add Neo4j connector dependency to your sbt build:

```
libraryDependencies += "com.github.music-of-the-ainur" %% "neo4j-almaren" % "0.1.2-2.4"
libraryDependencies += "com.github.music-of-the-ainur" %% "neo4j-almaren" % "0.1.3-2.4"
```

Neo4j Connector was implemented using [https://github.com/neo4j-contrib/neo4j-spark-connector](https://github.com/neo4j-contrib/neo4j-spark-connector).
For more details check the following [link](https://github.com/neo4j-contrib/neo4j-spark-connector).
To run in spark-shell:

```
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.3-$SPARK_VERSION,com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.2-$SPARK_VERSION"
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.8-2.4,com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-2.4"
```

Neo4j Connector is available in [Maven Central](https://mvnrepository.com/artifact/com.github.music-of-the-ainur)
repository.

| version | Connector Artifact |
|----------------------------|--------------------------------------------------------------|
| Spark 3.3.x and scala 2.13 | `com.github.music-of-the-ainur:neo4j-almaren_2.13:0.1.3-3.3` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:neo4j-almaren_2.11:0.1.3-2.4` |


## Source and Target

### Source
Expand Down
6 changes: 3 additions & 3 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ scalacOptions ++= Seq("-deprecation", "-feature")
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion % "provided",
"org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
"com.github.music-of-the-ainur" %% "almaren-framework" % s"0.9.3-${majorVersion}" % "provided",
"com.typesafe.scala-logging" %% "scala-logging" % "3.9.0",
"org.scalatest" %% "scalatest" % "3.0.5" % "test",
"com.github.music-of-the-ainur" %% "almaren-framework" % s"0.9.8-${majorVersion}" % "provided",
"com.typesafe.scala-logging" %% "scala-logging" % "3.9.5",
"org.scalatest" %% "scalatest" % "3.2.14" % "test",
"neo4j-contrib" %% "neo4j-connector-apache-spark" % "4.0.1"
)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ import org.apache.spark.sql.functions._
import com.github.music.of.the.ainur.almaren.Almaren
import com.github.music.of.the.ainur.almaren.builder.Core.Implicit
import com.github.music.of.the.ainur.almaren.neo4j.Neo4j.Neo4jImplicit
import org.scalatest.funsuite.AnyFunSuite

class Test extends FunSuite with BeforeAndAfter {
class Test extends AnyFunSuite with BeforeAndAfter {

val almaren = Almaren("neo4j-almaren")

Expand Down

0 comments on commit d7bf61a

Please sign in to comment.