Skip to content

Commit

Permalink
Merge pull request #9 from rohithvangapalli/spark-3.2
Browse files Browse the repository at this point in the history
Added the .githubactions.yml file and Updated the Readme.md file for Spark 3.2 branch
  • Loading branch information
badrinathpatchikolla authored Nov 1, 2022
2 parents 9541446 + bcdec1c commit 8663fc5
Show file tree
Hide file tree
Showing 5 changed files with 78 additions and 50 deletions.
51 changes: 51 additions & 0 deletions .github/workflows/neo4j.almaren-githubactions.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Neo4j.Almaren
on: [push, pull_request]

jobs:
Build:
runs-on: ubuntu-20.04
services:
neo4j:
image: neo4j:3.5
ports:
- 7474:7474
- 7687:7687
env:
neo4j-user: neo4j
neo4j-password: neo4j
postgres:
image: postgres:13.4
env:
POSTGRES_PASSWORD: postgres
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Update NEO4j password
run: |
curl -H "Content-Type: application/json" -XPOST -d '{"password":"neo4j1234"}' -u neo4j:neo4j http://localhost:7474/user/neo4j/password
- name : Check out repository code
uses: actions/checkout@v2
- name: Setup JDK
uses: actions/setup-java@v3
with:
distribution: temurin
java-version: 8
cache: sbt
- name: Build and test scala version
run: |
PGPASSWORD="postgres" psql -c 'create database almaren;' -U postgres -h localhost
PGPASSWORD="postgres" psql -c "ALTER USER postgres PASSWORD 'postgres' ;" -U postgres -h localhost
PGPASSWORD="postgres" psql -c 'create role runner;' -U postgres -h localhost
PGPASSWORD="postgres" psql -c 'ALTER ROLE "runner" WITH LOGIN SUPERUSER INHERIT CREATEDB CREATEROLE REPLICATION;' -U postgres -h localhost
sbt ++2.12.15 test
rm -rf "$HOME/.ivy2/local" || true
find $HOME/Library/Caches/Coursier/v1 -name "ivydata-*.properties" -delete || true
find $HOME/.ivy2/cache -name "ivydata-*.properties" -delete || true
find $HOME/.cache/coursier/v1 -name "ivydata-*.properties" -delete || true
find $HOME/.sbt -name "*.lock" -delete || true
42 changes: 0 additions & 42 deletions .travis.yml

This file was deleted.

24 changes: 21 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,36 @@
# Neo4j Connector

[![Build Status](https://travis-ci.com/modakanalytics/neo4j.almaren.svg?branch=master)](https://travis-ci.com/modakanalytics/neo4j.almaren)
[![Build Status](https://github.com/modakanalytics/neo4j.almaren/actions/workflows/neo4j.almaren-githubactions.yml/badge.svg)](https://github.com/modakanalytics/neo4j.almaren/actions/workflows/neo4j.almaren-githubactions.yml)

To add neo4j dependency to your sbt build:

```
libraryDependencies += "com.github.music-of-the-ainur" %% "neo4j-almaren" % "0.1.2-$SPARK_VERSION"
libraryDependencies += "com.github.music-of-the-ainur" %% "neo4j-almaren" % "0.1.3-3.2"
```

Neo4j Connector was implemented using [https://github.com/neo4j-contrib/neo4j-spark-connector](https://github.com/neo4j-contrib/neo4j-spark-connector).
For more details check the following [link](https://github.com/neo4j-contrib/neo4j-spark-connector).

To run in spark-shell:

```
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.3-$SPARK_VERSION,com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.2-$SPARK_VERSION"
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.8-3.2,com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.2"
```


Neo4j connector is available in the
[Maven Central](https://mvnrepository.com/artifact/com.github.music-of-the-ainur)
repository.

| versions | Connector Artifact |
|----------------------------|--------------------------------------------------------------|
| Spark 3.3.x and scala 2.13 | `com.github.music-of-the-ainur:neo4j-almaren_2.13:0.1.3-3.3` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:neo4j-almaren_2.11:0.1.3-2.4` |

## Source and Target

### Source
Expand Down
8 changes: 4 additions & 4 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ scalacOptions ++= Seq("-deprecation", "-feature")
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion % "provided",
"org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
"com.github.music-of-the-ainur" %% "almaren-framework" % s"0.9.4-${majorVersion}" % "provided",
"com.typesafe.scala-logging" %% "scala-logging" % "3.9.0",
"org.scalatest" %% "scalatest" % "3.0.5" % "test",
"neo4j-contrib" %% "neo4j-connector-apache-spark" % "4.0.1_for_spark_3"
"com.github.music-of-the-ainur" %% "almaren-framework" % s"0.9.8-${majorVersion}" % "provided",
"com.typesafe.scala-logging" %% "scala-logging" % "3.9.5",
"org.scalatest" %% "scalatest" % "3.2.14" % "test",
"org.neo4j" %% "neo4j-connector-apache-spark" % "4.1.5_for_spark_3"
)

enablePlugins(GitVersioning)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ import org.apache.spark.sql.functions._
import com.github.music.of.the.ainur.almaren.Almaren
import com.github.music.of.the.ainur.almaren.builder.Core.Implicit
import com.github.music.of.the.ainur.almaren.neo4j.Neo4j.Neo4jImplicit
import org.scalatest.funsuite.AnyFunSuite

class Test extends FunSuite with BeforeAndAfter {
class Test extends AnyFunSuite with BeforeAndAfter {

val almaren = Almaren("neo4j-almaren")

Expand Down

0 comments on commit 8663fc5

Please sign in to comment.