Skip to content

Commit

Permalink
Merge pull request #37 from rohithvangapalli/spark-3.2
Browse files Browse the repository at this point in the history
Updated the almaren and spark version
  • Loading branch information
badrinathpatchikolla authored Sep 22, 2023
2 parents c30eedd + 5063755 commit 5091963
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 11 deletions.
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@

To add Mongodb Almaren dependency to your sbt build::
```
libraryDependencies += "com.github.music-of-the-ainur" %% "mongodb-almaren" % "0.0.8-3.2"
libraryDependencies += "org.mongodb.spark" % "mongo-spark-connector" % "10.0.4"
libraryDependencies += "com.github.music-of-the-ainur" %% "mongodb-almaren" % "0.0.9-3.2"
libraryDependencies += "org.mongodb.spark" % "mongo-spark-connector" % "10.0.5"
```

To run in spark-shell:
```
spark-shell --master local[*] --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.8-3.2,com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.8-3.2,org.mongodb.spark:mongo-spark-connector:10.0.4"
spark-shell --master local[*] --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.10-3.2,com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.2,org.mongodb.spark:mongo-spark-connector:10.0.5"
```

MongoDB Connector is available in
Expand All @@ -19,11 +19,12 @@ repository.

| versions | Connector Artifact |
|----------------------------|----------------------------------------------------------------|
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.8-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.8-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.8-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.8-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:mongodb-almaren_2.11:0.0.8-2.4` |
| Spark 3.4.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.4` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:mongodb-almaren_2.11:0.0.9-2.4` |

## Source and Target

Expand Down
6 changes: 3 additions & 3 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ lazy val scala212 = "2.12.10"

ThisBuild / scalaVersion := scala212

val sparkVersion = "3.2.1"
val sparkVersion = "3.2.4"
val majorVersionReg = "([0-9]+\\.[0-9]+).{0,}".r

val majorVersionReg(majorVersion) = sparkVersion
Expand All @@ -15,8 +15,8 @@ scalacOptions ++= Seq("-deprecation", "-feature")
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion % "provided",
"org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
"com.github.music-of-the-ainur" %% "almaren-framework" % s"0.9.8-${majorVersion}" % "provided",
"org.mongodb.spark" % "mongo-spark-connector" % "10.0.4" % "provided",
"com.github.music-of-the-ainur" %% "almaren-framework" % s"0.9.10-${majorVersion}" % "provided",
"org.mongodb.spark" % "mongo-spark-connector" % "10.0.5" % "provided",
"com.typesafe.scala-logging" %% "scala-logging" % "3.9.5",
"org.scalatest" %% "scalatest" % "3.2.14" % "test"
)
Expand Down

0 comments on commit 5091963

Please sign in to comment.