Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TOREE-556] Support Scala 2.13 #218

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

[TOREE-556] Support Scala 2.13 #218

wants to merge 12 commits into from

Conversation

pan3793
Copy link
Member

@pan3793 pan3793 commented Aug 20, 2023

Co-authored-by: Neil Skinner [email protected]
Co-authored-by: Cheng Pan [email protected]

This PR is based on #199, and makes the project compatible with both Scala 2.12 and 2.13.

Usage:

sbt ++2.12 clean test
sbt ++2.13 clean test

or

SCALA_VERSION=2.12 sbt clean test
SCALA_VERSION=2.13 sbt clean test

or

make SCALA_VERSION=2.12 dev
make SCALA_VERSION=2.13 dev

CI is updated to cover both Scala 2.12 and 2.13 tests. I also tested some basic functionalities locally with the docker environment.

@pan3793 pan3793 marked this pull request as draft August 20, 2023 18:29
@pan3793 pan3793 force-pushed the PR_199 branch 2 times, most recently from 7d29aae to 2178e20 Compare August 29, 2023 04:06
@lresende
Copy link
Member

@pan3793 what is the status of this pr?

@pan3793
Copy link
Member Author

pan3793 commented Aug 26, 2024

@lresende I remember the coursier log API changed significantly, and we need to adapt the new API to recover the log display, let me find time these weeks to fix it and finish the scala 2.13 support.

@lresende
Copy link
Member

lresende commented Sep 2, 2024

Thank you for the updates @pan3793 I did not do much testing, but updated my env to spark 3.4.3 + scala 13 and at least it connects and establishes a spark session... will try to spend more time on validation during the week.

@@ -43,7 +50,6 @@ ThisBuild / scalacOptions ++= Seq(
"-deprecation",
"-unchecked",
"-feature",
"-Xfatal-warnings",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make the code support both Scala 2.12 and 2.13, we have to use some deprecated Scala API, so this scalac flag must be disabled.

object JavaConverters in package collection is deprecated (since 2.13.0): Use `scala.jdk.CollectionConverters` instead

@@ -16,4 +16,6 @@
*/
resolvers += Resolver.sonatypeRepo("releases")

addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full)
if (scalaBinaryVersion.value == "2.12") {
addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.1" cross CrossVersion.full)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/scalamacros/paradise

In Scala 2.13, the plugin's functionality has been included in the compiler directly under the -Ymacro-annotations flag.

lazy val scala213 = "2.13.8"
lazy val defaultScalaVersion = sys.env.get("SCALA_VERSION") match {
case Some("2.12") => scala212
case _ => scala213
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I make Scala 2.13 as the default version

@pan3793 pan3793 changed the title [DRAFT] Support Scala 2.13 [TOREE-556] Support Scala 2.13 Sep 4, 2024
@pan3793 pan3793 marked this pull request as ready for review September 4, 2024 12:42
@pan3793
Copy link
Member Author

pan3793 commented Sep 4, 2024

@lresende this PR is ready to review now.

@pan3793
Copy link
Member Author

pan3793 commented Sep 4, 2024

also cc @requaos

@pan3793
Copy link
Member Author

pan3793 commented Oct 18, 2024

kindly ping @lresende ~

@legiondean
Copy link

@pan3793, @lresende , @requaos
do we feel comfortable to merge the PR to support 2.13?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants