CNDB-11508: Enable our test config explicitly for JDK22 and apply some test fixes #1398

ekaterinadimitrova2 · 2024-10-30T23:41:22Z

What is the issue

We cannot:

compile and run main on JDK22
run tests on JDK22 with the current config

What does this PR fix and why was it fixed

Porting CASSANDRA-18467, CASSANDRA-18179, CASSANDRA-18258 and adding updates based on testing we already did in a POC branch will enable us to build and run Cassandra on JDK22 plus run all test suites on JDK22 from the main branch. We also added numerous fixes that get CI to a very good state on JDK22.

Removes the use of CASSANDRA_USE_JDK11 (we should verify that will not affect CI systems and other infra scripts in any way) and introduces the build.xml properties java.default and java.supported.
ant generate-idea-files now support JDK 8, JDK 11, and JDK 22.
To add support for another JDK the java-jvmargs property must be set for the JDK in question (see how it's done in build.xml for Java 11 and 22)

Other minor, but notable changes are:

test jvmargs are now added to idea run configurations
.idea dir and project iml file are first removed and then recreated during ant generate-idea-files
Adds some additional add-opens and add-exports needed and upgrades mockito for testing.

CI runs posted on the GH issue.

Checklist before you submit for review

Make sure there is a PR in the CNDB project updating the Converged Cassandra version
Use NoSpamLogger for log lines that may appear frequently in the logs
Verify test results on Butler
Test coverage for new/modified code is > 80%
Proper code formatting
Proper title for each commit staring with the project-issue number, like CNDB-1234
Each commit has a meaningful description
Each commit is not very long and contains related changes
Renames, moves and reformatting are in distinct commits

ekaterinadimitrova2 · 2024-10-30T23:42:10Z

build.xml

+    -->
+    <property name="java.default" value="11" />
+    <!-- Not sure whether we need 17 below, at least jvm options were added some time ago for 17 -->
+    <property name="java.supported" value="1.8,11,17,22" />


I should remove 1.8

Maybe add 21?

Is 1.8 actually still supported? I don't think it is. I would leave this as 11/17/22 as those are the JVMs we are using.

Yeah, I think what we agreed is the cleaning of dead config for JDK8 is done in main-5.0 but we can remove it from supported here.

Given we are compiling our binaries with 11, I think we should start making things show meaningful errors if someone uses 8 rather than the strange error you get when it actually tries to run.

Yes, I removed 1.8 and now it shows: Unsupported JDK version used: 1.8
Same as any other version which is not on that list of java.supported

ekaterinadimitrova2 · 2024-10-30T23:42:36Z

build.xml

+    <!-- JDKs supported.
+        All releases are built with the default JDK.
+        Builds with non-default JDKs are considered experimental and for development and testing purposes.
+        When building, javac's source and target flags are set to the jdk used, so lower JDKs are not supported at runtime.
+        The use of both CASSANDRA_USE_JDK11 and use-jdk11 is deprecated.


Not sure whether this is applicable for our CI, it was applicable for Apache, TBD

I don't think we need this comment

ekaterinadimitrova2 · 2024-10-30T23:43:20Z

build.xml

+        <string>-XX:G1RSetUpdatingPauseTimePercent=5</string>
+        <string>-XX:MaxGCPauseMillis=100</string>
+
+        <!-- Copy-paste for now these options from cndb/docker-entrypoint.sh; we will verify later whether we


This comment should be removed when things are settled

How do these all work? You mention these are for tests in the comment, why are there both _jvm22_arg_items and _jvm22_test_arg_items?

It would be nice if we did not need to have all these duplicated here and also in the server.options files, but that is probably something for another ticket.

It would be nice if we did not need to have all these duplicated here and also in the server.options files, but that is probably something for another ticket.

Yes and no. Good reference: CASSANDRA-18263

So pre-CMS being out - Apache Cassandra was using CMS for testing, but a different config for prod. Optimization for our tests or so. When we moved past JDK8 - CASSANDRA-18263 was born. Until someone determines what we do in tests, we will keep the same configuration as the server.options files for now. So far no one has reported any issues around that, and I am not sure we will get to differ anything again. Exactly the opposite, @jacek-lewandowski did very good points in favor to keep G1GC and the same config as prod.
But there is one more caveat - sometimes we need to open/export some JDK internals just for some of the test frameworks we use, but not for prod code. In those cases we will want that to be done only in build.xml.

TLDR - let's consider a change in a different ticket.

a different ticket, and OSS first in this case;

we should get server.properties + extra options for testing;

I believe CASSANDRA-18263 can be used for that and there is no need of additional one. I will leave a comment there that we may want to reconsider how we stream options between server.options and build.xml to reduce duplications and minimize the potential for bugs.

ekaterinadimitrova2 · 2024-10-30T23:43:27Z

build.xml

+        <!-- Below two are used in Cassandra, not in CNDB-->
+        <!--<string>-XX:MaxTenuringThreshold=1</string>
+        <string>-XX:G1HeapRegionSize=16m</string>-->


This comment should be removed when things are settled

those are really just for compilation/running tests, right?

The options in build.xml are only for testing, yes. In prod we would use the ones in server.properties files for self-managed. For Astra, they are at a different place.

ekaterinadimitrova2 · 2024-10-30T23:45:00Z

pylib/cassandra-cqlsh-tests.sh

@@ -50,9 +50,6 @@ fi
 if [[ -z "${java_version}" ]]; then
  java_version=$(java -version 2>&1 | awk -F '"' '/version/ {print $2}' | sed 's/^1\.//g' | awk -F. '{print $1}')
 fi
-if [[ ${java_version} -ge 11 ]]; then


Testing will show whether this is enough and whether CI is affected in some way by this change

if we use the new CCM, we should not observe problems; it should just use the Java version from the environment

I am not sure which the new CCM is, but so far I did not see any problems.

ekaterinadimitrova2 · 2024-10-30T23:45:49Z

The PR is not finished, but I wanted to do preliminary JDK11 testing to see whether I broke anything so far

tools/bin/cassandra.in.sh

JeremiahDJordan · 2024-11-04T21:46:39Z

tools/bin/cassandra.in.sh


-JAVA_VERSION=11
 if [ "$JVM_VERSION" = "1.8.0" ]  ; then


I think we should just require 11+ given we are doing our builds with 11.

JeremiahDJordan · 2024-11-04T21:48:41Z

I think this looks good for a full review. When you put it up for full review, please include links to 11/17/22 based CI runs.

ekaterinadimitrova2 · 2024-11-06T16:06:32Z

Cleaned JDK1.8 from allowed JDKs and fixed a bug in the tooling config.
CI links and additional business-related questions will be posted on the ticket for discussion in a bit.
Moving out of draft to trigger the JDK11 CI run.
For anyone following - please check the ticket before starting full review

…roperties `java.default` and `java.supported`. ant generate-idea-files now support JDK 8, JDK 11 and JDK 22. To add support of another JDK the java-jvmargs property must be set for the JDK in question (see how it's done in build.xml for Java 11 and 22) Other minor, but notable changes are: - test jvmargs are now added to idea run configurations - .idea dir and project iml file are first removed and then recreated during `ant generate-idea-files` Based on what was done in CASSANDRA-18467, CASSANDRA-18179, CASSANDRA-18258 for 17 plus additional stuff for 21 Co-authored-by: Ekaterina Dimitrova<[email protected]> Co-authored-by: Mick Semb Wever <[email protected]> Co-authored-by: Jakub Zytka <[email protected]>

…dit logger and vectorizedMismatch

ekaterinadimitrova2 · 2024-11-06T20:34:05Z

There was some issue with CI.
I rebased the branch and force-pushed it to trigger it again, hoping that it was some environmental temporary problem.
The old build was also in a weird state in Jenkins—it was shown as failed, but some jobs were still marked in progress. Hopefully, the new trigger clears the old run and triggers a fresh new one without problems.

bin/cassandra.in.sh

jacek-lewandowski · 2024-11-07T12:18:48Z

build.xml

+    <!-- JDKs supported.
+        All releases are built with the default JDK.
+        Builds with non-default JDKs are considered experimental and for development and testing purposes.
+        When building, javac's source and target flags are set to the jdk used, so lower JDKs are not supported at runtime.
+        The use of both CASSANDRA_USE_JDK11 and use-jdk11 is deprecated.


I don't think we need this comment

jacek-lewandowski · 2024-11-07T12:19:50Z

build.xml

+        When building, javac's source and target flags are set to the jdk used, so lower JDKs are not supported at runtime.
+        The use of both CASSANDRA_USE_JDK11 and use-jdk11 is deprecated.
+    -->
+    <property name="java.default" value="11" />


Not sure - why do we need this? Default Java version is the version of Java we run the build with

Exactly, it is a placeholder one-stop-place to bump when we start building instead with another version.
We also warn people if they build on their own with different version that it is not a default one we ship with.

jacek-lewandowski · 2024-11-07T12:20:43Z

build.xml

+        <condition><not><contains string="${java.supported}" substring="${ant.java.version}"/></not></condition>
+    </fail>
+    <condition property="is.java.default"><equals arg1="${ant.java.version}" arg2="${java.default}"/></condition>
+    <echo unless:true="${is.java.default}" message="Non default JDK version used: ${ant.java.version}"/>


We really don't need this warning

This is port from OSS where it makes sense because people may build something themselves. In our case it may not be needed that much, but I think it doesn't hurt too. It may be helpful for people experimenting with the codebase to know what is really supported at the moment and where it is on them to experiment without expectations.

jacek-lewandowski · 2024-11-07T12:22:12Z

build.xml

+        <!-- Below two are used in Cassandra, not in CNDB-->
+        <!--<string>-XX:MaxTenuringThreshold=1</string>
+        <string>-XX:G1HeapRegionSize=16m</string>-->


those are really just for compilation/running tests, right?

jacek-lewandowski · 2024-11-07T12:45:09Z

build.xml

        <javac
               debug="true" debuglevel="${debuglevel}" encoding="utf-8"
-               destdir="${build.classes.main}" includeantruntime="false" source="${source.version}" target="${target.version}">
+               destdir="${build.classes.main}" includeantruntime="false" source="${ant.java.version}" target="${ant.java.version}">


I wonder if we need to explicitly pass it, isn't it the default?

Not sure, but I prefer us to be explicit for things. Makes it easier to reason.

jacek-lewandowski · 2024-11-07T12:45:55Z

build.xml

@@ -1483,19 +1517,19 @@
        <jvmarg value="-Dcassandra.test.sstableformatdevelopment=true"/>
        <!-- The first time SecureRandom initializes can be slow if it blocks on /dev/random -->
        <jvmarg value="-Djava.security.egd=file:/dev/urandom" />
-        <jvmarg value="-Dcassandra.testtag=${tag}"/>
+        <jvmarg value="-Dcassandra.testtag=${tag}.jdk${ant.java.version}"/>


please don't tag it with Java version

We do it in a similar way in main-5.0 - https://github.com/datastax/cassandra/blob/main-5.0/build.xml#L1190? I find it helpful as we had troubles in the past when things were running in test with wrong JDKs. This makes it explicitly visible in CI.

jacek-lewandowski · 2024-11-07T12:46:59Z

build.xml

-    <replace file="${eclipse.project.name}.iml" token="JDK_1_8" value="JDK_11"/>
-    <replace file=".idea/misc.xml" token="JDK_1_8" value="JDK_11"/>
-    <replace file=".idea/misc.xml" token="1.8" value="11"/>
+  <target name="_maybe_update_idea_to_java11plus" depends="init" unless="java.version.8">


unless is probably not needed - the entire script should fail if we attempt to use 8

jacek-lewandowski · 2024-11-07T12:47:46Z

build.xml

-    <replace file=".idea/misc.xml" token="1.8" value="11"/>
+  <target name="_maybe_update_idea_to_java11plus" depends="init" unless="java.version.8">
+      <replace file="${eclipse.project.name}.iml" token="JDK_1_8" value="JDK_${ant.java.version}"/>
+      <replace file=".idea/misc.xml" token="JDK_1_8" value="JDK_${ant.java.version}"/>


perhaps the misc.xml should include 11 by default as well

You mean we should update the token in misc.xml? Or anything else?
I was trying to skip the cleaning from CASSANDRA-18255 on main as we already have it in main-5.0 to keep it simple. As less changes we have, the less we can expect missed issues. Let me know if you prefer to clean 8 completely too.

jacek-lewandowski · 2024-11-07T12:51:06Z

build.xml

@@ -2205,6 +2238,8 @@

  <!-- Generate IDEA project description files -->
  <target name="generate-idea-files" depends="init,maven-ant-tasks-init,resolver-dist-lib,gen-cql3-grammar,generate-jflex-java,createVersionPropFile" description="Generate IDEA files">
+    <delete dir=".idea"/>


please don't delete .idea. Whenever you want to just update your project you will lose all your local configuration, like configured tests, etc. Please just delete what we need to replace.

This is already in 5.0 after discussion on the mailing list - https://lists.apache.org/thread/o2fdkyv2skvf9ngy9jhpnhvo92qvr17m
Do you want to change it in datastax/cassandra all branches? It seems there was consensus.
By the way what config files for testing you have that may be useful and we can actually add for everyone? maybe?

If someone runs ant generate-idea-files that says to me they want to start from freshly generated files? Otherwise why are you running it?

We run it when we switch between JDK versions for example. Before this change I had to invalidate caches and restart IDE almost every time...

ekaterinadimitrova2 · 2024-11-07T22:16:40Z

Thank you for the reviews.
@jacek-lewandowski , I pushed a few small commits to address your feedback and I hope I responded to all your questions in comments. Please let me know.
Side note: The jdk22 server.options files mostly mimic what we had for JDK17 with additions for JDK22. Wondering what we want to leave as jvm options (other than add-exports/add-opens which are clear), probably just mimic what we would use in server.options for now? Same as we did in Apache?

bin/cassandra.in.sh

ekaterinadimitrova2 · 2024-11-08T17:12:26Z

CI is in a bad shape today going up and down and I am not being able to really check test results.
I am pushing some review feedback updates to docs and I will move the PRs to draft so they do not trigger useless CI triggers. I will move to another issue I am working on and check back later.

sonarcloud · 2024-11-12T22:32:50Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

cassci-bot · 2024-11-12T22:37:10Z

❌ Build ds-cassandra-pr-gate/PR-1398 rejected by Butler

4 new test failure(s) in 3 builds
See build details here

Found 4 new test failures

Test	Explanation	Branch history	Upstream history
...list,wide=false,scenario=MEMTABLE_QUERY]	regression	🔴🔵🔵	🔵🔵🔵🔵🔵🔵🔵
...ap<int,int>,wide=false,scenario=MEMTABLE_QUERY]	regression	🔴🔵🔵	🔵🔵🔵🔵🔵🔵🔵
o.a.c.i.s.p.PlanTest.testLazyAccessPropagation	regression	🔴🔴🔵	🔵🔵🔵🔵🔵🔵🔵
...hUtilsTest.testDeleteQuietlyIgnoresIOExceptions	regression	🔴🔴🔵	🔵🔵🔵🔵🔵🔵🔵

Found 13 known test failures

ekaterinadimitrova2 self-assigned this Oct 30, 2024

ekaterinadimitrova2 marked this pull request as draft October 30, 2024 23:41

ekaterinadimitrova2 commented Oct 30, 2024

View reviewed changes

ekaterinadimitrova2 force-pushed the CNDB-11508 branch from b68440f to 286bf05 Compare November 1, 2024 23:43

ekaterinadimitrova2 commented Nov 3, 2024

View reviewed changes

tools/bin/cassandra.in.sh Outdated Show resolved Hide resolved

JeremiahDJordan reviewed Nov 4, 2024

View reviewed changes

ekaterinadimitrova2 force-pushed the CNDB-11508 branch from 0131563 to 3512cbe Compare November 5, 2024 21:39

ekaterinadimitrova2 marked this pull request as ready for review November 6, 2024 16:06

ekaterinadimitrova2 marked this pull request as draft November 6, 2024 20:28

ekaterinadimitrova2 marked this pull request as ready for review November 6, 2024 20:28

michaelsembwever and others added 8 commits November 6, 2024 15:29

Fix add-opens and add-exports

fc90205

CNDB-11508: CQLSHLIB tests update

010f955

Fix EmptyValuesTest

a317f30

Fix sjk

075058d

Fix tools/bin/cassandra.in.sh; fix some JDK internals openings for au…

c08e0cf

…dit logger and vectorizedMismatch

Upgrade Mockito to fix UnifiedCompactionStrategyTest

94257c0

Get rid of JDK8 in scripts, not supported anywhere anymore

42378b4

ekaterinadimitrova2 force-pushed the CNDB-11508 branch from 3512cbe to 42378b4 Compare November 6, 2024 20:30

jacek-lewandowski reviewed Nov 7, 2024

View reviewed changes

ekaterinadimitrova2 added 2 commits November 7, 2024 16:06

Remove useless comment

d42e1c7

JdkProperties does not need to account for JDK8 anymore

4eb00e2

Sort alphabetically add-exports/add-opens as per review request

3f58f6c

JeremiahDJordan reviewed Nov 7, 2024

View reviewed changes

bin/cassandra.in.sh Show resolved Hide resolved

Update the Java 11 doc as per review agreeement

52e9b87

ekaterinadimitrova2 changed the title ~~CNDB-11508: Enable our test config explicitly for JDK22 - WIP branch, NO COMMIT, NO REVIEW NEEDED YET~~ CNDB-11508: Enable our test config explicitly for JDK22 Nov 8, 2024

ekaterinadimitrova2 changed the title ~~CNDB-11508: Enable our test config explicitly for JDK22~~ CNDB-11508: Enable our test config explicitly for JDK22 and apply some test fixes Nov 8, 2024

ekaterinadimitrova2 marked this pull request as draft November 8, 2024 17:12

ekaterinadimitrova2 marked this pull request as ready for review November 12, 2024 21:18

JeremiahDJordan approved these changes Nov 13, 2024

View reviewed changes

CNDB-11508: Enable our test config explicitly for JDK22 and apply some test fixes #1398

Are you sure you want to change the base?

CNDB-11508: Enable our test config explicitly for JDK22 and apply some test fixes #1398

Conversation

ekaterinadimitrova2 commented Oct 30, 2024 • edited Loading

What is the issue

What does this PR fix and why was it fixed

Checklist before you submit for review

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekaterinadimitrova2 Nov 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekaterinadimitrova2 commented Oct 30, 2024

Choose a reason for hiding this comment

JeremiahDJordan commented Nov 4, 2024

ekaterinadimitrova2 commented Nov 6, 2024 • edited Loading

ekaterinadimitrova2 commented Nov 6, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekaterinadimitrova2 Nov 7, 2024 • edited Loading

Choose a reason for hiding this comment

ekaterinadimitrova2 commented Nov 7, 2024

ekaterinadimitrova2 commented Nov 8, 2024

sonarcloud bot commented Nov 12, 2024

Quality Gate passed

cassci-bot commented Nov 12, 2024

❌ Build ds-cassandra-pr-gate/PR-1398 rejected by Butler

Found 4 new test failures

Found 13 known test failures

ekaterinadimitrova2 commented Oct 30, 2024 •

edited

Loading

ekaterinadimitrova2 Nov 7, 2024 •

edited

Loading

ekaterinadimitrova2 commented Nov 6, 2024 •

edited

Loading

ekaterinadimitrova2 Nov 7, 2024 •

edited

Loading