Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot find native jdk.internal.misc.CDS.isDumpingClassList0 #450

Closed
eklaDFF opened this issue May 3, 2024 · 43 comments
Closed

cannot find native jdk.internal.misc.CDS.isDumpingClassList0 #450

eklaDFF opened this issue May 3, 2024 · 43 comments

Comments

@eklaDFF
Copy link

eklaDFF commented May 3, 2024

Hi @cyrille-artho ,
While running ./gradle test, we have 38/1002 failed cases.

Below is one of the major error.

running jpf with args:
JavaPathfinder core system v8.0 (rev fd73ee185637c825bb49c944bdc90f5e165c44c2) - (C) 2005-2014 United States Government. All rights reserved.


====================================================== system under test
gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.runTestMethod()

====================================================== search started: 03/05/24, 4:24 pm
[WARNING] orphan NativePeer method: jdk.internal.misc.Unsafe.getUnsafe()Lsun/misc/Unsafe;
[WARNING] orphan NativePeer method: jdk.internal.reflect.Reflection.getCallerClass(I)Ljava/lang/Class;

====================================================== error 1
gov.nasa.jpf.vm.NoUncaughtExceptionsProperty
java.lang.UnsatisfiedLinkError: cannot find native jdk.internal.misc.CDS.isDumpingClassList0
	at jdk.internal.misc.CDS.isDumpingClassList0(no peer)
	at jdk.internal.misc.CDS.<clinit>(CDS.java:49)
	at java.util.ImmutableCollections.<clinit>(ImmutableCollections.java:80)
	at java.util.ImmutableCollections$Set12.<init>(ImmutableCollections.java:786)
	at java.util.Set.of(Set.java:470)
	at jdk.internal.reflect.Reflection.<clinit>(Reflection.java:49)
	at java.lang.invoke.MethodHandles$Lookup.<clinit>(MethodHandles.java:1433)
	at java.lang.invoke.MethodHandles.lookup(MethodHandles.java:116)
	at java.util.concurrent.ConcurrentSkipListMap.<clinit>(ConcurrentSkipListMap.java:3407)
	at gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.testFunctionality(ConcurrentSkipListMapTest.java:35)
	at java.lang.reflect.Method.invoke(gov.nasa.jpf.vm.JPF_java_lang_reflect_Method)
	at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)


====================================================== snapshot #1
thread java.lang.Thread:{id:0,name:main,status:RUNNING,priority:5,isDaemon:false,lockCount:0,suspendCount:0}
  owned locks:java.lang.Class@227,java.lang.Class@24c,java.lang.Class@26c,java.lang.Class@274,java.lang.Class@277
  call stack:
	at jdk.internal.misc.CDS.<clinit>(CDS.java:49)
	at java.util.ImmutableCollections.<clinit>(ImmutableCollections.java:80)
	at java.util.ImmutableCollections$Set12.<init>(ImmutableCollections.java:786)
	at java.util.Set.of(Set.java:470)
	at jdk.internal.reflect.Reflection.<clinit>(Reflection.java:49)
	at java.lang.invoke.MethodHandles$Lookup.<clinit>(MethodHandles.java:1433)
	at java.lang.invoke.MethodHandles.lookup(MethodHandles.java:116)
	at java.util.concurrent.ConcurrentSkipListMap.<clinit>(ConcurrentSkipListMap.java:3407)
	at gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.testFunctionality(ConcurrentSkipListMapTest.java:35)
	at java.lang.reflect.Method.invoke(Method.java)
	at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)


====================================================== results
error #1: gov.nasa.jpf.vm.NoUncaughtExceptionsProperty "java.lang.UnsatisfiedLinkError: cannot find native..."

====================================================== search finished: 03/05/24, 4:24 pm 
@eklaDFF
Copy link
Author

eklaDFF commented May 5, 2024

Hi @cyrille-artho ,
I do not have any idea about domain of the model classes. Please look into this issue.

@cyrille-artho
Copy link
Member

When JPF looks for a given class, it usually uses the class from the application or library. However, library classes often call native methods (such as in this example), which are then not supported in JPF.
With a model class, you replace the implementation of an entire class with a different version. This version can be free of native code, in which case it can fully execute inside JPF.
It is also possible to use native methods from model classes, in which case they have to declared as a native peer. The is a special mechanism for this: https://github.com/javapathfinder/jpf-core/wiki/Model-Java-Interface

When you encounter a case where a native method is not supported, a choice often arises about which class in the stack trace that leads to the problem to replace with a model class (perhaps augmented by a native peer). If you replace a class further down in the stack trace, that may be a use case that is covered by other code as well, so the implementation often has to be rather comprehensive. Classes higher up in the stack trace (closer to the failure) may be smaller and thus replaceable with less effort, but they may also integrate more closely with JVM-specific native code.

So there is no easy way to tell which class to replace: java.util.concurrent.ConcurrentSkipListMap or classes closer to the failure. It's worth looking at all these classes and their complexity.

If there is no obvious candidate after this, it may be best to fix a different test failure first and return to this one later. Perhaps other tests show overlap with this problem.

@eklaDFF
Copy link
Author

eklaDFF commented May 7, 2024

Talking particularly about the error java.lang.UnsatisfiedLinkError: cannot find native jdk.internal.misc.CDS.isDumpingClassList0, I am mentioning below two errors from different classes.

Class gov.nasa.jpf.test.java.io.ObjectStreamTest
.......
====================================================== error 1
gov.nasa.jpf.vm.NoUncaughtExceptionsProperty
java.lang.UnsatisfiedLinkError: cannot find native jdk.internal.misc.CDS.isDumpingClassList0
	at jdk.internal.misc.CDS.isDumpingClassList0(no peer)
	at jdk.internal.misc.CDS.<clinit>(CDS.java:49)
	at java.util.ImmutableCollections.<clinit>(ImmutableCollections.java:80)
	at java.util.ImmutableCollections$Set12.<init>(ImmutableCollections.java:786)
	at java.util.Set.of(Set.java:470)
	at jdk.internal.reflect.Reflection.<clinit>(Reflection.java:49)
	at java.lang.invoke.MethodHandles$Lookup.<clinit>(MethodHandles.java:1433)
	at java.lang.invoke.MethodHandles.lookup(MethodHandles.java:116)
	at java.util.concurrent.atomic.AtomicBoolean.<clinit>(AtomicBoolean.java:56)
	at java.io.ObjectInputFilter$Config.<clinit>(ObjectInputFilter.java:596)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:390)
	at gov.nasa.jpf.test.java.io.ObjectStreamTest.testLambdaSerialization(ObjectStreamTest.java:143)
	at java.lang.reflect.Method.invoke(gov.nasa.jpf.vm.JPF_java_lang_reflect_Method)
	at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)

and

Class gov.nasa.jpf.test.vm.reflection.MethodTest

====================================================== error 1
gov.nasa.jpf.vm.NoUncaughtExceptionsProperty
java.lang.UnsatisfiedLinkError: cannot find native jdk.internal.misc.CDS.isDumpingClassList0
	at jdk.internal.misc.CDS.isDumpingClassList0(no peer)
	at jdk.internal.misc.CDS.<clinit>(CDS.java:49)
	at jdk.internal.math.FDBigInteger.<clinit>(FDBigInteger.java:87)
	at jdk.internal.math.FloatingDecimal$BinaryToASCIIBuffer.dtoa(FloatingDecimal.java:448)
	at jdk.internal.math.FloatingDecimal.getBinaryToASCIIConverter(FloatingDecimal.java:1825)
	at jdk.internal.math.FloatingDecimal.toJavaFormatString(FloatingDecimal.java:80)
	at java.lang.Float.toString(Float.java:226)
	at java.lang.Float.toString(Float.java:611)
	at gov.nasa.jpf.ConsoleOutputStream.print(ConsoleOutputStream.java:73)
	at gov.nasa.jpf.test.vm.reflection.MethodTest.invokeTest(MethodTest.java:348)
	at gov.nasa.jpf.test.vm.reflection.MethodTest.argTestNumber(MethodTest.java:599)
	at java.lang.reflect.Method.invoke(gov.nasa.jpf.vm.JPF_java_lang_reflect_Method)
	at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)

Here we can see that class CDS is the most closest common ground. Is CDS best candidate to intercept the native method ?

@cyrille-artho
Copy link
Member

This is a good point, we've seen CDS appear in many stack traces now with your examples. There does not seem to be any good documentation on CDS (at least it's not listed prominently in a Google search); do you know a good source upon which we could base a model class? A copy of the source (this one being from Oracle's JDK) can be found here: https://code.yawk.at/java/17/java.base/jdk/internal/misc/CDS.java
From what I can see from other sources, CDS (class data sharing) primarily deals with the pre-initialization of static fields. Doing this correctly would require a careful synchronization of the state of a loaded class with the internal state of JPF. Not implementing this feature would only incur a performance penalty, which would be fine (as long as the behavior is otherwise equivalent). However, we need to understand well enough what happens and what the effects of the native methods are in order to be sure that JPF handles this in the right way, even if we re-initialize a class every time it is used.

@cyrille-artho
Copy link
Member

Thanks, these are good starting points for an overview of CDS. In our case, we also need to get information on the internal implementation. Luckily, we do not have to implement CDS itself, as long as we ensure that we achieve the same program state by loading an initializing classes as usual even when CDS is enabled.
CDS is also not typically a feature of interest when using JPF, because fast class load times are not relevant. So our main goal is to be able to run applications even if these new internal APIs are called internally. It could even be that an empty stub (or a nearly empty one) works for that.

@eklaDFF
Copy link
Author

eklaDFF commented May 8, 2024

I have made a try to understand the CDS (took help from different sources including ChatGPT). I am explaining this to you and then correct me if I am wrong. When we launch an application for the first time, JVM loads the class required by our application and when our application exit, the JVM archive the loaded classes. Now, when we re-launch our application, JVM will use the classes from archive to save startup time.

I have noticed something while reading articles[links of those articles are mentioned in previous comment] with the method jdk.internal.misc.CDS.isDumpingClassList0. I will inform you as soon as I'm assured about.

@eklaDFF
Copy link
Author

eklaDFF commented May 9, 2024

Your were right, we should create an empty stub. And far as for jdk.internal.misc.CDS.isDumpingClassList0 and more methods returning them true. jdk.internal.misc.CDS.isDumpingClassList0 is an indicator for dumping class list.

@eklaDFF
Copy link
Author

eklaDFF commented May 11, 2024

@cyrille-artho
Should we wait for our next meet up (next Friday) to work on this particular issue ?

@cyrille-artho
Copy link
Member

Try creating an empty stub for the missing native methods in CDS. This way, you can see if you manage to create the "mangled" name for native methods and if they work for the tests we have.

@eklaDFF
Copy link
Author

eklaDFF commented May 12, 2024

Few doubts :

  1. What rules we (JPF) follow for naming of stub classes?
  2. Where we have to put our stub class (source location)? Is it same as "native peers" ?
  3. How the JVM will execute our method/class instead of CDS? (way of linking according to JPF architecture)
  4. What rules we (JPF) follow for naming of stub class's methods ? Is is same as "mangled" name ?

@cyrille-artho

@cyrille-artho
Copy link
Member

  1. See https://github.com/javapathfinder/jpf-core/wiki/Mangling-for-MJI
  2. The Java side of stubs is in src/classes, native peers are in src/peers. It is probably fine to declare only a simple stub without it being native for the JPF version, so you may not need native peers as long as you do not have to interface with JPF internals.
  3. Code in src/classes replaces their equivalents in the existing Java library. The only way to still access the original functionality would be at the JPF VM level: a model class calls a native peer (through the MJI), which can then call the original Java library class of the host JVM.
    Note that this would access a different instance of that class; the host JVM's instance is completely independent of instances that are managed in JPF. https://link.springer.com/content/pdf/10.1007/978-3-031-57249-4_1
  4. A model class has to match the original class exactly in terms of module, package, and class name. It is, however, sufficient to implement only some methods (enough to cover the common use cases and our tests), as the effort to reimplement all methods would be too big.

@eklaDFF
Copy link
Author

eklaDFF commented May 14, 2024

Please take a look

Screenshot 2024-05-14 at 7 13 59 PM

As we only have to create stub so we have avoided mangling of names, because we do not have to interact with MJI.

But running ./gradlew build, the problem is still the same. Our stub class is not able to replace the original one. I think I am doing it wrong.

@cyrille-artho
Copy link
Member

Try building from the command line with Gradle; if that fails, please copy the (text of) build failure message.
A build from an IDE likely does not work in the same way and often fails due to not reproducing the setting correctly.

@eklaDFF
Copy link
Author

eklaDFF commented May 14, 2024

After building from command line

Test Execution: FAILURE
Summary: 1002 tests, 964 passed, 38 failed, 0 skipped

1002 tests completed, 38 failed

> Task :test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/ekla/GSOC/JPFjava17/jpf-core/build/reports/tests/test/index.html

BUILD FAILED in 2m 1s
19 actionable tasks: 6 executed, 13 up-to-date
ekla@Eklas-MacBook-Air jpf-core % 

And one of the failed report (file:///Users/ekla/GSOC/JPFjava17/jpf-core/build/reports/tests/test/index.html)


running jpf with args:
JavaPathfinder core system v8.0 (rev fd73ee185637c825bb49c944bdc90f5e165c44c2) - (C) 2005-2014 United States Government. All rights reserved.


====================================================== system under test
gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.runTestMethod()

====================================================== search started: 14/05/24, 10:47 pm
[WARNING] orphan NativePeer method: jdk.internal.misc.Unsafe.getUnsafe()Lsun/misc/Unsafe;
[WARNING] orphan NativePeer method: jdk.internal.reflect.Reflection.getCallerClass(I)Ljava/lang/Class;

====================================================== error 1
gov.nasa.jpf.vm.NoUncaughtExceptionsProperty
java.lang.UnsatisfiedLinkError: cannot find native jdk.internal.misc.CDS.isDumpingClassList0
	at jdk.internal.misc.CDS.isDumpingClassList0(no peer)
	at jdk.internal.misc.CDS.<clinit>(CDS.java:49)
	at java.util.ImmutableCollections.<clinit>(ImmutableCollections.java:80)
	at java.util.ImmutableCollections$Set12.<init>(ImmutableCollections.java:786)
	at java.util.Set.of(Set.java:470)
	at jdk.internal.reflect.Reflection.<clinit>(Reflection.java:49)
	at java.lang.invoke.MethodHandles$Lookup.<clinit>(MethodHandles.java:1433)
	at java.lang.invoke.MethodHandles.lookup(MethodHandles.java:116)
	at java.util.concurrent.ConcurrentSkipListMap.<clinit>(ConcurrentSkipListMap.java:3407)
	at gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.testFunctionality(ConcurrentSkipListMapTest.java:35)
	at java.lang.reflect.Method.invoke(gov.nasa.jpf.vm.JPF_java_lang_reflect_Method)
	at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)


====================================================== snapshot #1
thread java.lang.Thread:{id:0,name:main,status:RUNNING,priority:5,isDaemon:false,lockCount:0,suspendCount:0}
  owned locks:java.lang.Class@227,java.lang.Class@24c,java.lang.Class@26c,java.lang.Class@274,java.lang.Class@277
  call stack:
	at jdk.internal.misc.CDS.<clinit>(CDS.java:49)
	at java.util.ImmutableCollections.<clinit>(ImmutableCollections.java:80)
	at java.util.ImmutableCollections$Set12.<init>(ImmutableCollections.java:786)
	at java.util.Set.of(Set.java:470)
	at jdk.internal.reflect.Reflection.<clinit>(Reflection.java:49)
	at java.lang.invoke.MethodHandles$Lookup.<clinit>(MethodHandles.java:1433)
	at java.lang.invoke.MethodHandles.lookup(MethodHandles.java:116)
	at java.util.concurrent.ConcurrentSkipListMap.<clinit>(ConcurrentSkipListMap.java:3407)
	at gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.testFunctionality(ConcurrentSkipListMapTest.java:35)
	at java.lang.reflect.Method.invoke(Method.java)
	at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)


====================================================== results
error #1: gov.nasa.jpf.vm.NoUncaughtExceptionsProperty "java.lang.UnsatisfiedLinkError: cannot find native..."

====================================================== search finished: 14/05/24, 10:47 pm

@cyrille-artho
Copy link
Member

If the static initializer is not found, this means the class is not in the right place. I can't really see from your screenshot where you placed the file CDS.java, but it should be somewhere like src/classes/modules/java.base/jdk/internal/misc/.
The IDE shows something like jdk.internal, and I'm not sure if this is a strange way of pretty-printing directory names, or if this indeed a misnamed directory; follow instead the convention of one directory per package name level.

@eklaDFF
Copy link
Author

eklaDFF commented May 15, 2024

Screenshot 2024-05-15 at 4 25 05 PM
ekla@Eklas-MacBook-Air misc % ls
CDS.java	Unsafe.java
ekla@Eklas-MacBook-Air misc % pwd
/Users/ekla/GSOC/JPFjava17/jpf-core/src/classes/modules/java.base/jdk/internal/misc
ekla@Eklas-MacBook-Air misc % 

@cyrille-artho
Copy link
Member

Thanks. I see that method isDumpingClassList0 is missing so please add a stub for that method, too.
You can remove the static { } part if you have no code in it, as the empty is auto-generated by the compiler in that case.

@eklaDFF
Copy link
Author

eklaDFF commented May 15, 2024

java.lang.UnsatisfiedLinkError: cannot find native jdk.internal.misc.CDS.isDumpingClassList0
	at jdk.internal.misc.CDS.isDumpingClassList0(no peer)
	at jdk.internal.misc.CDS.<clinit>(CDS.java:49)
	at sun.util.locale.BaseLocale.<clinit>(BaseLocale.java:66)
	at java.util.Locale.createConstant(Locale.java:620)
	at java.util.Locale.<clinit>(Locale.java:582)
	at java.text.DateFormat.getDateTimeInstance(DateFormat.java:47)
	at gov.nasa.jpf.test.java.text.DateFormatTest.testSetAndGetTimeZone(DateFormatTest.java:71)
	at java.lang.reflect.Method.invoke(gov.nasa.jpf.vm.JPF_java_lang_reflect_Method)
	at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)

We are using above error as an example to see the problem.

What is happening behind the scenes :

  1. CDS.initializeFromArchive(Class<?> c) is being called.

Screenshot in context of our example error.
Screenshot 2024-05-15 at 6 26 11 PM

  1. As soon as it is called the static block is being executed where isDumpingClassList0 is being called.
Screenshot 2024-05-15 at 6 32 08 PM otherwise `isDumpingClassList0` is not being called. (`isDumpingClassList0` is a native method which is used to get the indication value from JVM at runtime).

So from here we can go with on stub of CDS.initializeFromArchive(Class<?> c) only ?

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Approach to add stub for isDumpingClassList0 method :
If stub CDS.java class is able replace the original one then should it ask for isDumpingClassList0 because our stub have empty static block ?

@cyrille-artho
Copy link
Member

Ah, I see now what you mean. I agree that the model class is not loaded/used, but I can't see why. Let's take a look at this when we have our meeting.

@cyrille-artho
Copy link
Member

It also helps a lot if you make a branch with your work in progress (in your forked repository) and share it with us. This way, we can inspect various things ourselves, which can help us confirm or rule out possible causes.

@eklaDFF
Copy link
Author

eklaDFF commented May 16, 2024

https://github.com/eklaDFF/jpf-core/tree/CDSstub

What I did :
I just added a stub class for CDS.java

@cyrille-artho
Copy link
Member

Thanks, I was able to run the tests on this branch. I get 16 test failures. Which one are you referring to above? The few that I looked at have java.lang.NoSuchMethodException: jdk.internal.misc.CDS.getRandomSeedForDumping()J in their output. This suggests that the new model class for CDS actually gets used, but we need a few more stubs to make it work with more tests.

@eklaDFF
Copy link
Author

eklaDFF commented May 16, 2024

Do not know why but still it showing 38 failures on CDSstub branch.

Screenshot 2024-05-17 at 1 15 25 AM

What is the reason ?

///////////////////////////////////////////////////////////////////////////////////////////////

Standard output stack trace of java.lang.NoSuchMethodException: jdk.internal.misc.CDS.getRandomSeedForDumping()J will help us more

@eklaDFF
Copy link
Author

eklaDFF commented May 18, 2024

PR #451

@cyrille-artho
Copy link
Member

cyrille-artho commented May 18, 2024

Thanks. As the next part, getRandomSeedForDumping is also a method that we have to provide a stub for. We don't support dumping loaded classes, so again, I assume that simply ignoring method calls (by providing an empty method) to all these methods will likely work.

@eklaDFF
Copy link
Author

eklaDFF commented May 21, 2024

Here is the implementation for getRandomSeedForDumping.
Screenshot 2024-05-21 at 10 02 22 PM

But we still have 16 failed cases. But this time different reason.

Attaching below as one of the example from 16 failed tests

Class gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest

all > gov.nasa.jpf.test.java.concurrent > ConcurrentSkipListMapTest
1tests1failures0ignored0.376sduration | 1tests | 1failures | 0ignored | 0.376sduration | 0%successful -- | -- | -- | -- | -- | -- 1tests | 1failures | 0ignored | 0.376sduration
  running jpf with args:
JavaPathfinder core system v8.0 (rev 629fbba1ca49064391347ba1e831825985d5d871) - (C) 2005-2014 United States Government. All rights reserved.

====================================================== system under test
gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.runTestMethod()

====================================================== search started: 19/05/24, 1:07 am
[WARNING] orphan NativePeer method: jdk.internal.misc.Unsafe.getUnsafe()Lsun/misc/Unsafe;
[WARNING] orphan NativePeer method: jdk.internal.reflect.Reflection.getCallerClass(I)Ljava/lang/Class;

====================================================== error 1
gov.nasa.jpf.vm.NoUncaughtExceptionsProperty
java.lang.NoSuchMethodError: java.lang.invoke.JPFFieldVarHandle.weakCompareAndSetRelease(Ljava/util/concurrent/atomic/Striped64;JJ)Z
at java.util.concurrent.atomic.Striped64.casBase(Striped64.java:181)
at java.util.concurrent.atomic.LongAdder.add(LongAdder.java:87)
at java.util.concurrent.ConcurrentSkipListMap.addCount(ConcurrentSkipListMap.java:441)
at java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:683)
at java.util.concurrent.ConcurrentSkipListMap.put(ConcurrentSkipListMap.java:1347)
at gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.testFunctionality(ConcurrentSkipListMapTest.java:37)
at java.lang.reflect.Method.invoke(gov.nasa.jpf.vm.JPF_java_lang_reflect_Method)
at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)

====================================================== snapshot #1
thread java.lang.Thread:{id:0,name:main,status:RUNNING,priority:5,isDaemon:false,lockCount:0,suspendCount:0}
call stack:
at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:650)

====================================================== results
error #1: gov.nasa.jpf.vm.NoUncaughtExceptionsProperty "java.lang.NoSuchMethodError: java.lang.invoke.JPFF..."

====================================================== search finished: 19/05/24, 1:07 am

Class gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest
all > gov.nasa.jpf.test.java.concurrent > ConcurrentSkipListMapTest
1
tests

1
failures

0
ignored

0.376s
duration

0%
successful

Failed tests
Tests
Standard output
running jpf with args:
JavaPathfinder core system v8.0 (rev 629fbba) - (C) 2005-2014 United States Government. All rights reserved.

====================================================== system under test
gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.runTestMethod()

====================================================== search started: 19/05/24, 1:07 am
[WARNING] orphan NativePeer method: jdk.internal.misc.Unsafe.getUnsafe()Lsun/misc/Unsafe;
[WARNING] orphan NativePeer method: jdk.internal.reflect.Reflection.getCallerClass(I)Ljava/lang/Class;

====================================================== error 1
gov.nasa.jpf.vm.NoUncaughtExceptionsProperty
java.lang.NoSuchMethodError: java.lang.invoke.JPFFieldVarHandle.weakCompareAndSetRelease(Ljava/util/concurrent/atomic/Striped64;JJ)Z
at java.util.concurrent.atomic.Striped64.casBase(Striped64.java:181)
at java.util.concurrent.atomic.LongAdder.add(LongAdder.java:87)
at java.util.concurrent.ConcurrentSkipListMap.addCount(ConcurrentSkipListMap.java:441)
at java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:683)
at java.util.concurrent.ConcurrentSkipListMap.put(ConcurrentSkipListMap.java:1347)
at gov.nasa.jpf.test.java.concurrent.ConcurrentSkipListMapTest.testFunctionality(ConcurrentSkipListMapTest.java:37)
at java.lang.reflect.Method.invoke(gov.nasa.jpf.vm.JPF_java_lang_reflect_Method)
at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)

====================================================== snapshot #1
thread java.lang.Thread:{id:0,name:main,status:RUNNING,priority:5,isDaemon:false,lockCount:0,suspendCount:0}
call stack:
at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:650)

====================================================== results
error #1: gov.nasa.jpf.vm.NoUncaughtExceptionsProperty "java.lang.NoSuchMethodError: java.lang.invoke.JPFF..."

====================================================== search finished: 19/05/24, 1:07 am

@eklaDFF
Copy link
Author

eklaDFF commented May 21, 2024

The above particular problem can be solved if implement the java.lang.invoke.JPFFieldVarHandle.weakCompareAndSetRelease(...) method. But the issue here is that it is difficult to find the implementation.

Trace :
at java.util.concurrent.atomic.LongAdder.add(LongAdder.java:87)
Screenshot 2024-05-21 at 11 01 12 PM

at java.util.concurrent.atomic.Striped64.casBase(Striped64.java:181)
Screenshot 2024-05-21 at 11 03 21 PM
(DataType of BASE is VarHandle)
and then

Screenshot 2024-05-21 at 11 04 39 PM

(VarHandle is an abstract class)

@cyrille-artho
Copy link
Member

Implementing VarHandle operations that interact with how the memory model behavior is resolved is going to be challenging. We may have to interface with JPF's choice generators when resolving possible effects of these memory updates. For this, an implementation of these operations on another JVM would not help much.

I recommend looking at other issues first.

@eklaDFF
Copy link
Author

eklaDFF commented May 22, 2024

One of the other failed tests were due to repeat() method in String class. Implemented them and then below is the result. [implementation of the repeat() method is different from in OpenJDK and is very simple, and it might be a case of longer time complexity than OpenJDK)

Summary: 1002 tests, 988 passed, 14 failed, 0 skipped

1002 tests completed, 14 failed

> Task :test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.

So before committing it, should I create PR for CDS.java ? And then for String.repeat(...) ?

@cyrille-artho
Copy link
Member

Yes, please create a PR for the work on CDS first (without the changes in String), and another one for String.repeat afterwards.
(You can also create them in parallel, as the changes do not affect each other.)

@eklaDFF
Copy link
Author

eklaDFF commented May 24, 2024

PR #452

@eklaDFF
Copy link
Author

eklaDFF commented May 24, 2024

String.repeat() PR #453

@cyrille-artho
Copy link
Member

cyrille-artho commented May 24, 2024 via email

@eklaDFF
Copy link
Author

eklaDFF commented May 24, 2024

Next error we got was toShortSignature() in Method.java. We implemented them. You can see here : https://github.com/eklaDFF/jpf-core/tree/toShortSignatureBranch

then we got new error and that is :

gov.nasa.jpf.vm.NoUncaughtExceptionsProperty
java.lang.NoSuchMethodError: java.lang.Class.descriptorString()Ljava/lang/String;
	at sun.invoke.util.BytecodeDescriptor.unparseSig(BytecodeDescriptor.java:155)
	at sun.invoke.util.BytecodeDescriptor.unparseMethod(BytecodeDescriptor.java:144)
	at java.lang.invoke.MethodType.toMethodDescriptorString(MethodType.java:1190)
	at java.lang.reflect.ProxyGenerator$ProxyMethod.generateMethod(ProxyGenerator.java:721)
	at java.lang.reflect.ProxyGenerator.generateClassFile(ProxyGenerator.java:500)
	at java.lang.reflect.ProxyGenerator.generateProxyClass(ProxyGenerator.java:178)
	at java.lang.reflect.Proxy.getProxyClass(Proxy.java:73)
	at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:85)
	at gov.nasa.jpf.test.vm.reflection.ProxyTest.testBasicProxy(ProxyTest.java:60)
	at java.lang.reflect.Method.invoke(gov.nasa.jpf.vm.JPF_java_lang_reflect_Method)
	at gov.nasa.jpf.util.test.TestJPF.runTestMethod(TestJPF.java:648)

java.lang.Class.descriptorString() uses some native methods like isPrimitive(), isArray(), and isHidden().

As our JPF intercepts the native methods, how will we will have to implement these ?

//////////////////////////////////////////////////////////////////////
Current Status of failed cases :

Test Execution: FAILURE
Summary: 1002 tests, 988 passed, 14 failed, 0 skipped

1002 tests completed, 14 failed

> Task :test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.

@cyrille-artho
Copy link
Member

JPF loads its own copy of the classes and has its own view of information on whether a method is primitive, where a field is an array, etc.
Implementing descriptorString will take a bit of time studying the internals of JPF, but it is not as difficult as VarHandle. I think it is a good next step to look into how JPF stores information on the classes it has loaded, so you can produce a correct descriptor string step by step.
The descriptor string is also useful for debugging.

For this, you will interface with JPF internals, but not with the equivalent functions of the host JVM. This is because JPF already has data structures in its own VM where all the metadata of the code is accessible.

@cyrille-artho
Copy link
Member

For the fix in toShortSignature, please make a PR as well so I can see the outcome.

@eklaDFF
Copy link
Author

eklaDFF commented May 24, 2024

#454 PR for toShortSignature

@eklaDFF
Copy link
Author

eklaDFF commented May 25, 2024

Now, please let me know about the next step as it is now more complex to me. You said earlier that JPF deals its own way for information about primitive, non primitive or interface.

@cyrille-artho
Copy link
Member

Please look at src/main/gov/nasa/jpf/vm/MethodInfo.java. This class stores the key information about methods. Similar classes exist to store class and field data.

@eklaDFF
Copy link
Author

eklaDFF commented May 30, 2024

For our descriptorString() method in our model class String.java, src/main/gov/nasa/jpf/vm/MethodInfo.java was not required much but looked into similar class src/main/gov/nasa/jpf/vm/ClassInfo.java which was very useful. There was declaration of methods like isPrimitive() and isArray(). And also these methods were declared in class String.java

I have just implemented the descriptorString() like the OpenJDK to maintain the similarity in pattern of description. I will change if required in case of License isssue.

Screenshot 2024-05-30 at 8 41 31 AM Screenshot 2024-05-30 at 8 55 03 AM

IMPORTANT :
I have left the implementation of isHidden() method here in our JPF because Hidden class feature has been added in Java 15.
We will have to write it because when we will write unit test for Hidden class this will cause problem.

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Current Status for ./gradlew clean build

Test Execution: FAILURE
Summary: 1002 tests, 994 passed, 8 failed, 0 skipped

1002 tests completed, 8 failed

> Task :test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/ekla/GSOC/JPFjava17/jpf-core/build/reports/tests/test/index.html

BUILD FAILED in 2m 9s
20 actionable tasks: 20 executed

All the errors related to descriptorString() method have been solved.

@cyrille-artho
Copy link
Member

Looks good! Please make a PR so I can take a look at the code.
Also, for anything relating to a new feature/issue/task, please create a new issue, so the list of comments does not grow indefinitely.

@eklaDFF
Copy link
Author

eklaDFF commented May 30, 2024

PR #455

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants