weird issue with snakeyaml #101

glelouet · 2023-09-27T14:52:36Z

The following test class

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.nio.charset.Charset;

import org.testng.annotations.Test;
import org.yaml.snakeyaml.LoaderOptions;

import com.helger.jcodemodel.JCodeModel;
import com.helger.jcodemodel.JCodeModelException;
import com.helger.jcodemodel.JDefinedClass;
import com.helger.jcodemodel.JMod;
import com.helger.jcodemodel.writer.JCMWriter;
import com.helger.jcodemodel.writer.OutputStreamCodeWriter;

import fr.lelouet.tools.compilation.inmemory.DynamicClassLoader;

public class SnakeYamlImportErrorTest {

	@Test
	public void testCompiling()
			throws JCodeModelException, ClassNotFoundException, InstantiationException, IllegalAccessException,
			IllegalArgumentException, InvocationTargetException, NoSuchMethodException, SecurityException, IOException {
		JCodeModel jcm = new JCodeModel();
		JDefinedClass cl = jcm._class("Test");
		cl.field(JMod.PUBLIC, LoaderOptions.class, "options");

		try {
			DynamicClassLoader dcl = new DynamicClassLoader(SnakeYamlImportErrorTest.class.getClassLoader()).withCode(jcm);
			Class<?> compiled = dcl.loadClass("Test");
			compiled.getConstructor().newInstance();
		} catch (Exception e) {
			ByteArrayOutputStream baos = new ByteArrayOutputStream();
			OutputStreamCodeWriter acw = new OutputStreamCodeWriter(baos, Charset.defaultCharset());
			new JCMWriter(jcm).build(acw);
			throw new UnsupportedOperationException(baos.toString(), e);
		}
	}

}

fails on snakeyaml dependency 2.0 or 2.3 .

compile diagnostic //Test.java:1: error: package org.yaml.snakeyaml does not exist
import org.yaml.snakeyaml.LoaderOptions;
^
compile diagnostic //Test.java:4: error: cannot find symbol
public LoaderOptions options;
^
symbol: class LoaderOptions
location: class Test
FAILED: testCompiling
java.lang.UnsupportedOperationException: import org.yaml.snakeyaml.LoaderOptions;

public class Test {
public LoaderOptions options;
}

at SnakeYamlImportErrorTest.testCompiling(SnakeYamlImportErrorTest.java:36)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:134)
at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:597)
at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:173)
at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:816)
at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:146)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.testng.TestRunner.privateRun(TestRunner.java:766)
at org.testng.TestRunner.run(TestRunner.java:587)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:384)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:378)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:337)
at org.testng.SuiteRunner.run(SuiteRunner.java:286)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1187)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1109)
at org.testng.TestNG.runSuites(TestNG.java:1039)
at org.testng.TestNG.run(TestNG.java:1007)
at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:115)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:251)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:77)
Caused by: java.lang.ClassNotFoundException: Test
at java.base/java.lang.ClassLoader.findClass(ClassLoader.java:718)
at fr.lelouet.tools.compilation.inmemory.DynamicClassLoader.findClass(DynamicClassLoader.java:53)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at SnakeYamlImportErrorTest.testCompiling(SnakeYamlImportErrorTest.java:30)
... 28 more

It however passes on snakeyaml 1.33 , which has critical security issues

I checked and snakeyaml is not imported with the dependencies (besides testng). If I do a main the same problem arises.
It may be a problem with my dynamic classloader, or the way the class is available in the jar (require module ?)

glelouet · 2023-09-27T15:43:57Z

I tried using other libs, or using different class in snakeyam.

result is, issue only appears snakeyaml v2.0+ and with other classes in that lib too.

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.nio.charset.Charset;

import org.testng.Assert;
import org.testng.annotations.Test;
import org.yaml.snakeyaml.LoaderOptions;
import org.yaml.snakeyaml.Yaml;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.helger.jcodemodel.JCodeModel;
import com.helger.jcodemodel.JCodeModelException;
import com.helger.jcodemodel.JDefinedClass;
import com.helger.jcodemodel.JMod;
import com.helger.jcodemodel.writer.JCMWriter;
import com.helger.jcodemodel.writer.OutputStreamCodeWriter;

import fr.lelouet.tools.compilation.inmemory.DynamicClassLoader;

public class SnakeYamlImportErrorTest {

	public void compileAndTest(Class<?> fieldClass) throws JCodeModelException, IOException {
		JCodeModel jcm = new JCodeModel();
		JDefinedClass cl = jcm._class("Test");
		cl.field(JMod.PUBLIC, fieldClass, "field");

		try {
			DynamicClassLoader dcl = new DynamicClassLoader(SnakeYamlImportErrorTest.class.getClassLoader()).withCode(jcm);
			Class<?> compiled = dcl.loadClass(cl.name());
			compiled.getConstructor().newInstance();
		} catch (Exception e) {
			ByteArrayOutputStream baos = new ByteArrayOutputStream();
			OutputStreamCodeWriter acw = new OutputStreamCodeWriter(baos, Charset.defaultCharset());
			new JCMWriter(jcm).build(acw);
			throw new UnsupportedOperationException(baos.toString(), e);
		}
	}

	@Test
	public void testCompilingLoader() throws JCodeModelException, IOException {
		compileAndTest(LoaderOptions.class);
	}

	@Test
	public void testCompilingYaml() throws JCodeModelException, IOException {
		compileAndTest(Yaml.class);
	}

	@Test
	public void testCompilingString() throws JCodeModelException, IOException {
		compileAndTest(String.class);
	}

	@Test
	public void testCompilingJCM() throws JCodeModelException, IOException {
		compileAndTest(JCodeModel.class);
	}

	@Test
	public void testCompilingJackson() throws JCodeModelException, IOException {
		compileAndTest(ObjectMapper.class);
	}

	@Test
	public void directAccess()
			throws JCodeModelException, ClassNotFoundException, InstantiationException, IllegalAccessException,
			IllegalArgumentException, InvocationTargetException, NoSuchMethodException, SecurityException, IOException {
		JCodeModel jcm = new JCodeModel();
		jcm._class("Test");

		DynamicClassLoader dcl = new DynamicClassLoader(SnakeYamlImportErrorTest.class.getClassLoader()).withCode(jcm);
		Class<?> lcl = dcl.loadClass(Yaml.class.getName());
		Assert.assertNotNull(lcl);
		Class<?> lcl2 = dcl.loadClass(LoaderOptions.class.getName());
		Assert.assertNotNull(lcl2);
	}

}

When I switch to snakeyaml v1.33 all tests pass ; when I switch to 2.0

[ERROR] Failures:
[ERROR] SnakeYamlImportErrorTest.testCompilingLoader:42->compileAndTest:36 UnsupportedOperation import org.yaml.snakeyaml.LoaderOptions;

public class Test {
public LoaderOptions field;
}

[ERROR] SnakeYamlImportErrorTest.testCompilingYaml:47->compileAndTest:36 UnsupportedOperation import org.yaml.snakeyaml.Yaml;

public class Test {
public Yaml field;
}

phax · 2023-09-27T15:48:05Z

Is it with head revision or the 3.x version?

glelouet · 2023-09-27T15:48:57Z

jcodemodel version is 3.4.1

I keep that class in my compiler to be sure the snakeyaml version allows to compile( when there is a huge number of classes to compile, I don't want it to fail at compile time)

glelouet · 2023-09-27T15:50:05Z

Note that it could as well be an issue with my own dynamicClassLoader code. I tried to find it in your source but could not :'( :'( :'( though I remembered having submitted a patch in that regard.

phax · 2023-09-27T15:51:09Z

Yes I know - I didn't do much in this project lately - and the proposed changes were so huge I couldn't understand them properly without investing more time.
Maybe Snakeyaml 2.x needs a later JVM version?

glelouet · 2023-09-27T15:51:21Z

I checked and there was no update here

https://mvnrepository.com/artifact/com.helger/jcodemodel

(I just spent a lot of time merging all my dependencies management in the same pom)

glelouet · 2023-09-27T15:55:15Z

sakeyaml uses java17 (which was specified in my projet java.version=17 ).
I tried bumping java.version=18, no positive effect.

phax · 2023-09-27T15:56:42Z

Snakeyaml 2.2 uses 1.7 according to https://search.maven.org/artifact/org.yaml/snakeyaml/2.2/bundle
As long as it is 1.7 and not 17 we should be okay

phax · 2023-09-27T15:59:16Z

Sorry can't help you out in any more detail - I an in front of a 4 week leave and need to finish tons of other things first :( Back in November. Good luck

glelouet · 2023-09-27T16:06:39Z

No problem :)
I've had this for a year now, so it's not important. Just wanted to be sure where it came from, so I isolated it.

java17 is the version specified in the root pom. Now java 21 :

	<properties>
		<java.version>21</java.version>
		<maven.compiler.source>${java.version}</maven.compiler.source>
		<maven.compiler.target>${java.version}</maven.compiler.target>
	</properties>
	<build>
		<pluginManagement>
			<plugins>
				<!-- update https://mvnrepository.com/artifact/org.apache.maven.plugins -->
				<plugin>
					<groupId>org.apache.maven.plugins</groupId>
					<artifactId>maven-compiler-plugin</artifactId>
					<version>3.11.0</version>
					<configuration>
						<release>${java.version}</release>
					</configuration>
				</plugin>

This is the newish (3years now?) way to specify java version
as shown by maven :

[INFO] --- maven-compiler-plugin:3.11.0:testCompile (default-testCompile) @ compiler ---
[INFO] Changes detected - recompiling the module! :dependency
[INFO] Compiling 3 source files with javac [debug release 21] to target/test-classes

Still same errors :

[ERROR] Failures:
[ERROR] SnakeYamlImportErrorTest.testCompilingLoader:42->compileAndTest:36 UnsupportedOperation import org.yaml.snakeyaml.LoaderOptions;

public class Test {
public LoaderOptions field;
}

[ERROR] SnakeYamlImportErrorTest.testCompilingYaml:47->compileAndTest:36 UnsupportedOperation import org.yaml.snakeyaml.Yaml;

public class Test {
public Yaml field;
}

stale · 2024-01-07T11:56:10Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

phax self-assigned this Sep 27, 2023

stale bot added the wontfix label Jan 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

weird issue with snakeyaml #101

weird issue with snakeyaml #101

glelouet commented Sep 27, 2023

glelouet commented Sep 27, 2023 •

edited

Loading

phax commented Sep 27, 2023

glelouet commented Sep 27, 2023

glelouet commented Sep 27, 2023

phax commented Sep 27, 2023

glelouet commented Sep 27, 2023

glelouet commented Sep 27, 2023

phax commented Sep 27, 2023 •

edited

Loading

phax commented Sep 27, 2023

glelouet commented Sep 27, 2023

stale bot commented Jan 7, 2024

weird issue with snakeyaml #101

weird issue with snakeyaml #101

Comments

glelouet commented Sep 27, 2023

glelouet commented Sep 27, 2023 • edited Loading

phax commented Sep 27, 2023

glelouet commented Sep 27, 2023

glelouet commented Sep 27, 2023

phax commented Sep 27, 2023

glelouet commented Sep 27, 2023

glelouet commented Sep 27, 2023

phax commented Sep 27, 2023 • edited Loading

phax commented Sep 27, 2023

glelouet commented Sep 27, 2023

stale bot commented Jan 7, 2024

glelouet commented Sep 27, 2023 •

edited

Loading

phax commented Sep 27, 2023 •

edited

Loading