Support for systems with different database schemas

[victorhugofr]

Hi @arcuri82,

I'm facing this error when trying to generate whitebox test cases with EvoMaster:

"ERROR - Failed to extract the SQL Database Schema: null java.lang.NullPointerException"

In my analysis, this happens because EvoMaster is only prepared to read tables from the database that are located in the "public" schema. In the system I'm trying to run, the entities are in various schemas, causing it to attempt to find a table in another schema within the "public" schema.

Is there any support for different schemas? Is it already on the radar? Or could it be some kind of misconfiguration?

Version of EM: 1.5.0
Running from JAR
Command-line options: java -jar evomaster.jar --outputFolder src/test/java
Database: PostgreSQL 14

[arcuri82]

Hi @victorhugofr ,
thanks for reporting this issue. I am not an expert in PostgreSQL, so it might be a misunderstanding on our part.
Can you provide some more details?

For example, how many different schemas you have and their name? are they all accessed with the same JDBC connection? How is the database structured / initialized?

is connection.getSchema() returning null here? or throwing an exception?

is the API you working on open-source, or closed-source?

thanks

[victorhugofr]

Hi @victorhugofr , thanks for reporting this issue. I am not an expert in PostgreSQL, so it might be a misunderstanding on our part. Can you provide some more details?

For example, how many different schemas you have and their name? are they all accessed with the same JDBC connection? How is the database structured / initialized?

is connection.getSchema() returning null here? or throwing an exception?

is the API you working on open-source, or closed-source?

thanks

Thanks for the reply,
The system has 5 schemas, as shown below:

They are automatically generated when the system is started with Hibernate/JDBC.

The API is closed-source (but I have access to the code).

Some important notes:
When I tried to run EvoMaster, the system returned an error indicating that it couldn't find the database. It is was launched in a new Docker environment, so the database didn't exist. Solution: after starting http://localhost:40100/controller/api, I entered the Docker container and ran the script "

psql -U postgres;" CREATE DATABASE databasename; \c databasename;
CREATE SCHEMA IF NOT EXISTS project AUTHORIZATION postgres;
CREATE SCHEMA IF NOT EXISTS public AUTHORIZATION postgres;
CREATE SCHEMA IF NOT EXISTS controle_acesso AUTHORIZATION postgres;
CREATE SCHEMA IF NOT EXISTS auditoria AUTHORIZATION postgres;

"CREATE SCHEMA" is because if I don't run this, the system doesn't create the tables correctly
and only then ran EvoMaster.

connection.getSchema() is returning "public" always

[victorhugofr]

In an attempt to mitigate the errors, I moved all entities to the 'public' schema.
Now, EvoMaster is unable to connect to the database, with the following error:

[victorhugofr]

In parallel, I attempted to run the blackbox, and the following error was returned:

java.lang.IllegalStateException: LimitObjectGene has no value
        at org.evomaster.core.search.gene.LimitObjectGene.getValueAsPrintableString(LimitObjectGene.kt:32)
        at org.evomaster.core.search.gene.Gene.getValueAsPrintableString$default(Gene.kt:267)
        at org.evomaster.core.search.gene.OptionalGene.getValueAsPrintableString(OptionalGene.kt:143)
        at org.evomaster.core.search.service.FitnessFunction.calculateIndividualWithPostHandling(FitnessFunction.kt:104)
        at org.evomaster.core.search.service.FitnessFunction.calculateCoverage(FitnessFunction.kt:56)
        at org.evomaster.core.search.service.FitnessFunction.calculateCoverage$default(FitnessFunction.kt:48)
        at org.evomaster.core.search.algorithms.RandomAlgorithm.searchOnce(RandomAlgorithm.kt:25)
        at org.evomaster.core.search.service.SearchAlgorithm.search(SearchAlgorithm.kt:66)
        at org.evomaster.core.Main$Companion.run(Main.kt:437)
        at org.evomaster.core.Main$Companion.initAndRun(Main.kt:150)
        at org.evomaster.core.Main$Companion.main(Main.kt:74)
        at org.evomaster.core.Main.main(Main.kt)

Command used: java -jar evomaster.jar --blackBox true --bbSwaggerUrl http://localhost:8080/project/v2/api-docs --outputFormat JAVA_JUNIT_4 --maxTime 30s --ratePerMinute 60

[arcuri82]

Hi,
thanks. A few things:

1. we have not tried yet multi-schemas... so that is definitively something we should investigate, and try to support

2. running that script shouldn't be necessary, as long as you open the connection after the API is started and initialized. Can you paste your EvoMaster driver/controller implementation here? (or send it to me privately if that is not possible)

3. the last point for black-box testing is a bug. But you used an old version, ie, 1.5.0. Can you check if you still get the same issue with latest 1.6.1? if yes, could you please open a different issue for it?

thanks!

[arcuri82]

also, if you are testing your own API on localhost, you do not need --ratePerMinute

[arcuri82]

for the time being, you can also try to disable SQL support, e.g. with --generateSqlDataWithSearch false --heuristicsForSQL false

[victorhugofr]

por enquanto, você também pode tentar desabilitar o suporte SQL, por exemplo, com--generateSqlDataWithSearch false --heuristicsForSQL false

Thank you very much for the support, it seems that we've made progress:
now, the execution returned a bug very similar to the blackbox version:

I will be opening a bug right now. #813