Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database verdictdbtemp already exists Exception in SelectQueryCoordinator #352

Open
sofiazxy opened this issue Mar 5, 2019 · 1 comment

Comments

@sofiazxy
Copy link

sofiazxy commented Mar 5, 2019

When attempting to execute the following 2 VerdictDB queries back to back:
VerdictSingleResult result = verdict.sql("select count() from mydefault.lineitem_scramble");
VerdictSingleResult result1 = verdict.sql("select count(
) from mydefault.lineitem_scramble where l_linenumber>2");

Encountered the following error:

rg.verdictdb.exception.VerdictDBDbmsException: Issued the following query: create schema verdictdbtemp
org.apache.hadoop.hive.metastore.api.AlreadyExistsException: Database verdictdbtemp already exists;

at org.verdictdb.connection.SparkConnection.executeSingle(SparkConnection.java:166)
at org.verdictdb.connection.SparkConnection.execute(SparkConnection.java:148)
at org.verdictdb.connection.CachedDbmsConnection.execute(CachedDbmsConnection.java:49)
at org.verdictdb.connection.DbmsConnection.execute(DbmsConnection.java:41)
at org.verdictdb.coordinator.SelectQueryCoordinator.process(SelectQueryCoordinator.java:128)
at org.verdictdb.coordinator.ExecutionContext.streamSelectQuery(ExecutionContext.java:400)
at org.verdictdb.coordinator.ExecutionContext.sqlSelectQuery(ExecutionContext.java:323)
at org.verdictdb.coordinator.ExecutionContext.sql(ExecutionContext.java:166)
at org.verdictdb.VerdictContext.sql(VerdictContext.java:388)
at org.verdictdb.connection.SparkQueryResultTest.testSparkQueryResultValues(SparkQueryResultTest.java:139)
@sofiazxy
Copy link
Author

sofiazxy commented Mar 5, 2019

Here's the problem diagnosis:
The Temporary schema gets created in SelectQueryCoordinator and problem is here:

    // create scratchpad schema if not exists
    if (!conn.getSchemas().contains(scratchpadSchema)) {
      log.info(
          String.format(
              "The schema for temporary tables (%s) does not exist; so we create it.",
              scratchpadSchema));
      CreateSchemaQuery createSchema = new CreateSchemaQuery(scratchpadSchema);
      conn.execute(createSchema);
    }```
so for Spark, 1) the check in the if statement is not working correctly; and 2) createSchema does not use `IF NOT EXISTS` hence causing the error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant