allow execution of queries without prepared statements

[tdawber]

Execute queries as prepared statements by default, but allow the option of executing them as standard statements.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling a1f8d3c on tdawber:non-prepared-statements into c2a160f on baztian:master.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling a1f8d3c on tdawber:non-prepared-statements into c2a160f on baztian:master.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling a1f8d3c on tdawber:non-prepared-statements into c2a160f on baztian:master.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling a1f8d3c on tdawber:non-prepared-statements into c2a160f on baztian:master.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling a1f8d3c on tdawber:non-prepared-statements into c2a160f on baztian:master.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling a1f8d3c on tdawber:non-prepared-statements into c2a160f on baztian:master.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling 935beae on tdawber:non-prepared-statements into c2a160f on baztian:master.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling 935beae on tdawber:non-prepared-statements into c2a160f on baztian:master.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling 935beae on tdawber:non-prepared-statements into c2a160f on baztian:master.

[coveralls]

Coverage increased (+0.3%) to 78.49% when pulling 935beae on tdawber:non-prepared-statements into c2a160f on baztian:master.

[tdawber]

@baztian, could you have a look over this and see if it's better suited than the other approach? Ta!

[baztian]

Thanks again for you pull request. I was proposing to change the connect method signature not the execute method signature. Or do you know dbs or situations where one statement should be prepared and the other not?

[tdawber]

There's an overhead in first preparing, then executing the query, both client-side and database-side. It's not big, but if you're firing off a lot of queries, it can have a cost, so I think being able to choose on a query-by-query basis whether you'll use prepared statements or not might be beneficial?

I ran a few simple tests using jaydebeapi and a local hsql database, and prepared statements were consistently 5% slower than direct queries.

So, personally, I'd prefer to choose the more appropriate option on a query-by-query basis.

[baztian]

Thanks for the clarification. I think it depends on the database and on the type of query you run and how often you run it. So your solution to have a choice at statement execution time seems to be good. Anyways it is not db-api standard.
MySql seems to have the parameter prepared=True at the cursor() method. Can you please have a look at other DB drivers and find if the is something like a de factor standard?
Also an interesting read: http://stackoverflow.com/questions/4614131/should-i-always-prefer-working-with-prepared-sql-statements-for-performance-ben

[baztian]

There is no support (and no fallback or warning) if parameters have been supplied.

[toby-coleman]

If you are using a library like Pandas to run SQL queries, it would be nice to be able to change this option on the connect method. Reason being that the execute method is called inside Pandas, so in this scenario you can't change it when running an individual query.

[baztian]

Interesting references: https://cx-oracle.readthedocs.io/en/latest/cursor.html?highlight=prepare
https://www.ibm.com/support/knowledgecenter/en/SSEPGG_9.5.0/com.ibm.db2.luw.apdv.python.doc/doc/t0054696.html
https://stackoverflow.com/questions/27649759/using-prepared-statements-with-mysql-in-python
https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursorprepared.html

[raelik]

I had just opened another PR similar to this (it simply used a regular statement when no parameters are given), but I closed it and figured I'd just add to the discussion here. Not all database drivers give the option to disable server-side prepared statements, and we've run into specific issues with Vertica when using some of their internal "procedures" within a prepared statement (Vertica doesn't have stored procedures. What you would normally do as one is instead implemented as a function with a side effect. Gross.)

[raelik]

I should also note that besides adding a level of protection against SQL injection, prepared statements are often used for performance purposes when the same query is being executed multiple times with different parameters (executemany() does this). But with JDBC, that requires that the original PreparedStatement object be used, which isn't possible in execute() due to how the Cursor closes and recreates the statement every time. You could achieve this by using a dictionary to hold the prepared statements, with the SQL acting as the key. This is really a completely separate topic though, but my point is that there is absolutely no reason, with the way that the Cursor currently implements execute(), to use a prepared statement there at all if there are no parameters being passed. executemany() is a different story, but you wouldn't likely be using that without parameters.

[koljamaier]

I have the exact same setup as @fmcmac with Drill. Is there a working solution with a parameter like use_prepared_statements, that won't wrap my queries with SELECT *...?

[thomaskelm]

Has there been any update on this Pull request? I'm also using the same Drill setup and you like to be able to run queries besides select statements

[jbelina]

It's been over 2 years since the last request for an update on this PR.

Can this update either be applied, or at least add a comment with the reason why it will not be merged in?

[keweishang]

@baztian We also run into this issue when PREPARE isn't needed. I think either passing this flag via the connect function or via the execute function would be great. I slightly prefer to use connect function. Please let us know if this PR or a new PR can be approved and merged? Thank you :)