[query] Expose references via ExecuteContext

[ehigham]

This change is split out from a larger refactoring effort on the various Backend
implementations. The goals of this effort are to provide query-level
configuration to the backend that's currently tied to the lifetime of a backend,
reduce code duplication and reduce state duplication.

In this change, I'm restoring references to the execute context [1] and
decoupling them from the backend. In a future change, they'll be lifted out of
the backend implementations altogether. This is to reduce the surface area of
the Backend interface to the details that are actually different.

Both the Local and Spark backend have state that's manipulated from python via
various py methods. These pollute the Backend interface [2] and so have been
extracted into the trait Py4JBackendExtensions. In future changes, this will
become a facade that owns state set in python.

Notes
[1] "Restoring" old behaviour I foolishly removed in fe5ed32
[2] "Pollute" in that they obfuscate what's different about backend query plan
and execution

[ehigham]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• [query] Lift backend state into {Service|Py4J}BackendApi #14698
• [query] Move LoweredTableReaderCoercer into ExecuteContext #14696
• [query] unify backend rpc #14693
• [query] Remove persistedIr from Backend interface #14692
• [query] Remove lookupOrCompileCachedFunction from Backend interface #14691
• [query] Remove BlockMatrix persist from Backend interface #14690
• [query] Expose references via ExecuteContext #14686 👈
• [query] Unify CloudCredentials and Modernise BatchService API #14684
• [query] Use sourcecode.Enclosing to handle timed blocks implicitly #14683
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @ehigham and the rest of your teammates on Graphite