Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible to use read_gbq() without bigquery.readsessions.create permissions? #340

Open
chrisperks opened this issue Jan 23, 2024 · 2 comments
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@chrisperks
Copy link

chrisperks commented Jan 23, 2024

Hi!

I'm using bigframes 0.19.2 and in particular, using pandas.read_gbq(..) to read an SQL statement to a Pandas DataFrame.

I don't have permissions to create a ReadSession on our BigQuery instance, so my query fails, reporting I don't have the relevant bigquery.readsessions.create permissions.

When using the regular BigQuery client, I can set the option create_bqstorage_client=False to get around this lack of permissions.

Can I use read_gbq in any similar way, or otherwise set options to remove the need for bigquery.readsessions.create permissions?

Thanks.
Chris.

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. label Jan 23, 2024
@tswast tswast added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Jan 23, 2024
@tswast
Copy link
Collaborator

tswast commented Jan 23, 2024

Thanks for the feedback @chrisperks. At the moment it is not possible to avoid using the BigQuery Storage Read API in BigQuery DataFrames to_pandas and other methods that could download data in bulk like __repr__ or .values.

Performance won't be ideal without the BQ Storage API, but it should be possible, as you have noted the BQ client library does support this.

I propose we add global use_bqstorage_read_api=True|False option to https://cloud.google.com/python/docs/reference/bigframes/latest/bigframes._config.bigquery_options.BigQueryOptions to avoid the BQ Storage API in I/O.

@chrisperks
Copy link
Author

Thanks for the feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

2 participants