Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] document workaround for CSV files with Hive connector #23973

Open
steveburnett opened this issue Nov 7, 2024 · 4 comments
Open

[docs] document workaround for CSV files with Hive connector #23973

steveburnett opened this issue Nov 7, 2024 · 4 comments

Comments

@steveburnett
Copy link
Contributor

Create table int type column with CSV format in hive catalog returns the following error

presto> create table  hive_data.hive_schema.intcsv ( type int ) with ( format = 'CSV' ) ;
Query failed: Hive CSV storage format only supports VARCHAR (unbounded). Unsupported columns: type integer

From @imjalpreet:

The limitation is due to the serde that Hive/Presto use for CSV, which is OpenCSVSerde. This serde deserializes columns from a CSV file into strings only. Therefore, when creating a CSV table in Presto, the columns can only have an unbounded varchar datatype.

Meanwhile, Trino describes it uses OpenCSVSerde

Trino / Hive :
https://trino.io/docs/current/connector/hive.html

CSV - using org.apache.hadoop.hive.serde2.OpenCSVSerde.

Expected Behavior or Use Case

Add documentation of the limitation and the workaround to the Hive Connector doc.

Workaround

  • create table in varchar
  • create view that cast the columns to their original datatypes.

Presto Component, Service, or Connector

Documentation.

Possible Implementation

Context

Help users understand how to use CSV files with the Hive Connector, due to the limitation of the serde that Hive/Presto uses for CSV.

@steveburnett
Copy link
Contributor Author

@imjalpreet, please review, comment, and edit this issue to correct any errors that I made writing it.

@imjalpreet
Copy link
Member

@steveburnett The description looks good.

@AsfarHorani
Copy link

Hi @steveburnett I am interested to work on it

@steveburnett
Copy link
Contributor Author

Hi @steveburnett I am interested to work on it

Sure, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🆕 Unprioritized
Development

No branches or pull requests

3 participants