Features planned for release

[gaborcsardi]

Definitely needed:

• Improve tests #4 (easy)
• Read & write dates, times, related types #8 (medium)
• Improve writing logicals #7 (easy)
• Support missing values in writer #6 (medium)
• Snappy compression in writer #5
• Better error messages #9 (easy)
• Read & write factors? #12 (some hard decisions)
• read_parquet is broken for boolean columns #15
• Test writer with larger files (easy)
• Decide on package name -> nanoparquet
• API: read_parquet() or parquet_read() or both? ... read_parquet() it is.
• Support dictionary pages in writer. Eg. for strings. Needed to keep ordering of factor levels. (hard) #18
• Writer: use smaller page sizes? (medium) #25
• Docs, pkgdown + ~~an article~~ #27

Would be nice:

• Function to read only the metadata #10 (medium)
• Convert strings to UTF-8 before writing (easy) #20
• Better printing of data frames (easy)
• Support big-endian platforms (hard) #21
• Read a subset of columns #39
• Dictionary encode other data types (i.e. not only factors)
• Choose automatically between dictionary and plain encoding

Optional:

• Improve reading metadata (e.g. DuckDB reads Parquet files 1 byte at a time (also bigger chunks) duckdb/duckdb#2051) it is fine for now
• LZO compression in reader #40
• ZSTD compression in reader #41
• LZO compression in writer #51
• ZSTD compression in writer #50
• Checksumming in writer #49
• Checksumming in reader #42
• Write statistics #43
• Read statistics #44
• Set parquet version better in writer #45
• Read from connection #46
• Write to connection #47
• Allow specifying column types #52
• Read a subset of rows #48

[gaborcsardi]

I am going to close this and open another one with the plans for the next release.