Skip to content
This repository has been archived by the owner on Jan 22, 2019. It is now read-only.

Could not crate .avro file #42

Closed
srini-daruna opened this issue Mar 23, 2016 · 5 comments
Closed

Could not crate .avro file #42

srini-daruna opened this issue Mar 23, 2016 · 5 comments

Comments

@srini-daruna
Copy link

Hi,

I created avro schema from class and got the byte array and want to write the data to avro file.
How can i do that.?

@dsyer
Copy link

dsyer commented Apr 1, 2016

The internet is full of utilities to copy a byte array to a file (e.g. look at Apache commons-io FileUtils, and Spring also has one).

@srini-daruna
Copy link
Author

If i have written the byte array to file and used spark-avro or avro-tools, it is not recognizing the file as avro. I have used same schema and byte array from jackson. Probably, i might have not pointed my question in right direction..

@dsyer
Copy link

dsyer commented Apr 1, 2016

Probably. Maybe you could share some code, and explain what you were trying to do in more detail?

@cowtowncoder
Copy link
Member

@srini-daruna For better or worse, Avro dictates different format for files than network transport -- Avro files are supposed to have both prefix and pre-written schema as JSON. Currently this module only does "raw" encoded content, and does not have means to add wrapper stuff.
Addition of header and avro schema would be a good addition I think.

Note, however, than when transferring Avro content over the wire, these are not supposed to be used. I don't know if spark (et al) would even allow use; perhaps they try to auto-detect which mode is used.

@cowtowncoder
Copy link
Member

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants