Feeding data from remote

[mguijarr]

Hello,

I have a quick question: I have an x-ray detector producing 2D images (= tensors), running on a host. Vineyard is running in another
host. What is the best way to send the data arrays to vineyard ? As far as I understand I cannot use an endpoint because it only works via UNIX domain socket. So I was thinking to manage data with a producer/consumer model, based on Apache Arrow Flight. The producer process on the detector host would send the data via Flight, the consumer would put it in Vineyard.

Would it be an ok approach like this ?

I found this MR: #215 , which seems to address the problem but I could not find how to use it.

Thanks in advance !

[sighingnow]

Hi @mguijarr,

I found this MR: #215 , which seems to address the problem but I could not find how to use it.

I think feed the data into vineyard using RPC directly would be more convenient in the case above.

I would try to add a tuturial about building objects using RPC client in Python later today.

[sighingnow]

I have open a issue #818 to track the progress.

[mguijarr]

Thanks a lot @sighingnow 😃

[sighingnow]

Hi @mguijarr,

I would let you know that we have just cut a new release vineyard-v0.6.1 that includes the API for blob creation and accessing via the RPC client.

We have a brief introduction about the usages in https://v6d.io/notes/data-accessing.html and https://v6d.io/notes/streams.html. Hope that could be helpful for you.

[mguijarr]

Thanks a lot !