How preallocate buffer through rendezvous protocol before ucp_tag_recv_nbx actually receiving?

[MoFHeka]

It seams there is no api about rendezvous in UCP. And there is no clear document or example for UCT.

[ivankochin]

Hi, what do you mean "preallocate buffer through rendezvous protocol"? AFAIK you need to pass preallocated buffer to ucp_tag_recv_nbx as second argument.

To better understand how do UCP or UCT API works you can check examples

[MoFHeka]

I would like to transmit the sparse tensors, and there is no way for two workers to know the size of the data in advance.

I don't know if I understand this correctly. In rendezvous protocol, ucp p2p sends the size of the data to be transmitted before the actual transmission begins. So when I use ucp, could I allocate a buffer based on the size of the rendezvous data? And then I pass this buffer into ucp_tag_recv_nbx.

Or is there a tag_recv API which can feed a allocator into.

I guess I need to refer to this to write my own API, right?
https://github.com/openucx/ucx/blob/049eed4d7f5f0f8388054fcf068813427d6287e8/src/ucp/tag/tag_rndv.h

[yosefe]

@MoFHeka for such case, it is recommended to use the active message API.
The sender can send its tensor using ucp_am_send_nbx, then a callback will be called on receiver with the length of the data and a "data" token. The callback can allocate a receive buffer and pass that token to ucp_am_recv_data_nbx, and complete the rendezvous data transfer. See https://github.com/openucx/ucx/blob/841b1b3/examples/ucp_client_server.c#L560 as example