Remove unused vars in sycl resource

[davidbeckingsale]

No description provided.

[trws]

This is freaking me out right now, because without the queue I don't think the event actually syncs anything. It's defined to create an already ready event. More oddly, there doesn't seem to be a sycl API for creating an event independently from running work on the device, looks like we'll have to pull the OpenCL queue out of the sycl queue and use clEnqueueMarkerWithWaitList to create an event that actually works.

That and add more tests, because there's no way this can possibly be working correctly.

[davidbeckingsale]

😂 yeah I just saw the unused stuff when I was building Umpire - we are not actually using any of the sycl resources at the moment. I think we probably need to run this by @homerdin ?

[trws]

Definitely, and need to get some dependency tests so we can actually verify all this stuff, I'll think about it. Maybe we can get something together that will be portable with a polymorphic resource? Not sure, but it would help.

[homerdin]

@trws Yeah, this event doesn't work currently (was focused on the resource). I think the main reason for the limited sycl event interface is from the out-of-order queue default. Relying on the OpenCL queue could create an issue when using level_zero.

If you expect camp events to only be used with the camp resources which have in-order queues we might be able to submit a do-nothing task to the queue and use that event (need to test). Otherwise creating an event to wait on everything in currently in the queue of an out-of-order queue might be non-deterministic.

I'm not familiar with all your use cases, would it be useful to have the camp sycl event store a set of dependencies that it needs to wait for? I'm not sure if having an option to build a find grained DAG would be useful.

[trws]

There are options to build a fine-grained dag at the raja level, but the semantics we provide from resources and events is that a resource represents an ordered queue. Does the level zero version now interoperate with the OpenCL interface? If that’s the case we might have a problem, but if it does then using the OpenCL marker support on an unordered queue works fine, it will wait on everything that was enqueued before the given point regardless of the settings on the queue.

…

On 13 Sep 2021, at 9:14, Brian Homerding wrote: @trws Yeah, this event doesn't work currently (was focused on the resource). I think the main reason for the limited sycl event interface is from the out-of-order queue default. Relying on the OpenCL queue could create an issue when using level_zero. If you expect camp events to only be used with the camp resources which have in-order queues we might be able to submit a do-nothing task to the queue and use that event (need to test). Otherwise creating an event to wait on everything in currently in the queue of an out-of-order queue might be non-deterministic. I'm not familiar with all your use cases, would it be useful to have the camp sycl event store a set of dependencies that it needs to wait for? I'm not sure if having an option to build a find grained DAG would be useful. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/LLNL/camp/pull/84*issuecomment-918352350__;Iw!!G2kpM7uM-TzIFchu!mtCGD2qasnRNvOhR2VTTNiZvrIWrJ6gwJFL7GkXohMz1H40pIwaV7uOWYQe2Bx66RA$

[trws]

@xtian-github, @RaviNarayanaswamy, sorry to bug you in an odd way for this, but it looks like we have a gap in the sycl api that concerns me a little bit. Is there a way to get an event handle that will wait on all tasks enqueued on a queue through dpc++? I know we can do this through OpenCL, but if for whatever reason that doesn't work on level zero or similar I don't see a way to do this through the actual sycl APIs. I'm asking on camp because this is where I ran into it, but this would be a problem for correct implementation of the OpenMP interop construct as well.

[xtian-github]

@trws do you mean to get native L0 event handle in DPC++? If so, the answer is yes. It is a feature we have just added.

[trws]

Not specifically, though that might work @xtian-github. The issue is that here in camp, and in interop for that matter, the logical model is that there is a way to wait on all the work that has been put onto a queue (or in the camp setup a resource) before a certain point. Basically what should go here to generate an event that will wait on everything previously submitted to qu.

As far as I can tell, the sycl::queue offers no way to create an event that does that. In OpenCL land I can create a marker event on the cl_queue object easily enough, and that works fine, but since the RAJA and camp backends are written in sycl and dpc++ doesn't seem to want to stick to OpenCL I'm not sure if that's a safe thing to do. If we have to do the OpenCL way in general and on DPC++ use L0 we could do that but I was hoping we could find a way to do this in the higher level APIs somehow, or at least keep it to one low-level API.

[xtian-github]

are you asking something like q.wait_all_events ?

[trws]

It looks like there's an extension proposal for this, that I somehow missed. We need q.submit_barrier() (proposal) to make this interface work.

To summarize it, we need to be able to get an object that we can later use to synchronize with all things that were enqueued before the object was created. The wait methods don't help us.

If this is in DPC++, which it seems to be, then I think the solution is we use submit_barrier on DPC++ and OpenCL's event marker elsewhere. That won't help us on hipsycl AFAICT, but I don't see any way to do it in current standard sycl.