Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPU node should request multiple jobs at once #11

Open
rvencu opened this issue Jul 12, 2021 · 3 comments
Open

GPU node should request multiple jobs at once #11

rvencu opened this issue Jul 12, 2021 · 3 comments

Comments

@rvencu
Copy link
Contributor

rvencu commented Jul 12, 2021

I need the ability to request multiple jobs at once from the GPU node such as

client.newJob(16)

to get a bundle of 16 jobs. this can be individual or composed shards as per #10 i.e. either 16 individual jobs or less when composed jobs are among them, while 8 composed jobs would be the minimum response to the above request

I guess client.downloadShard() would become client.downloadShards() or client.downloadJob() as for GPU node it makes no sense the naming downloadShard

@TheoCoombes
Copy link
Owner

Similarly to my response on #10, I'll add this to my todo list - it doesn't seem too hard to achieve.

@rvencu
Copy link
Contributor Author

rvencu commented Jul 12, 2021

for this client.invalidURL() we need a more elaborated protocol

  • say I request 16 shards
  • I try the download links, some fail
  • I send invalidURL() for those and I need to get new shards instead
  • looping the above until all 16 are successful or out of shards condition appears then continue to process them

I think this should be done at my end, if you provide a way to request a number of jobs (and limit that to 20 or so to avoid bad requests )

the problem to be solved here is, will you allocate 16 jobs at once to one client? when I send invalidURL I should specify shard id somehow

@rvencu
Copy link
Contributor Author

rvencu commented Jul 13, 2021

something like client.invalidURL('1b9437eb5f5144b9aeeae13d85909bb2')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants