Listing jobs

[pmenglund]

I've been looking for a way to list the jobs in the queues, but not found any way to do it apart from running a SQL query against Postgres. Is this something which should be supported in the river client, or is that supposed to be done through a subscription?

[brandur]

Hi Martin,

This is a feature we haven't thought about in depth — I was assuming to some degree at least that programs making use of River would have their own ORM or something like sqlc operating already and could examine things in the database that way according to their specific requirements.

That said, it'd probably make a decent client addition. A few things we'd have to think about:

• To make it infinitely flexible for anyone who'd want to use something like that, we'd have to include a lot of different filter options. Not insurmountable, but a light argument against it in that users who just write their own sqlc query can include whatever filter conditions they'd like.
• Would have to think about the default limit. If the limit defaulted to infinite then an easy way to accidentally lock a client and DB would be to list jobs on a gigantic jobs table.
• We may want to include an automatic pagination mechanism so that it's easy to list in batches and still iterate the entire jobs table.

To anyone else reading this: feel free to use the upvote emoticon to indicate interest in the feature so we know how to prioritize.

[brandur]

Thanks Martin. I think the API you proposed there looks pretty solid. Let me think about it a little more and get back to you on this one.

[bgentry]

@pmenglund just wanted to drop a note that I’ve been working on this and should have something to share in the next few days. Dynamic filtering and sorting is basically an anti pattern for sqlc so I’ve gone a different route.

[bgentry]

@pmenglund I have an initial version of this up in #117 — definitely interested in if this does what you're looking for!

[pmenglund]

This should take care of one of the things I want to do, but I also want to list the queues (which kinda can be done using ListJobs(), as long as there is a job in some state). I want to display all queues, with a list of the jobs for each queue, so I can display a simple dashboard for my application.

[DavidS-ovm]

As a happy sqlc user, I have no issues reaching directly into the database for this. Dealing with future river migrations will be ... interesting anyways.

[brandur]

@pmenglund As of River 0.0.17 a jobs list API is now available:

https://pkg.go.dev/github.com/riverqueue/river#Client.JobList

Try it out and let us know if it's roughly what you had in mind. Going to close this discussion as resolved for the time being.