Is it possible to make Dagster work like an event-driven workflows?

[sqkkyzx]

Hello,

I apologize for posing this question, as it seems Dagster is not primarily designed for real-time event-driven workflows. However, after a brief month of use, Dagster has certainly captivated me with its comfort and ease of use. I am reluctant to leave it behind in search of another task orchestration framework.

If I record events into a database and set the sensor's minimum_interval_seconds to 1, checking the database for new events every second, would this be a dangerous setup?

Or if not using such frequent sensors, is there a very simple way to trigger a job from the outside? I'm not looking for the CLI, but rather, is there a WebHook or REST API that allows a scheduler to run a job?
Thank you for your time and assistance.

[garethbrickman]

I don't think 1 second sensor evaluations would be feasible. Also note from in the docs about evaluation intervals:

It's important to note that this interval represents a minimum interval between runs of the sensor and not the exact frequency the sensor runs. If you have a sensor that takes 30 seconds to complete, but the minimum_interval_seconds is 5 seconds, the fastest Dagster daemon will run the sensor is every 35 seconds. The minimum_interval_seconds only guarantees that the sensor is not evaluated more frequently than the given interval.

The GraphQL API can be used to launch job executions and re-executions, allowing users to trigger executions on custom events. Check out #21131 for examples.