You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm very new to dagster platform - and am facing some difficulty in running job in dagit UI using cron scheduler. The job is running fine on Launchpad (and also from python script), however there are problems when I use scheduler. The job just won't run. I must be missing something and looking for some pointers to solve the problem.
This is what I've done so far:
In the job file in the 'jobs' directory, it is just calling a series of ops.
Stack Trace:
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/graphql/execution/executor.py", line 452, in resolve_or_error
return executor.execute(resolve_fn, source, info, **args)
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/graphql/execution/executors/sync.py", line 16, in execute
return fn(*args, **kwargs)
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/dagster_graphql/schema/schedules/schedules.py", line 101, in resolve_futureTicks
tick_times.append(next(time_iter).timestamp())
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/dagster/utils/schedules.py", line 27, in schedule_execution_time_iterator
check.invariant(len(cron_parts) == 5)
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/dagster/check/init.py", line 172, in invariant
raise CheckError("Invariant failed.")
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I'm very new to dagster platform - and am facing some difficulty in running job in dagit UI using cron scheduler. The job is running fine on Launchpad (and also from python script), however there are problems when I use scheduler. The job just won't run. I must be missing something and looking for some pointers to solve the problem.
This is what I've done so far:
In the job file in the 'jobs' directory, it is just calling a series of ops.
@job def user_and_zipcode_job():
In the Scheduler file (with cron schedule to run every 2 min):
@schedule(cron_schedule="*/2 * * * *", job=user_and_zipcode_job, execution_timezone="US/Pacific")
def user_and_zipcode_schedule(context):
return {
"ops": {
"get_random_user": {
"config": {"url": "https://randomuser.me/api/"}
},
"get_zipcode": {
"config": {"zipcode-url": "https://api.zippopotam.us/us/95391"}
},
"save_random_user_tofile": {
"config": {"random_user_log_dir": "./random_user/random_user_file"}
},
"save_zipcode_tofile": {
"config": {"zipcode_log_dir": "./zipcode/zipcode_file"}
}
}
}
And in the repository file:
@repository
def random_user_zipcode_repository():
"""Collection of user_and_zipcode jobs"""
return {
"jobs": {
"user_and_zipcode_job": user_and_zipcode_job,
},
"schedules": {"user_and_zipcode_schedule": user_and_zipcode_schedule},
}
In workspace.yaml:
load_from:
- python_file:
relative_path: repositories/user_zipcode_repository.py
working_directory: .
From the command line I run the command
$ dagit
I can see the Scheduler and toggle it to start. And then nothing happens (I was expecting to see the runs in Launchpad).
I also see this message when I click 'Scheduler' link in random_user_zipcode_repository@user_zipcode_repository.py :
Operation name: SchedulesRootQuery
Message: Invariant failed.
Path: ["repositoryOrError","schedules",0,"futureTicks"]
Locations: [{"line":92,"column":3}]
Stack Trace:
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/graphql/execution/executor.py", line 452, in resolve_or_error
return executor.execute(resolve_fn, source, info, **args)
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/graphql/execution/executors/sync.py", line 16, in execute
return fn(*args, **kwargs)
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/dagster_graphql/schema/schedules/schedules.py", line 101, in resolve_futureTicks
tick_times.append(next(time_iter).timestamp())
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/dagster/utils/schedules.py", line 27, in schedule_execution_time_iterator
check.invariant(len(cron_parts) == 5)
File "/Users/t316538/Documents/Olive/tests/dagster2/dgs2-env/lib/python3.9/site-packages/dagster/check/init.py", line 172, in invariant
raise CheckError("Invariant failed.")
Thank you all in advance.
Beta Was this translation helpful? Give feedback.
All reactions