Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: search all jobs by asset name or job id #193

Merged
merged 9 commits into from
Dec 13, 2024
Merged

feat: search all jobs by asset name or job id #193

merged 9 commits into from
Dec 13, 2024

Conversation

helen-m-lin
Copy link
Contributor

closes #180

This PR enables users to search all jobs by asset name or job id.

  • GET /api/v1/get_job_status_list?get_all_jobs=true: get all jobs from airflow that match default or provided status and submit time range
    • Uses httpx aysnc client to send requests concurrently to airflow. Takes ~3s to fetch ~1000 jobs.
  • Add "Advanced Search" options in job status page
    • Default table is still original table that gets 1 page of jobs at a time
    • If the user enables the advanced search toggle, the full jobs table is shown instead.
      • Full jobs table uses JS Datatables library to render and search full jobs list, fully in client-side.
      • Full jobs table has same UI rendering (status colors, task buttons, etc), pagination, etc.
    • User can search by asset name or job id in the search bar. Results are filtered as user types.

image

image

helen-m-lin
helen-m-lin commented Dec 12, 2024
View reviewed changes
src/aind_data_transfer_service/server.py
try:
url = os.getenv("AIND_AIRFLOW_SERVICE_JOBS_URL", "").strip("/")
get_one_job = request.query_params.get("dag_run_id") is not None
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_one_job was originally created to allow users to retrieve a job based on exact job id. This is not needed anymore since UI enables users to search fully client-side.

@helen-m-lin helen-m-lin marked this pull request as ready for review December 12, 2024 23:30
jtyoung84
jtyoung84 approved these changes Dec 13, 2024
View reviewed changes
Copy link
Contributor

@jtyoung84 jtyoung84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good for now. There's probably a way to avoid making extra calls when toggling the advanced search and then adding a filter by state. Something that can be implemented later.

@helen-m-lin helen-m-lin merged commit 1ffba85 into dev Dec 13, 2024
3 checks passed
@helen-m-lin helen-m-lin deleted the feat-180-search-asset-name branch December 13, 2024 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jtyoung84 jtyoung84 jtyoung84 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add ability to search by asset name in Jobs Status page
2 participants
@helen-m-lin @jtyoung84