[442] - Pagination added for logs

[huzaifmalik786]

No description provided.

[fatchat]

depends on DalgoT4D/prefect-proxy#154

[Ishankoradia]

@fatchat @huzaifmalik786 is there a webapp PR coming for this.

[huzaifmalik786]

@Ishankoradia , it has already been raised. Please see it here:
DalgoT4D/webapp#1143

Thanks!

[codecov]

Codecov Report

Attention: Patch coverage is 9.52381% with 19 lines in your changes missing coverage. Please review.

Project coverage is 75.11%. Comparing base (44114f8) to head (dc951e7).
Report is 12 commits behind head on main.

Files with missing lines	Patch %	Lines
ddpui/celeryworkers/tasks.py	0.00%	13 Missing ⚠️
ddpui/api/pipeline_api.py	0.00%	4 Missing ⚠️
ddpui/ddpprefect/prefect_service.py	33.33%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #832      +/-   ##
==========================================
- Coverage   75.22%   75.11%   -0.12%     
==========================================
  Files         184      184              
  Lines        7664     7678      +14     
==========================================
+ Hits         5765     5767       +2     
- Misses       1899     1911      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Ishankoradia]

@huzaifmalik786 works nicely.

For the ai summary, can you make sure we are fetching all logs in the api v1/flow_runs/{flow_run_id}/logsummary.
It internally calls a function summarize_logs that in turn calls get_flow_run_logs_v2 . maybe replace this with a common function that fetches all pages with some configurable limit.

Thanks. Nice work.

[huzaifmalik786]

@Ishankoradia , are you saying to use the same get_flow_run_logs function in the summarize_logs function?
I think get_flow_run_logs_v2 already satisfies our need. It returns all the logs without any pagination.