[Bug][gitlab] GitLab 11.1.4 APIs did not return merge_date and led the calculation of PR cycle time fail

[xuqiyu]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

My GitLab version is 11.1.4. Some pull_requests have been merged, but the GitLab APIs did not return merge_date, resulting in the merge_date field of the pull_request table being empty, and then there is a problem with the calculation logic behind it, the APIs returns the following payload:

[{"id":5,"iid":5,"project_id":1,"title":"feature620-1","description":"","state":"merged","created_at":"2023-06-19T16:50:55.534+08:00","updated_at":"2023-06-19T16:51:13.120+08:00","target_branch":"master","source_branch":"feature620","upvotes":0,"downvotes":0,"author":{"id":1,"name":"Administrator","username":"root","state":"active","avatar_url":"https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon","web_url":"http://10.89.168.63:8081/root"},"assignee":{"id":1,"name":"Administrator","username":"root","state":"active","avatar_url":"https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon","web_url":"http://10.89.168.63:8081/root"},"source_project_id":1,"target_project_id":1,"labels":[],"work_in_progress":false,"milestone":null,"merge_when_pipeline_succeeds":false,"merge_status":"can_be_merged","sha":"f44aebb41056fc9d5053599d0d6f270900b68dda","merge_commit_sha":"c68d591a86fcf8ccdf01b319fdb242708cb7340e","user_notes_count":0,"discussion_locked":null,"should_remove_source_branch":null,"force_remove_source_branch":false,"web_url":"http://10.89.168.63:8081/root/sample/merge_requests/5","time_stats":{"time_estimate":0,"total_time_spent":0,"human_time_estimate":null,"human_total_time_spent":null},"squash":false},{"id":4,"iid":4,"project_id":1,"title":"Feature618","description":"","state":"merged","created_at":"2023-06-18T14:15:00.749+08:00","updated_at":"2023-06-18T14:16:27.829+08:00","target_branch":"master","source_branch":"feature618","upvotes":0,"downvotes":0,"author":{"id":1,"name":"Administrator","username":"root","state":"active","avatar_url":"https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon","web_url":"http://10.89.168.63:8081/root"},"assignee":{"id":1,"name":"Administrator","username":"root","state":"active","avatar_url":"https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon","web_url":"http://10.89.168.63:8081/root"},"source_project_id":1,"target_project_id":1,"labels":[],"work_in_progress":false,"milestone":null,"merge_when_pipeline_succeeds":false,"merge_status":"can_be_merged","sha":"f425c59ed7f6825fa5b816333501c82008d8acd1","merge_commit_sha":"9ddfbb5e809d95ec44a70d4ab310e1d4c4f3fe2e","user_notes_count":1,"discussion_locked":null,"should_remove_source_branch":null,"force_remove_source_branch":true,"web_url":"http://10.89.168.63:8081/root/sample/merge_requests/4","time_stats":{"time_estimate":0,"total_time_spent":0,"human_time_estimate":null,"human_total_time_spent":null},"squash":false},{"id":3,"iid":3,"project_id":1,"title":"设置80端口","description":"设置80端口--完成","state":"merged","created_at":"2023-06-18T11:27:06.924+08:00","updated_at":"2023-06-18T11:27:11.581+08:00","target_branch":"master","source_branch":"feature-617","upvotes":0,"downvotes":0,"author":{"id":1,"name":"Administrator","username":"root","state":"active","avatar_url":"https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon","web_url":"http://10.89.168.63:8081/root"},"assignee":null,"source_project_id":1,"target_project_id":1,"labels":[],"work_in_progress":false,"milestone":null,"merge_when_pipeline_succeeds":false,"merge_status":"can_be_merged","sha":"58b4bbc198ffb1ab1eba42f4ebfdbbb3f75ad777","merge_commit_sha":"85c8e01001535672720d80751a6abd07bab4bc21","user_notes_count":0,"discussion_locked":null,"should_remove_source_branch":null,"force_remove_source_branch":false,"web_url":"http://10.89.168.63:8081/root/sample/merge_requests/3","time_stats":{"time_estimate":0,"total_time_spent":0,"human_time_estimate":null,"human_total_time_spent":null},"squash":false},{"id":2,"iid":2,"project_id":1,"title":"feature617-2 commit","description":"添加springboot基础配置","state":"merged","created_at":"2023-06-18T10:48:31.249+08:00","updated_at":"2023-06-18T10:48:36.565+08:00","target_branch":"master","source_branch":"feature-617","upvotes":0,"downvotes":0,"author":{"id":1,"name":"Administrator","username":"root","state":"active","avatar_url":"https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon","web_url":"http://10.89.168.63:8081/root"},"assignee":null,"source_project_id":1,"target_project_id":1,"labels":[],"work_in_progress":false,"milestone":null,"merge_when_pipeline_succeeds":false,"merge_status":"can_be_merged","sha":"8be390c067b3605d86d934b90ce2c9dd7704e069","merge_commit_sha":"4eeda69fb6338f787489f315a44f9ff819898aca","user_notes_count":0,"discussion_locked":null,"should_remove_source_branch":null,"force_remove_source_branch":false,"web_url":"http://10.89.168.63:8081/root/sample/merge_requests/2","time_stats":{"time_estimate":0,"total_time_spent":0,"human_time_estimate":null,"human_total_time_spent":null},"squash":false},{"id":1,"iid":1,"project_id":1,"title":"feature617-1 commit","description":"","state":"merged","created_at":"2023-06-17T22:51:21.079+08:00","updated_at":"2023-06-17T22:51:26.507+08:00","target_branch":"master","source_branch":"feature-617","upvotes":0,"downvotes":0,"author":{"id":1,"name":"Administrator","username":"root","state":"active","avatar_url":"https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon","web_url":"http://10.89.168.63:8081/root"},"assignee":null,"source_project_id":1,"target_project_id":1,"labels":[],"work_in_progress":false,"milestone":null,"merge_when_pipeline_succeeds":false,"merge_status":"can_be_merged","sha":"e72fa99c78d87cfca6f6b40ecb16fafa65648373","merge_commit_sha":"47c685aefff3fa26d2cf37966c02d3dd34d20ddd","user_notes_count":0,"discussion_locked":null,"should_remove_source_branch":null,"force_remove_source_branch":false,"web_url":"http://10.89.168.63:8081/root/sample/merge_requests/1","time_stats":{"time_estimate":0,"total_time_spent":0,"human_time_estimate":null,"human_total_time_spent":null},"squash":false}]

What do you expect to happen

In case this version of gitlab does not return merge_date, consider other solutions to the merge_date problem.

How to reproduce

Use DevLake to collect GitLab v11.1.4 data

Anything else

No response

Version

v0.17

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[likyh]

incubator-devlake/backend/plugins/gitlab/impl/impl.go

Lines 115 to 135 in 5ae16b9

	tasks.EnrichMergeRequestsMeta,
	tasks.CollectAccountsMeta,
	tasks.ExtractAccountsMeta,
	tasks.ConvertAccountsMeta,
	tasks.ConvertProjectMeta,
	tasks.ConvertApiMergeRequestsMeta,
	tasks.ConvertMrCommentMeta,
	tasks.ConvertApiMrCommitsMeta,
	tasks.ConvertIssuesMeta,
	tasks.ConvertIssueLabelsMeta,
	tasks.ConvertMrLabelsMeta,
	tasks.ConvertCommitsMeta,
	tasks.ConvertPipelineMeta,
	tasks.ConvertPipelineCommitMeta,
	tasks.ConvertJobMeta,
	tasks.CollectApiCommitsMeta,
	tasks.ExtractApiCommitsMeta,
	tasks.ExtractApiMergeRequestDetailsMeta,
	tasks.CollectTagMeta,
	tasks.ExtractTagMeta,
	}

I think ExtractApiMergeRequestDetailsMeta should move before ConvertApiMergeRequestsMeta.

[Startrekzky]

@xuqiyu

1. The issue has been translated into English to follow the community guidelines. Please use English next time you created an issue for better communication.
2. Is it a typo you typed GitLab 1.11.4 initially? I've changed it to v11.1.4, which looks more like the right version.
3. I noticed that you checked this. Do you want to fix the issue?
   

[klesh]

Look like this is the same problem to #5486