Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unit test for each queue jobs class #9899

Open
43 of 46 tasks
touhidurabir opened this issue Apr 19, 2024 · 16 comments
Open
43 of 46 tasks

Add unit test for each queue jobs class #9899

touhidurabir opened this issue Apr 19, 2024 · 16 comments
Assignees
Labels
Housekeeping:2:Urgent Any dependency management or refactor that must be done soon to fix or avoid a significant problem.
Milestone

Comments

@touhidurabir
Copy link
Member

touhidurabir commented Apr 19, 2024

Describe the problem
Starting from 3.4.0 we have queue jobs to handle long running tasks or for tasks that need to done later . Our queue jobs implementation is an extended implementation of Laravel queue jobs.

The problem arise or will arise when any of the jobs structure will update but there is one or more pending jobs with older structure already dispatched in queue waiting to get executed . This problem get more complex with the upgrade of newer version when older structure based job still in queue as they may continue to fail and no way to resolve it after upgrade which may lead to undesirable outcome.

Describe the proposal
The initial plan was to force to run all pending queue jobs before upgrade which proposed at Add pre-flight check for pending jobs before running upgrade . This was a very simple and easy approach but may not suited for all possible cases . Also a job class structure or handle implementation may need to update based on requirement.

After some detailed discussion, we decided to add unit tests for each job class to make sure the consistence update the job class structure and handle method implementation . The idea is to try to mock each of jobs class to make sure it's on even after any update to it.

Impacted version
OJS, OMP or OPS version 3.4.0 and main

Additional information
Details discussion at Right approach to handle pending queue jobs at upgrade

Job classes to have unit tests

Stable-3.4.0

OJS
  • APP\jobs\doi\DepositIssue
  • APP\jobs\notifications\IssuePublishedNotifyUsers
  • APP\jobs\notifications\OpenAccessMailUsers
  • APP\jobs\statistics\CompileCounterSubmissionDailyMetrics
  • APP\jobs\statistics\CompileCounterSubmissionInstitutionDailyMetrics
  • APP\jobs\statistics\CompileIssueMetrics
  • APP\jobs\statistics\CompileSubmissionGeoDailyMetrics
  • APP\jobs\statistics\CompileUniqueInvestigations
  • APP\jobs\statistics\CompileUniqueRequests
  • APP\jobs\statistics\CompileUsageStatsFromTemporaryRecords [deprecated 3.4.0.5]
  • APP\jobs\statistics\DeleteUsageStatsTemporaryRecords
  • APP\jobs\statistics\ProcessUsageStatsLogFile
OMP
  • APP\jobs\statistics\CompileCounterSubmissionDailyMetrics
  • APP\jobs\statistics\CompileCounterSubmissionInstitutionDailyMetrics
  • APP\jobs\statistics\CompileSeriesMetrics
  • APP\jobs\statistics\CompileSubmissionGeoDailyMetrics
  • APP\jobs\statistics\CompileUniqueInvestigations
  • APP\jobs\statistics\CompileUniqueRequests
  • APP\jobs\statistics\CompileUsageStatsFromTemporaryRecords [deprecated 3.4.0.5]
  • APP\jobs\statistics\DeleteUsageStatsTemporaryRecords
  • APP\jobs\statistics\ProcessUsageStatsLogFile
OPS
  • APP\jobs\statistics\CompileCounterSubmissionDailyMetrics
  • APP\jobs\statistics\CompileCounterSubmissionInstitutionDailyMetrics
  • APP\jobs\statistics\CompileSubmissionGeoDailyMetrics
  • APP\jobs\statistics\CompileUniqueInvestigations
  • APP\jobs\statistics\CompileUniqueRequests
  • APP\jobs\statistics\CompileUsageStatsFromTemporaryRecords [deprecated 3.4.0.5]
  • APP\jobs\statistics\DeleteUsageStatsTemporaryRecords
  • APP\jobs\statistics\ProcessUsageStatsLogFile
PKP-LIB
  • PKP\jobs\bulk\BulkEmailSender
  • PKP\jobs\doi\DepositContext
  • PKP\jobs\doi\DepositSubmission
  • PKP\jobs\email\EditorialReminder
  • PKP\jobs\metadata\BatchMetadataChangedJob
  • PKP\jobs\metadata\MetadataChangedJob
  • PKP\jobs\notifications\NewAnnouncementNotifyUsers
  • PKP\jobs\notifications\StatisticsReportMail
  • PKP\jobs\notifications\StatisticsReportNotify
  • PKP\jobs\statistics\ArchiveUsageStatsLogFile
  • PKP\jobs\statistics\CompileContextMetrics
  • PKP\jobs\statistics\CompileMonthlyMetrics
  • PKP\jobs\statistics\CompileSubmissionMetrics
  • PKP\jobs\statistics\RemoveDoubleClicks
  • PKP\jobs\submissions\RemoveSubmissionFileFromSearchIndexJob
  • PKP\jobs\submissions\RemoveSubmissionFromSearchIndexJob
  • PKP\jobs\submissions\UpdateSubmissionSearchJob

PRs (Merged)
pkp-lib --> #9981
ojs --> pkp/ojs#4285
omp --> pkp/omp#1666
ops --> pkp/ops#745

PR Patch
pkp-lib --> #10368
ojs --> pkp/ojs#4426
ops --> pkp/ops#763
omp --> pkp/omp#1693

@touhidurabir touhidurabir added the Housekeeping:2:Urgent Any dependency management or refactor that must be done soon to fix or avoid a significant problem. label Apr 19, 2024
@touhidurabir touhidurabir added this to the 3.4.0-6 milestone Apr 19, 2024
@touhidurabir touhidurabir modified the milestones: 3.4.0-6, 3.4.0-x Apr 19, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 30, 2024
…d PKP\jobs\bulk\BulkEmailSender along with mockery package version update
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 30, 2024
…omSearchIndexJob and PKP\jobs\submissions\RemoveSubmissionFileFromSearchIndexJob
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Jun 6, 2024
@touhidurabir touhidurabir self-assigned this Jun 9, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 9, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
…d PKP\jobs\bulk\BulkEmailSender along with mockery package version update
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
…omSearchIndexJob and PKP\jobs\submissions\RemoveSubmissionFileFromSearchIndexJob
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/omp that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/omp that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/omp that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/omp that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/omp that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ops that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ops that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ops that referenced this issue Sep 27, 2024
touhidurabir added a commit to touhidurabir/ops that referenced this issue Sep 27, 2024
@touhidurabir
Copy link
Member Author

@asmecher I have updated the tests to backup and restore mocked DAO and container keys but unable to reproduce the issue that happening in your machine as we had seen in our last test . And it seems that it was happening for https://github.com/pkp/jatsTemplate/blob/main/tests/functional/ArticleBodyTest.php which only present in the main branch but not in the stable-3.4.0 branch . in fact for https://github.com/pkp/jatsTemplate in stable-3.4.0 , there is no tests at all . Any idea what I am missing .

I was able to complete all tests (not only jobs test) for stable-3.4.0 branch for OJS, OMP and OPS without any issue .

touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Oct 1, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Oct 1, 2024
touhidurabir added a commit to touhidurabir/omp that referenced this issue Oct 1, 2024
touhidurabir added a commit to touhidurabir/ops that referenced this issue Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Housekeeping:2:Urgent Any dependency management or refactor that must be done soon to fix or avoid a significant problem.
Projects
None yet
Development

No branches or pull requests

4 participants