-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add logging to the licence-import import jobs
https://eaflood.atlassian.net/browse/WATER-4024 https://eaflood.atlassian.net/browse/WATER-4039 This follows the same pattern used to get [non-noisy logging included in the NALD import jobs](#677). Because the import process is run through jobs, it means for each company and licence to be imported, a 'job' is created. The legacy code would then log each time this happened. But when each entails creating approximately 70K jobs (140K in total!) the logs just become a sea of noise. We implemented a pattern of including information in the job data about how many jobs in total there are, and what was the index of the current job when added to the queue. We can then use this information to get log when the first job is started, and the last is completed. In this way we are only logging the start and end of the 'job' (even though there are thousands of them). This change applies the same pattern to the licence-import company and licence import jobs. We also do some tweaking to bring both NALD import and licence import inline.
- Loading branch information
1 parent
6eb378d
commit 766b932
Showing
16 changed files
with
275 additions
and
92 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.