Issues with generation of thumbnail

[mahalakshme]

https://avni.freshdesk.com/a/tickets/3455

Context:

There are already cards to monitor lambda failures but was not picked up because of bandwidth issues. As part of fixing the issue in the ticket, let know your suggestions to prevent such unnotified failure cases by identifying the rootcause for this issue. Should probably get fixed by making the lambda to run for the dates for which it didn't run.

[1t5j0y]

Thumbnail generation for the images for the entities mentioned in the ticket for the org in question were failing with the error:
[Error: VipsJpeg: Corrupt JPEG data: 196224 extraneous bytes before marker 0xdb VipsJpeg: Corrupt JPEG data: 196224 extraneous bytes before marker 0xdb VipsJpeg: Corrupt JPEG data: 196224 extraneous bytes before marker 0xdb ]

Fixed the thumbnail generation to be more tolerant of warnings.
Sharp Library documentation, discussion about this issue

Additionally during investigation, found instances of thumbnail generation failures for other orgs due to insufficient memory or timeout so those have been bumped as well.

After this fix is deployed to prod, scripts/invoke-instant-lambda-for-files.sh needs to be run for the known impacted files.

[1t5j0y]

Deployed instant thumbnail lambdas to staging and prerelease for testing.

[1t5j0y]

Deployed to prod and invoked instant lambda for the known impacted files.

[himeshr]

@1t5j0y regarding invoking instant lambda for known impacted files.. what were the criteria for known impacted files.?
Could we run this for whole of Goonj org in prod.?

[1t5j0y]

@himeshr criteria was the subjects for whom this issue was reported. We can't currently run this for an entire org due to which we created #163 .

Noticed some other missing thumbnails for goonj and I'm working on restoring as many as possible.

[1t5j0y]

Additionally invoked instant lambda for few more media. List extracted by browsing through avni-media and capturing images that do not load.
filenames2-uniq.txt

There are still quite a few cases where the source image itself is not present (60 cases - shows up as ERROR AccessDenied: Access Denied in the cloudwatch logs) or other failure (39 cases - shows up as Error: source: bad seek to 3817552 heif: Unsupported feature: Unsupported codec (4.3000) in the cloudwatch logs - Related Issue on sharp library)

[himeshr]

Additionally invoked instant lambda for few more media. List extracted by browsing through avni-media and capturing images that do not load. filenames2-uniq.txt

Yes, this was the concern, that there could be other entries that weren't reported but could be recovered.
Ideally, we should have a job, that does this in an automated manner for the specified org. Created a card for this, to save us the manual effort for the next time.