Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image import is super super insanely slow in 103.4.1 #638

Open
jurvi opened this issue Apr 4, 2023 · 1 comment
Open

Image import is super super insanely slow in 103.4.1 #638

jurvi opened this issue Apr 4, 2023 · 1 comment

Comments

@jurvi
Copy link

jurvi commented Apr 4, 2023

Environment and configuration

  1. Magento 2.4.4-p3
  2. akeneo/module-magento2-connector-community 103.4.1
  3. akeneo/api-php-client v10.0.0

Steps to reproduce

  1. Run product import for a family that has large amount of products and images

Expected result

  1. Import goes OK and finishes in few minutes as expected and it has done for a long time

Actual result

  1. Import takes insanely long time, hours. This must be related to the "Add localisable and scopable image attribute import" which I had to revert to old version to actually be able to use 103.4.1.
@indykoning
Copy link

I believe it's because of the synchronous nature of the image downloads.

For every image for every product it will need to download one image, then go on to the next.

So imagine you've got ~10 images per product and have around 8k product in your import.
It will have to download 80,000 images.
Let's say this takes ~100ms per image, it will take ~2 hours to import them all.

Maybe this package could benefit from concurrent requests
https://docs.guzzlephp.org/en/stable/quickstart.html#concurrent-requests

for instance making the image requests per product concurrent would reduce this hypothetical to ~15 minutes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants