Question about reducing browser restart frequency with scrapy-playwright

[SH-zwhy]

Hi,

I'm using scrapy-playwright for data scraping, where URLs are provided through a txt file. I've noticed that every time a URL is scraped, the browser restarts, which significantly reduces scraping efficiency.

Is there a way to avoid restarting the browser for each URL or to reduce the frequency of browser restarts to improve scraping performance?

Thanks!

[elacuesta]

That's not the way the package works by default, you might be starting a new job for each URL. By default a new page, not browser, is created for each URL, however you can reuse pages as explained at https://github.com/scrapy-plugins/scrapy-playwright?tab=readme-ov-file#playwright_page.

Please share your code and logs as requested in the Reporting issues section.