Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrapy playwright will just not return 'page' in metadata #319

Open
rubmz opened this issue Sep 19, 2024 · 1 comment
Open

Scrapy playwright will just not return 'page' in metadata #319

rubmz opened this issue Sep 19, 2024 · 1 comment

Comments

@rubmz
Copy link

rubmz commented Sep 19, 2024

I am using scrapy-playwright with latest versions on the webkit browser on ubuntu 22.04.
I can start and debug the spider once or twice. Trying to stop it using the debugger "stop" button (Ctrl+Break) will only half stop it, it will linger open forever if not stopped again with another Ctrl+Break.
After once or twice that the scrapy spider is working fine, it will stop returning 'page' in the meta field of the response object.
There are no errors in the logs.
Sometimes I manage to restart the process again. At other times I am forced to restart the machine...
I tried killing the WPBrowser* processes but no luck...
Any idea how to deal with this?
Do I need to add some handler to stop the debug more gracefully? Is it related at all?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants