fix docip bug & add support for python 3.11 #205
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
1. Fix: docip crawler bug
As mentioned in issue #204 , the bug in docip crawler will block the getter thread and thus cannot get proxy normally.
This PR is to fix the host:port parse logic in
proxypool/crawlers/public/docip.py
.2. Add: support for python3.11
2.1 update requirements.txt
After python3.11, there is a variable name change for CFrame, which will cause the building failure when installing old version of gevent.
e.g.
Check this commit in CPython: [bpo-45431]Rename CFrame to _PyCFrame in the C API
They rename it from
CFrame *cframe;
to_PyCFrame *cframe;
and this results in a large number of packages needing to be modified to adapt to CPython updates.As Python 3.11 has been available for over a year, and Python 3.12 has been available for over two months, I think it's time to bump the requirements to support newer version of Python.
2.2 update proxypool/processors/tester.py
After python3.11, directly passing coroutines to asyncio loop is forbidden, which will cause TypeError on
proxypool/processors/tester.py:85
as
I changed it to newer style for supporting newer Python version.
This is also mentioned in issue #201.
I tested these changes under Python 3.11 & Python 3.12 environment and verified they are running steadily.
Regards,
B1ACK917