Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
目前没有整个进程的最大运行时间控制,因此实现此功能;
但无法达到精准的控制,只能在秒级别内,比如设置20秒,会在20-23秒左右停止
配置:
./crawlergo --max-run-time=20 www.xxx.com
MaxRunTime
默认最大执行时间为 3600秒控制点
task_main.go:addTask2Pool()
: 若检测超时则无法再添加新的url 创建新的tabtask_main.go:Task()
:每个tab的最大超时时间由
TabRunTimeout
控制,因此这里的逻辑是:进程剩余时间和tab最大超时时间 取最小,作为 tab的超时时间。
如果没时间了,则取消创建。