Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

README example seems to leave lingering processes #36

Open
dgmora opened this issue Jan 3, 2025 · 0 comments
Open

README example seems to leave lingering processes #36

dgmora opened this issue Jan 3, 2025 · 0 comments

Comments

@dgmora
Copy link

dgmora commented Jan 3, 2025

Hello, I have tried the README example with the latest main branch in the rails console and it seems to leave some lingering processes:

$> QuotesToScrapeCom.run { |q| quotes << q }
$> puts JSON.generate(quotes)
I, [2025-01-03T10:31:24.614667 #73722]  INFO -- : Driver: visiting https://quotes.toscrape.com/tag/humor/, last_request = 0
D, [2025-01-03T10:31:26.034991 #73722] DEBUG -- : Engine: crawler starts processing https://quotes.toscrape.com/tag/humor/ with :parse
D, [2025-01-03T10:31:26.102485 #73722] DEBUG -- : Engine: :parse enqueued requests ["https://quotes.toscrape.com/tag/humor/page/2/"]
D, [2025-01-03T10:31:26.102624 #73722] DEBUG -- : Engine: crawler stopped processing https://quotes.toscrape.com/tag/humor/
I, [2025-01-03T10:31:26.158699 #73722]  INFO -- : Driver: visiting https://quotes.toscrape.com/tag/humor/page/2/, last_request = 1735896686
D, [2025-01-03T10:31:26.619436 #73722] DEBUG -- : Engine: crawler starts processing https://quotes.toscrape.com/tag/humor/page/2/ with :parse
D, [2025-01-03T10:31:26.630985 #73722] DEBUG -- : Engine: crawler stopped processing https://quotes.toscrape.com/tag/humor/page/2/
[{"author":"Jane Austen","text":"“The person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.”"},{"author":"Steve Martin","text":"“A day without sunshine is like, you know, night.”"},{"author":"Garrison Keillor","text":"“Anyone who thinks sitting in church can make you a Christian must also think that sitting in a garage can make you a car.”"},{"author":"Jim Henson","text":"“Beauty is in the eye of the beholder and it may be necessary from time to time to give a stupid or misinformed beholder a black eye.”"},{"author":"Charles M. Schulz","text":"“All you need is love. But a little chocolate now and then doesn't hurt.”"},{"author":"Suzanne Collins","text":"“Remember, we're madly in love, so it's all right to kiss me anytime you feel like it.”"},{"author":"Charles Bukowski","text":"“Some people never go crazy. What truly horrible lives they must lead.”"},{"author":"Terry Pratchett","text":"“The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it.”"},{"author":"Dr. Seuss","text":"“Think left and think right and think low and think high. Oh, the thinks you can think up if only you try!”"},{"author":"George Carlin","text":"“The reason I talk to myself is because I’m the only one whose answers I accept.”"},{"author":"W.C. Fields","text":"“I am free of all prejudice. I hate everyone equally. ”"},{"author":"Jane Austen","text":"“A lady's imagination is very rapid; it jumps from admiration to love, from love to matrimony in a moment.”"}]
=> nil
$> I, [2025-01-03T10:31:28.516039 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:31:32.520740 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:31:36.523737 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
$> I, [2025-01-03T10:31:40.525877 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:31:44.531464 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:31:48.536225 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:31:52.537976 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:31:56.543284 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:00.545688 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:04.552105 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:08.557909 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:12.562917 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:16.565751 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:20.571640 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:24.577255 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:28.582404 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:32.587462 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:36.594109 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:40.598564 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:44.604204 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:48.609430 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:52.614856 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:32:56.618541 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:33:00.625810 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:33:04.631233 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:33:08.636769 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:33:12.642164 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:33:16.644030 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true
I, [2025-01-03T10:33:20.649950 #73722]  INFO -- : Cargo: req_enqueued=2, res_dequeued=2, res_handled=2, item_pipelined=12, item_processed=12, item_sent=12, item_rejected=0, idling=true

Is that expected? I have tried with 0.2.0 and that README's example and that does not happen (at least there are no logs). Since 0.2.0 is quite old, I'd prefer to use the latest changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant