Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue when when first running the bot #224

Open
developedbyalex opened this issue Dec 14, 2022 · 114 comments
Open

Issue when when first running the bot #224

developedbyalex opened this issue Dec 14, 2022 · 114 comments

Comments

@developedbyalex
Copy link

C:\Users\Administrator.virtualenvs\reddit-karma-farming-bot-4.0.4-irMUKD2V\Lib\site-packages\psaw\PushshiftAPI.py:192: UserWarning: Got non 200 code 404
warnings.warn("Got non 200 code %s" % response.status_code)
C:\Users\Administrator.virtualenvs\reddit-karma-farming-bot-4.0.4-irMUKD2V\Lib\site-packages\psaw\PushshiftAPI.py:180: UserWarning: Unable to connect to pushshift.io. Retrying after backoff.
warnings.warn("Unable to connect to pushshift.io. Retrying after backoff.")

@lckynmbrsvn
Copy link

lckynmbrsvn commented Dec 28, 2022

I am getting the same issue

Edit: apparently it might be because pushshift is changing servers, still looking into it

@lckynmbrsvn
Copy link

lckynmbrsvn commented Jan 2, 2023

it seems the pushshift api is not working at the moment and some of it is down: https://stats.uptimerobot.com/l8RZDu1gBG

if the pushshift API does come back online, the /src/apis/pushshift.py will need to be updated with the new pushshift api endpoint

@Gullysl
Copy link

Gullysl commented Jan 2, 2023

yeah, seems like pushift is down at the moment and has been for a few days. Fingers crossed it comes back online.

@Gullysl
Copy link

Gullysl commented Jan 2, 2023

I am getting the same issue

Edit: apparently it might be because pushshift is changing servers, still looking into it

did you find any more info out?

@lckynmbrsvn
Copy link

It seems that pushshift changed or updated their servers and also made changes to their api. I had a hard time piecing together what is going on from the threads in the pushshift subreddit.

@Gullysl
Copy link

Gullysl commented Jan 3, 2023

Ah ok, do you think we would be able to update the code ourselves considering that the original author doesn't look like he's going to release an update?

@Gullysl
Copy link

Gullysl commented Jan 3, 2023

I think this could help https://www.reddit.com/r/pushshift/comments/zkggt0/update_on_colo_switchover_bug_fixes_reindexing/
looks like they are in the process of transferring servers. Their status page now shows them as online. I think their domain changed.

here are some more sources (i will update as i find more info): https://www.reddit.com/r/pushshift/comments/zlryw1/comment/j0bss25/

@Gullysl
Copy link

Gullysl commented Jan 3, 2023

Think I may've fixed it. Just running now to check.

EDIT: Nope, :( I have managed to get it to successfully connect to pushift, but it cannot find any posts. I'll keep trying to figure it out.

@Gullysl
Copy link

Gullysl commented Jan 3, 2023

image
Looks like the python library itself will not work with the current API and is not expected to be updated. There is an alternative library which it could be updated to work, but will probably require some time and knowledge. Ideally the author of this project will decide to update it but seems unlikely for now.

@lckynmbrsvn
Copy link

That is unfortunate. What is the alternate library? Maybe it won’t be to hard to update.

@Gullysl
Copy link

Gullysl commented Jan 5, 2023

That is unfortunate. What is the alternate library? Maybe it won’t be to hard to update.

image
I think the api may return data in a slightly different way than before, but it may be possible.

@Gullysl
Copy link

Gullysl commented Jan 6, 2023

Do you @lckynmbrsvn have any idea of how the code functions? I have a basic understanding, but I think to implement the new API, deeper knowledge might be needed. There is a code walkthrough on YouTube.

@Gullysl
Copy link

Gullysl commented Jan 7, 2023

@sheepsushis Got any idea? I see you made the tutorial and stuff.

@lckynmbrsvn
Copy link

@gully32 I haven't had a chance to look through anything yet, I want to take a look but I'm not the best programmer. I want to give it a shot though.

@Gullysl
Copy link

Gullysl commented Jan 7, 2023

@gully32 I haven't had a chance to look through anything yet, I want to take a look but I'm not the best programmer. I want to give it a shot though.

Yeah same, think it should be possible. Update me if you find anything and we can try and work it out. I’ve figured out how to change the url for the initial ping test but not sure how to make requests etc

@gabrielmajosi
Copy link
Contributor

The bot correctly implements the use of PSAW. I concur that pushshift changed something on their end, causing PSAW to break. While PMAW certainly will not be a drop-in replacement, it should not be too difficult to migrate the bot to it. Another way we could solve this is by forking PSAW, and applying fixes. However, this has a major caveat: Somebody will have to maintain the fork. From my personal standpoint, migrating to PMAW is a better route as it will likely have future updates for a long period of time.

@Gullysl
Copy link

Gullysl commented Jan 8, 2023

The bot correctly implements the use of PSAW. I concur that pushshift changed something on their end, causing PSAW to break. While PMAW certainly will not be a drop-in replacement, it should not be too difficult to migrate the bot to it. Another way we could solve this is by forking PSAW, and applying fixes. However, this has a major caveat: Somebody will have to maintain the fork. From my personal standpoint, migrating to PMAW is a better route as it will likely have future updates for a long period of time.

How feasible is this? I happy to offer help although it may be limited lol, is there any guides on how to migrate? Thx for the reasponse btw

@sesseor
Copy link
Contributor

sesseor commented Jan 8, 2023

The bot correctly implements the use of PSAW. I concur that pushshift changed something on their end, causing PSAW to break. While PMAW certainly will not be a drop-in replacement, it should not be too difficult to migrate the bot to it. Another way we could solve this is by forking PSAW, and applying fixes. However, this has a major caveat: Somebody will have to maintain the fork. From my personal standpoint, migrating to PMAW is a better route as it will likely have future updates for a long period of time.

How feasible is this? I happy to offer help although it may be limited lol, is there any guides on how to migrate? Thx for the reasponse btw

major functionalities of the bot have to be re-written/updates.. that gives 2 options

  1. re-write the bot from scratch/update funcs
  2. create a new bot

both options are feasible with time at hand

@gabrielmajosi
Copy link
Contributor

gabrielmajosi commented Jan 8, 2023

major functionalities of the bot have to be re-written/updates.. that gives 2 options

  1. re-write the bot from scratch/update funcs
  2. create a new bot

both options are feasible with time at hand

I disagree, the codebase does not have to be entirely rewritten. The bot does a lot more than grabbing data from pushshift, and it's more reasonable to only replace parts of the code.

@sesseor
Copy link
Contributor

sesseor commented Jan 8, 2023

most important parts of the bot are broken..smh

@gabrielmajosi
Copy link
Contributor

@gully32 I am going to make a fork of the repository, and slowly start working on it. I'll link it here somehow, but I don't know the best method.

@Gullysl
Copy link

Gullysl commented Jan 8, 2023

@gully32 I am going to make a fork of the repository, and slowly start working on it. I'll link it here somehow, but I don't know the best method.

Thank you!! Keep me updated and tell me if you need anything

@Mohamed6935
Copy link

Any luck?

@Gullysl
Copy link

Gullysl commented Jan 13, 2023

@gully32 I am going to make a fork of the repository, and slowly start working on it. I'll link it here somehow, but I don't know the best method.

How is it going? Figured anything out yet ?

@gabrielmajosi
Copy link
Contributor

@Gullysl
Copy link

Gullysl commented Feb 1, 2023

We have identified and resolved the issue with PSAW by implementing PMAW as a replacement, ensuring seamless migration and long-term support for the bot's functionality. I have also evaluated the option of forking PSAW and applying fixes, but determined that migrating to PMAW would be a more optimal solution for sustained progress and maintenance.

So it should work now?

@0x17d9ef78
Copy link

Where can we find the solution?

@Gullysl
Copy link

Gullysl commented Mar 26, 2023

How's it going?

@Sonpink
Copy link

Sonpink commented Mar 27, 2023

I am also getting the same error https://gyazo.com/2188063b6784b56ff509c753ff273ba7

Hope someone give solution for this issue.

@sesseor
Copy link
Contributor

sesseor commented Apr 1, 2023

I am also getting the same error https://gyazo.com/2188063b6784b56ff509c753ff273ba7

Hope someone give solution for this issue.

you should delete that really fast .. your account credentials are exposed

@Gullysl
Copy link

Gullysl commented Apr 10, 2023

UPDATE:
Tested it, it does work as expected.

is there any easy way to update? or do i have to restart? appreciate the fix tho!

For now, you'll have to copy the entire master directory from my fork. I'm currently thinking about creating an updater script that'll update the entire directory with the changes from the fork. I'll have to figure out the logistics of that though.

alr I will give it a go

@Gullysl
Copy link

Gullysl commented Apr 10, 2023

UPDATE:
Tested it, it does work as expected.

is there any easy way to update? or do i have to restart? appreciate the fix tho!

For now, you'll have to copy the entire master directory from my fork. I'm currently thinking about creating an updater script that'll update the entire directory with the changes from the fork. I'll have to figure out the logistics of that though.

alr I will give it a go

done. I will update if any issus arise

@Gullysl
Copy link

Gullysl commented Apr 10, 2023

UPDATE:
Tested it, it does work as expected.

is there any easy way to update? or do i have to restart? appreciate the fix tho!

For now, you'll have to copy the entire master directory from my fork. I'm currently thinking about creating an updater script that'll update the entire directory with the changes from the fork. I'll have to figure out the logistics of that though.

alr I will give it a go

done. I will update if any issus arise

getting this now when the bot tries to repost. 21:39:11 Error: crosspost subreddit name is None

@L-mbda
Copy link

L-mbda commented Apr 10, 2023

Hmm, the script should still work.

@Gullysl
Copy link

Gullysl commented Apr 11, 2023

Hmm, the script should still work.

I have disabled crossposting and used this guide to keep the program from crashing: https://www.peterbe.com/plog/run-forever.sh

@Gullysl
Copy link

Gullysl commented Apr 11, 2023

Hmm, the script should still work.

I have disabled crossposting and used this guide to keep the program from crashing: https://www.peterbe.com/plog/run-forever.sh

hmm I still seem to be getting 09:43:38 reposting post: u1v4h4 09:43:38 REPOST ERROR: SUBMIT_VALIDATION_FLAIR_REQUIRED: 'Your post must contain post flair.' on field 'flair' and Info: skipping post; it was malformed or date indicated

@L-mbda
Copy link

L-mbda commented Apr 11, 2023

Hmm, the script should still work.

I have disabled crossposting and used this guide to keep the program from crashing: https://www.peterbe.com/plog/run-forever.sh

hmm I still seem to be getting 09:43:38 reposting post: u1v4h4 09:43:38 REPOST ERROR: SUBMIT_VALIDATION_FLAIR_REQUIRED: 'Your post must contain post flair.' on field 'flair' and Info: skipping post; it was malformed or date indicated

It will still work with the bot

@ElReeey
Copy link

ElReeey commented Apr 12, 2023

@de-y @gully32 I was able to install the fork with the help of your past messages. However I have run into the following issue:

15:36:35 comments need to be initialized
15:36:35 intiializing comments
15:36:35 using cobe to generate comments
15:36:35 cobe db size is: 0mb, need 12mb - learning...
15:36:35 picking a random subreddit
15:36:36 checking subreddit: xyz
15:36:36 learning from /r/xyz

Unfortunately it stays there and nothing happens.
I tried to set the min db size to 0mb however the comments generated where "I don't know enough to answer you yet!"

Do you have an idea why this is happening?

@Gullysl
Copy link

Gullysl commented Apr 12, 2023

@de-y @gully32 I was able to install the fork with the help of your past messages. However I have run into the following issue:

15:36:35 comments need to be initialized 15:36:35 intiializing comments 15:36:35 using cobe to generate comments 15:36:35 cobe db size is: 0mb, need 12mb - learning... 15:36:35 picking a random subreddit 15:36:36 checking subreddit: xyz 15:36:36 learning from /r/xyz

Unfortunately it stays there and nothing happens. I tried to set the min db size to 0mb however the comments generated where "I don't know enough to answer you yet!"

Do you have an idea why this is happening?

How long have you left it for? It takes a long period of time to learn and download to the database. If you want to disable commenting with cobe, set the comment chance to zero in reddit_config

@ElReeey
Copy link

ElReeey commented Apr 12, 2023

@de-y @gully32 I was able to install the fork with the help of your past messages. However I have run into the following issue:
15:36:35 comments need to be initialized 15:36:35 intiializing comments 15:36:35 using cobe to generate comments 15:36:35 cobe db size is: 0mb, need 12mb - learning... 15:36:35 picking a random subreddit 15:36:36 checking subreddit: xyz 15:36:36 learning from /r/xyz
Unfortunately it stays there and nothing happens. I tried to set the min db size to 0mb however the comments generated where "I don't know enough to answer you yet!"
Do you have an idea why this is happening?

How long have you left it for? It takes a long period of time to learn and download to the database. If you want to disable commenting with cobe, set the comment chance to zero in reddit_config

I thought it was stuck, but it seems to gather information. I will let it sit for a while and report back in case I encounter any issues. Thank you!

@Gullysl
Copy link

Gullysl commented Apr 12, 2023 via email

@ElReeey
Copy link

ElReeey commented Apr 12, 2023

No problem! How big is your mum db size mine is at 8 at the comments seem pretty nonsensical. I wonder how big of an increase is needed. Also make sure it doesn’t learn off some subreddits like mental health or drug related stuff.

On Wed, 12 Apr 2023 at 15:15, ElReeey @.> wrote: @de-y https://github.com/De-y @gully32 https://github.com/gully32 I was able to install the fork with the help of your past messages. However I have run into the following issue: 15:36:35 comments need to be initialized 15:36:35 intiializing comments 15:36:35 using cobe to generate comments 15:36:35 cobe db size is: 0mb, need 12mb - learning... 15:36:35 picking a random subreddit 15:36:36 checking subreddit: xyz 15:36:36 learning from /r/xyz Unfortunately it stays there and nothing happens. I tried to set the min db size to 0mb however the comments generated where "I don't know enough to answer you yet!" Do you have an idea why this is happening? How long have you left it for? It takes a long period of time to learn and download to the database. If you want to disable commenting with cobe, set the comment chance to zero in reddit_config I thought it was stuck, but it seems to gather information. I will let it sit for a while and report back in case I encounter any issues. Thank you! — Reply to this email directly, view it on GitHub <#224 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIE4XVSAOBH6UEQLZMF2ODTXA22JXANCNFSM6AAAAAAS65RX7U . You are receiving this because you were mentioned.Message ID: @.>

So far it's only 1mb after the first run. It's quite slow, I might consider switching of comments at some point.

@Gullysl
Copy link

Gullysl commented Apr 12, 2023 via email

@ElReeey
Copy link

ElReeey commented Apr 12, 2023

Yeah, comments do make it less bot like though. Might try training for longer

I disabled comments for now but I'm having a really hard time getting the bot to work properly. I receive either one of the following two errors:

  1. REPOST ERROR: SUBMIT_VALIDATION_FLAIR_REQUIRED: 'Your post must contain post flair.' on field 'flair'
  2. raise InvalidURL(f"Invalid URL {url!r}: No host supplied")
    requests.exceptions.InvalidURL: Invalid URL 'https://': No host supplied

After the 2nd one the bot is stuck and I need to restart. Any solutions for that? I tried to change the list of subreddits on reddit_sub_lists however it doesn't seem to work, to make sure a flair is not required.

@Gullysl
Copy link

Gullysl commented Apr 12, 2023

Yeah, comments do make it less bot like though. Might try training for longer

I disabled comments for now but I'm having a really hard time getting the bot to work properly. I receive either one of the following two errors:

  1. REPOST ERROR: SUBMIT_VALIDATION_FLAIR_REQUIRED: 'Your post must contain post flair.' on field 'flair'
  2. raise InvalidURL(f"Invalid URL {url!r}: No host supplied")
    requests.exceptions.InvalidURL: Invalid URL 'https://': No host supplied

After the 2nd one the bot is stuck and I need to restart. Any solutions for that? I tried to change the list of subreddits on reddit_sub_lists however it doesn't seem to work, to make sure a flair is not required.

I've had the same issue, I tried to make a script to keep the bot running but I couldn't figure it out. Maybe if I spent longer I could but for now im just using a macro to press up and enter on the console every so often lol

@ElReeey
Copy link

ElReeey commented Apr 12, 2023

I've had the same issue, I tried to make a script to keep the bot running but I couldn't figure it out. Maybe if I spent longer I could but for now im just using a macro to press up and enter on the console every so often lol

Do you experience the second error? I have not managed to make a single post yet.

@Gullysl
Copy link

Gullysl commented Apr 12, 2023

I've had the same issue, I tried to make a script to keep the bot running but I couldn't figure it out. Maybe if I spent longer I could but for now im just using a macro to press up and enter on the console every so often lol

Do you experience the second error? I have not managed to make a single post yet.

How long have you run it for? I've made several posts although a lot of them are auto removed. One has over 3k upvotes but I have been running all day and overnight.

@ElReeey
Copy link

ElReeey commented Apr 12, 2023

How long have you run it for? I've made several posts although a lot of them are auto removed. One has over 3k upvotes but I have been running all day and overnight.

I made some changes to code and seems to work now.

Regarding the crashes I did this:

  1. Open Editor, enter the following code and save as name.py
import os

while(os.system('cmd /c run_windows.bat') == 0):
    os.system('cmd /c run_windows.bat')
  1. cd directory
  2. .\.venv\Scripts\activate
  3. python name.py

Seems to stop the crashes for now

@L-mbda
Copy link

L-mbda commented Apr 12, 2023

How long have you run it for? I've made several posts although a lot of them are auto removed. One has over 3k upvotes but I have been running all day and overnight.

I made some changes to code and seems to work now.

Regarding the crashes I did this:

1. Open Editor, enter the following code and save as name.py
import os

while(os.system('cmd /c run_windows.bat') == 0):
    os.system('cmd /c run_windows.bat')
2. `cd directory`

3. `.\.venv\Scripts\activate`

4. `python name.py`

Seems to stop the crashes for now

I did attach a script for running it that should stop the crashes, but your one also works

@Gullysl
Copy link

Gullysl commented Apr 12, 2023

You got a script for linux?

@L-mbda
Copy link

L-mbda commented Apr 12, 2023 via email

@Gullysl
Copy link

Gullysl commented Apr 12, 2023

It should be under run_linux.sh in my fork

On Wed, Apr 12, 2023 at 12:21 PM gully32 @.> wrote: You got a script for linux? — Reply to this email directly, view it on GitHub <#224 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOXR4X6ULYRWIA4RS6V7GF3XA36FHANCNFSM6AAAAAAS65RX7U . You are receiving this because you were mentioned.Message ID: @.>

Thank you!!

@Gullysl
Copy link

Gullysl commented Apr 12, 2023

It should be under run_linux.sh in my fork

On Wed, Apr 12, 2023 at 12:21 PM gully32 @.> wrote: You got a script for linux? — Reply to this email directly, view it on GitHub <#224 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOXR4X6ULYRWIA4RS6V7GF3XA36FHANCNFSM6AAAAAAS65RX7U . You are receiving this because you were mentioned.Message ID: _@**.**_>

Thank you!!

still crashes unfortunatly :( raise InvalidURL(f"Invalid URL {url!r}: No host supplied") requests.exceptions.InvalidURL: Invalid URL 'https://': No host supplied

@ElReeey
Copy link

ElReeey commented Apr 12, 2023

still crashes unfortunatly :( raise InvalidURL(f"Invalid URL {url!r}: No host supplied") requests.exceptions.InvalidURL: Invalid URL 'https://': No host supplied

Same for me btw. Unfortunately 2 accounts shadowbanned today aswell.

@Gullysl
Copy link

Gullysl commented Apr 12, 2023

still crashes unfortunatly :( raise InvalidURL(f"Invalid URL {url!r}: No host supplied") requests.exceptions.InvalidURL: Invalid URL 'https://': No host supplied

Same for me btw. Unfortunately 2 accounts shadowbanned today aswell.

i've had no accounts shadowbanned, did you just create them? Mines been around for 2 years

@ElReeey
Copy link

ElReeey commented Apr 12, 2023

i've had no accounts shadowbanned, did you just create them? Mines been around for 2 years

Yeah those were two brand new accounts. Didn't want my aged ones to get banned, while figuring this bot out.

@Gullysl
Copy link

Gullysl commented Apr 12, 2023

i've had no accounts shadowbanned, did you just create them? Mines been around for 2 years

Yeah those were two brand new accounts. Didn't want my aged ones to get banned, while figuring this bot out.

fair enough, I found much more success when running the .sh rather than the .py, have you tried running the other windows script?

@ElReeey
Copy link

ElReeey commented Apr 13, 2023

fair enough, I found much more success when running the .sh rather than the .py, have you tried running the other windows script?

Yeah I've only been running the windows.bat not the .py
I'm sure the accounts got reported for low quality posts/comments

@L-mbda
Copy link

L-mbda commented Apr 13, 2023 via email

@DavidHlavacek
Copy link

PMAW

You should open the cobe config and change the min database size to at least 100mb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests