Bestbuy API implementation

[TheTabKey]

Also here is a bestbuy api:
https://www.bestbuy.com/api/tcfb/model.json?paths=%5B%5B%22shop%22%2c%22buttonstate%22%2c%22v5%22%2c%22item%22%2c%22skus%22%2c6429440%2c%22conditions%22%2c%22NONE%22%2c%22destinationZipCode%22%2c92582%2c%22storeId%22%2c%201441%2c%22context%22%2c%22cyp%22%2c%22addAll%22%2c%22false%22%5D%5D&method=get

sku can be replaced.....and between using these two apis, it will reduce bandwidth usage by a great margin, be faster, and reduce issues compared to trying to load product page. Url was cleaned up so it only shows the important information. I know lots of people were complaining about bandwidth usage so I am doing all i can.

example of item in stock:
https://www.bestbuy.com/api/tcfb/model.json?paths=%5B%5B%22shop%22%2c%22buttonstate%22%2c%22v5%22%2c%22item%22%2c%22skus%22%2c6260915%2c%22conditions%22%2c%22NONE%22%2c%22destinationZipCode%22%2c92582%2c%22storeId%22%2c%201441%2c%22context%22%2c%22cyp%22%2c%22addAll%22%2c%22false%22%5D%5D&method=get

Both apis load instantly and have no rate limits.

[jef]

Awesome! Thanks a lot for scraping this information. I appreciate the help! Hopefully we can get this in here soon enough!

Also, how can you be sure about this?

Both apis load instantly and have no rate limits.

I know normally requests get sent back some sort of rate limit header, but some APIs do not. Have you sent N requests to test?

[TheTabKey]

Awesome! Thanks a lot for scraping this information. I appreciate the help! Hopefully we can get this in here soon enough!

Also, how can you be sure about this?

Both apis load instantly and have no rate limits.

I know normally requests get sent back some sort of rate limit header, but some APIs do not. Have you sent N requests to test?

I have done the tests that I can, and no issue.....I even tried spamming for a while but no rate limits were detected as far as I can see.

[TheTabKey]

Is there any other configuration needed for the newegg api implementation? Ill run constant hits all night

[jef]

Nope! Should just be able to set the STORES=newegg and be good to go. I'll run it myself! Good thinking.

[jef]

Right now, the code is doing both lookups, API and browser. I suppose we should probably only do API. I'll make a note of that here and when we implement bestbuy, we can update that as well.

[jbwilliams1]

I would caution against polling the real time API unless the website indicates possible stock. I happen to know some DevOps folks at Best Buy and their origins are... not great.

Can always try though. Otherwise I can implement the same thing I did for Newegg, for Best Buy

[TheTabKey]

I would caution against polling the real time API unless the website indicates possible stock. I happen to know some DevOps folks at Best Buy and their origins are... not great.

Can always try though. Otherwise I can implement the same thing I did for Newegg, for Best Buy

If you want to create a beta branch with bestbuy api implemented, Ill run it for a day and see how it is. Just make sure to make it so it only checks the api page and not the main product page for both newegg and bestbuy.

[jbwilliams1]

I can take this, maybe today. I'm going to first add it as a redundancy for the normal store page, but also will add the API only option as a different store

[TheTabKey]

I can take this, maybe today. I'm going to first add it as a redundancy for the normal store page, but also will add the API only option as a different store

Yeah thats what I was thinking, make it newegg-api like how nvidia is setup

[jef]

but also will add the API only option as a different store

I think that's a good idea with newegg as well.

I won't step over you if you'd like to work on this. Thanks!

[TheTabKey]

Asus definately times you out fast if you use both.

[jbwilliams1]

@TheTabKey nah, Asus has been throwing a ton of 503s and routing you to their other servers. I think they are having outages, I was seeing 503s even when manually browsing w/ no snatcher running

[jxjftw]

Just an FYI, I've been scraping the bestbuy API for a bit over a month now, the API seems to be a bit behind what is actually displayed on the page, could be 20 seconds behind or 2 minutes. Also there are 2 implementations you can do, the authenticated API will rate limit you, the non-authed seemed good for about 2 hits a second.

[TheTabKey]

Just an FYI, I've been scraping the bestbuy API for a bit over a month now, the API seems to be a bit behind what is actually displayed on the page, could be 20 seconds behind or 2 minutes. Also there are 2 implementations you can do, the authenticated API will rate limit you, the non-authed seemed good for about 2 hits a second.

That api tells the page what to display....Maybe its a mix of check intervals and the queue system implementation? Unless your referring to cooperate api? After investigation though, the api above tells the page what button to display.