You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current scraper generates a static cache file which we cannot update. This means we cannot provide any historical data on an archetype which limits our utility.
A better deck scraper would acquire decks from mtgtop8, massage the data, and put everything into a series of sane tables in postgres.
Additionally, a video scraper from mtgcoverage looks possible and would provide a lot of value.
The text was updated successfully, but these errors were encountered:
Structure the scraper ala priceWriter with two asymmetric sources that handle the database access separately. Make sure that any panics are recovered inside the scrapers to avoid full crashes on markup changes.
Deck names need to be normalized between mtgtop8 and mtgcoverage.
mtgtop8 provides extremely broad categories, ie UrzaTron for U Tron. Granular naming by looking for specific cards, ie snapcaster or sad bot in U tron, could do it.
The current scraper generates a static cache file which we cannot update. This means we cannot provide any historical data on an archetype which limits our utility.
A better deck scraper would acquire decks from mtgtop8, massage the data, and put everything into a series of sane tables in postgres.
Additionally, a video scraper from mtgcoverage looks possible and would provide a lot of value.
The text was updated successfully, but these errors were encountered: