Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorporate news APIs #8

Open
MatthewChatham opened this issue May 18, 2018 · 1 comment
Open

Incorporate news APIs #8

MatthewChatham opened this issue May 18, 2018 · 1 comment

Comments

@MatthewChatham
Copy link
Owner

We need to investigate the APIs available to us.

NYTimes: https://developer.nytimes.com
Allows you to set date ranges and search terms, returns WAY more than I'm scraping: https://developer.nytimes.com/article_search_v2.json#/Documentation/GET/articlesearch.json

general api: https://newsapi.org/

news API looks like the best bet

@MatthewChatham
Copy link
Owner Author

Is there a reason to use scraping over an API, other than the ability to practice and show off some skills?

Probably not, if the only goal is to produce a machine learning dataset. If I wanted to study some questions about the behavior of these publications' online outlets it would make sense to scrape them (e.g., to learn how long articles tend to stay up).

@MatthewChatham MatthewChatham changed the title APIs Incorporate news APIs May 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant