Solution to the Devprojects Web Scraper --> https://www.codementor.io/projects/tool/web-scraper-to-get-news-article-content-atx32d46qe
We want to build a simple web scraper that will return the content of a news article when given a specific URL. Some examples of real products which use similar technologies include price-tracking websites and SEO audit tools which may scrape top search results. This project may take you around 4 to 8 hours to complete.
Requirements Choose one news website - see article examples below for inspiration. Given a specific article URL from the website of your choice, return the title and content of the article to the user.
Examples article URLs:
https://www.nytimes.com/2020/09/02/opinion/remote-learning-coronavirus.html https://www.washingtonpost.com/technology/2020/09/25/privacy-check-blacklight/ https://edition.cnn.com/travel/article/scenic-airport-landings-2020/index.html https://www.reuters.com/article/us-health-coronavirus-global-deaths/global-coronavirus-deaths-pass-agonizing-milestone-of-1-million-idUSKBN26K08Y
For an extra challenge: Parse out information such as the article title, updated date, and byline to return separately to the user.