Simple Web Fetcher in Ruby

Description

A simple web fetcher that fetches and saves the target website and its metadata into the folder with the same name as the website domain.

./fetch https://www.google.com -a

Installation & Run

Run and get into the terminal of the container

docker-compose run web-fetcher bash

(Inside Container) Call the program as a command line tool:

./fetch https://www.google.com

(Inside Container) Call the program with flags:

• Help Information

./fetch -h

./fetch --help

• Display Metadata

./fetch https://www.google.com -m

./fetch https://www.google.com --metadata

• Archive Asset

./fetch https://www.google.com -a

./fetch https://www.google.com --archive

You can also directly run the program by appending the command after docker-compose run

(This way will create a new container every time, not recommended)

docker-compose run web-fetcher ./fetch https://www.google.com/ -m