Skip to content

Latest commit

 

History

History
47 lines (31 loc) · 1.35 KB

README.md

File metadata and controls

47 lines (31 loc) · 1.35 KB
Raw

derzis Node.js CI

A path-aware distributed linked data crawler.

Run

The easiest way of deploying Derzis is using Docker, but it can also be executed without using containers.

Docker

Dependencies

  • docker
  • docker-compose

Running

  • Edit src/config/index.ts to modify the crawler parameters.
  • Run docker-compose up --build.
  • The web app will be accessible at http://localhost:5432/processes

Locally

Install dependencies

  • MongoDB
  • Redis
  • Node.js v20
  • Run npm install on the root folder

Run in development mode

  • Open a terminal and run npm run worker:dev.
  • Open another terminal and run npm run manager:dev.
  • The web app will be accessible at http://localhost:5432/processes

Build and run

  • Run npm run build on the root folder
  • Make sure you have Redis and MongoDB running.
  • Edit dist/src/config/index.js to modify the crawler parameters.
  • Open a terminal and run node ./dist/src/manager/bin/manager.js.
  • Open another terminal and run ./dist/src/worker/bin/worker-pool.
  • The web app will be accessible at http://localhost:5432/processes