v0.5.6

0.5.6 - 2024-03-06

Added

• Added optional ENV variable REDIS_DB.

v0.5.5

0.5.5 - 2023-11-13

Fixed

• Fixed Content-Length calculation method in POST requests to callbackUri.

v0.5.4

0.5.4 - 2023-10-27

Added

• Added the scenario option config.options.proxyUrls.
• Added action evaluate.
• Added strategy evaluate in the action collectData.

v0.5.3

0.5.3 - 2023-10-23

Changed

• Database connections now always have UTC timezone.

v0.5.2

0.5.2 - 2023-10-18

Added

• Added errors logging in the Crawler's errorHandler handler.

Fixed

• Fixed the value of the field "finishedAt" in the result returned from Crawler.

v0.5.1

0.5.1 - 2023-09-20

Fixed

• Fixed cookies transferring between requests with different sessions.

v0.5.0

0.5.0 - 2023-07-10

Added

• Added field active for scenario schedulers. The field is required for POST/PUT endpoints, and it is also returned by GET endpoints.
• Added endpoints PUT /api/scenario-schedulers/:scenarioSchedulerId/activate and PUT /api/scenario-schedulers/:scenarioSchedulerId/activate.

v0.4.3

0.4.3 - 2023-07-10

Changed

• Changed pm2 configuration file for docker images 68publishers/crawler:{version}-app. The application is launched in the fork mode instead of the cluster mode.

Fixed

• Fixed zombie chrome processes - worker container is run with the option --init - the process should be used as the PID 1 in the container.

v0.4.2

0.4.2 - 2023-07-03

Added

• Added separated docker images for development (dev-app, dev-scheduler, dev-worker).
• Added separated docker images for production (prod-app, prod-scheduler, prod-worker). Existing all-in-one image has now target prod-all.

Changed

• Log files in var/log are now kept for 7 days only.
• The crawler waits for the request queue to empty before exiting (until now, it only waited during an aborted scenario).
• Changed Release GitHub Action. Images for specific entrypoints are now also released with pattern 68publishers/crawler:{$version}-{app|scheduler|worker}.

v0.4.1

0.4.1 - 2023-06-23

Changed

• Developer containers have been resized (chrome is installed for the worker image only).