Skip to content

Scrape Data Every 7 Hours #540

Scrape Data Every 7 Hours

Scrape Data Every 7 Hours #540

Workflow file for this run

name: Scrape Data Every 7 Hours
on:
schedule:
- cron: '0 */7 * * *' # Runs every 7 hours
workflow_dispatch: # Allows manual triggering
jobs:
scrape:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: '20' # Using Node.js version 20
- name: Install dependencies
run: npm install
- name: Get chmod
run: chmod +x backend/scrape.mjs # brev eat a fokin deck
# If your scraper depends on an API running, then keep this step
- name: Start API server
run: |
nohup node backend/api.js & # Starts the API server in the background
sleep 60
# Run the scraper
- name: Run scraper script # Eat a d**ck
run: node backend/scrape.mjs # Ensure this script handles scraping and storing the data
# If your scraper updates files that need to be committed back to the repository
- name: Commit and push changes
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add .
git commit -m "Auto-update data file"
git push