Skip to content

Commit

Permalink
Merge pull request #47 from NeuromatchAcademy/update-robots.txt-action
Browse files Browse the repository at this point in the history
add `Undocumented AI Agent` type to Dark Visitors API call
  • Loading branch information
hobgoblina authored May 7, 2024
2 parents 7f91e30 + 0ad7eef commit 93eca10
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion .github/workflows/update-robots-txt.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
curl --location 'https://api.darkvisitors.com/robots-txts' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $API_KEY" \
--data '{ "agent_types": [ "AI Data Scraper", "AI Assistant", "AI Search Crawler" ], "disallow": "/" }' >> public/robots.txt
--data '{ "agent_types": [ "AI Data Scraper", "AI Assistant", "AI Search Crawler", "Undocumented AI Agent" ], "disallow": "/" }' >> public/robots.txt
- name: Create pull request
uses: peter-evans/create-pull-request@v6
with:
Expand Down

0 comments on commit 93eca10

Please sign in to comment.