i136-create-s3-download-to-bulkrax-import-rake-task

[aprilrieger]

Ref Ticket #scientist-softserv#136

Adding a script to download obj from s3 bucket folder and create a bulkrax importer that was from when Rob migrated the objs previously but added in a reusable manner.

How to Test

Set these env vars in the .env locally (don't commit secrets)

AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_S3_BUCKET="ams.americanarchive.org"
AWS_REGION=us-east-1
S3_FOLDER="april-test-script"

In the docker container docker compose exec web bash run rake ams:copy_files_from_s3

Once the script completes you will be able to navigate to the Bulkrax importers page and see some new importers that are in status Pending with titles like AMS1Importer_#{start_at}-#{end_at}"

They will only be run when you manually run them. You will need to go to the importer's page and select Edit at bottom of page and click the Update and Import button to run the importer.

[afred]

The glob #{filepath/*xml} is the same as "tmp/imports/#{batch_name}/*xml", and I don't think anything in that glob is going to have leading or trailing white space that would result in landing in the condition file != file.strip.... iow, i don't think this will ever actually move anything. What are the conditions for which files were needing to be moved?

Also, the output "File renamed..." should probably be inside the conditional that does the renaming, right?