Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ETL can't perform clean up due to S3 bad request #38

Open
mattfullerton opened this issue Dec 7, 2015 · 0 comments
Open

ETL can't perform clean up due to S3 bad request #38

mattfullerton opened this issue Dec 7, 2015 · 0 comments

Comments

@mattfullerton
Copy link
Contributor

After finally finding somewhere for the bucket where data can be uploaded, the app doesn't appear to be able to get the data back out. The user has S3 full access. I suppose there is a small chance this is the reason for #35...

Debug:
[2015-12-07 23:11:32,967: WARNING/Worker-2] Failed: Clean up source data
[2015-12-07 23:11:33,693: ERROR/MainProcess] Task spendb.tasks.load_from_source[14d7ea38-b1c1-4540-b9df-9d14fca22263] raised unexpected: S3ResponseError: 400 Bad Request
Traceback (most recent call last):
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(_args, *_kwargs)
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/celery/app/trace.py", line 438, in protected_call
return self.run(_args, *_kwargs)
File "/Volumes/MacintoshHD2/Users/mattbirgit/checkouts/spendb/spendb/tasks.py", line 36, in load_from_source
source = tasks.transform_source(dataset, source_name)
File "/Volumes/MacintoshHD2/Users/mattbirgit/checkouts/spendb/spendb/etl/job.py", line 74, in wrapper
job.failed()
File "/Volumes/MacintoshHD2/Users/mattbirgit/checkouts/spendb/spendb/etl/job.py", line 53, in failed
self.end(Run.STATUS_FAILED)
File "/Volumes/MacintoshHD2/Users/mattbirgit/checkouts/spendb/spendb/etl/job.py", line 39, in end
self.log_handler.archive()
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/loadkit/logger.py", line 27, in archive
logfile = LogFile(self.package, name)
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/archivekit/resource.py", line 21, in init
self.meta = ResourceMetaData(self)
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/archivekit/manifest.py", line 38, in init
self.manifest = resource.package.manifest
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/archivekit/package.py", line 42, in manifest
self._manifest = Manifest(obj)
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/archivekit/manifest.py", line 14, in init
self.load()
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/archivekit/manifest.py", line 17, in load
if self.object.exists():
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/archivekit/store/s3.py", line 98, in exists
self._key = self.store.bucket.get_key(self._key_name)
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/archivekit/store/s3.py", line 34, in bucket
self._bucket = self.conn.get_bucket(self.bucket_name)
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/boto/s3/connection.py", line 502, in get_bucket
return self.head_bucket(bucket_name, headers=headers)
File "/Volumes/MacintoshHD2/Users/mattbirgit/venvs/spendb/lib/python2.7/site-packages/boto/s3/connection.py", line 549, in head_bucket
response.status, response.reason, body)
S3ResponseError: S3ResponseError: 400 Bad Request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant