Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error 500 when trying to use the goodtable options #46

Open
renatocron opened this issue Sep 17, 2019 · 0 comments
Open

Error 500 when trying to use the goodtable options #46

renatocron opened this issue Sep 17, 2019 · 0 comments

Comments

@renatocron
Copy link

Hello,

I'm new to CKAN, so maybe I'm missing something on the config, but I'm unable to make even the sample README "id;group;measurement" to work.

I'm running CKAN 2.7.5 and downloaded the extensions from the branch master (both ckanext-validation.git and ckanext-scheming, the only issue I had was necessary to run pip install six==1.11.0 before installing ckanext-validation)

I added the presets resource_schema, validation_options and hidden_in_form to /usr/lib/ckan/default/src/./ckanext-scheming/ckanext/scheming/presets.json
and my config looks like this:

scheming.dataset_schemas=ckanext.validation.examples:ckan_default_schema.json
scheming.presets =			ckanext.scheming:presets.json

scheming.dataset_fallback = false

ckanext.validation.run_on_create_sync = True
ckanext.validation.run_on_update_sync = True

When I try to upload

<2 blank lines>
id;group;measurement
# 2017
1;A;23
2;B;24
# 2016
3;C;23
4;C;25

using no schema selected, and { "headers": 3, "delimiter": ";", "skip_rows": ["#"], "skip_checks": ["blank-rows"] }
as Validation options
I have an error 500, and the log goes as:

URL: http://fgv-ckan-aws.appcivico.com/dataset/new_resource/test-dt
File '/usr/lib/ckan/default/lib/python2.7/site-packages/weberror/errormiddleware.py', line 171 in __call__
  app_iter = self.application(environ, sr_checker)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
  resp = self.call_func(req, *args, **self.kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
  return self.func(req, *args, **kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/fanstatic/publisher.py', line 234 in __call__
  return request.get_response(self.app)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
  application, catch_exc_info=False)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
  app_iter = application(self.environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
  resp = self.call_func(req, *args, **self.kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
  return self.func(req, *args, **kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/fanstatic/injector.py', line 54 in __call__
  response = request.get_response(self.app)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
  application, catch_exc_info=False)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
  app_iter = application(self.environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/config/middleware/pylons_app.py', line 268 in inner
  result = application(environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/beaker/middleware.py', line 73 in __call__
  return self.app(environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/beaker/middleware.py', line 156 in __call__
  return self.wrap_app(environ, session_start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/routes/middleware.py', line 131 in __call__
  response = self.app(environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/config/middleware/common_middleware.py', line 80 in __call__
  return self.app(environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/wsgiapp.py', line 125 in __call__
  response = self.dispatch(controller, environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/wsgiapp.py', line 324 in dispatch
  return controller(environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 212 in __call__
  res = WSGIController.__call__(self, environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 221 in __call__
  response = self._dispatch_call()
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 172 in _dispatch_call
  response = self._inspect_call(func)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 107 in _inspect_call
  result = self._perform_call(func, args)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 60 in _perform_call
  return func(**args)
File '/usr/lib/ckan/default/src/ckan/ckan/controllers/package.py', line 681 in new_resource
  get_action('resource_create')(context, data)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py', line 457 in wrapped
  result = _action(context, data_dict, **kw)
File '/usr/lib/ckan/default/src/ckan/ckanext-validation/ckanext/validation/logic.py', line 489 in resource_create
  resource_id, local_upload=is_local_upload, new_resource=True)
File '/usr/lib/ckan/default/src/ckan/ckanext-validation/ckanext/validation/logic.py', line 637 in _run_sync_validation
  if not report['valid']:
TypeError: 'NoneType' object has no attribute '__getitem__'

I have another csv, using this schema: https://gist.githubusercontent.com/renatocron/7c84d31196a0a0da34fa8f9514253ac6/raw/df279c3b92cf7c19f376d1cb9143a54aa71a03f1/gistfile1.txt
and when I try the options { "skip_checks": ["blank-rows", "duplicate-headers"], "checks": ["sequential-value": {"column": "id"} ], "headers": 3}
I also get an error 500, but with a different message:

Error - <class 'goodtables.exceptions.GoodtablesException'>: Check "column" is not registered
URL: http://fgv-ckan-aws.appcivico.com/dataset/new_resource/test-dt
File '/usr/lib/ckan/default/lib/python2.7/site-packages/weberror/errormiddleware.py', line 171 in __call__
  app_iter = self.application(environ, sr_checker)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
  resp = self.call_func(req, *args, **self.kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
  return self.func(req, *args, **kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/fanstatic/publisher.py', line 234 in __call__
  return request.get_response(self.app)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
  application, catch_exc_info=False)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
  app_iter = application(self.environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
  resp = self.call_func(req, *args, **self.kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
  return self.func(req, *args, **kwargs)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/fanstatic/injector.py', line 54 in __call__
  response = request.get_response(self.app)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
  application, catch_exc_info=False)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
  app_iter = application(self.environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/config/middleware/pylons_app.py', line 268 in inner
  result = application(environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/beaker/middleware.py', line 73 in __call__
  return self.app(environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/beaker/middleware.py', line 156 in __call__
  return self.wrap_app(environ, session_start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/routes/middleware.py', line 131 in __call__
  response = self.app(environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/config/middleware/common_middleware.py', line 80 in __call__
  return self.app(environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/wsgiapp.py', line 125 in __call__
  response = self.dispatch(controller, environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/wsgiapp.py', line 324 in dispatch
  return controller(environ, start_response)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 212 in __call__
  res = WSGIController.__call__(self, environ, start_response)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 221 in __call__
  response = self._dispatch_call()
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 172 in _dispatch_call
  response = self._inspect_call(func)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 107 in _inspect_call
  result = self._perform_call(func, args)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 60 in _perform_call
  return func(**args)
File '/usr/lib/ckan/default/src/ckan/ckan/controllers/package.py', line 681 in new_resource
  get_action('resource_create')(context, data)
File '/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py', line 457 in wrapped
  result = _action(context, data_dict, **kw)
File '/usr/lib/ckan/default/src/ckan/ckanext-validation/ckanext/validation/logic.py', line 489 in resource_create
  resource_id, local_upload=is_local_upload, new_resource=True)
File '/usr/lib/ckan/default/src/ckan/ckanext-validation/ckanext/validation/logic.py', line 624 in _run_sync_validation
  u'async': False})
File '/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py', line 457 in wrapped
  result = _action(context, data_dict, **kw)
File '/usr/lib/ckan/default/src/ckan/ckanext-validation/ckanext/validation/logic.py', line 131 in resource_validation_run
  run_validation_job(resource)
File '/usr/lib/ckan/default/src/ckan/ckanext-validation/ckanext/validation/jobs.py', line 89 in run_validation_job
  report = _validate_table(source, _format=_format, schema=schema, **options)
File '/usr/lib/ckan/default/src/ckan/ckanext-validation/ckanext/validation/jobs.py', line 122 in _validate_table
  report = validate(source, format=_format, schema=schema, **options)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/goodtables/validate.py', line 56 in validate
  report = inspector.inspect(source, preset=preset, **options)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/goodtables/inspector.py', line 80 in inspect
  table_warnings, table_report = task.get()
File '/usr/lib/python2.7/multiprocessing/pool.py', line 567 in get
  raise self._value
GoodtablesException: Check "column" is not registered

I tried to look at that code try to understand what it's supposed to be registered but I don't get it yet, I'm actually a perl programmer, python is new for me!

JVickery-TBS pushed a commit to JVickery-TBS/ckanext-validation that referenced this issue Jul 23, 2024
CHDEXCHAE-57 / check if there is a schema before validation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant