Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Materialization crashes on TIMESTAMP-typed fields #17

Open
Felix-neko opened this issue Feb 2, 2022 · 0 comments
Open

Materialization crashes on TIMESTAMP-typed fields #17

Felix-neko opened this issue Feb 2, 2022 · 0 comments

Comments

@Felix-neko
Copy link
Contributor

Hi again! I've faced yet another problem = (

I've tried to use a Hive table where one of the field had TIMESTAMP type. When I tried to run feast materialize incremental ... it crashed with the following stacktrace.

Traceback (most recent call last):
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/bin/feast", line 8, in <module>
    sys.exit(cli())
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/cli.py", line 469, in materialize_incremental_command
    end_date=utils.make_tzaware(datetime.fromisoformat(end_ts)),
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/usage.py", line 280, in wrapper
    raise exc.with_traceback(traceback)
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/usage.py", line 269, in wrapper
    return func(*args, **kwargs)
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/feature_store.py", line 884, in materialize_incremental
    tqdm_builder=tqdm_builder,
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/infra/passthrough_provider.py", line 151, in materialize_single_feature_view
    rows_to_write = _convert_arrow_to_proto(batch, feature_view, join_keys)
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/infra/provider.py", line 317, in _convert_arrow_to_proto
    for column, dtype in columns
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/infra/provider.py", line 317, in <dictcomp>
    for column, dtype in columns
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/type_map.py", line 332, in python_values_to_proto_values
    return _python_value_to_proto_value(value_type, values)
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/type_map.py", line 293, in _python_value_to_proto_value
    return [ProtoValue(int64_val=int(value)) for value in values]
  File "/home/felix/.cache/pypoetry/virtualenvs/feast-hive-tests-NQ4GOj-a-py3.7/lib/python3.7/site-packages/feast/type_map.py", line 293, in <listcomp>
    return [ProtoValue(int64_val=int(value)) for value in values]
TypeError: int() argument must be a string, a bytes-like object or a number, not 'datetime.datetime'

My test repo and base population script to reproduce the error could be found here:
https://github.com/Felix-neko/feast_sandbox/tree/timestamp_field_checkup/repos/timestamp_checkup_repo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant