Skip to content

Commit

Permalink
fix: index out of range error in feast integration (#68)
Browse files Browse the repository at this point in the history
* fix: index out of range error in feast integration

* switch to logging with logger

* version bump

* update gitignore

* fix lint errors
  • Loading branch information
emgeee authored Dec 10, 2024
1 parent d5a056c commit d86a124
Show file tree
Hide file tree
Showing 6 changed files with 43 additions and 8 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@
.DS_Store
.ipynb_checkpoints/
Untitled.ipynb
*.o
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion py-denormalized/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "denormalized-python"
version = "0.0.11"
version = "0.0.12"
edition = "2021"
homepage = "https://github.com/probably-nothing-labs/denormalized.git"
repository = "https://github.com/probably-nothing-labs/denormalized.git"
Expand Down
1 change: 1 addition & 0 deletions py-denormalized/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ dev = [
"jupyterlab>=4.3.0",
"pdoc>=15.0.0",
"pip>=24.3.1",
"ruff>=0.8.2",
]
docs = [
"pdoc>=15.0.0",
Expand Down
16 changes: 11 additions & 5 deletions py-denormalized/python/denormalized/feast_data_stream.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import inspect
import logging
from typing import Any, TypeVar, Union, cast, get_type_hints

import pyarrow as pa
Expand All @@ -10,6 +11,8 @@

from .data_stream import DataStream

logger = logging.getLogger(__name__)

T = TypeVar("T")


Expand Down Expand Up @@ -108,10 +111,13 @@ def write_feast_feature(
"""

def _sink_to_feast(rb: pa.RecordBatch):
df = rb.to_pandas()
try:
feature_store.push(source_name, df, to=PushMode.ONLINE)
except Exception as e:
print(e)
if len(rb):
df = rb.to_pandas()
try:
feature_store.push(source_name, df, to=PushMode.ONLINE)
except Exception as e:
logger.error(
f"Failed to push to Feast feature store: {e}", exc_info=True
)

self.ds.sink_python(_sink_to_feast)
29 changes: 28 additions & 1 deletion py-denormalized/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit d86a124

Please sign in to comment.