Skip to content

Commit

Permalink
fix: Do not use E escape in Redshift batch exports (#18706)
Browse files Browse the repository at this point in the history
fix: Do not use E escape
  • Loading branch information
tomasfarias authored Nov 17, 2023
1 parent c1084ba commit 3b4471e
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions posthog/temporal/workflows/redshift_batch_export.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,10 @@ async def insert_records_to_redshift(
pre_query_str = pre_query.as_string(cursor).encode("utf-8")

async def flush_to_redshift(batch):
await cursor.execute(pre_query_str + b",".join(batch))
rows_exported.add(len(batch) - 1)
values = b",".join(batch).replace(b" E'", b" '")

await cursor.execute(pre_query_str + values)
rows_exported.add(len(batch))
# It would be nice to record BYTES_EXPORTED for Redshift, but it's not worth estimating
# the byte size of each batch the way things are currently written. We can revisit this
# in the future if we decide it's useful enough.
Expand Down

0 comments on commit 3b4471e

Please sign in to comment.