SurveyCTO stream without SubmissionDate

[fatchat]

Dwani has a SurveyCTO stream without a SubmissionDate field

2024-08-13 09:50:28 source > Marking stream education_infrastructure_ as STARTED
2024-08-13 09:50:28 source > Encountered an exception while reading stream education_infrastructure_
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/abstract_source.py", line 135, in read
    yield from self._read_stream(
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/abstract_source.py", line 216, in _read_stream
    stream_instance.state = stream_state  # type: ignore # we check that state in the dir(stream_instance)
  File "/airbyte/integration_code/source_surveycto/source.py", line 79, in state
    self._cursor_value = value[self.cursor_field]
KeyError: 'SubmissionDate'

Our connector assumes the existence of this field. Is this an incorrect assumption?

• Can we start by looking at their API response

Update: @jakejh advises us to prefer CompletionDate to SubmissionDate

[fatchat]

ugh

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/abstract_source.py", line 133, in read
    yield from self._read_stream(
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/abstract_source.py", line 225, in _read_stream
    stream_instance.state = stream_state  # type: ignore # we check that state in the dir(stream_instance)
  File "/airbyte/integration_code/source_surveycto/source.py", line 174, in state
    self._cursor_value = value[self.cursor_field]
KeyError: 'CompletionDate'

Important: This command can only be run by one of the repository admins:
Usage: ./tools/bin/get_repo_admins.sh <personal_access_token>

[fatchat]

correction: the connector wasn't looking in the API response - which does contain the SubmissionDate - it was looking in the (empty) saved state

putting in a check fixes it

Important: This command can only be run by one of the repository admins:
Usage: ./tools/bin/get_repo_admins.sh <personal_access_token>