Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Something went wrong in the connector - "source-github" #399

Open
obaidmuneer opened this issue Sep 27, 2024 · 2 comments
Open

Something went wrong in the connector - "source-github" #399

obaidmuneer opened this issue Sep 27, 2024 · 2 comments

Comments

@obaidmuneer
Copy link

obaidmuneer commented Sep 27, 2024

import airbyte as ab
source = ab.get_source(
    "source-github",
    install_if_missing=True,
    config={
        "repositories": ["airbytehq/quickstarts"],
        "credentials": {
            "personal_access_token": ab.get_secret("GITHUB_PERSONAL_ACCESS_TOKEN"),
        },
    },
)
source.check()
source.set_streams(["pull_requests", "issues", "reviews", "stargazers"])
cache = ab.get_default_cache()
result = source.read(cache=cache)

error:

Writing PyAirbyte logs to file: C:\Users\OBAIDM~1\AppData\Local\Temp\airbyte\logs\2024-09-27\airbyte-log-J8SKMYTPJ.log
Writing `source-github` logs to file: C:\Users\OBAIDM~1\AppData\Local\Temp\airbyte\logs\source-github\source-github-log-J8SKMYTQ6.log
Connection check succeeded for `source-github`.
C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\github.py:17: DeprecationWarning: The 'set_streams' method is deprecated and will be removed in a future version. Please use the 'select_streams' method instead.
  source.set_streams(["pull_requests", "issues", "reviews", "stargazers"])
ERROR: Something went wrong in the connector. See the logs for more details.
Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\github.py", line 17, in <module>
    source.set_streams(["pull_requests", "issues", "reviews", "stargazers"])
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 99, in set_streams
    self.select_streams(streams)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 147, in select_streams
    available_streams = self.get_available_streams()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 219, in get_available_streams
    return [s.name for s in self.discovered_catalog.streams]
                            ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 325, in discovered_catalog
    self._discovered_catalog = self._discover()
                               ^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 209, in _discover
    for msg in self._execute(["discover", "--config", config_file]):
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\_connector_base.py", line 440, in _execute
    raise exc.AirbyteConnectorFailedError(
airbyte.exceptions.AirbyteConnectorFailedError: Connector failed. (AirbyteConnectorFailedError)
------------------------------------------------------------
AirbyteConnectorFailedError: Connector failed.
    Please review the log file for more information.
    Connector Name: 'source-github'
    Exit Code: 1
    Log file: C:\Users\OBAIDM~1\AppData\Local\Temp\airbyte\logs\source-github\source-github-log-J8SKMYTQ6.log

would be apricated for help

@obaidmuneer
Copy link
Author

Log file: C:\Users\OBAIDM~1\AppData\Local\Temp\airbyte\logs\source-github\source-github-log-J8SKMYTQ6.log
2024-09-27 16:25:25 - INFO - Check succeeded
2024-09-27 16:25:28 - INFO - [WinError 267] The directory name is invalid: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'
Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 252, in __subject__
    return self.cache
           ^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 176, in __getattribute__
    return _oga(self, attr)
           ^^^^^^^^^^^^^^^^
AttributeError: 'JsonRef' object has no attribute 'cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\jsonref.py", line 178, in callback
    base_doc = self.loader(uri)
               ^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 33, in __call__
    with open(uri) as f:
         ^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: ''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\entrypoint.py", line 117, in run
    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.discover(source_spec, config))
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\entrypoint.py", line 152, in discover
    catalog = self.source.discover(self.logger, config)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\abstract_source.py", line 74, in discover
    streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\abstract_source.py", line 74, in <listcomp>
    streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\streams\core.py", line 286, in as_airbyte_stream
    json_schema=dict(self.get_json_schema()),
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\streams\core.py", line 281, in get_json_schema
    return ResourceSchemaLoader(package_name_from_class(self.__class__)).get_schema(self.name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 140, in get_schema
    return self._resolve_schema_references(raw_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 156, in _resolve_schema_references
    resolved = resolve_ref_links(resolved)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 58, in resolve_ref_links
    return {k: resolve_ref_links(v) for k, v in obj.items()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 58, in <dictcomp>
    return {k: resolve_ref_links(v) for k, v in obj.items()}
               ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 58, in resolve_ref_links
    return {k: resolve_ref_links(v) for k, v in obj.items()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 58, in <dictcomp>
    return {k: resolve_ref_links(v) for k, v in obj.items()}
               ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 49, in resolve_ref_links
    obj = resolve_ref_links(obj.__subject__)
                            ^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 176, in __getattribute__
    return _oga(self, attr)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 254, in __subject__
    self.cache = super(LazyProxy, self).__subject__
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 240, in __subject__
    return self.callback()
           ^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\jsonref.py", line 180, in callback
    self._error("%s: %s" % (e.__class__.__name__, unicode(e)), cause=e)
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\jsonref.py", line 223, in _error
    raise JsonRefError(
jsonref.JsonRefError: FileNotFoundError: [Errno 2] No such file or directory: ''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 620, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 878, in onerror
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Scripts\source-github.exe\__main__.py", line 7, in <module>
    sys.exit(run())
             ^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\source_github\run.py", line 17, in run
    launch(source, sys.argv[1:])
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\entrypoint.py", line 234, in launch
    for message in source_entrypoint.run(parsed_args):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\entrypoint.py", line 99, in run
    with tempfile.TemporaryDirectory() as temp_dir:
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 904, in __exit__
    self.cleanup()
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 908, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 622, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 881, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 603, in _rmtree_unsafe
    onerror(os.scandir, path, sys.exc_info())
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 600, in _rmtree_unsafe
    with os.scandir(path) as scandir_it:
         ^^^^^^^^^^^^^^^^
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'
2024-09-27 16:25:28 - ERROR - Something went wrong in the connector. See the logs for more details.
2024-09-27 16:25:28 - INFO - {"type":"TRACE","trace":{"type":"ERROR","emitted_at":1727436328748.815,"error":{"message":"Something went wrong in the connector. See the logs for more details.","internal_message":"[WinError 267] The directory name is invalid: 'C:\\\\Users\\\\OBAIDM~1\\\\AppData\\\\Local\\\\Temp\\\\tmpuvxwz6tc\\\\comments.sqlite'","stack_trace":"Traceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 252, in __subject__\n    return self.cache\n           ^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 176, in __getattribute__\n    return _oga(self, attr)\n           ^^^^^^^^^^^^^^^^\nAttributeError: 'JsonRef' object has no attribute 'cache'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\jsonref.py\", line 178, in callback\n    base_doc = self.loader(uri)\n               ^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 33, in __call__\n    with open(uri) as f:\n         ^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: ''\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\entrypoint.py\", line 117, in run\n    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.discover(source_spec, config))\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\entrypoint.py\", line 152, in discover\n    catalog = self.source.discover(self.logger, config)\n              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\abstract_source.py\", line 74, in discover\n    streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]\n              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\abstract_source.py\", line 74, in <listcomp>\n    streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\streams\\core.py\", line 286, in as_airbyte_stream\n    json_schema=dict(self.get_json_schema()),\n                     ^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\streams\\core.py\", line 281, in get_json_schema\n    return ResourceSchemaLoader(package_name_from_class(self.__class__)).get_schema(self.name)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 140, in get_schema\n    return self._resolve_schema_references(raw_schema)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 156, in _resolve_schema_references\n    resolved = resolve_ref_links(resolved)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 58, in resolve_ref_links\n    return {k: resolve_ref_links(v) for k, v in obj.items()}\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 58, in <dictcomp>\n    return {k: resolve_ref_links(v) for k, v in obj.items()}\n               ^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 58, in resolve_ref_links\n    return {k: resolve_ref_links(v) for k, v in obj.items()}\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 58, in <dictcomp>\n    return {k: resolve_ref_links(v) for k, v in obj.items()}\n               ^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 49, in resolve_ref_links\n    obj = resolve_ref_links(obj.__subject__)\n                            ^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 176, in __getattribute__\n    return _oga(self, attr)\n           ^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 134, in wrapper\n    return method(self, *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 254, in __subject__\n    self.cache = super(LazyProxy, self).__subject__\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 134, in wrapper\n    return method(self, *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 240, in __subject__\n    return self.callback()\n           ^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 134, in wrapper\n    return method(self, *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\jsonref.py\", line 180, in callback\n    self._error(\"%s: %s\" % (e.__class__.__name__, unicode(e)), cause=e)\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 134, in wrapper\n    return method(self, *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\jsonref.py\", line 223, in _error\n    raise JsonRefError(\njsonref.JsonRefError: FileNotFoundError: [Errno 2] No such file or directory: ''\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 620, in _rmtree_unsafe\n    os.unlink(fullname)\nPermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\OBAIDM~1\\\\AppData\\\\Local\\\\Temp\\\\tmpuvxwz6tc\\\\comments.sqlite'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 878, in onerror\n    _os.unlink(path)\nPermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\OBAIDM~1\\\\AppData\\\\Local\\\\Temp\\\\tmpuvxwz6tc\\\\comments.sqlite'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"<frozen runpy>\", line 198, in _run_module_as_main\n  File \"<frozen runpy>\", line 88, in _run_code\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Scripts\\source-github.exe\\__main__.py\", line 7, in <module>\n    sys.exit(run())\n             ^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\source_github\\run.py\", line 17, in run\n    launch(source, sys.argv[1:])\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\entrypoint.py\", line 234, in launch\n    for message in source_entrypoint.run(parsed_args):\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\entrypoint.py\", line 99, in run\n    with tempfile.TemporaryDirectory() as temp_dir:\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 904, in __exit__\n    self.cleanup()\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 908, in cleanup\n    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 890, in _rmtree\n    _shutil.rmtree(name, onerror=onerror)\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 759, in rmtree\n    return _rmtree_unsafe(path, onerror)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 622, in _rmtree_unsafe\n    onerror(os.unlink, fullname, sys.exc_info())\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 881, in onerror\n    cls._rmtree(path, ignore_errors=ignore_errors)\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 890, in _rmtree\n    _shutil.rmtree(name, onerror=onerror)\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 759, in rmtree\n    return _rmtree_unsafe(path, onerror)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 603, in _rmtree_unsafe\n    onerror(os.scandir, path, sys.exc_info())\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 600, in _rmtree_unsafe\n    with os.scandir(path) as scandir_it:\n         ^^^^^^^^^^^^^^^^\nNotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\\\Users\\\\OBAIDM~1\\\\AppData\\\\Local\\\\Temp\\\\tmpuvxwz6tc\\\\comments.sqlite'\n","failure_type":"system_error","stream_descriptor":null}}}


@aaronsteers
Copy link
Contributor

aaronsteers commented Sep 28, 2024

Hi, @obaidmuneer . Thanks for raising this! I have not seen this issue before but I do see this error which is probably related to root cause:

NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'

This appears to be originating from the source connector itself (source-github) and is likely related to some python cross-OS compatibility issues for how Windows treats temp files.

If you have access to WSL or a linux box, it is likely this would run correctly on one of those platforms. If this is the correct root cause, a "full" fix likely would require an update to the source itself.

We do try to keep Windows working well for PyAirbyte, and we have a robust test matrix covering Windows as well as Linux, but the connectors themselves sometimes will have OS-level compatibility issues.

Let me know if this sounds right, and if you have any other environments you can test from.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants