Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux deployment #284

Merged
merged 14 commits into from
Aug 18, 2023
Merged

Linux deployment #284

merged 14 commits into from
Aug 18, 2023

Conversation

CodyCBakerPhD
Copy link
Collaborator

Replaces #257

Sister PR to #283

I suggest, since we all work on different devices, and since Mac is more touchy about signing issues than the other platforms, we should focus on debugging this PR first on the DANDI Hub (remote desktop), which is the common linux environment we all have access to

Only problem right now is getting the deployment working, could use some help on it: https://github.com/NeurodataWithoutBorders/nwb-guide/actions/workflows/Build-and-deploy-linux.yml

This was referenced Aug 11, 2023
@CodyCBakerPhD CodyCBakerPhD changed the title Fix linux deployment 2 Linux deployment Aug 11, 2023
@CodyCBakerPhD
Copy link
Collaborator Author

Confirmed the app fails to connect to backend on the DANDI Hub

To test it for yourself, just download the release from the webpage and run chmod +x NWB-GUIDE-0.0.3.AppImage from the terminal, then execute it from the remote desktop

The downside of the .AppImage container is it's harder to introspect the inner workings; I can't 100% confirm it's the scipy issue we see elsewhere but the behavior is the same

@garrettmflynn
Copy link
Member

garrettmflynn commented Aug 12, 2023 via email

@CodyCBakerPhD
Copy link
Collaborator Author

Would you be able to access and run the executable with the test file?

No dice; the test runs fine, spits out this massive log

Found file true
[spawn]: [1433:0814/162628.231448:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[1433:0814/162628.231571:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory

[spawn]: [1433:0814/162628.299235:ERROR:zygote_host_impl_linux.cc(263)] Failed to adjust OOM score of renderer with pid 1480: Permission denied (13)

Time to Start: 1709.00ms
[spawn]: Killing all previous processes

[spawn]: libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)

[spawn]: [1480:0814/162628.684712:ERROR:viz_main_impl.cc(186)] Exiting GPU process due to errors during initialization

[spawn]: [1433:0814/162628.803549:ERROR:zygote_host_impl_linux.cc(263)] Failed to adjust OOM score of renderer with pid 1549: Permission denied (13)

[spawn]: [1433:0814/162630.023527:ERROR:zygote_host_impl_linux.cc(263)] Failed to adjust OOM score of renderer with pid 1567: Permission denied (13)

[spawn]: libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)

[spawn]: [1567:0814/162631.005456:ERROR:viz_main_impl.cc(186)] Exiting GPU process due to errors during initialization

[spawn]: [1433:0814/162631.020587:ERROR:zygote_host_impl_linux.cc(263)] Failed to adjust OOM score of renderer with pid 1628: Permission denied (13)

[spawn]: libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)

[spawn]: [1628:0814/162633.622429:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported.

[spawn]: [nwb-guide:4242]: Traceback (most recent call last):
File "app.py", line 8, in

[spawn]: [nwb-guide:4242]: File "", line 1007, in _find_and_load

[spawn]: [nwb-guide:4242]: File "", line 986, in _find_and_load_unlocked

[spawn]: [nwb-guide:4242]: File "", line 680, in _load_unlocked
File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]: [nwb-guide:4242]: File "namespaces/init.py", line 1, in

[spawn]: [nwb-guide:4242]: File "", line 1007, in _find_and_load

[spawn]: [nwb-guide:4242]: File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]: [nwb-guide:4242]: File "namespaces/configureNamespaces.py", line 1, in
File "", line 1007, in _find_and_load

[spawn]: [nwb-guide:4242]: File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
File "flask_restx/init.py", line 1, in

[spawn]: [nwb-guide:4242]: File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked

[spawn]: [nwb-guide:4242]: File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
File "flask_restx/reqparse.py", line 15, in
File "", line 1007, in _find_and_load

[spawn]: [nwb-guide:4242]: File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
File "flask_restx/model.py", line 13, in
File "", line 1007, in _find_and_load

[spawn]: [nwb-guide:4242]: File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]: [nwb-guide:4242]: File "jsonschema/init.py", line 16, in

[spawn]: [nwb-guide:4242]: File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked

[spawn]: [nwb-guide:4242]: File "", line 680, in _load_unlocked
File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]: [nwb-guide:4242]: File "jsonschema/validators.py", line 19, in
File "", line 1007, in _find_and_load

[spawn]: [nwb-guide:4242]: File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked

[spawn]: [nwb-guide:4242]: File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]: [nwb-guide:4242]: File "jsonschema_specifications/init.py", line 11, in

[spawn]: [nwb-guide:4242]: File "referencing/_core.py", line 297, in rmatmul

[spawn]: [nwb-guide:4242]: File "jsonschema_specifications/_core.py", line 27, in _schemas

[spawn]: [nwb-guide:4242]: File "pathlib.py", line 1160, in iterdir

[spawn]: [nwb-guide:4242]: FileNotFoundError: [Errno 2] No such file or directory: '/tmp/.mount_NWB-GUi7YR9M/resources/flask/jsonschema_specifications/schemas'
[1575] Failed to execute script 'app' due to unhandled exception!

[spawn]: [nwb-guide:flask] exit code 1

[spawn]: Checking for update

[spawn]: Error: Error: Cannot find latest-linux.yml in the latest release artifacts (https://github.com/catalystneuro/nwb-guide/releases/download/v0.0.1/latest-linux.yml): HttpError: 404
"method: GET url: https://github.com/NeurodataWithoutBorders/nwb-guide/releases/download/v0.0.1/latest-linux.yml\n\nPlease double check that your authentication token is correct. Due to security reasons actual status maybe not reported, but 404.\n"
Headers: {
"server": "GitHub.com",
"date": "Mon, 14 Aug 2023 16:26:38 GMT",
"content-type": "text/plain; charset=utf-8",
"vary": "X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, Accept-Encoding, Accept, X-Requested-With",
"cache-control": "no-cache",
"strict-transport-security": "max-age=31536000; includeSubdomains; preload",
"x-frame-options": "deny",
"x-content-type-options": "nosniff",
"x-xss-protection": "0",
"referrer-policy": "no-referrer-when-downgrade",
"content-security-policy": "default-src 'none'; base-uri 'self'; connect-src 'self'; form-action 'self'; img-src 'self' data:; script-src 'self'; style-src 'unsafe-inline'",
"content-encoding": "gzip",
"content-length": "29",
"x-github-request-id": "A0B4:29DC:A07A:E551:64DA55BE"
}
at createHttpError (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:4291:12)
at ElectronHttpExecutor.handleResponse (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:4375:16)
at ClientRequest. (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:4346:18)
at ClientRequest.emit (node:events:526:28)
at SimpleURLLoaderWrapper. (node:electron/js2c/browser_init:101:6917)
at SimpleURLLoaderWrapper.emit (node:events:526:28)
at Object.newError (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:6586:21)
at GitHubProvider.getLatestVersion (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:15224:40)
at async AppImageUpdater.getUpdateInfoAndProvider (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:15669:15)
at async AppImageUpdater.doCheckForUpdates (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:15683:22)

[spawn]: (node:1433) UnhandledPromiseRejectionWarning: Error: Cannot find latest-linux.yml in the latest release artifacts (https://github.com/catalystneuro/nwb-guide/releases/download/v0.0.1/latest-linux.yml): HttpError: 404
"method: GET url: https://github.com/NeurodataWithoutBorders/nwb-guide/releases/download/v0.0.1/latest-linux.yml\n\nPlease double check that your authentication token is correct. Due to security reasons actual status maybe not reported, but 404.\n"
Headers: {
"server": "GitHub.com",
"date": "Mon, 14 Aug 2023 16:26:38 GMT",
"content-type": "text/plain; charset=utf-8",
"vary": "X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, Accept-Encoding, Accept, X-Requested-With",
"cache-control": "no-cache",
"strict-transport-security": "max-age=31536000; includeSubdomains; preload",
"x-frame-options": "deny",
"x-content-type-options": "nosniff",
"x-xss-protection": "0",
"referrer-policy": "no-referrer-when-downgrade",
"content-security-policy": "default-src 'none'; base-uri 'self'; connect-src 'self'; form-action 'self'; img-src 'self' data:; script-src 'self'; style-src 'unsafe-inline'",
"content-encoding": "gzip",
"content-length": "29",
"x-github-request-id": "A0B4:29DC:A07A:E551:64DA55BE"
}
at createHttpError (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:4291:12)
at ElectronHttpExecutor.handleResponse (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:4375:16)
at ClientRequest. (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:4346:18)
at ClientRequest.emit (node:events:526:28)
at SimpleURLLoaderWrapper. (node:electron/js2c/browser_init:101:6917)
at SimpleURLLoaderWrapper.emit (node:events:526:28)
at Object.newError (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:6586:21)
at GitHubProvider.getLatestVersion (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:15224:40)
at async AppImageUpdater.getUpdateInfoAndProvider (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:15669:15)
at async AppImageUpdater.doCheckForUpdates (/tmp/.mount_NWB-GUi7YR9M/resources/app.asar/build/main/main.js:15683:22)
(Use nwb-guide --trace-warnings ... to show where the warning was created)

[spawn]: (node:1433) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 203)

but of course that is actually launching and running the entire app itself (just the front-end in this case); it does not capture the errors evidenced in the dev console of the app where it fails to connect to the backend

@garrettmflynn What I would recommend for now is that you try building and debugging the stand-alone distributable PyInstaller executable on the DANDI Hub; that's the shared environment we both have access to and the issue seems to be the same over there as on Windows

In the meantime I'll keep trying to see if I can enhance the CI deployment to catch this kind of thing quicker

What do you think?

@garrettmflynn
Copy link
Member

Awesome. Already on it

@garrettmflynn
Copy link
Member

garrettmflynn commented Aug 14, 2023

Working my way through the Pyinstaller issues. Posting the progress so far:

Issue 1: jsonschema_specifications

[spawn]: Traceback (most recent call last):
  File "app.py", line 8, in <module>

[spawn]:   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

[spawn]:   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked

[spawn]:   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "namespaces/__init__.py", line 1, in <module>

[spawn]:   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

[spawn]:   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "namespaces/configureNamespaces.py", line 1, in <module>

[spawn]:   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

[spawn]:   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "flask_restx/__init__.py", line 1, in <module>

[spawn]:   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked

[spawn]:   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "flask_restx/reqparse.py", line 15, in <module>

[spawn]:   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

[spawn]:   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "flask_restx/model.py", line 13, in <module>

[spawn]:   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

[spawn]:   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "jsonschema/__init__.py", line 16, in <module>

[spawn]:   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked

[spawn]:   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "jsonschema/validators.py", line 19, in <module>

[spawn]:   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load

[spawn]:   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "jsonschema_specifications/__init__.py", line 11, in <module>

[spawn]:   File "referencing/_core.py", line 297, in __rmatmul__

[spawn]:   File "jsonschema_specifications/_core.py", line 27, in _schemas

[spawn]:   File "pathlib.py", line 1160, in iterdir

[spawn]: FileNotFoundError: [Errno 2] No such file or directory: '/home/jovyan/nwb-guide/build/flask/nwb-guide/jsonschema_specifications/schemas'
[15240] Failed to execute script 'app' due to unhandled exception!

[spawn] exit: 1

This was noted and fixed by Ryan earlier (#89) by adding --collect-data jsonschema_specifications to the Pyinstaller command. However, I'd removed that because it wasn't essential for me to build on Mac.

Issue 2: Pydantic Email Validator

[spawn]: Traceback (most recent call last):
  File "pydantic/networks.py", line 578, in pydantic.networks.import_email_validator

[spawn]: ModuleNotFoundError: No module named 'email_validator'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 16, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "setupUtils/__init__.py", line 3, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "setupUtils/configureRouteHandlers.py", line 1, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "apis/__init__.py", line 2, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "apis/apiNeuroConv.py", line 6, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "manageNeuroconv/__init__.py", line 1, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "manageNeuroconv/manage_neuroconv.py", line 2, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "neuroconv/__init__.py", line 5, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "neuroconv/tools/yaml_conversion_specification/__init__.py", line 1, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "neuroconv/tools/yaml_conversion_specification/yaml_conversion_specification.py", line 7, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "dandi/metadata.py", line 25, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module

[spawn]:   File "dandischema/__init__.py", line 2, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "dandischema/metadata.py", line 17, in <module>

[spawn]:   File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "dandischema/models.py", line 640, in <module>

[spawn]:   File "pydantic/main.py", line 197, in pydantic.main.ModelMetaclass.__new__

[spawn]:   File "pydantic/fields.py", line 506, in pydantic.fields.ModelField.infer

[spawn]:   File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__

[spawn]:   File "pydantic/fields.py", line 557, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 831, in pydantic.fields.ModelField.populate_validators
  File "pydantic/networks.py", line 591, in __get_validators__
  File "pydantic/networks.py", line 580, in pydantic.networks.import_email_validator
ImportError: email-validator is not installed, run `pip install pydantic[email]`
[21132] Failed to execute script 'app' due to unhandled exception!

I'll attempt to install the version of pydantic with email included and go from there.

Edit 1: Related DANDI Issue

May be related to dandi/dandi-archive#405 faced by the DANDI team on Heroku.

It looks like the email option simply adds email-validator>=1.0.3 as a dependency, so I've added this to the Linux dependency list and will see if the build works.

@CodyCBakerPhD
Copy link
Collaborator Author

ImportError: email-validator is not installed, run pip install pydantic[email]

I've also seen this one before, not sure why it sometimes occurs and sometimes doesn't, but I always just manually install it and then it's fine

Base automatically changed from build-mac to main August 16, 2023 17:35
@CodyCBakerPhD
Copy link
Collaborator Author

@garrettmflynn
Copy link
Member

Ah my bad. Was prototyping changes on the DANDI Hub and never pushed

@CodyCBakerPhD CodyCBakerPhD marked this pull request as ready for review August 18, 2023 18:39
@CodyCBakerPhD CodyCBakerPhD self-assigned this Aug 18, 2023
@CodyCBakerPhD
Copy link
Collaborator Author

@garrettmflynn I'm OK with merging this in as a fix to the jsonschema issue

Then a separate PR can continue debugging the remaining issues that might remain after #302, but that should cover the email validator problem

Agreed?

Copy link
Member

@garrettmflynn garrettmflynn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet let's do it

@CodyCBakerPhD CodyCBakerPhD merged commit 0723062 into main Aug 18, 2023
@CodyCBakerPhD CodyCBakerPhD deleted the fix_linux_deployment_2 branch August 18, 2023 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants