[BUG] Incomputable histogram crashes Jupyter kernel

[duytnguyendtn]

Jdaviz component

Imviz

Description

Hello Jdaviz devs!

This bug report is for the histogram crashing bug we suspect happening in Imviz. The following data product reliably crashes my kernel when opening the Plot Options plugin in the Data Analysis tray:

from jdaviz import Imviz
imviz = Imviz()
imviz.show()
imviz.load_data('http://archive.stsci.edu/pub/hlsp/candels/goods-s/gsd12/v0.5/hlsp_candels_hst_wfc3_gsd12-sect32_f160w_v0.5_drz.fits')

I have been able to reproduce this issue on two independent Windows 11 machines, on different browsers, on Jupyter Notebook and Lab, as well as on main as of Wednesday.

During our code review, we investigated the possibility it was a file size constraint, but after some digging around, it appears to be more of a data product specific error with the above file. I previously had two files I was testing with, but in further testing, the other data product we were testing (this one) loads okay. I am able to reproduce the issue loading only the second data file (the one in the snippet). Of particular note, the working file is 1.5GB, whereas the broken file is only 262MB, suggesting it isn't a filesize issue. We theorized it may be related to the large number of nans in the data product.

Also looking closer at the state of the plugin at the time of crash, the more likely it seems to be the histogram, as the spinner is stuck on the histogram when the kernel crashes:

I discovered this in investigating Cami's issue reported in my pyvo plugin PR here: #2872 (comment). I can't really begin debugging Cami's issue since I can't even open the Plot Options menu 😅

How to Reproduce

1. Launch Imviz
2. Load data product via url: http://archive.stsci.edu/pub/hlsp/candels/goods-s/gsd12/v0.5/hlsp_candels_hst_wfc3_gsd12-sect32_f160w_v0.5_drz.fits
3. Open Plot Options menu in UI
4. Wait roughly 5 seconds
5. Kernel dies

Expected behavior

Histogram loads successfully and kernel does not die

Browser

Chromium 128.0.6613.138 (64-bit)

Jupyter

(See Freeze below)

(Also tested and verified on Jupyter Lab 4.2.0)

Software versions

Full pip freeze

anyio==4.3.0
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asdf==3.2.0
asdf-astropy==0.6.1
asdf_coordinates_schemas==0.3.0
asdf_standard==1.1.1
asdf_transform_schemas==0.5.0
asdf_wcs_schemas==0.4.0
asteval==0.9.32
astropy==6.1.0
astropy-iers-data==0.2024.5.6.0.29.28
astroquery==0.4.7
asttokens==2.4.1
async-lru==2.0.4
attrs==23.2.0
Babel==2.15.0
backports.tarfile==1.1.1
beautifulsoup4==4.12.3
bleach==6.1.0
bqplot==0.12.43
bqplot-gl==0.0.0
bqplot-image-gl==1.4.11
cachetools==5.5.0
casa-formats-io==0.3.0
cdshealpix==0.6.5
certifi==2024.2.2
cffi==1.16.0
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
cloudpickle==3.0.0
colorama==0.4.6
comm==0.2.2
contourpy==1.2.1
coverage==7.6.0
cycler==0.12.1
dask==2024.5.0
debugpy==1.8.1
decorator==5.1.1
defusedxml==0.7.1
dill==0.3.8
distlib==0.3.8
echo==0.8.0
et-xmlfile==1.1.0
exceptiongroup==1.2.1
executing==2.0.1
fast-histogram==0.14
fastjsonschema==2.19.1
filelock==3.15.4
fonttools==4.51.0
fqdn==1.5.1
freetype-py==2.4.0
fsspec==2024.3.1
glue-astronomy==0.10.0
glue-core==1.21.0
glue-jupyter==0.21.0
glue-qt==0.3.1
glue-vispy-viewers==1.1.0
gwcs==0.21.0
h11==0.14.0
h5py==3.11.0
hsluv==5.0.4
html5lib==1.1
httpcore==1.0.5
httpx==0.27.0
hypothesis==6.108.2
idna==3.7
imageio==2.34.1
importlib_metadata==7.1.0
iniconfig==2.0.0
ipydatawidgets==4.3.5
ipygoldenlayout==0.4.0
ipykernel==6.29.4
ipympl==0.9.4
ipypopout==1.2.1
ipysplitpanes==0.2.0
ipython==8.24.0
ipython-genutils==0.2.0
ipyvolume==0.6.3
ipyvue==1.11.1
ipyvuetify==1.9.4
ipywebrtc==0.6.0
ipywidgets==8.1.2
isoduration==20.11.0
jaraco.classes==3.4.0
jaraco.context==5.3.0
jaraco.functools==4.0.1
-e git+https://github.com/duytnguyendtn/jdaviz.git@fd89b644391a1b9333e38fc9897891d4b92c2e2e#egg=jdaviz
jedi==0.19.1
Jinja2==3.1.4
jmespath==1.0.1
joblib==1.4.2
json5==0.9.25
jsonpointer==2.4
jsonschema==4.22.0
jsonschema-specifications==2023.12.1
jupyter-events==0.10.0
jupyter-lsp==2.2.5
jupyter_client==8.6.1
jupyter_core==5.7.2
jupyter_server==2.14.0
jupyter_server_terminals==0.5.3
jupyterlab==4.1.8
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.1
jupyterlab_widgets==3.0.10
keyring==25.2.0
kiwisolver==1.4.5
lazy_loader==0.4
locket==1.0.0
MarkupSafe==2.1.5
matplotlib==3.8.4
matplotlib-inline==0.1.7
mistune==3.0.2
mocpy==0.13.1
more-itertools==10.2.0
mpl-scatter-density==0.7
nbclient==0.7.4
nbconvert==7.16.4
nbformat==5.10.4
ndcube==2.2.0
nest-asyncio==1.6.0
networkx==3.3
notebook==7.1.3
notebook_shim==0.2.4
numpy==1.26.4
openpyxl==3.1.2
overrides==7.7.0
packaging==24.1
pandas==2.2.2
pandocfilters==1.5.1
parso==0.8.4
partd==1.4.2
photutils==1.12.0
pillow==10.3.0
platformdirs==4.2.2
pluggy==1.5.0
prometheus_client==0.20.0
prompt-toolkit==3.0.43
psutil==5.9.8
pure-eval==0.2.2
pvextractor==0.4
pycparser==2.22
pyerfa==2.0.1.4
Pygments==2.18.0
PyOpenGL==3.1.7
pyparsing==3.1.2
pyproject-api==1.7.1
pytest==8.2.2
pytest-arraydiff==0.6.1
pytest-astropy==0.11.0
pytest-astropy-header==0.2.2
pytest-cov==5.0.0
pytest-doctestplus==1.2.1
pytest-filter-subpackage==0.2.0
pytest-mock==3.14.0
pytest-remotedata==0.4.1
python-dateutil==2.9.0.post0
python-json-logger==2.0.7
pythreejs==2.4.2
pytz==2024.1
-e git+https://github.com/duytnguyendtn/pyvo@d70d9bc4762e587466b1f15a76e54b6e39c69259#egg=pyvo
pywin32==306
pywin32-ctypes==0.2.2
pywinpty==2.0.13
PyYAML==6.0.1
pyzmq==26.0.3
qtconsole==5.5.2
QtPy==2.4.1
radio-beam==0.3.7
referencing==0.35.1
regions==0.9
requests==2.31.0
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rpds-py==0.18.1
scikit-image==0.23.2
scipy==1.13.0
semantic-version==2.10.0
Send2Trash==1.8.3
shapely==2.0.4
sidecar==0.7.0
six==1.16.0
sniffio==1.3.1
sortedcontainers==2.4.0
soupsieve==2.5
specreduce==1.3.0
spectral-cube==0.6.5
specutils==1.15.0
stack-data==0.6.3
stdatamodels==1.10.1
synphot==1.4.0
terminado==0.18.1
tifffile==2024.5.10
tinycss2==1.3.0
tomli==2.0.1
toolz==0.12.1
tornado==6.4
tox==4.18.0
traitlets==5.14.3
traittypes==0.2.1
types-python-dateutil==2.9.0.20240316
typing_extensions==4.11.0
tzdata==2024.1
uri-template==1.3.0
urllib3==2.2.1
virtualenv==20.26.3
vispy==0.14.2
voila==0.4.4
wcwidth==0.2.13
webcolors==1.13
webencodings==0.5.1
websocket-client==1.8.0
websockets==12.0
widgetsnbextension==4.0.10
xlrd==2.0.1
zipp==3.18.1

[duytnguyendtn]

One additional data point I was able to find, I was able to reproduce the crashing kernel even if I extract the data array itself and just load that into Imviz:

from astropy.io import fits
from jdaviz import Imviz
imviz = Imviz()
fits_hdul = fits.open('http://archive.stsci.edu/pub/hlsp/candels/goods-s/gsd12/v0.5/hlsp_candels_hst_wfc3_gsd12-sect32_f160w_v0.5_drz.fits')
imviz.show()
imviz.load_data(fits_hdul [0].data)
# Open Plot Options plugin here

[pllim]

I remember us discussing this at some point. Was there a resolution?

[duytnguyendtn]

I don't recall there being one; I think this is still an open bug