You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As an electrophysiology manager and author on a Patch-Seq paper, I want to understand why potential cells for October 2020 release are not progressing through the processing pipeline, so that I can make the decision to fail them or take corrective action.
4674 potential October 2020 cells have been run through the pipeline.
Four failure modes are listed below for three processing pipelines:
'EPHYS_NWB_STIMULUS_SUMMARY_V3_QUEUE' (4 + 1 failures )
'EPHYS_QC_V3_QUEUE' ( 60 failures )
'EPHYS_FEATURE_EXTRACTION_V3_QUEUE' ( 284 + 52 + 9 + 5 + 22 + 1 = 385 failures )
Acceptance criteria:
A very brief analysis on each type of failure
Whether modifications to the code are a reasonable strategy to address the failure
Whether any action taken by data generators (e.g. patch-clamp team or MIES developer) could address the issue
Create new issue to address action that could be taken
For each failure, the following information is listed: cell_specimen_id ; cell_specimen_name ; ephys_roi_result_id
EPHYS_NWB_STIMULUS_SUMMARY_V3_QUEUE
ValueError: Found multiple stimulus series {[s.name for s in matching_series]} for sweep number {sweep_number}
637823753;"Htr3a-Cre_NO152;Ai14-349020.03.02.01";637823738
637824915;"Htr3a-Cre_NO152;Ai14-349020.03.02.02";637824890
637833144;"Htr3a-Cre_NO152;Ai14-349020.03.02.03";637833126
637839160;"Htr3a-Cre_NO152;Ai14-349020.03.02.04";637838825
Exception: Could not construct VoltageClampSeries object due to: VoltageClampSeries.__init__: missing argument 'gain'
767875138;"Nos1-CreERT2;Ai14-421031.03.01.01";767874754
EPHYS_QC_V3_QUEUE
marshmallow.exceptions.ValidationError: {'sweep_features': {15: {'stimulus_duration': ['Field may not be null.']}}}
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 109, in <module>
if __name__ == "__main__": main()
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 105, in main
module.args["cell_features"])
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 54, in run_feature_extraction
cell_state, feature_states) = dsft.extract_data_set_features(data_set)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/data_set_features.py", line 378, in extract_data_set_features
sweep_features[s['sweep_number']]['peak_deflect'] = s['peak_deflect']
KeyError: 4
9 with this error e.g. id = 1005657758
Traceback (most recent call last):
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 109, in <module>
if __name__ == "__main__": main()
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 105, in main
module.args["cell_features"])
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 41, in run_feature_extraction
raise er.FeatureError("There are no QC-passed sweeps available to analyze")
ipfx.error.FeatureError: There are no QC-passed sweeps available to analyze
5 with this error e.g. ids 1005108832
1005032767
1003299623
1001443326
1001424563
Traceback (most recent call last):
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 109, in <module>
if __name__ == "__main__": main()
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 105, in main
module.args["cell_features"])
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 82, in run_feature_extraction
plotqc.display_features(qc_fig_dir, data_set, feature_data)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 769, in display_features
make_sweep_page(data_set, qc_fig_dir)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 715, in make_sweep_page
sweep_files = plot_sweep_figures(data_set, image_dir, sizes)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 207, in plot_sweep_figures
v, i, t, r, dt = load_sweep(data_set, sweep_number)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 36, in load_sweep
dt = sweep.t[1] - sweep.t[0]
IndexError: index 1 is out of bounds for axis 0 with size 0
22 with this error
1000612453,999319545,920015861,926949081,912201092,870849339,874356674,880976423,874570247,873810658,875020807,880542017,87552,386,875486563,872282934,872248672,875139943,874444852,852974739,839931440,941908038,751935222
Traceback (most recent call last):
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 109, in <module>
if __name__ == "__main__": main()
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 105, in main
module.args["cell_features"])
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 82, in run_feature_extraction
plotqc.display_features(qc_fig_dir, data_set, feature_data)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 769, in display_features
make_sweep_page(data_set, qc_fig_dir)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 715, in make_sweep_page
sweep_files = plot_sweep_figures(data_set, image_dir, sizes)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 174, in plot_sweep_figures
if r_init[0] <= 0:
TypeError: 'NoneType' object is not subscriptable
ValueError: Axis limits cannot be NaN or Inf
ephys_roi_result.id = 660049701
Failure [limsweb7.corp.alleninstitute.org]: Exit status is 1 [log file: /allen/programs/celltypes/production/mousecelltypes/prod58/Ephys_Roi_Result_660049701/202008031836_EPHYS_FEATURE_EXTRACTION_V3_QUEUE_660049701.log]: (last 25 lines)...
) File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 105, in main
module.args["cell_features"])
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/bin/run_feature_extraction.py", line 82, in run_feature_extraction
plotqc.display_features(qc_fig_dir, data_set, feature_data)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 770, in display_features
make_cell_page(data_set, feature_data, qc_fig_dir)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 728, in make_cell_page
cell_files = plot_cell_figures(data_set, feature_data, image_dir, sizes)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 620, in plot_cell_figures
plot_ramp_figures(data_set, cell_features, lims_features, sweep_features, image_dir, sizes, cell_image_files)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 455, in plot_ramp_figures
figs = plot_single_ap_values(data_set, ramps_sweeps, lims_features, sweep_features, cell_features, "ramp")
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/ipfx/plot_qc_figures.py", line 147, in plot_single_ap_values
plt.xlim(spikes[0]["threshold_t"] - 0.002, spikes[0]["fast_trough_t"] + 0.01)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/matplotlib/pyplot.py", line 1396, in xlim
ret = ax.set_xlim(*args, **kwargs)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 3185, in set_xlim
right = self._validate_converted_limits(right, self.convert_xunits)
File "/allen/aibs/technology/conda/production/fx/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 3101, in _validate_converted_limits
raise ValueError("Axis limits cannot be NaN or Inf")
ValueError: Axis limits cannot be NaN or Inf
The text was updated successfully, but these errors were encountered:
As an electrophysiology manager and author on a Patch-Seq paper, I want to understand why potential cells for October 2020 release are not progressing through the processing pipeline, so that I can make the decision to fail them or take corrective action.
4674 potential October 2020 cells have been run through the pipeline.
Four failure modes are listed below for three processing pipelines:
'EPHYS_NWB_STIMULUS_SUMMARY_V3_QUEUE' (4 + 1 failures )
'EPHYS_QC_V3_QUEUE' ( 60 failures )
'EPHYS_FEATURE_EXTRACTION_V3_QUEUE' ( 284 + 52 + 9 + 5 + 22 + 1 = 385 failures )
Acceptance criteria:
For each failure, the following information is listed:
cell_specimen_id ; cell_specimen_name ; ephys_roi_result_idEPHYS_NWB_STIMULUS_SUMMARY_V3_QUEUE
EPHYS_QC_V3_QUEUE
marshmallow.exceptions.ValidationError: {'sweep_features': {15: {'stimulus_duration': ['Field may not be null.']}}}
EPHYS_FEATURE_EXTRACTION_V3_QUEUE
if r[0] <= 0: # This happens when stimulus starts less than 0.5 s after test pulse
TypeError: 'NoneType' object is not subscriptable
52 with this error: e.g. err.id 1004779043
9 with this error e.g. id = 1005657758
5 with this error e.g. ids 1005108832
1005032767
1003299623
1001443326
1001424563
22 with this error
1000612453,999319545,920015861,926949081,912201092,870849339,874356674,880976423,874570247,873810658,875020807,880542017,87552,386,875486563,872282934,872248672,875139943,874444852,852974739,839931440,941908038,751935222
ValueError: Axis limits cannot be NaN or Inf
ephys_roi_result.id = 660049701
The text was updated successfully, but these errors were encountered: