Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic tutorial #304

Merged
merged 40 commits into from
Oct 22, 2024
Merged

Basic tutorial #304

merged 40 commits into from
Oct 22, 2024

Conversation

mzouink
Copy link
Member

@mzouink mzouink commented Oct 16, 2024

@pattonw
Adds a simple tutorial start-to-finish tutorial for DaCapo.
Still a few things left to resolve before its ready for merging:

  • Loss behaving strangely after the first validation (temporary solution involves giving an extra singleton dimension to the provided data)
  • Validation fails (we want to show validation loss/scores, validation outputs across iterations, and how to finally take the best iteration and apply it with the post processing to a volume manually)
  • stdout too verbose
  • plotting functions should be built into DaCapo (there are plotting functions, not sure how to include bokeh plots so I didn't dive too deep into this)

mzouink and others added 30 commits September 23, 2024 17:36
There appear to be some python formatting errors in
4583723. This pull request
uses the [psf/black](https://github.com/psf/black) formatter to fix
these issues.
There appear to be some python formatting errors in
d899c63. This pull request
uses the [psf/black](https://github.com/psf/black) formatter to fix
these issues.
There appear to be some python formatting errors in
525f2fe. This pull request
uses the [psf/black](https://github.com/psf/black) formatter to fix
these issues.
Adds a simple tutorial start-to-finish tutorial for DaCapo.
Still a few things left to resolve before its ready for merging:

- [ ] Loss behaving strangely after the first validation (temporary
solution involves giving an extra singleton dimension to the provided
data)
- [ ] Validation fails (we want to show validation loss/scores,
validation outputs across iterations, and how to finally take the best
iteration and apply it with the post processing to a volume manually)
- [ ] stdout too verbose
- [ ] plotting functions should be built into `DaCapo` (there are
plotting functions, not sure how to include bokeh plots so I didn't dive
too deep into this)
@mzouink
Copy link
Member Author

mzouink commented Oct 18, 2024

i added few changes.
The current example is working with distance task because i am not using BlockwisseTask here

task = daisy.Task(
f"threshold_{output_array.dataset}",
total_roi=self.prediction_array.roi,
read_roi=read_roi,
write_roi=read_roi,
process_function=process_block,
check_function=None,
read_write_conflict=False,
fit="overhang",
max_retries=0,
timeout=None,
)
return daisy.run_blockwise([task], multiprocessing=False)

changed to just daisy local
but affinity is failing because watershed_postprocess is using segment_blockwise
segment_blockwise(
segment_function_file=str(
Path(Path(dacapo.blockwise.__file__).parent, "watershed_function.py")
),
context=parameters.context,
total_roi=self.prediction_array.roi,
read_roi=read_roi.grow(parameters.context, parameters.context),
write_roi=read_roi,
num_workers=num_workers,
max_retries=2, # TODO: make this an option
timeout=None, # TODO: make this an option
######
input_array_identifier=self.prediction_array_identifier,
output_array_identifier=output_array_identifier,
parameters=pars,
)

I don't know which is the better solution. i recommend using the old code of watershed for now.
Validation are failing because of BlockwiseTask too

@pattonw @rhoadesScholar

mzouink and others added 10 commits October 18, 2024 09:53
There appear to be some python formatting errors in
25f3812. This pull request
uses the [psf/black](https://github.com/psf/black) formatter to fix
these issues.
Add a basic in-memory postprocessor for watershed.

It doesn't seem to be working that well because prediction seems broken.
Here's a picture:

![Figure_1](https://github.com/user-attachments/assets/01c209e3-914e-44a2-85b4-70343b1a984f)

I expect the postprocessor to work significantly better if the
predictions are done with enough context to avoid the block artifacts
and the predictions are post processed properly so that high affinity
within objects leads to an affinity of 1 (it looks like its set to 0
here), and a low affinity in the background leads to an affinity of 0
(looks like it is set to 0.5 here)
@mzouink mzouink merged commit f1fdef3 into main Oct 22, 2024
3 of 4 checks passed
@mzouink mzouink deleted the basic_tutorial branch October 22, 2024 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants