bugfix: database breakage during update #511
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, updating a database created from the beta cisTEM version would result in a crash at the outset as output_pixel_size was being improperly added to REFINEMENT_PACKAGE_CONTAINED_PARTICLES_ tables (it was out of order). Additionally, REFINEMENT_RESULTS_ and CLASSIFICATION_RESULTS_ have pixel_size, aberration, microscope voltage, and amplitude contrast updated as well, making all operations usable post-update.
Some columns still remained empty; updated to either give appropriate (already existent) value, or filled with 0.0.
Using an interface that MainFrame inherits from and Database is aware of, the schema update is now accurately tracked when upgrading a database originally created in cisTEM to the current devel via a OneSecondProgress dialog.
twagner9
force-pushed
the
output_pixel_size_db_update_fix
branch
from
8e73ce4 to
68ae856
Compare
bHimes
requested changes
Oct 9, 2024
-Made dedicated custom dialog for updating the database -Retained same dialog format when no schema changes are detected -Added Database::CopyDatabase for backing up the database -Functionalized the update tracking logic as it can now be used for Update only and Backup and Update
-Changed backup function name in database.cpp/h -Changed how failing backup is handled in MainFrame.cpp
|
looks good |
bHimes
approved these changes
Oct 14, 2024
|
This is great, you can close #388 with this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The corresponding PR to Issue #510.
Fixes #388
Edit: It took me awhile to find a suitable method for tracking this update via the GUI, and I landed on an interface as the best medium to avoid exposing any GUI code to the database, avoiding increasing compilation time and strange dependency issues.
I also had been underestimating the scale of this upgrade; for a single refinement package with ~1.2 million particles, if any classification and refinements have been run using it, it’s much more accurate to imagine the number of column changes on the scale of 100 million or more, into the billions – each
CLASSIFICATION_RESULT_%andREFINEMENT_RESULT_%table has several columns that need to be changed for every row, which would be every particle in the package.Because of this, upgrade times are substantially slower. I was able to use a project that was being used for 3D classification. It only contained
REFINEMENT_RESULT_%tables (noCLASSIFICATION_RESULT_%tables), and refinement packages contained anywhere from 10,000 to 200,000+ particles, for a total of ~460 million row updates. The upgrade takes around 1h:30m to complete. I spoke with @timothygrant80 about this, and he said he thinks this is an uncommon enough case that this upgrade time is okay.I have rebased my feature branch to be current with the master branch using to minimize conflicts and headaches
Which compilers were tested
These changes are isolated to the
How has the functionality been tested?
Please describe the tests that you ran to verify your changes. Please also note any relevant details for your test configuration.
Checklist: