Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optional validation step (and documentation improvements) #107

Merged
merged 28 commits into from
Jan 30, 2023

Conversation

fsoubelet
Copy link
Member

@fsoubelet fsoubelet commented Jan 27, 2023

This addresses #106

The main changes are:

  • Version bump to 3.3.0
  • Addition of a validate boolean argument in the reader, which defaults to True, enabling the user to skip the step.
  • Addition of a validate boolean argument in the writer, which defaults to True, enabling the user to skip the step.
  • Addition of tests (works similarly with/without the flag & expected warnings not issued when validation is skipped)
  • Slight improvements to the documentation (few extra sphinx pluggins, admonitions, small examples of code, more info in some docstrings)

Worthy of noting:
This is a (very) significant speedup on large files. One can see a x4 speedup reading a typical LHC model twiss without validation, and a ~x11 speedup(!) on a 873M file (sliced fcc, 171k rows and 236 columns). See below a quick test.

Validation No Validation
validate no_validate

To be discussed:

On a side note, if someone know why the new documentation builds are weirdly expanding a lot in the index table of contents, I'd love to fix it too.

@fsoubelet fsoubelet added Type: Feature A (suggetion for a) new feature or enhancement in functionality. To be discussed Type: Documentation Improvements, updates and fixes to the documentation. labels Jan 27, 2023
@fsoubelet fsoubelet self-assigned this Jan 27, 2023
@fsoubelet fsoubelet linked an issue Jan 27, 2023 that may be closed by this pull request
tests/test_reader.py Outdated Show resolved Hide resolved
tests/test_writer.py Outdated Show resolved Hide resolved
tfs/reader.py Outdated Show resolved Hide resolved
tfs/writer.py Outdated Show resolved Hide resolved
@fsoubelet fsoubelet requested a review from JoschD January 30, 2023 11:13
@fsoubelet fsoubelet merged commit 0b9e385 into master Jan 30, 2023
@fsoubelet fsoubelet deleted the optional_validation branch January 30, 2023 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Documentation Improvements, updates and fixes to the documentation. Type: Feature A (suggetion for a) new feature or enhancement in functionality.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request]: Make validation optional
2 participants