Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update test suite #205

Merged
merged 22 commits into from
Jun 9, 2023
Merged

Conversation

kavitharaju
Copy link
Collaborator

Pulls the latest test samples, after the fixes and clarification given at committee's test suite, as on June 7, 2023.

Testing stats on parsing(Using the existing grammar implementation of usfm-grammar)

  • Parsing tests: All 380 tests passing
    • pass/fail values given for 34 samples are over-ridded (dropped from 88 before)
    • 4 common causes for the above 34 and few one-off cases identifed
      • Missing significant spaces after markers like \p, \q, \m and \b
      • Attributes (custom and others) not used as per spec
      • Missing content in markers like \ide, \rem, \toc1, \ip
      • Missing \p or other body paragraph markers (usually after \s)
  • JSON Conversion tests, with existing JSON schema(not USJ) and grammar: All passing expect 2
  • List conversion: All passing expect 1
  • USX conversion: 20 failing, with the rest passing.
  • Comparing generated USX with testsuite's: 73 failing

This PR also include the following changes

  • Temporarily changes the tests on CI/CD to run only parsing tests, not conversion test
  • Consider "MISSING" in syntax tree as a valid error report in test (until we are able to report them as error in parser itself Missing elements are not reported as error #173 )
  • Handle use of \qt(as not milestone), \k-s, \k-e and independant \xt in tests were checking for all markers present in syntax tree.
  • Remove old list of doubtful USFMs, that were being excluded from test

@kavitharaju kavitharaju added test This has to do with testing. v3-tree-sitter labels Jun 8, 2023
@kavitharaju kavitharaju requested a review from joelthe1 June 8, 2023 08:11
@joelthe1
Copy link
Collaborator

joelthe1 commented Jun 9, 2023

LGTM 👍

@joelthe1 joelthe1 merged commit ac5f4e1 into Bridgeconn:version-3 Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test This has to do with testing. v3-tree-sitter
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants