Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

console output helper in Cakefile for test case selection and parser generation statistics #5473

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

cosmicexplorer
Copy link
Contributor

@cosmicexplorer cosmicexplorer commented Nov 15, 2024

This PR purely modifies the Cakefile in this repo and should not affect any users of the language.

Problem

I was playing around with changes to the parser and language semantics in e.g. #5471 (comment). During the process I found it somewhat difficult to test my changes:

  1. It's not possible to select a subset of tests to run with cake test.
  • Some of them fail on main for example--I'll fix that in a separate change, but it's why I wanted to select specific tests.
  1. It takes a while to build the parser, and it's not easy to distinguish from a hang.
  2. There's no mechanism to select what output should be logged when performing a console task with cake.

Solution

  • Add -f/--file, -d/--desc flags to select which tests to run using regex.
    • There are also --negFile and --negDesc flags to filter out matches.
  • Add -l/--level flag to select which console outputs to display.
    • This requires subclassing console.Console, which requires some hijinx.
  • Using the new log levels to make it ignorable, add some console output for the parser build process:
# By default, we still print nothing:
> cake build
# With 'info', we print more, including parser build statistics.
> cake -l info build
parser created (206 symbols, 107 terminals, 406 productions)
loading grammar: 11127.603219999999 ms
parser generation: 27.266960000000836 ms
full parser build time: 11154.87018 ms
# This runs tests, ignoring ones that are currently failing:
> cake --negFile sourcemap --negDesc 'assert keyword' --negDesc 'dynamic import assertion' --negDesc 4558 --negDesc 4418 test
test '#4418: stack traces for compiled files reference the correct line number' was filtered out by patterns
test '#4418: stack traces for compiled strings reference the correct line number' was filtered out by patterns
test '#4558: compiling a string inside a script doesn’t screw up stack trace line number' was filtered out by patterns
test 'dynamic import assertion' was filtered out by patterns
test 'assert keyword' was filtered out by patterns
file 'sourcemap.coffee' was filtered out by patterns
passed 1460 tests in 4.51 seconds 
# At level 'info', each passing test is also printed out:
> cake -l info --negFile sourcemap --negDesc 'assert keyword' --negDesc 'dynamic import assertion' --negDesc 4558 --negDesc 4418 test
# (lots of output omitted...)
passed: tagged template literal with backticks in test/tagged_template_literals.coffee
passed: tagged template literal with escaped backticks in test/tagged_template_literals.coffee
passed: tagged template literal with unnecessarily escaped backticks in test/tagged_template_literals.coffee
passed: tagged template literal with ES interpolation in test/tagged_template_literals.coffee
passed: tagged template literal with both ES and CoffeeScript interpolation in test/tagged_template_literals.coffee
passed: tagged template literal with escaped ES interpolation in test/tagged_template_literals.coffee
passed: tagged template literal with unnecessarily escaped ES interpolation in test/tagged_template_literals.coffee
passed: tagged template literal special escaping in test/tagged_template_literals.coffee
passed: #4467: tagged template literal call recognized as a callable function in test/tagged_template_literals.coffee
passed 1460 tests in 4.64 seconds 

@cosmicexplorer
Copy link
Contributor Author

Proposed a small and focused fix to jison at zaach/jison#408 which improves parser build time by 3x!

@cosmicexplorer cosmicexplorer force-pushed the regex-match-test branch 2 times, most recently from 31a901e to b913d1c Compare November 24, 2024 07:40
- add summary stats for parser generation
- add a perf counter for grammar build time
- add negative regex matching
- move Console to build-support
- move PatternSet to build-support
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant