History.txt

=== 3.1.1 / 2012-12-19

* 1 minor enhancement:

  * Added MOVE_TIMEOUT env var for ruby_parse_extract_error to move slow files to a sibling directory

* 4 bug fixes:

  * 1.9: Fixed lexing of "0o". (whitequark)
  * 1.9: Fixed parsing of unary plus on literals. (whitequark)
  * Added timeout arg to RubyParser#process to pass through to the real parser
  * Updated Synopsis to reflect new options for running RP. (louismullie)

=== 3.1.0 / 2012-12-06

* 2 minor enhancements:

  * Added RubyParser.for_current_ruby to provide a parser that matches your runtime. (neilconway)
  * Duck-typed IDENT_CHAR_RE instead of using RUBY_VERSION

* 3 bug fixes:

  * Cleared out body comments in class/module/defn/defs
  * Flipped lexer tests to US-ASCII to avoid encoding hell
  * yyerror is now an alias for syntax_error

=== 3.0.4 / 2012-11-26

* 1 bug fix:

  * RPStringScanner#lineno was still using byte offset, not char offset. (brynary)

=== 3.0.3 / 2012-11-23

* 1 minor enhancement:

  * Improved error output on invalid char in expression.

* 1 bug fix:

  * Fixed lexing of no-name ivars. (whitequark)

=== 3.0.2 / 2012-11-21

52 down makes 99.9767% or 3.7σ. 130 files failed to parse out of 558k.

* 4 minor enhancements:

  * Added RP_TIMEOUT env var to override default timeout of 10 seconds.
  * Minor optimization to RubyLexer#parse_number
  * Only output parseerror output to stderr if $DEBUG.
  * ruby_parse_extract_error modified to include 'it' blocks in its search.

* 7 bug fixes:

  * 1.9: Fixed args in dot-call forms (eg f.(...)).
  * 1.9: Fixed lexing stabby lambda w/ do/end
  * Deal better with DOS files. Ugh.
  * Fix line number of production after heredoc.
  * Fixed RubyParser#process to reuse parser instances across calls.
  * Fixed line numbers for several productions.
  * new_call sets line number to smallest line number of members.

=== 3.0.1 / 2012-11-02

* 2 bug fixes -- both based on MRI bugs... Yay?:

  * 1.9.2 doesn't have String#byteslice... so I have to do the *HORRIBLE*.
  * Lexing regexps w/ bad character encodings and no declared options will fall back to /.../n.

=== 3.0.0 / 2012-11-02

I've hit 99.967% success rate against 558k files! 3.6σ!! 182 files
failed to parse and spot checking them shows that they're pretty much
lost causes. I'm sure I missed some stuff, but it is more important at
this point to release a version to get more people using it in more
diverse ways than I can come up with.

* 3 minor enhancements:

  * Added RBStringScanner#charpos using #string_to_pos.
  * Added RBStringScanner#string_to_pos using String#byteslice.
  * Optimized regexp used for escape chars in strings. (presidentbeef)

* 3 bug fixes:

  * Fixed current_line and unread_many depending on StringScanner#pos
  * Fixed parsing of 'a[] = b'
  * Fixed the worst abuse of heredocs EVER. Just because it is valid doesn't mean you should.

=== 3.0.0.a10 / 2012-10-26

* 1 major enhancement:

  * iter nodes are now structurally the same as defs. Block args are COMPLETELY different.

* 4 minor enhancements:

  * 1.8 and 1.9 now treat f { |(...)| } differently, per MRI.
  * 1.9: Added __ENCODING__ support. (witlessbird)
  * 1.9: Finished coverage for masgn variants in block args
  * 1.9: covered all forms of block args: optional, splat, block, and destructuring

=== 3.0.0.a9 / 2012-10-22

Holy crap! I've hit a 99.92% success rate against 558k files! 492
failed parses to go. NOW is the time to start testing ruby_parser
against your code!

* 16 minor enhancements:

  * 1.9 parser: Finished magic encoding and utf-8 bom support.
  * 1.9: Added leading dot support eg "a\n.b".
  * 1.9: Added partial handling for *arg and opt=val args in block args.
  * 1.9: OMFG Encoding is the worst thing ever. Overhauled encoding support once again. It is much better at doing last ditch guessing of the encoding of the source file. 
  * 1.9: added String#grep for legacy support.
  * Added Sexp#block_pass?
  * Added ability to delete slow files when they timeout.
  * Added block_dup_check and refactored grammar code to use it for block arg handling.
  * Added in_lex_state?(*states) to clean up lexer code.
  * Added tentative support for optional block args. THIS IS SUBJECT TO CHANGE!!
  * Added toggleable debugging to StackState to make my life easier
  * All untested uses have not moved but have been laced with poison. 
  * Finally got good tests for most forms of double-block arg errors.
  * Moved all _known_ uses of SyntaxError to RubyParser::SyntaxError
  * f_block_optarg now always returns a block node.
  * ruby_parse_extract_error uses Find.find instead of globbing so dot files aren't missed.

* 12 bug fixes:

  * 1.9: Completely ignore IndexError in unread_many because... you know... it sucks.
  * 1.9: Fixed lex state after lexing ? in trinary.
  * 1.9: Fixed lex state in some ternarys.
  * 1.9: Fixed parsing of "1 ? b('') : 2\na d: 3"... ARE YOU NOT GLAD?!?!
  * Fix Timeout differences between 1.8 and 1.9 :(
  * Fixed emacs-style encodings to deal with no whitespace.
  * Fixed error message for bad % codes. (whitequark)
  * Fixed lexing of :a==>b vs :a===b vs :a==>b. P.S. Your space bar is broken. chump.
  * Fixed lexing of rare numeric formats.
  * Fixed magic comment encodings on DOS files ... :/
  * Fixed ruby_parse_extract_error to exit non-zero on errors.
  * Removed Symbol#is_argument and switch usage to is_arg?

=== 3.0.0.a8 / 2012-09-26

* 1 bug fix:

  * Remove method #d from global namespace. (alexch)

=== 3.0.0.a7 / 2012-09-21

* 3 minor enhancements:

  * Reorganized ruby_parse_extract_error so it will start much faster with a bunch of glob directories
  * RubyParserStuff#process takes optional time arg and raises Timeout::Error if it goes too long. You should rescue that, ya know...
  * ruby_parse_extract_error now checks *.rake and Rakefile on dir scan.

* 3 bug fixes:

  * 1.9: Fixed ternary state tracking so {a:f{f()},b:nil} won't trip up the lexer.
  * Fixed optional values in block args (no20/no21)
  * ruby_parse_extract_error skips non-files. Some asshats put .rb on their dirs. :(

=== 3.0.0.a6 / 2012-08-20

* 2 minor enhancements:

  * 1.8: Added basic encoding support to 1.8 parser. Assumes -Ku.
  * 1.9: Added encoding magic comment support to 1.9 parser.

* 8 bug fixes:

  * 1.9: Fixed lexing of -1 w/in conditionals. yeah... I dunno.
  * 1.9: Fixed parsing of a do | | end.
  * 1.9: Fixed parsing of not(x).
  * 1.9: Fixed parsing of op_asgn + rescue: 'a ||= b rescue nil'
  * 1.9: added \r to the EOL backslash handler. dos files blow
  * 1.9: hacked in a workaround for 1.9 specific regexps running in 1.8.
  * Added #reset to RubyParser proxy class
  * Fixed lexing of conditional w/ %() tokens

=== 3.0.0.a5 / 2012-07-31

* 5 bug fixes:

  * 1.9: Fix construction of 'f(:x, y: nil,)' w/ trailing comma.
  * 1.9: cleaned up lexing exit lex_state handling. Fixes bug parsing 'if f :x; end'
  * 1.9: fixed building of right-leaning masgns: 'f { |a, (b, c)| }'
  * 1.9: fixed lexing 'when *splat'
  * 1.9: fixed lexing of regexps in whens

=== 3.0.0.a4 / 2012-07-26

* 10 minor enhancements:

  * 'rake debug' defaults to 1.9 parser since that's all I'm doing these days
  * 1.9: Fixed f { |(a, b, ...), ...| ... } handling.
  * Added 'rake extract F=path' task to quickly extract errors from large files
  * Added on_error handler to provide more readable error message.
  * Aliased #process to #parse.
  * Renamed #parse to #process (legacy name), added default path of '(string)'
  * cleaned ruby_parse_extract_error  output and fixed to 1.9 parser
  * ruby_parse_extract_error expands shell globs from ARGV
  * ruby_parse_extract_error should also capture RuntimeError
  * yyerror(msg) now warns with the message instead of ignoring it.

* 3 bug fixes:

  * 1.9: Fixed bug lexing/parsing [ in rhs.
  * 1.9: Fixed f { |((a, b), c)| ... } handling
  * 1.9: fixed newline handling during expr_value

=== 3.0.0.a3 / 2012-07-03

* 1 major enhancement:

  * OMG! Implemented all the 1.9 arg crap and I went from 500 lines of
    structural diff to 32!

* 17 minor enhancements:

  * 1.9: !(...) is now a method call. ugh
  * 1.9: Added __ENCODING__ keyword (returns Unsupported atm).
  * 1.9: Added support for "a.()" thingy. whatever you call it. Have I ever mentioned that ruby has too much syntax? I mean really...
  * 1.9: Moved kRETURN/kBREAK/kNEXT from command_call to command
  * 1.9: Removed when_args and refactored into args production.
  * 1.9: Support for not/! being a call to #!. why matz? why?!?
  * 1.9: Support for trailing comma support in opt call args.
  * 1.9: Switched keyword support to deal with 1.8 vs 1.9 differences.
  * 1.9: refactored and created command_asgn
  * 1.9: removed case/else from primary
  * 1.9: reworked mlhs and family. Now horrible things like 'a, *, c = f' work
  * 1.9: reworked top level rules from program to expr. Lots of lame refactorings in mri.
  * Moved 'a[]' from primary down to method_call
  * Removed 'args trailer' production from assoc_list (eg {1, 2}).
  * Removed 1.9 support for : as a terminator in do/then
  * RubyParser no longer subclasses the 18 parser, so I had to change some consts around.
  * Undeprecated RubyParser - but now it is a compound parser that tries 1.9 first.

* 3 bug fixes:

  * 1.9: fixed bang method name.
  * Fixed case/when/splat handling on both 1.8 and 1.9.
  * Removed lambda_body production from lambda

=== 3.0.0.a2 / 2012-06-19

* 1 minor enhancement:

  * Updated to sexp_processor ~> 4.0

* 1 bug fix:

  * Fixed new_module to remove scope/block. Not enough tests on module variance.

* 1 known issue:

  * No parsing of "a.()" thingy. (reported by julian7) (patches welcome!)

=== 3.0.0.a1 / 2012-05-22

This is the first alpha release of the 3.0.0 series. It is probably
woefully incomplete, bug ridden, and hasn't showered in several days.
Please please please beat the crap out of it and send
bugs/patches/complaints/suggestions.

* 5 major enhancements:

  * 1.9 parsing! Thanks to EVERYONE for submitting patches for this!
  * Removed :arglist from everything but :op_asgn1
  * Removed :block from resbody
  * Removed :block from when
  * Removed :block nodes inside of scope nodes (defn/defs/class/sclass).
  * Removed :scope nodes in defn/defs/class/sclass nodes.
  * (probably more sexp cleanup to come before 3.0.0 final)

* 25 minor enhancements:

  * 1.9: Fix \!a. (wanabe)
  * 1.9: Method calling with postargs. (wanabe)
  * 1.9: Method definition with postargs. (wanabe)
  * 1.9: Support lambda args without parentheses. (wanabe)
  * Added R arg to `rake debug` to debug ruby straight up
  * Added RubyParser, subclassing Ruby18Parser but warning on instantiation.
  * Added backref_assign_error (needs tests)
  * Added bin/ruby_parse_extract_error to help with error submissions
  * Added debug task to help quickly get into a bug
  * Added more 18 vs 19 lexing tests for ?c.
  * Added ruby_parser.rb that pulls everything together in proper order.
  * Added tLABEL. (brynary)
  * Branched ruby_parser.y to ruby18_parser.y
  * Fix to pass test_lambda_args_block__19 test. (mrmargolis)
  * Got rid of one instance of unread_many. (Confusion)
  * Moved everything from RubyParser to RubyParserStuff and included module in both.
  * Refactored 1.9 args handling
  * Refactored and added new_resbody to ruby_parser_extras.
  * Refactored and added new_when
  * Refactored tests infrastructure and added both 1.8 and 1.9 test branches.
  * Removed unused methods: unread, begin_of_line? was_begin_of_line. (YAY!) (Confusion)
  * Renamed ruby_parser.y to ruby19_parser.y
  * RubyLexer now takes a version specifier.
  * Started doing comparative refactoring between MRI's 1.9 parser and RP's. Shouldn't differ functionally except where we missed stuff in RP.
  * `rake debug` prints the output if it succeeds.

* 21 bug fixes:

  * Added missing gvar arg error. (1.8)
  * Attach parser files to isolate to ensure they can build
  * Conditionalize handling of tLABEL to ruby19+. Fixes issue #33.
  * DOH. I deactivated some tests and never reactivated them. (Confusion ftw)
  * Duplicate the input so that heredoc processing doesn't morph original. (banister)
  * Entirely reworked block arg handling. (1.8)
  * Fix ?x char literal. (nobu)
  * Fixed 4/5 of literal lambda tests (jamie)
  * Fixed deps for parser
  * Fixed lexing of ?c for ruby 1.8 and 1.9.
  * Fixed more Ruby 1.9 args tests (brynary)
  * Fixed reswords to match MRI (1.8, 1.9)
  * Fixed symbols with no spaces in method calls (e.g. foo:bar) (YAY! brynary)
  * Fixed ternary_nil_no_space and other ternary edge cases for 1.9. (lastobelus)
  * Fixed test_call_not_equal__19. First bug bounty! (albus522)
  * Made lambda w/o arg list zero out the arg slot.
  * Renamed awords to qwords to match stupid MRI naming. (1.8, 1.9) :(
  * Rolled out brynary's symbols-no-spaces (foo:bar) changes when parsing 1.8 code
  * Split 1.8 from 1.9 open paren lexer. Gawd that's ugly code.
  * Split block_var from for_var. (1.8, 1.9)
  * Use binread (and emulate in ruby 1.8) to avoid encoding issues

=== 2.3.1 / 2011-09-21

* 2 bug fixes:

  * Fixed line numbers at end of special var+whitespace (larsch)
  * Holy crap I was smokin' something good... Fixed 1.9.3 warning

=== 2.3.0 / 2011-09-06

* 2 minor enhancements:

  * Add -g flag to parser compile if DEBUG
  * Lexer now embeds line number in yacc_value for keywords, helping fix up line numbers

* 3 bug fixes:

  * Fix method line numbers when no args and no parens (quix)
  * Fixed line numbers on return/break/next w/ result expr. (pjnz)
  * Fixed some lexing state in order to parse: 'f (1), 2' as 'f(1, 2)'. (invernizzi)

=== 2.2.0 / 2011-08-23

* 2 minor enhancements:

  * Moved Keyword, Environment, and StackState inside of RubyParser
  * Added proper dsym and dsym->sym support.

* 3 bug fixes:

  * Added extra (failing) tests for call/iter line number checking (quix)
  * Fixed line numbers for certain call/iter edge cases
  * Fixed parsing of: alias :"<<" :">>".

=== 2.1.0 / 2011-08-15

* 2 minor enhancements:

  * Added new accessor canonicalize_conditions to toggle conditional canonicalization (on by default). (confused)
  * Awesome cleanup: Replaced call to append_block by block_append. (Confusion)

* 2 bug fixes:

  * Fixed handling last line of =begin/=end. (raybaxter)
  * Fixed source line numbers after heredocs. (jbarreneche)

=== 2.0.6 / 2011-02-18

* 1 minor enhancement:

  * Switched to hoe's racc plugin to clean up rakefile and builds

* 1 bug fix:

  * Fixed empty =begin/end.

=== 2.0.5 / 2010-09-01

* 1 minor enhancement:

  * Started merging like lexical cases to try to squeeze some optimization out

=== 2.0.4 / 2009-08-18

* 1 minor enhancement:

  * Changed requires around to be more accurate.

* 4 bug fixes:

  * Fixed .autotest for minitest
  * Fixed emacs escape lexing bug: "\C-\\" (maglev/gemstone)
  * Fixed octal lexing edgecases. (maglev/gemstone)
  * Fixed regexp lexing edgecases. (maglev/gemstone)

=== 2.0.3 / 2009-06-23

* 4 minor enhancements:

  * Removed dead code handling d/regex match node cruft.
  * Switched to minitest
  * Updated .autotest and rakefile wrt rcov for new hoe capabilities
  * Updated hoe for new capabilities

* 4 bug fixes:

  * Environment#all now deals with strange edge cases in RAD.
  * Fixed packaging/compilation issue.
  * Minor 1.9 fixes
  * hoe -> flay -> rubyparser rakefile circularity fixed

=== 2.0.2 / 2009-01-20

* 2 minor enhancements:

  * Added gauntlet_rubyparser plugin. YAY for easy massive bug-hunting.
  * Promoted Sexp's file/line/comments to sexp_processor.

* 4 bug fixes:

  * Fixed and improved the readme
  * Fixed lexing heredoc newlines.
  * Fixed line numbers on defns.
  * Fixed rdoc generation bug pointed out by hugh sasse (who rocks)

=== 2.0.1 / 2008-11-04

* 2 minor enhancements:

  * Updated for changes to splat node in many contexts.
  * Made PT a developer dep

=== 2.0.0 / 2008-10-22

* 1 major enhancement

  * Brought on the AWESOME! 4x faster! no known lexing/parsing bugs!

* 71 minor enhancements

  * 1.9: Added Fixnum#ord.
  * 1.9: Added missing Regexp constants and did it so it'd work on 1.9.
  * Added #store_comment and #comments
  * Added StringScanner #begin_of_line?
  * Added a bunch of tests for regexp escape chars, #parse_string, #read_escape, ? numbers, ? whitespace.
  * Added a hack for rubinius' r2l eval bug.
  * Added a new token type tSTRING that bypasses tSTRING_BEG/END entirely. Only does non-interpolated strings and then falls back to the old way. MUCH cleaner tho.
  * Added bin/ruby_parse
  * Added compare rule to Rakefile.
  * Added coverage files/dirs to clean rule.
  * Added file and line numbers to all sexp nodes. Column/ranges to come.
  * Added lex_state change for lvars at the end of yylex.
  * Added lexed comments to defn/defs/class/module nodes.
  * Added stats gathering for yylex. Reordered yylex for avg data
  * Added tSYMBOL token type and parser rule to speed up symbol lexing.
  * Added tally output for getch, unread, and unread_many.
  * Added tests for ambigous uminus/uplus, backtick in cmdarg, square and curly brackets, numeric gvars, eos edge cases, string quoting %<> and %%%.
  * All cases throughout yylex now return directly if they match, no passthroughs.
  * All lexer cases now slurp entire token in one swoop.
  * All zarrays are now just empty arrays.
  * Changed s(:block_arg, :blah) to :"&blah" in args sexp.
  * Cleaned up lexer error handling. Now just raises all over.
  * Cleaned up read_escape and regx_options
  * Cleaned up tokadd_string (for some definition of cleaned).
  * Converted single quoted strings to new tSTRING token type.
  * Coverage is currently 94.4% on lexer.
  * Done what I can to clean up heredoc lexing... still sucks.
  * Flattened resbodies in rescue node. Fixed .autotest file.
  * Folded lex_keywords back in now that it screams.
  * Found very last instanceof ILiteralNode in the code. haha!
  * Got the tests subclassing PTTC and cleaned up a lot. YAY
  * Handle yield(*ary) properly
  * MASSIVELY cleaned out =begin/=end comment processor.
  * Massive overhaul on Keyword class. All hail the mighty Hash!
  * Massively cleaned up ident= edge cases and fixed a stupid bug from jruby.
  * Merged @/@@ scanner together, going to try to do the same everywhere.
  * Refactored fix_arg_lex_state, common across the lexer.
  * Refactored new_fcall into new_call.
  * Refactored some code to get better profile numbers.
  * Refactored some more #fix_arg_lex_state.
  * Refactored tail of yylex into its own method.
  * Removed Module#kill
  * Removed Token, replaced with Sexp.
  * Removed all parse_number and parse_quote tests.
  * Removed argspush, argscat. YAY!
  * Removed as many token_buffer.split(//)'s as possible. 1 to go.
  * Removed begins from compstmts
  * Removed buffer arg for tokadd_string.
  * Removed crufty (?) solo '@' token... wtf was that anyhow?
  * Removed most jruby/stringio cruft from StringScanner.
  * Removed one unread_many... 2 to go. They're harder.
  * Removed store_comment, now done directly.
  * Removed token_buffer. Now I just use token ivar.
  * Removed use of s() from lexer. Changed the way line numbers are gathered.
  * Renamed *qwords to *awords.
  * Renamed StringScanner to RPStringScanner (a subclass) to fix namespace trashing.
  * Renamed parse to process and aliased to parse.
  * Renamed token_buffer to string_buffer since that arcane shit still needs it.
  * Resolved the rest of the lexing issues I brought up w/ ruby-core.
  * Revamped tokadd_escape.
  * Rewrote Keyword and KWtable.
  * Rewrote RubyLexer using StringScanner.
  * Rewrote tokadd_escape. 79 lines down to 21.
  * Split out lib/ruby_parser_extras.rb so lexer is standalone.
  * Started to clean up the parser and make it as skinny as possible
  * Stripped out as much code as possible.
  * Stripped yylex of some dead code.
  * Switched from StringIO to StringScanner.
  * Updated rakefile for new hoe.
  * Uses pure ruby racc if ENV['PURE_RUBY'], otherwise use c.
  * Wrote a ton of lexer tests. Coverage is as close to 100% as possible.
  * Wrote args to clean up the big nasty args processing grammar section.
  * lex_strterm is now a plain array, removed RubyLexer#s(...).
  * yield and super now flatten args.

* 21+ bug fixes:

  * I'm sure this list is missing a lot:
  * Fixed 2 bugs both involving attrasgn (and ilk) esp when lhs is an array.
  * Fixed a bug in the lexer for strings with single digit hex escapes.
  * Fixed a bug parsing: a (args) { expr }... the space caused a different route to be followed and all hell broke loose.
  * Fixed a bug with x\n=beginvar not putting begin back.
  * Fixed attrasgn to have arglists, not arrays.
  * Fixed bug in defn/defs with block fixing.
  * Fixed class/module's name slot if colon2/3.
  * Fixed dstr with empty interpolation body.
  * Fixed for 1.9 string/char changes.
  * Fixed lexer BS wrt determining token type of words.
  * Fixed lexer BS wrt pass through values and lexing words. SO STUPID.
  * Fixed lexing of floats.
  * Fixed lexing of identifiers followed by equals. I hope.
  * Fixed masgn with splat on lhs
  * Fixed new_super to deal with block_pass correctly.
  * Fixed parser's treatment of :colon2 and :colon3.
  * Fixed regexp scanning of escaped numbers, ANY number is valid, not just octs.
  * Fixed string scanning of escaped octs, allowing 1-3 chars.
  * Fixed unescape for \n
  * Fixed: omg this is stupid. '()' was returning bare nil
  * Fixed: remove_begin now goes to the end, not sure why it didn't before.

=== 1.0.0 / 2007-12-20

* 1 major enhancement
  * Birthday!