Unable to process files with non-UTF-8 characters

[laynemcnish]

When the rails5-spec-converter encounters non-UTF-8 characters, it is unable to continue parsing.

Stack trace:

Processing: spec/unit/services/csv_processor_spec.rb
/Users/layne.mcnish/.rvm/gems/ruby-2.3.1/gems/rails5-spec-converter-1.0.15/lib/rails5/spec_converter/text_transformer.rb:28:in `transform': undefined method `each_node' for false:FalseClass (NoMethodError)
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/gems/rails5-spec-converter-1.0.15/lib/rails5/spec_converter/cli.rb:52:in `block (2 levels) in run'
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/gems/rails5-spec-converter-1.0.15/lib/rails5/spec_converter/cli.rb:47:in `glob'
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/gems/rails5-spec-converter-1.0.15/lib/rails5/spec_converter/cli.rb:47:in `block in run'
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/gems/rails5-spec-converter-1.0.15/lib/rails5/spec_converter/cli.rb:46:in `each'
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/gems/rails5-spec-converter-1.0.15/lib/rails5/spec_converter/cli.rb:46:in `run'
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/gems/rails5-spec-converter-1.0.15/exe/rails5-spec-converter:6:in `<top (required)>'
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/bin/rails5-spec-converter:22:in `load'
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/bin/rails5-spec-converter:22:in `<main>'
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `eval'
    from /Users/layne.mcnish/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `<main>'

After a process of trial and elimination, this is the spec that's causing the problem:

    context "when the csv has rows with non-UTF-8 characters" do
      let(:csv_text) { <<-CSV }
        FIRST_NAME,LAST_NAME,ADDRESS_1,ADDRESS_2
        Libby,David,1 Oak St\xA6,Apt 4
      CSV

      subject do
        processor.pre_process_and_adapt(method :convert_row) do |row|
          @ingested_row = row
        end
      end

      it "does not blow up and removes invalid characters" do
        expect { subject }.not_to raise_error
        expect(@ingested_row[:address_1]).to eq "1 Oak St"
      end
    end

I think is a similar issue as to what Rubocop is experiencing with Parser: whitequark/parser#283

[tjgrathwell]

For now, I've released 1.0.17 which will simply skip files that fail Parser's parsing for any reason.

Previously something as simple as an unmatched do...end would cause the whole thing to explode, so it seems like an improvement.