Avoid trying to interpret escape sequences inside comments. #95
Conversation
| # In case of error, just let's pick the code | ||
| # from the `input_line` attribute of | ||
| # prescanner: | ||
| self.code = self.prescanner.input_line |
There was a problem hiding this comment.
Actually, we can replace the whole try/except block by lines 476 and 477.
There was a problem hiding this comment.
If that works, it would be simpler.
There was a problem hiding this comment.
I tested against the comment that produces the syntax error in GS1.m and works. Then I tested on each GS?.m and we still have one test wrong on GS2.m and GS3.m, but it seems it is not related to this issue.
There was a problem hiding this comment.
Looking again, at found a problem with the "simpler" approach: it could happen that after the comment finishes, some code that needs to take into account the escape characters comes. For example,
(*
Hi, this is a test \|<-this would produce an error if we pass by `replace_escape_sequences`
*)
a="\|AAAAAA"
(*The previous line would require the use of replace_escape_sequences*)
So I was wrong with the extra simplification.
There was a problem hiding this comment.
Still this fix does not cover a corner case of the form
(* corner case *) A="\|AAAAAA"
So maybe we need to track this, and reload the last line. I will change the comment to mention this.
There was a problem hiding this comment.
All of these approaches feel hacky and not the way scanners usually work. Handling escape sequences is not something that is typically done separately from initial scanning. But right now this is not something I want to spend time on.
So right now I would prefer defer addressing until such time when we can do this properly. The code base has too many "quick hacks" and could use better-thought-out and more standard solutions.
There was a problem hiding this comment.
So maybe we need to track this, and reload the last line. I will change the comment to mention this.
Yes, please open an issue for this. Long comments on specific bugs are better put in the issue tracker. There, one can be extremely verbose, more so than in comments.
|
LGTM - merge when you are satisfied. Let's leave the more-complete rework for later though. |
This should fix #94