Skip to content

Commit

Permalink
Avoid trying to interpret escape sequences inside comments. (#95)
Browse files Browse the repository at this point in the history
* fix #94
  • Loading branch information
mmatera authored Nov 25, 2024
1 parent b97a62c commit 106d81d
Showing 1 changed file with 14 additions and 4 deletions.
18 changes: 14 additions & 4 deletions mathics_scanner/tokeniser.py
Original file line number Diff line number Diff line change
Expand Up @@ -472,10 +472,20 @@ def _skip_blank(self):
try:
self.incomplete()
except ValueError:
# Funny symbols like | in comments can cause a ValueError.
# Until we have a better fix -- like noting we are inside a
# comment and should not try to substitute symbols -- ignore.
pass
# `incomplete` tries to parse substrings like `\|AAAAA`
# that can be interpreted as a character reference.
# To do that, it tries to get the
# new line using the method
# `Prescanner.replace_escape_sequences()`
# Inside a comment, the special meaning of escape sequences
# like `\|` should not be taken into account.
#
# In case of error, just let's pick the code
# from the `input_line` attribute of
# prescanner:
self.code = self.prescanner.input_line
# TODO: handle the corner case where the rest of the line
# include escaped sequences, out of the comment.
else:
break
if comment:
Expand Down

0 comments on commit 106d81d

Please sign in to comment.