From 5ee062e563b3e8eab4adde2906be64d25a02b22e Mon Sep 17 00:00:00 2001 From: Jordan Date: Sun, 5 Jul 2020 18:43:29 -0700 Subject: [PATCH] Reason V4 [Stacked Diff 2/n #2599] [String Template Literals] Summary:This diff implements string template literals. Test Plan: Reviewers: CC: --- docs/TEMPLATE_LITERALS.md | 146 ++++++++++++++ .../expected_output/templateStrings.re | 190 ++++++++++++++++++ .../typeCheckedTests/input/templateStrings.re | 159 +++++++++++++++ src/reason-parser/dune | 2 +- src/reason-parser/reason_attributes.ml | 1 + .../reason_declarative_lexer.mll | 89 +++++++- src/reason-parser/reason_errors.ml | 3 + src/reason-parser/reason_errors.mli | 1 + src/reason-parser/reason_lexer.ml | 82 ++++++-- src/reason-parser/reason_location.ml | 2 +- src/reason-parser/reason_parser.mly | 63 +++++- src/reason-parser/reason_parser_explain.ml | 15 +- src/reason-parser/reason_pprint_ast.ml | 110 ++++++++-- src/reason-parser/reason_syntax_util.cppo.ml | 40 ++++ src/reason-parser/reason_syntax_util.cppo.mli | 8 + src/reason-parser/reason_template.ml | 97 +++++++++ src/reason-parser/reason_template.mli | 19 ++ 17 files changed, 978 insertions(+), 49 deletions(-) create mode 100644 docs/TEMPLATE_LITERALS.md create mode 100644 formatTest/typeCheckedTests/expected_output/templateStrings.re create mode 100644 formatTest/typeCheckedTests/input/templateStrings.re create mode 100644 src/reason-parser/reason_template.ml create mode 100644 src/reason-parser/reason_template.mli diff --git a/docs/TEMPLATE_LITERALS.md b/docs/TEMPLATE_LITERALS.md new file mode 100644 index 000000000..12134f5bf --- /dev/null +++ b/docs/TEMPLATE_LITERALS.md @@ -0,0 +1,146 @@ + +Contributors: Lexing and Parsing String Templates: +=================================================== +Supporting string templates requires coordination between the lexer, parser and +printer. The lexer (as always) creates a token stream, but when it encounters a +backtick, it begins a special parsing mode that collects the (mostly) raw text, +until either hitting a closing backtick, or a `${`. If it encounters the `${` +(called an "interpolation region"), it will temporarily resume the "regular" +lexing approach, instead of collecting the raw text - until it hits a balanced +`}`, upon which it will enter the "raw text" mode again until it hits the +closing backtick. + +- Parsing of raw text regions and regular tokenizing: Handled by + `reason_declarative_lexer.ml`. +- Token balancing: Handled by `reason_lexer.ml`. + +The output of lexing becomes tokens streamed into the parser, and the parser +`reason_parser.mly` turns those tokens into AST expressions. + +## Lexing: + +String templates are opened by: +- A backtick. +- Followed by any whitespace character (newline, or space/tab). + +- Any whitespace character (newline, or space/tab). +- Followed by a backtick + +```reason +let x = ` hi this is my string template ` +let x = ` +The newline counts as a whitespace character both for opening and closing. +` + +``` + +Within the string template literal, there may be regions of non-string +"interpolation" where expressions are lexed/parsed. + +```reason +let x = ` hi this is my ${expressionHere() ++ "!"} template ` +``` + +Template strings are lexed into tokens, some of those tokens contain a string +"payload" with portions of the string content. +The opening backtick, closing backtick, and `${` characters do not become a +token that is fed to the parser, and are not included in the text payload of +any token. The Right Brace `}` closing an interpolation region `${` _does_ +become a token that is fed to the parser. There are three tokens that are +produced when lexing string templates. + +- `STRING_TEMPLATE_TERMINATED(string)`: A string region that is terminated with + closing backtick. It may be the entire string template contents if there are + no interpolation regions `${}`, or it may be the final string segment after + an interpolation region `${}`, as long as it is the closing of the entire + template. +- `STRING_TEMPLATE_SEGMENT_LBRACE(string)`: A string region occuring _before_ + an interpolation region `${`. The `string` payload of this token is the + contents up until (but not including) the next `${`. +- `RBRACE`: A `}` character that terminates an interpolation region that + started with `${`. + +Simple example: + + STRING_TEMPLATE_TERMINATED + | | + ` lorem ipsum lorem ipsum bla ` + ^ ^ + | | + | The closing backtick also doesn't show up in the token + | stream, but the last white space is part of the lexed + | STRING_TEMPLATE_TERMINATED token + | (it is used to compute indentation, but is stripped from + | the string constant, or re-inserted in refmting if not present) + | + The backtick doesn't show up anywhere in the token stream. The first + single white space after backtick is also not part of the lexed tokens. + +Multiline example: + + All of this leading line whitespace remains parts of the tokens' payloads + but it is is normalized and stripped when the parser converts the tokens + into string expressions. + | + | This newline not part of any token + | | + | v + | ` + +-> lorem ipsum lorem + ipsum bla + ` + ^ + | + All of this white space on final line is part of the token as well. + + +For interpolation, the token `STRING_TEMPLATE_SEGMENT_LBRACE` represents the +string contents (minus any single/first white space after backtick), up to the +`${`. As with non-interpolated string templates, the opening and closing +backtick does not show up in the token stream, the first white space character +after opening backtick is not included in the lexed string contents, the final +white space character before closing backtick *is* part of the lexed string +token (to compute indentation), but that final white space character, along +with leading line whitespace is stripped from the string expression when the +parsing stage converts from lexed tokens to AST string expressions. + + ` lorem ipsum lorem ipsum bla${expression}lorem ipsum lorem ip lorem` + | | || | + STRING_TEMPLATE_TERMINATED |STRING_TEMPLATE_TERMINATED + RBRACE +## Parsing: + +The string template tokens are turned into normal AST expressions. +`STRING_TEMPLATE_SEGMENT_LBRACE` and `STRING_TEMPLATE_TERMINATED` lexed tokens +contains all of the string contents, plus leading line whitespace for each +line, including the final whitespace before the closing backtick. These are +normalized in the parser by stripping that leading whitespace including two +additional spaces for nice indentation, before turning them into some +combination of string contants with a special attribute on the AST, or string +concats with a special attribute on the concat AST node. + +```reason + +// This: +let x = ` + Hello there +`; +// Becomes: +let x = [@reason.template] "Hello there"; + +// This: +let x = ` + ${expr} Hello there +`; +// Becomes: +let x = [@reason.template] (expr ++ [@reason.template] "Hello there"); + +``` + +User Documentation: +=================== +> This section is the user documentation for string template literals, which +> will be published to the [official Reason Syntax +> documentation](https://reasonml.github.io/) when + +TODO diff --git a/formatTest/typeCheckedTests/expected_output/templateStrings.re b/formatTest/typeCheckedTests/expected_output/templateStrings.re new file mode 100644 index 000000000..39ba021d1 --- /dev/null +++ b/formatTest/typeCheckedTests/expected_output/templateStrings.re @@ -0,0 +1,190 @@ +[@reason.version 3.7]; +/** + * Comments: + */ + +let addTwo = (a, b) => string_of_int(a + b); +let singleLineConstant = ` + Single line template +`; +let singleLineInterpolate = ` + Single line ${addTwo(1, 2)}! +`; + +let multiLineConstant = ` + Multi line template + Multi %a{x, y}line template + Multi line template + Multi line template +`; + +let printTwo = (a, b) => { + print_string(a); + print_string(b); +}; + +let templteWithAttribute = + [@attrHere] + ` + Passing line template + Passing line template + Passing line template + Passing line template + `; + +let result = + print_string( + ` + Passing line template + Passing line template + Passing line template + Passing line template + `, + ); + +let resultPrintTwo = + printTwo( + "short one", + ` + Passing line template + Passing line template + Passing line template + Passing line template + `, + ); + +let hasBackSlashes = ` + One not escaped: \ + Three not escaped: \ \ \ + Two not escaped: \\ + Two not escaped: \\\ + One not escaped slash, and one escaped tick: \\` + Two not escaped slashes, and one escaped tick: \\\` + Two not escaped slashes, and one escaped dollar-brace: \\\${ + One not escaped slash, then a close tick: \ +`; + +let singleLineInterpolateWithEscapeTick = ` + Single \`line ${addTwo(1, 2)}! +`; + +let singleLineConstantWithEscapeDollar = ` + Single \${line template +`; + +// The backslash here is a backslash literal. +let singleLineInterpolateWithBackslashThenDollar = ` + Single \$line ${addTwo(2, 3)}! +`; + +let beforeExpressionCommentInNonLetty = ` + Before expression comment in non-letty interpolation: + ${/* Comment */ string_of_int(1 + 2)} +`; + +let beforeExpressionCommentInNonLetty2 = ` + Same thing but with comment on own line: + ${ + /* Comment */ + string_of_int(10 + 8) + } +`; +module StringIndentationWorksInModuleIndentation = { + let beforeExpressionCommentInNonLetty2 = ` + Same thing but with comment on own line: + ${ + /* Comment */ + string_of_int(10 + 8) + } + `; +}; + +let beforeExpressionCommentInNonLetty3 = ` + Same thing but with text after final brace on same line: + ${ + /* Comment */ + string_of_int(20 + 1000) + }TextAfterBrace +`; + +let beforeExpressionCommentInNonLetty3 = ` + Same thing but with text after final brace on next line: + ${ + /* Comment */ + string_of_int(100) + } + TextAfterBrace +`; + +let x = 0; +let commentInLetSequence = ` + Comment in letty interpolation: + ${ + /* Comment */ + let x = 200 + 49; + string_of_int(x); + } +`; + +let commentInLetSequence2 = ` + Same but with text after final brace on same line: + ${ + /* Comment */ + let x = 200 + 49; + string_of_int(x); + }TextAfterBrace +`; + +let commentInLetSequence3 = ` + Same but with text after final brace on next line: + ${ + /* Comment */ + let x = 200 + 49; + string_of_int(x); + } + TextAfterBrace +`; + +let reallyCompicatedNested = ` + Comment in non-letty interpolation: + + ${ + /* Comment on first line of interpolation region */ + + let y = (a, b) => a + b; + let x = 0 + y(0, 2); + // Nested string templates + let s = ` + asdf${addTwo(0, 0)} + alskdjflakdsjf + `; + s ++ s; + }same line as brace with one space + and some more text at the footer no newline +`; + +let reallyLongIdent = "!"; +let backToBackInterpolations = ` + Two interpolations side by side: + ${addTwo(0, 0)}${addTwo(0, 0)} + Two interpolations side by side with leading and trailing: + Before${addTwo(0, 0)}${addTwo(0, 0)}After + + Two interpolations side by side second one should break: + Before${addTwo(0, 0)}${ + reallyLongIdent + ++ reallyLongIdent + ++ reallyLongIdent + ++ reallyLongIdent + }After + + Three interpolations side by side: + Before${addTwo(0, 0)}${ + reallyLongIdent + ++ reallyLongIdent + ++ reallyLongIdent + ++ reallyLongIdent + }${ + "" + }After +`; diff --git a/formatTest/typeCheckedTests/input/templateStrings.re b/formatTest/typeCheckedTests/input/templateStrings.re new file mode 100644 index 000000000..f036df30e --- /dev/null +++ b/formatTest/typeCheckedTests/input/templateStrings.re @@ -0,0 +1,159 @@ +/** + * Comments: + */ + + +let addTwo = (a, b) => string_of_int(a + b); +let singleLineConstant = ` Single line template `; +let singleLineInterpolate = ` Single line ${addTwo(1, 2)} ! `; + +let multiLineConstant = ` + Multi line template + Multi %a{x, y}line template + Multi line template + Multi line template +`; + +let printTwo = (a, b) => { + print_string(a) + print_string(b) +}; + +let templteWithAttribute = [@attrHere] ` + Passing line template + Passing line template + Passing line template + Passing line template +`; + + +let result = print_string(` + Passing line template + Passing line template + Passing line template + Passing line template +`); + +let resultPrintTwo = printTwo("short one", ` + Passing line template + Passing line template + Passing line template + Passing line template +`); + +let hasBackSlashes = ` + One not escaped: \ + Three not escaped: \ \ \ + Two not escaped: \\ + Two not escaped: \\\ + One not escaped slash, and one escaped tick: \\` + Two not escaped slashes, and one escaped tick: \\\` + Two not escaped slashes, and one escaped dollar-brace: \\\${ + One not escaped slash, then a close tick: \ `; + +let singleLineInterpolateWithEscapeTick = ` Single \`line ${addTwo(1, 2)} ! `; + +let singleLineConstantWithEscapeDollar = ` Single \${line template `; + +// The backslash here is a backslash literal. +let singleLineInterpolateWithBackslashThenDollar = ` Single \$line ${addTwo(2, 3)} ! `; + +let beforeExpressionCommentInNonLetty = ` + Before expression comment in non-letty interpolation: + ${/* Comment */ string_of_int(1 + 2)} +`; + +let beforeExpressionCommentInNonLetty2 = ` + Same thing but with comment on own line: + ${ + /* Comment */ + string_of_int(10 + 8) + } +`; +module StringIndentationWorksInModuleIndentation = { + let beforeExpressionCommentInNonLetty2 = ` + Same thing but with comment on own line: + ${ + /* Comment */ + string_of_int(10 + 8) + } + `; +}; + +let beforeExpressionCommentInNonLetty3 = ` + Same thing but with text after final brace on same line: + ${ + /* Comment */ + string_of_int(20 + 1000) + }TextAfterBrace +`; + +let beforeExpressionCommentInNonLetty3 = ` + Same thing but with text after final brace on next line: + ${ + /* Comment */ + string_of_int(100) + } + TextAfterBrace +`; + +let x = 0; +let commentInLetSequence = ` + Comment in letty interpolation: + ${ + /* Comment */ + let x = 200 + 49; + string_of_int(x); + } +`; + +let commentInLetSequence2 = ` + Same but with text after final brace on same line: + ${ + /* Comment */ + let x = 200 + 49; + string_of_int(x); + }TextAfterBrace +`; + +let commentInLetSequence3 = ` + Same but with text after final brace on next line: + ${ + /* Comment */ + let x = 200 + 49; + string_of_int(x); + } + TextAfterBrace +`; + +let reallyCompicatedNested = ` + Comment in non-letty interpolation: + + ${ + /* Comment on first line of interpolation region */ + + let y = (a, b) => a + b; + let x = 0 + y(0, 2); + // Nested string templates + let s = ` + asdf${addTwo(0, 0)} + alskdjflakdsjf + `; + s ++ s + } same line as brace with one space + and some more text at the footer no newline +`; + +let reallyLongIdent = "!"; +let backToBackInterpolations = ` + Two interpolations side by side: + ${addTwo(0, 0)}${addTwo(0, 0)} + Two interpolations side by side with leading and trailing: + Before${addTwo(0, 0)}${addTwo(0, 0)}After + + Two interpolations side by side second one should break: + Before${addTwo(0, 0)}${reallyLongIdent ++ reallyLongIdent ++ reallyLongIdent ++ reallyLongIdent}After + + Three interpolations side by side: + Before${addTwo(0, 0)}${reallyLongIdent ++ reallyLongIdent ++ reallyLongIdent ++ reallyLongIdent}${""}After +`; diff --git a/src/reason-parser/dune b/src/reason-parser/dune index 9214362b3..ed7c91640 100644 --- a/src/reason-parser/dune +++ b/src/reason-parser/dune @@ -94,5 +94,5 @@ reason_parser reason_single_parser reason_multi_parser merlin_recovery reason_recover_parser reason_declarative_lexer reason_lexer reason_oprint reason_parser_explain_raw reason_parser_explain reason_parser_recover - reason_string) + reason_template reason_string) (libraries ocaml-migrate-parsetree menhirLib reason.easy_format reason.version)) diff --git a/src/reason-parser/reason_attributes.ml b/src/reason-parser/reason_attributes.ml index 74d0e4395..f583930cf 100644 --- a/src/reason-parser/reason_attributes.ml +++ b/src/reason-parser/reason_attributes.ml @@ -19,6 +19,7 @@ let is_stylistic_attr = function * affect printing *) | { attr_name = {txt="ocaml.ppwarn"}; _} | { attr_name = {txt="reason.preserve_braces"}; _} -> true + | { attr_name = {txt="reason.template"}; _} -> true | _ -> false diff --git a/src/reason-parser/reason_declarative_lexer.mll b/src/reason-parser/reason_declarative_lexer.mll index e144191c7..bd135b1e7 100644 --- a/src/reason-parser/reason_declarative_lexer.mll +++ b/src/reason-parser/reason_declarative_lexer.mll @@ -45,8 +45,8 @@ * *) -(* This is the Reason lexer. As stated in src/README, there's a good section in - Real World OCaml that describes what a lexer is: +(* This is the Reason lexer. As stated in docs/GETTING_STARTED_CONTRIBUTING.md + * there's a good section in Real World OCaml that describes what a lexer is: https://realworldocaml.org/v1/en/html/parsing-with-ocamllex-and-menhir.html @@ -154,6 +154,11 @@ type state = { txt_buffer : Buffer.t; } +type string_template_parse_result = + | TemplateTerminated + | TemplateNotTerminated + | TemplateInterpolationMarker + let get_scratch_buffers { raw_buffer; txt_buffer } = Buffer.reset raw_buffer; Buffer.reset txt_buffer; @@ -376,6 +381,15 @@ rule token state = parse try Hashtbl.find keyword_table s with Not_found -> LIDENT s } + | "`" (lowercase | uppercase) identchar * + { let s = Lexing.lexeme lexbuf in + let word = String.sub s 1 (String.length s - 1) in + match Hashtbl.find keyword_table word with + | exception Not_found -> NAMETAG word + | _ -> + raise_error (Location.curr lexbuf) (Keyword_as_tag word); + LIDENT "thisIsABugReportThis" + } | lowercase_latin1 identchar_latin1 * { Ocaml_util.warn_latin1 lexbuf; LIDENT (Lexing.lexeme lexbuf) } | uppercase identchar * @@ -423,6 +437,17 @@ rule token state = parse let txt = flush_buffer raw_buffer in STRING (txt, None, Some delim) } + | "`" newline + { + (* Need to update the location in the case of newline so line counts are + * correct *) + update_loc lexbuf None 1 false 0; + token_in_template_string_region state lexbuf + } + | "`" (' ' | '\t') + { + token_in_template_string_region state lexbuf + } | "'" newline "'" { (* newline can span multiple characters (if the newline starts with \13) @@ -460,7 +485,6 @@ rule token state = parse } | "&" { AMPERSAND } | "&&" { AMPERAMPER } - | "`" { BACKQUOTE } | "'" { QUOTE } | "(" { LPAREN } | ")" { RPAREN } @@ -775,6 +799,7 @@ and comment buffer firstloc nestedloc = parse { store_lexeme buffer lexbuf; comment buffer firstloc nestedloc lexbuf } + | "'" newline "'" { store_lexeme buffer lexbuf; update_loc lexbuf None 1 false 1; @@ -912,6 +937,64 @@ and quoted_string buffer delim = parse quoted_string buffer delim lexbuf } +and template_string_region buffer = parse + | newline + { store_lexeme buffer lexbuf; + update_loc lexbuf None 1 false 0; + template_string_region buffer lexbuf + } + | eof + { TemplateNotTerminated } + | "`" + { + TemplateTerminated + } + | "${" + { + (* set_lexeme_length lexbuf 1; *) + TemplateInterpolationMarker + } + | '\\' '`' + { + Buffer.add_char buffer '`'; + template_string_region buffer lexbuf + } + | '\\' '$' '{' + { + Buffer.add_char buffer '$'; + Buffer.add_char buffer '{'; + template_string_region buffer lexbuf + } + | _ as c + { + Buffer.add_char buffer c; + template_string_region buffer lexbuf + } + + +and token_in_template_string_region state = parse + | _ + { + (* Unparse it, now go run the template string parser with a buffer *) + set_lexeme_length lexbuf 0; + let string_start = lexbuf.lex_start_p in + let start_loc = Location.curr lexbuf in + let raw_buffer, _ = get_scratch_buffers state in + match template_string_region raw_buffer lexbuf with + | TemplateNotTerminated -> raise_error start_loc Unterminated_string; + STRING_TEMPLATE_TERMINATED + "This should never be happen. If you see this string anywhere, file a bug to the Reason repo." + | TemplateTerminated -> + lexbuf.lex_start_p <- string_start; + let txt = flush_buffer raw_buffer in + STRING_TEMPLATE_TERMINATED txt + | TemplateInterpolationMarker -> + lexbuf.lex_start_p <- string_start; + let txt = flush_buffer raw_buffer in + STRING_TEMPLATE_SEGMENT_LBRACE txt + } + + and skip_sharp_bang = parse | "#!" [^ '\n']* '\n' [^ '\n']* "\n!#\n" { update_loc lexbuf None 3 false 0 } diff --git a/src/reason-parser/reason_errors.ml b/src/reason-parser/reason_errors.ml index 7df0a1b9f..af49e86aa 100644 --- a/src/reason-parser/reason_errors.ml +++ b/src/reason-parser/reason_errors.ml @@ -19,6 +19,7 @@ type lexing_error = | Unterminated_string | Unterminated_string_in_comment of Location.t * Location.t | Keyword_as_label of string + | Keyword_as_tag of string | Literal_overflow of string | Invalid_literal of string @@ -78,6 +79,8 @@ let format_lexing_error ppf = function Ocaml_util.print_loc loc | Keyword_as_label kwd -> fprintf ppf "`%s' is a keyword, it cannot be used as label name" kwd + | Keyword_as_tag kwd -> + fprintf ppf "`%s' is a keyword, it cannot be used as tag name" kwd | Literal_overflow ty -> fprintf ppf "Integer literal exceeds the range of representable \ integers of type %s" ty diff --git a/src/reason-parser/reason_errors.mli b/src/reason-parser/reason_errors.mli index fc265e8a4..ac6c0105e 100644 --- a/src/reason-parser/reason_errors.mli +++ b/src/reason-parser/reason_errors.mli @@ -17,6 +17,7 @@ type lexing_error = | Unterminated_string | Unterminated_string_in_comment of Location.t * Location.t | Keyword_as_label of string + | Keyword_as_tag of string | Literal_overflow of string | Invalid_literal of string diff --git a/src/reason-parser/reason_lexer.ml b/src/reason-parser/reason_lexer.ml index f68831c02..c8f3e1a0d 100644 --- a/src/reason-parser/reason_lexer.ml +++ b/src/reason-parser/reason_lexer.ml @@ -31,16 +31,23 @@ let init ?insert_completion_ident lexbuf = let lexbuf state = state.lexbuf -let rec token state = - match - Reason_declarative_lexer.token - state.declarative_lexer_state state.lexbuf + +let rec comment_capturing_tokenizer tokenizer = + fun state -> + match tokenizer state.declarative_lexer_state state.lexbuf with | COMMENT (s, comment_loc) -> state.comments <- (s, comment_loc) :: state.comments; - token state + comment_capturing_tokenizer tokenizer state | tok -> tok +let token a = (comment_capturing_tokenizer Reason_declarative_lexer.token) a + +let token_after_interpolation_region state = + Reason_declarative_lexer.token_in_template_string_region + state.declarative_lexer_state + state.lexbuf + (* Routines for manipulating lexer state *) let save_triple lexbuf tok = @@ -56,6 +63,25 @@ exception Lex_balanced_failed of token positioned list * exn option let closing_of = function | LPAREN -> RPAREN | LBRACE -> RBRACE + | LBRACKET + | LBRACKETLESS + | LBRACKETGREATER + | LBRACKETAT + | LBRACKETPERCENT + | LBRACKETPERCENTPERCENT -> RBRACKET + | STRING_TEMPLATE_SEGMENT_LBRACE _ -> RBRACE + | _ -> assert false + +let continuer_after_closing = function + | LBRACKET + | LBRACKETLESS + | LBRACKETGREATER + | LBRACKETAT + | LBRACKETPERCENT + | LBRACKETPERCENTPERCENT + | LPAREN + | LBRACE -> fun a -> token a + | STRING_TEMPLATE_SEGMENT_LBRACE _ -> fun a-> token_after_interpolation_region a | _ -> assert false let inject_es6_fun = function @@ -77,17 +103,16 @@ let rec lex_balanced_step state closing acc tok = raise (Lex_balanced_failed (acc, None)) | (( LBRACKET | LBRACKETLESS | LBRACKETGREATER | LBRACKETAT - | LBRACKETPERCENT | LBRACKETPERCENTPERCENT ), _) -> - lex_balanced state closing (lex_balanced state RBRACKET acc) - | ((LPAREN | LBRACE), _) -> + | LBRACKETPERCENT | LBRACKETPERCENTPERCENT ) as l, _) -> + lex_balanced state closing (lex_balanced state (closing_of l) acc) + | ((LPAREN | LBRACE | STRING_TEMPLATE_SEGMENT_LBRACE _), _) -> let rparen = try lex_balanced state (closing_of tok) [] with (Lex_balanced_failed (rparen, None)) -> raise (Lex_balanced_failed (rparen @ acc, None)) in - begin match token state with - | exception exn -> - raise (Lex_balanced_failed (rparen @ acc, Some exn)) + begin match (continuer_after_closing tok) state with + | exception exn -> raise (Lex_balanced_failed (rparen @ acc, Some exn)) | tok' -> let acc = if is_triggering_token tok' then inject_es6_fun acc else acc in lex_balanced_step state closing (rparen @ acc) tok' @@ -118,8 +143,7 @@ let rec lex_balanced_step state closing acc tok = and lex_balanced state closing acc = match token state with - | exception exn -> - raise (Lex_balanced_failed (acc, Some exn)) + | exception exn -> raise (Lex_balanced_failed (acc, Some exn)) | tok -> lex_balanced_step state closing acc tok let lookahead_esfun state (tok, _, _ as lparen) = @@ -145,6 +169,25 @@ let lookahead_esfun state (tok, _, _ as lparen) = ) end +let rec lookahead_in_template_string_interpolation state (tok, _, _ as lparen) = + match lex_balanced state (closing_of tok) [] with + | exception (Lex_balanced_failed (tokens, exn)) -> + state.queued_tokens <- List.rev tokens; + state.queued_exn <- exn; + lparen + | tokens -> + (* tokens' head will be the RBRACE *) + (* Change the below to parse "remaining template string" entrypoint *) + begin match token_after_interpolation_region state with + | exception exn -> + state.queued_tokens <- List.rev tokens; + state.queued_exn <- Some exn; + lparen + | token -> + state.queued_tokens <- List.rev (save_triple state.lexbuf token :: tokens); + lparen + end + let token state = let lexbuf = state.lexbuf in match state.queued_tokens, state.queued_exn with @@ -155,8 +198,12 @@ let token state = lookahead_esfun state lparen | [(LBRACE, _, _) as lparen], None -> lookahead_esfun state lparen + | [(STRING_TEMPLATE_SEGMENT_LBRACE s, _, _) as template_seg], None -> + lookahead_in_template_string_interpolation state template_seg | [], None -> begin match token state with + | (STRING_TEMPLATE_SEGMENT_LBRACE _) as tok -> + lookahead_in_template_string_interpolation state (save_triple state.lexbuf tok) | LPAREN | LBRACE as tok -> lookahead_esfun state (save_triple state.lexbuf tok) | (LIDENT _ | UNDERSCORE) as tok -> @@ -166,10 +213,17 @@ let token state = state.queued_exn <- Some exn; tok | tok' -> + (* On finding an identifier or underscore in expression position, if + * the next token is "triggering" =>/:, then only return a + * ficticious (fake_triple) ES6_FUN marker, but queue up both the + * identifier tok as well as whatever was "triggering" into + * queued_tokens *) if is_triggering_token tok' then ( state.queued_tokens <- [tok; save_triple lexbuf tok']; fake_triple ES6_FUN tok ) else ( + (* Otherwise return the identifier token, but queue up the token + * that didn't "trigger" *) state.queued_tokens <- [save_triple lexbuf tok']; tok ) @@ -179,7 +233,9 @@ let token state = | x :: xs, _ -> state.queued_tokens <- xs; x +(* Tokenize with support for IDE completion *) let token state = + (* The _last_ token's end position *) let space_start = state.last_cnum in let (token', start_p, curr_p) as token = token state in let token_start = start_p.Lexing.pos_cnum in diff --git a/src/reason-parser/reason_location.ml b/src/reason-parser/reason_location.ml index 809d3096d..528daaeec 100644 --- a/src/reason-parser/reason_location.ml +++ b/src/reason-parser/reason_location.ml @@ -8,7 +8,7 @@ module Range = struct lnum_end: int } - (** + (* * make a range delimited by [loc1] and [loc2] * 1| let a = 1; * 2| diff --git a/src/reason-parser/reason_parser.mly b/src/reason-parser/reason_parser.mly index 35eeb54e3..789ef163a 100644 --- a/src/reason-parser/reason_parser.mly +++ b/src/reason-parser/reason_parser.mly @@ -173,6 +173,9 @@ let make_ghost_loc loc = { let ghloc ?(loc=dummy_loc ()) d = { txt = d; loc = (make_ghost_loc loc) } +let reloc_expr exp startpos endpos = + {exp with pexp_loc = {exp.pexp_loc with loc_start = startpos; loc_end = endpos}} + (** * turn an object into a real *) @@ -309,11 +312,11 @@ let mkoperator {Location. txt; loc} = let ghunit ?(loc=dummy_loc ()) () = mkexp ~ghost:true ~loc (Pexp_construct (mknoloc (Lident "()"), None)) -let mkinfixop arg1 op arg2 = - mkexp(Pexp_apply(op, [Nolabel, arg1; Nolabel, arg2])) +let mkinfixop ?loc ?attrs arg1 op arg2 = + mkexp ?loc ?attrs (Pexp_apply(op, [Nolabel, arg1; Nolabel, arg2])) -let mkinfix arg1 name arg2 = - mkinfixop arg1 (mkoperator name) arg2 +let mkinfix ?loc ?attrs arg1 name arg2 = + mkinfixop ?loc ?attrs arg1 (mkoperator name) arg2 let neg_string f = if String.length f > 0 && f.[0] = '-' @@ -1121,6 +1124,7 @@ let add_brace_attr expr = %token AS %token ASSERT %token BACKQUOTE +%token NAMETAG [@recover.expr ""] [@recover.cost 2] %token BANG %token BAR %token BARBAR @@ -1231,6 +1235,12 @@ let add_brace_attr expr = %token STAR %token STRING [@recover.expr ("", None, None)] [@recover.cost 2] + +%token STRING_TEMPLATE_TERMINATED + [@recover.expr ("")] [@recover.cost 2] +%token STRING_TEMPLATE_SEGMENT_LBRACE + [@recover.expr ("")] [@recover.cost 2] + %token STRUCT %token THEN %token TILDE @@ -3030,6 +3040,11 @@ parenthesized_expr: *) %inline simple_expr_template(E): | as_loc(val_longident) { mkexp (Pexp_ident $1) } + | template_string + { + let (_indent, expr) = $1 in + expr + } | constant { let attrs, cst = $1 in mkexp ~attrs (Pexp_constant cst) } | jsx { $1 } @@ -4670,6 +4685,44 @@ constant: } ; +(* + * Important: Read docs/TEMPLATE_LITERALS.md to understand this. + * TODO: In STRING_TEMPLATE_TERMINATED case, detect empty string and return a + * None. + *) +template_string: + | STRING_TEMPLATE_TERMINATED { + let split_by_newlines = Reason_syntax_util.split_by_newline ~keep_empty:true $1 in + let revLines = List.rev split_by_newlines in + let (indent, revLines) = + Reason_template.Parse.normalize_or_remove_last_line revLines in + let txt = + Reason_template.Parse.strip_leading_for_non_last ~indent "" revLines in + ( indent, + Ast_helper.Exp.constant (Pconst_string (txt, Some "reason.template"))) + } + | STRING_TEMPLATE_SEGMENT_LBRACE seq_expr RBRACE template_string + { + let indent, tmplt = $4 in + let op1 = mkloc "++" (mklocation $endpos($1) $startpos($2)) in + let op2 = mkloc "++" (mklocation $startpos($3) $startpos($4)) in + (* Right associative, unlike the future ++ will be parsed in next breaking + * change. We will keep this right assoc though to make it easy to print *) + let attrs = simple_ghost_text_attr "reason.template" in + let seq_expr = reloc_expr $2 $endpos($1) $startpos($3) in + if String.length $1 == 0 then + (indent, mkinfix ~attrs seq_expr op2 tmplt) + else + let split_by_newlines = Reason_syntax_util.split_by_newline ~keep_empty:true $1 in + let revLines = List.rev split_by_newlines in + let txt = + Reason_template.Parse.strip_leading_for_non_last ~indent "" revLines in + let left = (Ast_helper.Exp.constant (Pconst_string (txt, None))) in + (indent, mkinfix ~attrs left op1 (mkinfix ~attrs seq_expr op2 tmplt)) + (* TODO: Perform the string concat or printf depending *) + } +; + signed_constant: | constant { $1 } | MINUS INT { let (n, m) = $2 in ([], Pconst_integer("-" ^ n, m)) } @@ -4870,7 +4923,7 @@ toplevel_directive: opt_LET_MODULE: MODULE { () } | LET MODULE { () }; -%inline name_tag: BACKQUOTE ident { $2 }; +%inline name_tag: NAMETAG { $1 }; %inline label: LIDENT { $1 }; diff --git a/src/reason-parser/reason_parser_explain.ml b/src/reason-parser/reason_parser_explain.ml index b3f25024c..9944db575 100644 --- a/src/reason-parser/reason_parser_explain.ml +++ b/src/reason-parser/reason_parser_explain.ml @@ -84,15 +84,15 @@ let unclosed_parenthesis is_opening_symbol closing_symbol check_function env = None let check_unclosed env = - let check (message, opening_symbols, closing_symbol, check_function) = + let check (message, open_msg, opening_symbols, closing_symbol, check_function) = match unclosed_parenthesis (fun x -> List.mem x opening_symbols) closing_symbol check_function env with | None -> None | Some (loc_start, _) -> - Some (Format.asprintf "Unclosed %S (opened line %d, column %d)" - message loc_start.pos_lnum + Some (Format.asprintf "Unclosed %S (%s on line %d, column %d)" + message open_msg loc_start.pos_lnum (loc_start.pos_cnum - loc_start.pos_bol)) in let rec check_list = function @@ -103,13 +103,16 @@ let check_unclosed env = | Some result -> result in check_list [ - ("(", Interp.[X (T T_LPAREN)], + ("${", "for string region beginning ", Interp.[X (T T_STRING_TEMPLATE_SEGMENT_LBRACE)], + Interp.X (T T_RBRACE), + Raw.transitions_on_rbrace); + ("(", "opened", Interp.[X (T T_LPAREN)], Interp.X (T T_RPAREN), Raw.transitions_on_rparen); - ("{", Interp.[X (T T_LBRACE); X (T T_LBRACELESS)], + ("{", "opened", Interp.[X (T T_LBRACE); X (T T_LBRACELESS)], Interp.X (T T_RBRACE), Raw.transitions_on_rbrace); - ("[", Interp.[ X (T T_LBRACKET); X (T T_LBRACKETAT); + ("[", "opened", Interp.[ X (T T_LBRACKET); X (T T_LBRACKETAT); X (T T_LBRACKETBAR); X (T T_LBRACKETGREATER); X (T T_LBRACKETLESS); X (T T_LBRACKETPERCENT); X (T T_LBRACKETPERCENTPERCENT); ], diff --git a/src/reason-parser/reason_pprint_ast.ml b/src/reason-parser/reason_pprint_ast.ml index 44a167e04..8a6e89f88 100644 --- a/src/reason-parser/reason_pprint_ast.ml +++ b/src/reason-parser/reason_pprint_ast.ml @@ -1194,21 +1194,13 @@ let rec beginsWithStar_ line length idx = let beginsWithStar line = beginsWithStar_ line (String.length line) 0 -let rec numLeadingSpace_ line length idx accum = - if idx = length then accum else - match String.get line idx with - | '\t' | ' ' -> numLeadingSpace_ line length (idx + 1) (accum + 1) - | _ -> accum - -let numLeadingSpace line = numLeadingSpace_ line (String.length line) 0 0 - (* Computes the smallest leading spaces for non-empty lines *) let smallestLeadingSpaces strs = let rec smallestLeadingSpaces curMin strs = match strs with | [] -> curMin | ""::tl -> smallestLeadingSpaces curMin tl | hd::tl -> - let leadingSpace = numLeadingSpace hd in + let leadingSpace = Reason_syntax_util.num_leading_space hd in let nextMin = min curMin leadingSpace in smallestLeadingSpaces nextMin tl in @@ -1269,7 +1261,7 @@ let formatComment_ txt = | Some num -> num + 1 in let padNonOpeningLine s = - let numLeadingSpaceForThisLine = numLeadingSpace s in + let numLeadingSpaceForThisLine = Reason_syntax_util.num_leading_space s in if String.length s == 0 then "" else (String.make leftPad ' ') ^ (string_after s (min attemptRemoveCount numLeadingSpaceForThisLine)) in @@ -1852,7 +1844,7 @@ let semiTerminated term = makeList [term; atom ";"] (* postSpace is so that when comments are interleaved, we still use spacing rules. *) let makeLetSequence ?(wrap=("{", "}")) letItems = makeList - ~break:Always_rec + ~break:Layout.Always_rec ~inline:(true, false) ~wrap ~postSpace:true @@ -1896,6 +1888,60 @@ let formatAttributed ?(labelBreak=`Auto) x y = (makeList ~inline:(true, true) ~postSpace:true y) x + +(** Utility for creating several lines (in reverse order) where each line is of + * the form lbl(a, lbl(b, lbl(c ...) and the line never breaks unless the a, b, + * c were to break. + * + * This is useful for printing string interpolation syntax, but may be useful + * for many other things. + * TODO: Put this in its own module when we can refactor. + *) +module Reason_template_layout = struct + let labelLinesBreakRight ?(flushLine=false) revLines itm = + match flushLine, revLines with + | false, last_ln :: firsts -> + label ~break:`Never last_ln itm :: firsts + | _, [] + | true, _ -> itm :: revLines + + let lets ~wrap loc let_list = + if List.length let_list > 1 then source_map ~loc (makeLetSequence ~wrap let_list) + else source_map ~loc (makeList ~break:IfNeed ~wrap let_list) + + let rec append_lines ?(flushLine=false) acc lines = + match lines with + | [] -> acc + | hd :: tl -> append_lines ~flushLine:true (labelLinesBreakRight ~flushLine acc (atom hd)) tl + + let append_lines acc txt = + let escapedString = Reason_template.Print.escape_string_template txt in + let next_lines = Reason_syntax_util.split_by_newline ~keep_empty:true escapedString in + append_lines acc next_lines + + let rec format_simple_string_template_string_concat letListMaker acc e1 e2 = ( + match e1.pexp_attributes, e1.pexp_desc with + | [], Pexp_constant(Pconst_string (s, (None | Some("reason.template")))) -> + format_simple_string_template letListMaker (append_lines acc s) e2 + | _ -> + format_simple_string_template letListMaker ( + labelLinesBreakRight acc (lets e1.pexp_loc ~wrap:("${", "}") (letListMaker e1)) + ) e2 + ) + and format_simple_string_template letListMaker acc x = ( + let {stdAttrs; jsxAttrs; stylisticAttrs} = partitionAttributes x.pexp_attributes in + match (x.pexp_desc) with + | Pexp_constant(Pconst_string (s, (None | Some("reason.template")))) -> append_lines acc s + | ( + Pexp_apply ({pexp_desc=Pexp_ident ({txt = Longident.Lident("++")})}, [(Nolabel, e1); (Nolabel, e2)]) + ) when Reason_template.Print.is_template_style stylisticAttrs -> + format_simple_string_template_string_concat letListMaker acc e1 e2 + | _ -> + labelLinesBreakRight acc (lets x.pexp_loc ~wrap:("${", "}") (letListMaker x)) + ) +end + + (* For when the type constraint should be treated as a separate breakable line item itself not docked to some value/pattern label. fun x @@ -1915,7 +1961,6 @@ let formatCoerce expr optType coerced = | Some typ -> label ~space:true (makeList ~postSpace:true [formatTypeConstraint expr typ; atom ":>"]) coerced - (* Standard function application style indentation - no special wrapping * behavior. * @@ -2022,11 +2067,21 @@ let constant_string_for_primitive ppf s = let tyvar ppf str = Format.fprintf ppf "'%s" str +(* Constant string template that has no interpolation *) +let constant_string_template s = + let next_lines = Reason_syntax_util.split_by_newline ~keep_empty:true s in + let at_least_two = match next_lines with _ :: _ :: _ -> true | _ -> false in + makeList + ~wrap:("`", "`") + ~pad:(true, true) + ~break:(if at_least_two then Always_rec else IfNeed) + (List.map (fun s -> atom(Reason_template.Print.escape_string_template s)) next_lines) + (* In some places parens shouldn't be printed for readability: * e.g. Some((-1)) should be printed as Some(-1) * In `1 + (-1)` -1 should be wrapped in parens for readability *) -let constant ?raw_literal ?(parens=true) ppf = function +let single_token_constant ?raw_literal ?(parens=true) ppf = function | Pconst_char i -> Format.fprintf ppf "%C" i | Pconst_string (i, None) -> @@ -2036,6 +2091,9 @@ let constant ?raw_literal ?(parens=true) ppf = function | None -> Format.fprintf ppf "\"%s\"" (Reason_syntax_util.escape_string i) end + (* Ideally, this branch should never even be hit *) + | Pconst_string (i, Some "reason.template") -> + Format.fprintf ppf "` %s `" (Reason_template.Print.escape_string_template i) | Pconst_string (i, Some delim) -> Format.fprintf ppf "{%s|%s|%s}" delim i delim | Pconst_integer (i, None) -> @@ -2452,8 +2510,12 @@ let printer = object(self:'self) method longident_loc (x:Longident.t Location.loc) = source_map ~loc:x.loc (self#longident x.txt) - method constant ?raw_literal ?(parens=true) = - wrap (constant ?raw_literal ~parens) + method constant ?raw_literal ?(parens=true) c = + match c with + (* The one that isn't a "single token" *) + | Pconst_string (s, Some "reason.template") -> constant_string_template s + (* Single token constants *) + | _ -> ensureSingleTokenSticksToLabel (wrap (single_token_constant ?raw_literal ~parens) c) method constant_string_for_primitive = wrap constant_string_for_primitive method tyvar = wrap tyvar @@ -3511,11 +3573,20 @@ let printer = object(self:'self) e2; atom cls; ] - - method simple_get_application x = - let {stdAttrs; jsxAttrs} = partitionAttributes x.pexp_attributes in + let {stdAttrs; jsxAttrs; stylisticAttrs} = partitionAttributes x.pexp_attributes in match (x.pexp_desc, stdAttrs, jsxAttrs) with + | ( + Pexp_apply ({pexp_desc=Pexp_ident ({txt = Longident.Lident("++")})}, [(Nolabel, e1); (Nolabel, e2)]), + [], + [] + ) when Reason_template.Print.is_template_style stylisticAttrs -> + let revAllLines = + Reason_template_layout.format_simple_string_template_string_concat self#letList [] e1 e2 in + Some( + source_map ~loc:x.pexp_loc + (makeList ~pad:(true, true) ~wrap:("`", "`") ~break:Always_rec (List.rev (revAllLines))) + ) | (_, _::_, []) -> None (* Has some printed attributes - not simple *) | (Pexp_apply ({pexp_desc=Pexp_ident loc}, l), [], _jsx::_) -> ( (* TODO: Soon, we will allow the final argument to be an identifier which @@ -6412,8 +6483,7 @@ let printer = object(self:'self) | Pexp_constant c -> (* Constants shouldn't break when to the right of a label *) let raw_literal, _ = extract_raw_literal x.pexp_attributes in - Some (ensureSingleTokenSticksToLabel - (self#constant ?raw_literal c)) + Some ((self#constant ?raw_literal c)) | Pexp_pack me -> Some ( makeList diff --git a/src/reason-parser/reason_syntax_util.cppo.ml b/src/reason-parser/reason_syntax_util.cppo.ml index 33946529a..8a26c5049 100644 --- a/src/reason-parser/reason_syntax_util.cppo.ml +++ b/src/reason-parser/reason_syntax_util.cppo.ml @@ -279,6 +279,11 @@ let split_by ?(keep_empty=false) is_delim str = in loop [] len (len - 1) +let is_newline c = c == '\n' + +let split_by_newline ?keep_empty str = split_by ?keep_empty is_newline str + +(* Returns the index of the first white space at the end of the string *) let rec trim_right_idx str idx = if idx = -1 then 0 else @@ -296,6 +301,29 @@ let trim_right str = str else String.sub str 0 index +(* Returns the index of the last white space after which we start the substring. + * max is the max size of white spaces to remove. *) +let rec trim_left_idx ~max_trim_size str idx = + let len = String.length str in + if (idx == len) || (idx == max_trim_size) then idx - 1 + else + match String.get str idx with + | '\t' | ' ' | '\n' | '\r' -> trim_left_idx ~max_trim_size str (idx + 1) + | _ -> idx - 1 + +let trim_left ?(max_trim_size) str = + let length = String.length str in + let max_trim_size = + match max_trim_size with None -> length | Some mts -> mts in + if length = 0 then "" + else + let index = trim_left_idx ~max_trim_size str 0 in + if index = length - 1 then "" + else if index = -1 then + str + else String.sub str (index + 1) (length - (index + 1)) + + let processLine line = let rightTrimmed = trim_right line in @@ -733,6 +761,18 @@ let location_contains loc1 loc2 = loc1.loc_start.Lexing.pos_cnum <= loc2.loc_start.Lexing.pos_cnum && loc1.loc_end.Lexing.pos_cnum >= loc2.loc_end.Lexing.pos_cnum +let rec last_index_rec_opt s lim i c = + if i <= lim then None else + if String.unsafe_get s i == c then Some i else last_index_rec_opt s lim (i - 1) c + +let rec num_leading_space_ line length idx accum = + if idx = length then accum else + match String.get line idx with + | '\t' | ' ' -> num_leading_space_ line length (idx + 1) (accum + 1) + | _ -> accum + +let num_leading_space line = num_leading_space_ line (String.length line) 0 0 + #if OCAML_VERSION >= (4, 8, 0) let split_compiler_error (err : Location.error) = (err.main.loc, Format.asprintf "%t" err.main.txt) diff --git a/src/reason-parser/reason_syntax_util.cppo.mli b/src/reason-parser/reason_syntax_util.cppo.mli index 01f13ae62..e985c46d0 100644 --- a/src/reason-parser/reason_syntax_util.cppo.mli +++ b/src/reason-parser/reason_syntax_util.cppo.mli @@ -32,6 +32,8 @@ val pick_while : ('a -> bool) -> 'a list -> 'a list * 'a list val split_by : ?keep_empty:bool -> (char -> bool) -> string -> string list +val split_by_newline : ?keep_empty:bool -> string -> string list + val processLineEndingsAndStarts : string -> string val isLineComment : string -> bool @@ -87,9 +89,15 @@ val location_is_before : Location.t -> Location.t -> bool val location_contains : Location.t -> Location.t -> bool +val trim_right : string -> string + +val trim_left : ?max_trim_size:int -> string -> string + val split_compiler_error : Location.error -> Location.t * string val explode_str : string -> char list + +val num_leading_space : string -> int #endif module Clflags : sig diff --git a/src/reason-parser/reason_template.ml b/src/reason-parser/reason_template.ml new file mode 100644 index 000000000..46c07e628 --- /dev/null +++ b/src/reason-parser/reason_template.ml @@ -0,0 +1,97 @@ +(** + * This module should include utilities for parsing and printing template + * strings, but should not have any dependencies on any printing framework + * (like Easy_format or Reason_layout). For that, make another module. This + * file should be shared between the printer and the parser, so avoiding + * dependencies on printing frameworks, makes it easy to bundle just the parser + * if necessary. + *) + +open Migrate_parsetree +open Ast_408 + +module Parse = struct + (* Normalizes the last line: + * + * ` + * abc + * 123 + * xyz` + * + * Into: + * + * ` + * abc + * 123 + * xyz + * ` + * ^ ^ + * | | + * | one less space than previously (or zero if already had zero) + * white spaces 2 fewer than observed indent of last line. + * + * Or doesn't do anything if the last line is all white space. + * + * ` + * abc + * 123 + * xyz + * ` + * + * Into: + * + * ` + * abc + * 123 + * xyz + * ` + * ^ ^ + * | | + * | Doesn't remove any trailing white space on last line in this case. + * Undisturbed + * + * Removes a final line filled only with whitespace, returning its indent, + * or does not remove the final line if it has non-whitespace, but returns + * a normalized version of that final line if it is not terminated with a newline *) + let normalize_or_remove_last_line rev_lst = + match rev_lst with + | [] -> (0, []) + | s :: tl -> + let indent = Reason_syntax_util.num_leading_space s in + let len = String.length s in + if indent == len then (indent, tl) + else + (* Else, the last line contains non-white space after white space *) + let withoutFinalWhite = + (* Also, trim one single final white space *) + match String.get s (len - 1) with + | '\t' | ' ' | '\n' | '\r' -> String.sub s 0 (len - 1) + | _ -> s + in + (indent, withoutFinalWhite :: tl) + + let rec strip_leading_for_non_last ~indent acc revLst = + match revLst with + | [] -> acc + | hd::tl -> + (* The first line doesn't get a newline before it *) + let ln = Reason_syntax_util.trim_left ~max_trim_size:(indent+2) hd in + let next = match tl with | [] -> ln ^ acc | _ -> "\n" ^ ln ^ acc in + strip_leading_for_non_last ~indent next tl +end + + +module Print = struct + let escape_string_template str = + let buf = Buffer.create (String.length str) in + let strLen = String.length str in + String.iteri (fun i c -> + match c with + | '`' -> Buffer.add_string buf "\\`" + | '$' when i + 1 < strLen && str.[i + 1] == '{' -> Buffer.add_string buf "\\$" + | c -> Buffer.add_char buf c + ) str; + Buffer.contents buf + let is_template_style lst = + match lst with [({Parsetree.attr_name = {txt="reason.template"}; _ })] -> true | _ -> false +end diff --git a/src/reason-parser/reason_template.mli b/src/reason-parser/reason_template.mli new file mode 100644 index 000000000..047e9d7d0 --- /dev/null +++ b/src/reason-parser/reason_template.mli @@ -0,0 +1,19 @@ +(** + * This module should include utilities for parsing and printing template + * strings, but should not have any dependencies on any printing framework + * (like Easy_format or Reason_layout). For that, make another module. This + * file should be shared between the printer and the parser, so avoiding + * dependencies on printing frameworks, makes it easy to bundle just the parser + * if necessary. + *) + +module Parse : sig + val normalize_or_remove_last_line: string list -> int * string list + + val strip_leading_for_non_last: indent:int -> string -> string list -> string +end + +module Print : sig + val escape_string_template : string -> string + val is_template_style : Migrate_parsetree.Ast_408.Parsetree.attribute list -> bool +end