v2.2.0-beta

This is the compiler for Yarn Spinner. If you want to use Yarn Spinner in a Unity game, please see the releases page for Yarn Spinner for Unity!

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Added

• Added DeclarationBuilder and FunctionTypeBuilder classes. These classes allow external libraries to construct new Declaration and FunctionType objects, without having to have access to the internal setters.
• CompilationResult.DebugInfo now provides per-instruction positional debug information.
  • This allows users of the Compiler class to access positional information for each instruction, which is an important first step for source-level debugging.
• Made Diagnostic and Declaration serializable, for easier communication with language servers and other utilities.
• The compiler now does a last-line-before-options tagging pass.
  • This will add a #lastline tag onto any dialogue line that immediately precedes a block of options.
  • This is intended to used by other parts of the game to modify dialogue view behaviours.

Changed

• Declaration and Diagnostic now provide position information via a Range object, which specifies the start and end position of the relevant parts of the document.
• Fixed an issue where attempting to access the value of a variable with insufficient context to figure out its type would crash the compiler. (This could happen when you used a variable in a line, like Variable: {$myVar} with no other uses of $myVar.)
• Fixed an issue where an option condition with no expression (for example: -> Option one <<if>>) would crash the compiler.
• The compiler will no longer attempt to generate code if the Yarn script contains errors. (Previously, it was generating code, and then discarding it, but this allows for potential errors and crashes if code-generation is attempted on an invalid parse tree.)
• Typechecker now does partial backwards type inference, allowing for functions and variables to inform the type of the other regardless of them being the l- or r-value in an expression.

Removed

v2.1.0

This is the compiler for Yarn Spinner. If you want to use Yarn Spinner in a Unity game, please see the releases page for Yarn Spinner for Unity!

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Added

• The <<jump>> statement can now take an expression.

<<set $myDestination = "Home">>
<<jump {$myDestination}>>

• Previously, the jump statement required the name of a node. With this change, it can now also take an expression that resolves to the name of a node.
• Jump expressions may be a constant string, a variable, a function call, or any other type of expression.
• These expressions must be wrapped in curly braces ({ }), and must produce a string.

v2.0.2

This is the compiler for Yarn Spinner. If you want to use Yarn Spinner in a Unity game, please see the releases page for Yarn Spinner for Unity!

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Changed

• Fixed an error when a constant float value inside a marker was parsed and the user's current locale doesn't use a period (.) as the decimal separator.

v2.0.1

This is the compiler for Yarn Spinner. If you want to use Yarn Spinner in a Unity game, please see the releases page for Yarn Spinner for Unity!

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Added

• The v1 to v2 language upgrader now renames node names that have a period (.) in their names to use underscores (_) instead. Jumps and options are also updated to use these new names.

Changed

• Fixed a crash in the compiler when producing an error message about an undeclared function.
• Fixed an error when a constant float value (such as in a <<declare>> statement) was parsed and the user's current locale doesn't use a period (.) as the decimal separator.

v2.0.0

This is the compiler for Yarn Spinner. If you want to use Yarn Spinner in a Unity game, please see the releases page for Yarn Spinner for Unity! If you're not sure what this means, check the documentation.

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Yarn Spinner 2.0

Yarn Spinner 2.0 is a major new release, and contains a large number of new features and improvements.

New syntax for jumping to a different node.

• We have added a <<jump Destination>> command, which replaces the [[Destination]] jump syntax.

• Accordingly, the [[Destination]] and [[Option|Destination]] syntax has been removed from the language.
• Instead of using [[Option|Destination]] syntax, combine the new <<jump Destination>> command with shortcut -> options instead. For example:

// Before
Kim: You want a bagel?
[[Yes, please!|GiveBagel]]
[[No, thanks!|DontWantBagel]]

// After
Kim: You want a bagel?
-> Yes, please!
  <<jump GiveBagel>>
-> No, thanks!
  <<jump DontWantBagel>>

• The old syntax was inherited from the original Yarn language, which itself inherited it from Twine.
  - We removed it for four reasons:
  1. it conflated jumps and options, which are very different operations, with too-similar syntax
  2. the Option-destination syntax for declaring options involved the management of non-obvious state (that is, if an option statement was inside an if branch that was never executed, it was not presented, and the runtime needed to keep track of that)
  3. it was not obvious that options accumulated and were only presented at the end of the node
  4. finally, shortcut options provide a cleaner way to present the same behaviour.
• No change to the bytecode is made here; these changes only affect the compiler.

Automatic upgrader for Yarn Spinner 1.0 variables.

• An automatic upgrader has been added that attempts to determine the types of variables in Yarn Spinner 1.0, and generates <<declare>> statements for variables.

• This upgrader infers the type of a variable based on the values that are assigned to it, and the values of expressions that it participates in.
  • If the upgrader cannot determine the type of a variable, it generates a declaration of the form <<declare $variable_name as undefined>>. The word undefined is not a valid type in Yarn Spinner, which means that these declarations will cause an error in compilation (which is a signal to the developer that the script needs to be manually updated.)
  • For example: given the following script:

<<set $const_string = "foo">>
<<set $const_number = 2>>
<<set $const_bool = true>>

• The upgrader will generate the following variable declarations:

    <<declare $const_string = "" as string>>
    <<declare $const_number = 0 as number>>
    <<declare $const_bool = false as bool>>

• The upgrader is able to make use of type even when it appears later in the program, and is able to make inferences about type using indirect information.

// These variables are participating in expressions that include
// variables we've derived the type for earlier in this program, so they
// will be bound to that type
{$derived_expr_const_string + $const_string}
{$derived_expr_const_number + $const_number}
{$derived_expr_const_bool && $const_bool}

// These variables are participating in expressions that include
// variables that we define a type for later in this program. They will
// also be bound to that type.
{$derived_expr_const_string_late + $const_string_late}
{$derived_expr_const_number_late + $const_number_late}
{$derived_expr_const_bool_late && $const_bool_late}

<<set $const_string_late = "yes">>
<<set $const_number_late = 1>>
<<set $const_bool_late = true>>

• The upgrader will also make in-line changes to any if or elseif statements where the expression is determined to use a number rather than a bool will be rewritten so that the expression evaluates to a bool:

// Define some variables whose type is known before the expressions are
// hit
<<set $some_num_var = 1>>
<<set $some_other_num_var = 1>>

// This will be converted to a bool expression
<<if $some_num_var>>
<<elseif $some_other_num_var>>
<<endif>>

* Will be rewritten to:

<<elseif $some_other_num_var != 0>>
<<endif>>

You can use characters that the parser uses in scripts!

• Characters can now be escaped in lines and options.

• The \ character can be used to write characters that the parser would otherwise use.
• The following characters can be escaped: { } < > # / \
  • The / and < characters don't usually need to be escaped if they're appearing on their own (they're only meaningful when they appear in pairs), but this allows you to escape things like commands and comments.

Identifiers now support a wider range of characters.

This includes most multilingual letters and numbers, as well as symbols and emoji.

Made line conditions control the IsAvailable flag on options that are sent to the game.

• This change was made in order to allow games to conditionally present, but disallow, options that the player can't choose. For example, consider the following script:

TD-110: Let me see your identification.
-> Of course... um totally not General Kenobi and the son of Darth Vader.
    Luke: Wait, what?!
    TD-110: Promotion Time!
-> You don't need to see his identification. <<if $learnt_mind_trick is true>>
    TD-110: We don't need to see his identification.

• If the variable $learnt_mind_trick is false, a game may want to show the option but not allow the player to select it (i.e., show that this option could have been chosen if they'd learned how to do a mind trick.)
  • In previous versions of Yarn Spinner, if a line condition failed, the entire option was not delivered to the game. With this change, all options are delivered, and the OptionSet.Option.IsAvailable variable contains false if the condition was not met, and true if it was (or was not present.)
  • It's entirely up to the game to decide what to do with this information. To re-create the behaviour from previous Yarn Spinner versions, simply don't show any options whose IsAvailable value is false.

Variable declarations are now automatically determined, where possible

• If a variable is not declared (i.e. it doesn't have a <<declare>> statement), the compiler will now attempt to infer its declaration.
• When a variable doesn't have a declaration, the compiler will try to figure out the type based on how the variable is being used. It will always try to figure out the single type that the variable must be; if it's ambiguous, or no information is available at all, it will report an error, and you will have to add a declaration.

Variable declaration descriptions use comments

• Declarations have their descriptions set using a triple-slash (///) comment:

/// The number of coins the player has
<<declare $coins = 0>>

• These documentation comments can be before a declaration, or on the same line as a declaration:

<<declare $player_likes_dogs = true>> /// Whether the player likes dogs or not

• Multiple-line documentation comments are also supported:

/// Whether these are the droids that the 
/// guards are looking for.
<<declare $are_the_droids_we're_looking_for = false>>

A new type system has been added.

• The type-checking system in Yarn Spinner now supports types with supertypes and methods. This change has no significant impact on users writing Yarn scripts, but it enables the development of more advanced language features.
  • The main impact on users of this library (such as, for example, Yarn Spinner for Unity) is that the Yarn.Type enumeration has been removed, and is now replaced with the Yarn.IType interface and the BuiltinTypes class.
  • The type checker no longer hard-codes which operations can be run on which types; this decision is now determined by the types themselves.

Better Error Messages

The Compiler will no longer throw a ParseException, TypeException or CompilerException when an error is encountered during compilation. Instead, CompilationResult.Diagnostics contains a collection of Diagnostic objects, which represent errors, warnings, or other diagnostic information related to the compiled program.

• This change was implemented so that if multiple problems can be detected in a program, they can all be reported at once, rather than the compiler stopping at the first one.
• This also allows the compiler to issue non-fatal diagnostic messages, like warnings, that do not prevent the script from being compiled, but might indicate a problem with the code.
• Exceptions will continue to be thrown if the compiler encounters an internal error (in other words, if Yarn Spinner itself has a bug.)
  • If an error is encountered during compilation, CompilationResult.Program will be null.
  • This change means that compilation failures will not cause Compiler.Compile() to throw an exception; code that was previously using a try...catch to detect problems will need to be rewritten to check the CompilationResult.Diagnostics property to find the actual problem.

Fixes and Smaller Changes

• Fixed a crash in LineParser if a null inpu...

v2.0.0-rc1

This is the first release candidate of the compiler for Yarn Spinner, v2.0. If you want to check out the v2.0 beta for Yarn Spinner for Unity, you should head to the beta release!

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

v2.0.0-rc1 contains no user-facing features or bug fixes; it exists to be in sync with the corresponding v2.0.0-rc1 tag for Yarn Spinner for Unity.

v2.0.0-beta6

This is the beta release of the compiler for Yarn Spinner. If you want to check out the v2.0 beta for Yarn Spinner for Unity, you should head to the beta release!

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Added

• The Compiler will no longer throw a ParseException, TypeException or CompilerException when an error is encountered during compilation. Instead, CompilationResult.Diagnostics contains a collection of Diagnostic objects, which represent errors, warnings, or other diagnostic information related to the compiled program.
  • This change was implemented so that if multiple problems can be detected in a program, they can all be reported at once, rather than the compiler stopping at the first one.
  • This also allows the compiler to issue non-fatal diagnostic messages, like warnings, that do not prevent the script from being compiled, but might indicate a problem with the code.
  • Exceptions will continue to be thrown if the compiler encounters an internal error (in other words, if Yarn Spinner itself has a bug.)
• If an error is encountered during compilation, CompilationResult.Program will be null.
• This change means that compilation failures will not cause Compiler.Compile() to throw an exception; code that was previously using a try...catch to detect problems will need to be rewritten to check the CompilationResult.Diagnostics property to find the actual problem.

Changed

• Made the lexer not use semantic predicates when lexing the TEXT rule, which reduces the amount of C# code present in the grammar file.
• Markup can now be escaped, using the \ character:

\[b\]hello\[/b\]
// will appear to the user as "[b]hello[/b]", and will not 
// be treated as markup

• Dialogue.SetSelectedOption can now be called within the options handler itself.

  • If you do this, the Dialogue will continue executing after the options handler returns, and you do not need to call Continue.

• The compiler now generates better error messages for syntax errors. For example, given the following code (note the lack of an <<endif>> at the end):

<<if $has_key>>
  Guard: You found the key! Let me unlock the door.

The compiler will produce the following error message:

Expected an <<endif>> to match the <<if>> statement on line 1

• The compiler's new error messages now also report additional information about the context of a syntax error. For example, given the following code:

<<if hasCompletedObjective("find_key" >>
  // error! we forgot to add an ')'!
<<endif>>

The compiler will produce the following error message:

Unexpected ">>" while reading a function call

• VirtualMachine.executionState has been renamed to VirtualMachine.CurrentExecutionState.

• It is now a compiler error if the same line ID is used on more than one line.

• Dialogue.VariableStorage is now public.

Removed

• The ParseException, TypeException and CompilerException classes have been removed.

v2.0.0 Beta 5

This is the beta release of the compiler for Yarn Spinner. If you want to check out the v2.0 beta for Yarn Spinner for Unity, you should head to the beta release!

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Added

Variable declarations are now automatically determined, where possible

• If a variable is not declared (i.e. it doesn't have a <<declare>> statement), the compiler will now attempt to infer its declaration.
• When a variable doesn't have a declaration, the compiler will try to figure out the type based on how the variable is being used. It will always try to figure out the single type that the variable must be; if it's ambiguous, or no information is available at all, it will report an error, and you will have to add a declaration.

Variable declaration descriptions now use comments

• Declarations now have their descriptions set using a triple-slash (///) comment:

/// The number of coins the player has
<<declare $coins = 0>>

• These documentation comments can be before a declaration, or on the same line as a declaration:

<<declare $player_likes_dogs = true>> /// Whether the player likes dogs or not

• Multiple-line documentation comments are also supported:

/// Whether these are the droids that the 
/// guards are looking for.
<<declare $are_the_droids_we're_looking_for = false>>

A new type system has been added.

• The type-checking system in Yarn Spinner now supports types with supertypes and methods. This change has no significant impact on users writing Yarn scripts, but it enables the development of more advanced language features.
  • The main impact on users of this library (such as, for example, Yarn Spinner for Unity) is that the Yarn.Type enumeration has been removed, and is now replaced with the Yarn.IType interface and the BuiltinTypes class.
  • The type checker no longer hard-codes which operations can be run on which types; this decision is now determined by the types themselves.

Changed

• Variable declaration upgrader now generates .yarnproject files, not .yarnprogram files.
• Line tagger now adds line tags before any // comment in the line.
• Dialogue: LogErrorMessage and LogDebugMessage now perform null-checks before being invoked.
• Utility.GenerateYarnFileWithDeclarations now generates files that use triple-slash (///) comments.
• Fixed a bug where expressions inside an if statement or elseif statement would not be type-checked.
• The keywords enum, endenum and case are now reserved.
• The type-conversion functions, string, number and bool, are no longer built-in special-case functions; they are now regular built-in functions that take a value of Any type.

Removed

• In previous betas, variable descriptions were done by adding a string. This has been removed:

// This will no longer work:
<<declare $coins = 0 "The number of coins the player has">>

v2.0.0 Beta4

This is the beta release of the compiler for Yarn Spinner. If you want to check out the v2.0 beta for Yarn Spinner for Unity, you should head to the beta release!

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Added

• Characters can now be escaped in lines and options.
  • The \ character can be used to write characters that the parser would otherwise use.
  • The following characters can be escaped: { } < > # / \
    • The / and < characters don't usually need to be escaped if they're appearing on their own (they're only meaningful when they appear in pairs), but this allows you to escape things like commands and comments.
• Identifiers now support a wider range of characters, including most multilingual letters and numbers, as well as symbols and emoji.

Changed

• Made line conditions control the IsAvailable flag on options that are sent to the game.
• This change was made in order to allow games to conditionally present, but disallow, options that the player can't choose. For example, consider the following script:

TD-110: Let me see your identification.
-> Of course... um totally not General Kenobi and the son of Darth Vader.
    Luke: Wait, what?!
    TD-110: Promotion Time!
-> You don't need to see his identification. <<if $learnt_mind_trick is true>>
    TD-110: We don't need to see his identification.

• If the variable $learnt_mind_trick is false, a game may want to show the option but not allow the player to select it (i.e., show that this option could have been chosen if they'd learned how to do a mind trick.)

• In previous versions of Yarn Spinner, if a line condition failed, the entire option was not delivered to the game. With this change, all options are delivered, and the OptionSet.Option.IsAvailable variable contains false if the condition was not met, and true if it was (or was not present.)

• It's entirely up to the game to decide what to do with this information. To re-create the behaviour from previous Yarn Spinner versions, simply don't show any options whose IsAvailable value is false.

• Fixed a crash in LineParser if a null input was provided to it.

• Fixed a crash in FormatFunctionUpgrader (which upgrades v1 Yarn scripts to v2) if an invalid format format function was encountered.

v2.0.0 Beta 2

This is the beta release of the compiler for Yarn Spinner. If you want to check out the v2.0 beta for Yarn Spinner for Unity, you should head to the beta release!

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Added

• The [[Destination]] and [[Option|Destination]] syntax has been removed from the language.
  • This syntax was inherited from the original Yarn language, which itself inherited it from Twine.
  • We removed it for four reasons:
    • it conflated jumps and options, which are very different operations, with too-similar syntax;
    • the Option-destination syntax for declaring options involved the management of non-obvious state (that is, if an option statement was inside an if branch that was never executed, it was not presented, and the runtime needed to keep track of that);
    • it was not obvious that options accumulated and were only presented at the end of the node;
    • finally, shortcut options provide a cleaner way to present the same behaviour.
  • We have added a <<jump Destination>> command, which replaces the [[Destination]] jump syntax.
  • No change to the bytecode is made here; these changes only affect the compiler.
  • Instead of using [[Option|Destination]] syntax, use shortcut options instead. For example:

// Before
Kim: You want a bagel?
[[Yes, please!|GiveBagel]]
[[No, thanks!|DontWantBagel]]

// After
Kim: You want a bagel?
-> Yes, please!
  <<jump GiveBagel>>
-> No, thanks!
  <<jump DontWantBagel>>

• An automatic upgrader has been added that attempts to determine the types of variables in Yarn Spinner 1.0, and generates <<declare>> statements for variables.

  • This upgrader infers the type of a variable based on the values that are assigned to it, and the values of expressions that it participates in.
  • If the upgrader cannot determine the type of a variable, it generates a declaration of the form <<declare $variable_name as undefined>>. The word undefined is not a valid type in Yarn Spinner, which means that these declarations will cause an error in compilation (which is a signal to the developer that the script needs to be manually updated.)

• For example: given the following script:

<<set $const_string = "foo">>
<<set $const_number = 2>>
<<set $const_bool = true>>

• The upgrader will generate the following variable declarations:

    <<declare $const_string = "" as string>>
    <<declare $const_number = 0 as number>>
    <<declare $const_bool = false as bool>>

The upgrader is able to make use of type even when it appears later in the program, and is able to make inferences about type using indirect information.

// These variables are participating in expressions that include
// variables we've derived the type for earlier in this program, so they
// will be bound to that type
{$derived_expr_const_string + $const_string}
{$derived_expr_const_number + $const_number}
{$derived_expr_const_bool && $const_bool}

// These variables are participating in expressions that include
// variables that we define a type for later in this program. They will
// also be bound to that type.
{$derived_expr_const_string_late + $const_string_late}
{$derived_expr_const_number_late + $const_number_late}
{$derived_expr_const_bool_late && $const_bool_late}

<<set $const_string_late = "yes">>
<<set $const_number_late = 1>>
<<set $const_bool_late = true>>

• The upgrader will also make in-line changes to any if or elseif statements where the expression is determined to use a number rather than a bool will be rewritten so that the expression evaluates to a bool:

// Define some variables whose type is known before the expressions are
// hit
<<set $some_num_var = 1>>
<<set $some_other_num_var = 1>>

// This will be converted to a bool expression
<<if $some_num_var>>
<<elseif $some_other_num_var>>
<<endif>>

Will be rewritten to:

<<if $some_num_var != 0>>
<<elseif $some_other_num_var != 0>>
<<endif>>

Changed

• The internal structure of the LanguageUpgrader system has been updated to make it easier to add future upgrade passes.