Merge branch 'master' into develop

Build/Sprache.proj

@@ -50,7 +50,7 @@
 
         <Delete Files="@(FilesToDelete)" />
 
-        <Copy SourceFiles="@(FilesToCopy)" DestinationFolder="$(Package)\lib\net40" />
+        <Copy SourceFiles="@(FilesToCopy)" DestinationFolder="$(Package)\lib\portable-net4+netcore45+win8+wp8+sl5+MonoAndroid+Xamarin.iOS10+MonoTouch" />
 
         <GetAssemblyIdentity AssemblyFiles="$(OutputDir)\Sprache.dll">
             <Output TaskParameter="Assemblies" ItemName="AsmInfo" />

README.md

@@ -5,40 +5,62 @@ It doesn't compete with "industrial strength" language workbenches - it fits som
 Usage
 -----
 
+[![Join the chat at https://gitter.im/sprache/Sprache](https://badges.gitter.im/sprache/Sprache.svg)](https://gitter.im/sprache/Sprache?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
+
 Unlike most parser-building frameworks, you use Sprache directly from your program code, and don't need to set up any build-time code generation tasks. Sprache itself is a single tiny assembly.
 
 A simple parser might parse a sequence of characters:
 
-    // Parse any number of capital 'A's in a row
-    var parseA = Parse.Char('A').AtLeastOnce();
+```csharp
+// Parse any number of capital 'A's in a row
+var parseA = Parse.Char('A').AtLeastOnce();
+```
 
 Sprache provides a number of built-in functions that can make bigger parsers from smaller ones, often callable via Linq query comprehensions:
 
-    Parser<string> identifier =
-        from leading in Parse.Whitespace.Many()
-        from first in Parse.Letter.Once()
-        from rest in Parse.LetterOrDigit.Many()
-        from trailing in Parse.Whitespace.Many()
-        select new string(first.Concat(rest).ToArray());
+```csharp
+Parser<string> identifier =
+    from leading in Parse.WhiteSpace.Many()
+    from first in Parse.Letter.Once()
+    from rest in Parse.LetterOrDigit.Many()
+    from trailing in Parse.WhiteSpace.Many()
+    select new string(first.Concat(rest).ToArray());
+
+var id = identifier.Parse(" abc123  ");
 
-    var id = identifier.Parse(" abc123  ");
+Assert.AreEqual("abc123", id);
+```
 
-    Assert.AreEqual("abc123", id);
-
-Background and Tutorials
-------------------------
+Examples and Tutorials
+----------------------
 
-The best place to start is [this introductory article.](http://nblumhardt.com/2010/01/building-an-external-dsl-in-c/)
+The best place to start is [this introductory article](http://nblumhardt.com/2010/01/building-an-external-dsl-in-c/).
 
-Examples included with the source demonstrate:
+**Examples** included with the source demonstrate:
 
 * [Parsing XML directly to a Document object](https://github.com/sprache/Sprache/blob/master/src/XmlExample/Program.cs)
-* [Parsing numeric expressions to System.Linq.Expression objects](https://github.com/sprache/Sprache/blob/master/src/LinqyCalculator/ExpressionParser.cs)
-* [Parsing comma-separated (CSV) 'files' into lists of strings](https://github.com/sprache/Sprache/blob/master/src/Sprache.Tests/Scenarios/CsvTests.cs)
+* [Parsing numeric expressions to `System.Linq.Expression` objects](https://github.com/sprache/Sprache/blob/master/src/LinqyCalculator/ExpressionParser.cs)
+* [Parsing comma-separated values (CSV) into lists of strings](https://github.com/sprache/Sprache/blob/master/src/Sprache.Tests/Scenarios/CsvTests.cs)
+
+**Tutorials** explaining Sprache:
+
+ * A great [CodeProject article by Alexey Yakovlev ](http://www.codeproject.com/Articles/795056/Sprache-Calc-building-yet-another-expression-evalu) (and [in Russian](http://habrahabr.ru/post/228037/))
+ * Mike Hadlow's example of [parsing connection strings](http://mikehadlow.blogspot.com.au/2012/09/parsing-connection-string-with-sprache.html)
+
+**Real-world** parsers built with Sprache:
+
+ * The [template parser](https://github.com/OctopusDeploy/Octostache/blob/master/source/Octostache/Templates/TemplateParser.cs) in [Octostache](https://github.com/OctopusDeploy/Octostache), the variable substitution language of [Octopus Deploy](https://octopus.com)
+ * The [XAML binding expression parser](https://github.com/SuperJMN/OmniXAML/blob/master/Source/OmniXaml/Parsers/MarkupExtensions/MarkupExtensionParser.cs) in [OmniXaml](https://github.com/SuperJMN/OmniXAML), the cross-platform XAML implementation
+ * The query parser in [Seq](https://getseq.net), a structured log server for .NET
+ * The [connection string parser](https://github.com/EasyNetQ/EasyNetQ/blob/master/Source/EasyNetQ/ConnectionString/ConnectionStringGrammar.cs) in [EasyNetQ](http://easynetq.com/), a .NET API for RabbitMQ
+ * Sprache appears in the [credits for JetBrains ReSharper](https://confluence.jetbrains.com/display/ReSharper/Third-Party+Software+Shipped+With+ReSharper#Third-PartySoftwareShippedWithReSharper-Sprache)
+
+Background
+----------
 
-Parser combinators are covered extensively on the web. The original paper describing the monadic implementation by [Graham Hutton and Eric Meijer](http://www.cs.nott.ac.uk/~gmh/monparsing.pdf) is very readable. Sprache grew out of some exercises in [Hutton's Haskell book](http://www.amazon.com/Programming-Haskell-Graham-Hutton/dp/0521692695).
+Parser combinators are covered extensively on the web. The original paper describing the monadic implementation by [Graham Hutton and Eric Meijer](http://www.cs.nott.ac.uk/~gmh/monparsing.pdf) is very readable. Sprache was originally written by [Nicholas Blumhardt](http://nblumhardt.com) and grew out of some exercises in [Hutton's Haskell book](http://www.amazon.com/Programming-Haskell-Graham-Hutton/dp/0521692695).
 
-Sprache itself draws on some great C# tutorials:
+The implementation of Sprache draws on ideas from:
 
 * [Luke Hoban's Blog](http://blogs.msdn.com/b/lukeh/archive/2007/08/19/monadic-parser-combinators-using-c-3-0.aspx)
 * [Brian McNamara's Blog](http://lorgonblog.wordpress.com/2007/12/02/c-3-0-lambda-and-the-first-post-of-a-series-about-monadic-parser-combinators/)

src/LinqyCalculator/LinqyCalculator.csproj

@@ -80,7 +80,7 @@
   </ItemGroup>
   <ItemGroup>
     <ProjectReference Include="..\Sprache\Sprache.csproj">
-      <Project>{DF5FE6F0-5ABE-4363-9184-EB6EF64F0F61}</Project>
+      <Project>{82cc8cf8-b770-4756-850a-95dd14dd312f}</Project>
       <Name>Sprache</Name>
     </ProjectReference>
   </ItemGroup>

src/Sprache.Tests/AssertInput.cs

@@ -6,7 +6,7 @@ namespace Sprache.Tests
 {
     static class AssertInput
     {
-        public static Input AdvanceMany(this Input input, int count)
+        public static IInput AdvanceMany(this IInput input, int count)
         {
             for (int i = 0; i < count; i++)
             {
@@ -16,14 +16,14 @@ public static Input AdvanceMany(this Input input, int count)
             return input;
         }
 
-        public static Input AdvanceAssert(this Input input, Action<Input, Input> assertion)
+        public static IInput AdvanceAssert(this IInput input, Action<IInput, IInput> assertion)
         {
             var result = input.Advance();
             assertion(input, result);
             return result;
         }
 
-        public static Input AdvanceManyAssert(this Input input, int count, Action<Input, Input> assertion)
+        public static IInput AdvanceManyAssert(this Input input, int count, Action<IInput, IInput> assertion)
         {
             var result = input.AdvanceMany(count);
             assertion(input, result);

src/Sprache.Tests/AssertParser.cs

@@ -31,7 +31,7 @@ public static void SucceedsWith<T>(Parser<T> parser, string input, Action<T> res
             parser.TryParse(input)
                 .IfFailure(f =>
                 {
-                    Assert.Fail("Parsing of \"{0}\" failed unexpectedly at position {1}: {2}", input, f.Remainder.Position, f.Message);
+                    Assert.Fail("Parsing of \"{0}\" failed unexpectedly. {1}", input, f);
                     return f;
                 })
                 .IfSuccess(s =>

src/Sprache.Tests/DecimalTests.cs

@@ -8,6 +8,7 @@ namespace Sprache.Tests
     public class DecimalTests
     {
         private static readonly Parser<string> DecimalParser = Parse.Decimal.End();
+        private static readonly Parser<string> DecimalInvariantParser = Parse.DecimalInvariant.End();
 
         private CultureInfo _previousCulture;
 
@@ -49,5 +50,12 @@ public void Letters()
         {
             DecimalParser.Parse("1A.5");
         }
+
+        [Test]
+        public void LeadingDigitsInvariant()
+        {
+            Assert.AreEqual("12.23", DecimalInvariantParser.Parse("12.23"));
+        }
+
     }
 }

src/Sprache.Tests/InputTests.cs

@@ -96,7 +96,7 @@ public void AdvancingInputAtEOL_ResetsColumnNumber()
         [Test]
         public void LineCountingSmokeTest()
         {
-            var i = new Input("abc\ndef");
+            IInput i = new Input("abc\ndef");
             Assert.AreEqual(0, i.Position);
             Assert.AreEqual(1, i.Line);
             Assert.AreEqual(1, i.Column);

src/Sprache.Tests/ParseTests.cs

@@ -1,4 +1,5 @@
-using System.Collections.Generic;
+using System;
+using System.Collections.Generic;
 using System.Linq;
 using NUnit.Framework;
 
@@ -152,12 +153,6 @@ from rest in Parse.Char('a').Once()
              select first.Concat(rest))
             .Or(Parse.Char('a').Once());
 
-        [Test, Ignore("Not Implemented")]
-        public void CanParseLeftRecursiveGrammar()
-        {
-            AssertParser.SucceedsWith(ASeq.End(), "a,a,a", r => Assert.AreEqual(new string(r.ToArray()), "aaa"));
-        }
-
         [Test]
         public void DetectsLeftRecursion()
         {
@@ -176,12 +171,6 @@ from rest in Parse.Char('b').Once()
              select first.Concat(rest))
             .Or(Parse.Char('b').Once());
 
-        [Test, Ignore("Not Implemented")]
-        public void CanParseMutuallyLeftRecursiveGrammar()
-        {
-            AssertParser.SucceedsWithAll(ABSeq.End(), "baba");
-        }
-
         [Test]
         public void DetectsMutualLeftRecursion()
         {
@@ -270,6 +259,26 @@ public void RegexParserDoesNotConsumeInputOnFailedMatch()
             Assert.AreEqual(0, r.Remainder.Position);
         }
 
+        [Test]
+        public void RegexMatchParserConsumesInputOnSuccessfulMatch()
+        {
+            var digits = Parse.RegexMatch(@"\d(\d*)");
+            var r = digits.TryParse("123d");
+            Assert.IsTrue(r.WasSuccessful);
+            Assert.AreEqual("123", r.Value.Value);
+            Assert.AreEqual("23", r.Value.Groups[1].Value);
+            Assert.AreEqual(3, r.Remainder.Position);
+        }
+
+        [Test]
+        public void RegexMatchParserDoesNotConsumeInputOnFailedMatch()
+        {
+            var digits = Parse.RegexMatch(@"\d+");
+            var r = digits.TryParse("d123");
+            Assert.IsFalse(r.WasSuccessful);
+            Assert.AreEqual(0, r.Remainder.Position);
+        }
+
         [Test]
         public void PositionedParser()
         {
@@ -348,6 +357,57 @@ public void RepeatParserCanParseWithCountOfZero()
             Assert.AreEqual(0, r.Remainder.Position);
         }
 
+        [Test]
+        public void RepeatParserCanParseAMinimumNumberOfValues()
+        {
+            var repeated = Parse.Char('a').Repeat(4, 5);
+
+            // Test failure.
+            var r = repeated.TryParse("aaa");
+            Assert.IsFalse(r.WasSuccessful);
+            Assert.AreEqual(0, r.Remainder.Position);
+
+            // Test success.
+            r = repeated.TryParse("aaaa");
+            Assert.IsTrue(r.WasSuccessful);
+            Assert.AreEqual(4, r.Remainder.Position);
+        }
+
+        [Test]
+        public void RepeatParserCanParseAMaximumNumberOfValues()
+        {
+            var repeated = Parse.Char('a').Repeat(4, 5);
+
+            var r = repeated.TryParse("aaaa");
+            Assert.IsTrue(r.WasSuccessful);
+            Assert.AreEqual(4, r.Remainder.Position);
+
+            r = repeated.TryParse("aaaaa");
+            Assert.IsTrue(r.WasSuccessful);
+            Assert.AreEqual(5, r.Remainder.Position);
+
+            r = repeated.TryParse("aaaaaa");
+            Assert.IsTrue(r.WasSuccessful);
+            Assert.AreEqual(5, r.Remainder.Position);
+        }
+
+        [Test]
+        public void RepeatParserErrorMessagesAreReadable()
+        {
+            var repeated = Parse.Char('a').Repeat(4, 5);
+
+            var expectedMessage = "Parsing failure: Unexpected 'end of input'; expected 'a' between 4 and 5 times, but found 3";
+
+            try
+            {
+                var r = repeated.Parse("aaa");
+            }
+            catch(ParseException ex)
+            {
+                Assert.That(ex.Message, Is.StringStarting(expectedMessage));
+            }
+        }
+
         [Test]
         public void CanParseSequence()
         {
@@ -357,6 +417,17 @@ public void CanParseSequence()
             Assert.IsTrue(r.Remainder.AtEnd);
         }
 
+        [Test]
+        public void FailGracefullyOnSequence()
+        {
+            var sequence = Parse.Char('a').XDelimitedBy(Parse.Char(','));
+            AssertParser.FailsWith(sequence, "a,a,b", result =>
+            {
+                StringAssert.Contains("unexpected 'b'", result.Message);
+                CollectionAssert.Contains(result.Expectations, "a");
+            });
+        }
+
         [Test]
         public void CanParseContained()
         {

src/Sprache.Tests/RegexTests.cs

@@ -0,0 +1,90 @@
+using System;
+using System.Reflection;
+using System.Text.RegularExpressions;
+using NUnit.Framework;
+
+namespace Sprache.Tests
+{
+    /// <summary>
+    /// These tests exist in order to verify that the modification that is applied to
+    /// the regex passed to every call to the <see cref="Parse.Regex(string,string)"/>
+    /// or <see cref="Parse.Regex(Regex,string)"/> methods does not change the results
+    /// in any way.
+    /// </summary>
+    public class RegexTests
+    {
+        private const string _startsWithCarrot = "^([a-z]{3})([0-9]{3})$";
+        private const string _alternation = "(this)|(that)|(the other)";
+
+        private static readonly MethodInfo _optimizeRegexMethod = typeof(Parse).GetMethod("OptimizeRegex", BindingFlags.NonPublic | BindingFlags.Static);
+
+        [Test]
+        public void OptimizedRegexIsNotSuccessfulWhenTheMatchIsNotAtTheBeginningOfTheInput()
+        {
+            var regexOriginal = new Regex("[a-z]+");
+            var regexOptimized = OptimizeRegex(regexOriginal);
+
+            const string input = "123abc";
+
+            Assert.That(regexOriginal.IsMatch(input), Is.True);
+            Assert.That(regexOptimized.IsMatch(input), Is.False);
+        }
+
+        [Test]
+        public void OptimizedRegexIsSuccessfulWhenTheMatchIsAtTheBeginningOfTheInput()
+        {
+            var regexOriginal = new Regex("[a-z]+");
+            var regexOptimized = OptimizeRegex(regexOriginal);
+
+            const string input = "abc123";
+
+            Assert.That(regexOriginal.IsMatch(input), Is.True);
+            Assert.That(regexOptimized.IsMatch(input), Is.True);
+        }
+
+        [TestCase(_startsWithCarrot, RegexOptions.None, "abc123", TestName = "Starts with ^, no options, success")]
+        [TestCase(_startsWithCarrot, RegexOptions.ExplicitCapture, "abc123", TestName = "Starts with ^, explicit capture, success")]
+        [TestCase(_startsWithCarrot, RegexOptions.None, "123abc", TestName = "Starts with ^, no options, failure")]
+        [TestCase(_startsWithCarrot, RegexOptions.ExplicitCapture, "123abc", TestName = "Starts with ^, explicit capture, failure")]
+        [TestCase(_alternation, RegexOptions.None, "abc123", TestName = "Alternation, no options, success")]
+        [TestCase(_alternation, RegexOptions.ExplicitCapture, "that", TestName = "Alternation, explicit capture, success")]
+        [TestCase(_alternation, RegexOptions.None, "that", TestName = "Alternation, no options, failure")]
+        [TestCase(_alternation, RegexOptions.ExplicitCapture, "that", TestName = "Alternation, explicit capture, failure")]
+        public void RegexOptimizationDoesNotChangeRegexBehavior(string pattern, RegexOptions options, string input)
+        {
+            var regexOriginal = new Regex(pattern, options);
+            var regexOptimized = OptimizeRegex(regexOriginal);
+
+            var matchOriginal = regexOriginal.Match(input);
+            var matchModified = regexOptimized.Match(input);
+
+            Assert.That(matchModified.Success, Is.EqualTo(matchOriginal.Success));
+            Assert.That(matchModified.Value, Is.EqualTo(matchOriginal.Value));
+            Assert.That(matchModified.Groups.Count, Is.EqualTo(matchOriginal.Groups.Count));
+
+            for (int i = 0; i < matchModified.Groups.Count; i++)
+            {
+                Assert.That(matchModified.Groups[i].Success, Is.EqualTo(matchOriginal.Groups[i].Success));
+                Assert.That(matchModified.Groups[i].Value, Is.EqualTo(matchOriginal.Groups[i].Value));
+            }
+        }
+
+        /// <summary>
+        /// Calls the <see cref="Parse.OptimizeRegex"/> method via reflection.
+        /// </summary>
+        private static Regex OptimizeRegex(Regex regex)
+        {
+            // Reflection isn't the best way of verifying behavior,
+            // but cluttering the public api sucks worse.
+
+            if (_optimizeRegexMethod == null)
+            {
+                throw new Exception("Unable to locate a private static method named " +
+                                    "\"OptimizeRegex\" in the Parse class. Has it been renamed?");
+            }
+
+            var optimizedRegex = (Regex)_optimizeRegexMethod.Invoke(null, new object[] { regex });
+            return optimizedRegex;
+        }
+    }
+}