Skip to content

Commit

Permalink
add post about number equality in json
Browse files Browse the repository at this point in the history
  • Loading branch information
gregsdennis committed Nov 14, 2023
1 parent 1dca41d commit 83ff89b
Show file tree
Hide file tree
Showing 3 changed files with 89 additions and 1 deletion.
2 changes: 1 addition & 1 deletion _posts/2023/2023-07-26-jsonnode-odd-api.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "JsonNode's Odd API"
date: 2023-07-26 09:00:00 +1200
tags: [json-path, json-pointer]
tags: [json-path, json-pointer, json-node, oddity]
toc: true
pin: false
---
Expand Down
81 changes: 81 additions & 0 deletions _posts/2023/2023-11-09-decimals-are-weird.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
title: ".Net Decimals are Weird"
date: 2023-11-14 09:00:00 +1200
tags: [json-node, oddity]
toc: true
pin: false
---

I've discovered another odd consequence of what is probably fully intentional code: `4m != 4.0m`.

Okay, that's not strictly true, but it does seem so if you're comparing the values in JSON.

```C#
var a = 4m;
var b = 4.0m;

JsonNode jsonA = a;
JsonNOde jsonB = b;

// use .IsEquivalentTo() from Json.More.Net
Assert.True(jsonA.IsEquivalentTo(jsonB)); // fails!
```

What?!

_This took me so long to find..._

## What's happening ([brother](https://www.youtube.com/watch?v=tvjrSU9RaPs))

The main insight is contained in [this StackOverflow answer](https://stackoverflow.com/a/13770183/878701). `decimal` has the ability to retain significant digits! Even if those digits are expressed in code!!

So when we type `4.0m` in C# code, the compiler tells `System.Decimal` that the `.0` is important. When the value is printed (e.g. via `.ToString()`), even without specifying a format, you get `4.0` back. And this includes when serializing to JSON. If you debug the code above, you'll see that `a` has a value of `4` while `b` has a value of `4.0`. Even before it gets to the `JsonNode` assignments.

While this doesn't affect _numeric_ equality, it could affect equality that relies on the string representation of the number (like in JSON).

## How this bit me

In developing a new library for [JSON-e](https://json-e.js.org/) support (spoiler, I guess), I found a test that was failing, and I couldn't understand why.

I won't go into the full details here, but JSON-e supports expressions, and one of the tests has the expression `4 == 3.2 + 0.8`. Simple enough, right? So why was I failing this?

When getting numbers from JSON throughout all of my libraries, I chose to use `decimal` because I felt it was more important to support JSON's arbitrary precision with `decimal`'s higher precision rather than using `double` for a bit more range. So when parsing the above expression, I get a tree that looks like this:

```
==
/ \
4 +
/ \
3.2 0.8
```

where each of the numbers are represented as `JsonNode`s with `decimals` underneath.

When the system processes `3.2 + 0.8`, it gives me `4.0`. As I said before, numeric comparisons between `decimal`s work fine. But in these expressions, `==` doesn't compare just numbers; it compares `JsonNode`s. And it does so using my `.IsEquivalentTo()` extension method.

## What's wrong with the extension?

When I built the extension method, I already had one for `JsonElement`. (It handles everything correctly, too.) However `JsonNode` doesn't always store `JsonElement` underneath. It can also store the raw value.

This has an interesting nuance to the problem in that if the `JsonNode`s are parsed:

```C#
var jsonA = JsonNode.Parse("4");
var jsonB = JsonNode.Parse("4.0");

Assert.True(jsonA.IsEquivalentTo(jsonB));
```

the assertion passes because parsing into `JsonNode` just stores `JsonElement`, and the comparison works for that.

So instead of rehashing all of the possibilities of checking strings, booleans, and all of the various numeric types, I figured it'd be simple enough to just `.ToString()` the node and compare the output.

And it worked... until I tried the expression above. For **18 months** it's worked without any problems. Such is software development, I suppose.

## It's fixed now

So now I check explicitly for numeric equality by calling `.GetNumber()`, which checks all of the various .Net number types returns a `decimal?` (null if it's not a number).

There's a new package available for those impacted by this (I didn't receive any reports).

And that's the story of how creating a new package to support a new JSON functionality showed me how 4 is not always 4.
7 changes: 7 additions & 0 deletions _posts/2023/2023-11-09-updating-vocabs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
title: "Why I'm Updating My JSON Schema Vocabularies"
date: 2023-11-09 09:00:00 +1200
tags: [json-schema, vocab, vocabulary]
toc: true
pin: false
---

0 comments on commit 83ff89b

Please sign in to comment.