Skip to content

Commit

Permalink
Merge pull request #9 from gregsdennis/pointer/low-alloc
Browse files Browse the repository at this point in the history
add post about performance improvements
  • Loading branch information
gregsdennis authored May 1, 2024
2 parents 4c6ff53 + 5d3a6bf commit 022557a
Show file tree
Hide file tree
Showing 3 changed files with 147 additions and 0 deletions.
Binary file modified .jekyll-metadata
Binary file not shown.
143 changes: 143 additions & 0 deletions _posts/2024/2024-04-17-better-json-pointer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
---
title: "Better JSON Pointer"
date: 2024-04-30 09:00:00 +1200
tags: [json-pointer, architecture, performance, learning]
toc: true
pin: false
---

This post was going to be something else, and somewhat more boring. Be glad you're not reading that.

In the midst of updating _JsonPointer.Net_, instead of blindly forging on when metrics looked decent but the code was questionable, I stopped to consider whether I actually wanted to push out the changes I had made. In the end, I'm glad I hesitated.

In this post and at least the couple that follow, I will cover my experience trying to squeeze some more performance out of a simple, immutable type.

## In the before times

The `JsonPointer` class is a typical object-oriented approach to implementing the JSON Pointer specification, RFC 6901.

Syntactically, a JSON Pointer is nothing more a series of string segments separated by forward slashes. All of the pointer segments follow the same rule: any tildes (`~`) or forward slashes (`/`) need to be escaped; otherwise, just use the string as-is.

A class is created to model a segment (`PointerSegment`), and then another class is created to house a series of them (`JsonPointer`). Easy.

Tack on some functionality for parsing, evaluation, and maybe some pointer math (combining and building pointers), and you have a full implementation.

## An idea is formed

In thinking about how the model could be better, I realized that the class is immutable, and it doesn't directly hold a lot of data. What if it were a struct? Then it could live on the stack, eliminating a memory allocation.

Then, instead of holding a collection of strings, it could hold just the full string and a collection of `Range` objects could indicate the segments as sort of "zero-allocation substrings": one string allocation instead of an array of objects that hold strings.

This raises a question of whether the string should hold pointer-encoded segments. If it did, then `.ToString()` could just return the string, eliminating the need to build it, and I could provide new allocation-free string comparison methods that accounted for encoding so that users could still operate on segments.

I implemented all of this, and it worked! It actually worked quite well:

| Version| n | Mean | Error | StdDev | Gen0 | Allocated |
|------- |------ |-----------:|----------:|-----------:|---------:|----------:|
| v4.0.1 | 1 | 2.778 us | 0.0546 us | 0.1025 us | 4.1962 | 8.57 KB |
| v5.0.0 | 1 | 1.718 us | 0.0335 us | 0.0435 us | 1.4915 | 3.05 KB |
| v4.0.1 | 10 | 26.749 us | 0.5000 us | 0.7330 us | 41.9617 | 85.7 KB |
| v5.0.0 | 10 | 16.719 us | 0.3219 us | 0.4186 us | 14.8926 | 30.47 KB |
| v4.0.1 | 100 | 286.995 us | 5.6853 us | 12.5983 us | 419.4336 | 857.03 KB |
| v5.0.0 | 100 | 157.159 us | 2.5567 us | 2.1350 us | 149.1699 | 304.69 KB |

... for parsing. Pointer math was a bit different:

| Version| n | Mean | Error | StdDev | Gen0 | Allocated |
|------- |------ |------------:|------------:|------------:|---------:|----------:|
| v4.0.1 | 1 | 661.2 ns | 12.86 ns | 11.40 ns | 1.1473 | 2.34 KB |
| v5.0.0 | 1 | 916.3 ns | 17.46 ns | 15.47 ns | 1.1120 | 2.27 KB |
| v4.0.1 | 10 | 6,426.4 ns | 124.10 ns | 121.88 ns | 11.4746 | 23.44 KB |
| v5.0.0 | 10 | 9,128.2 ns | 180.82 ns | 241.39 ns | 11.1237 | 22.73 KB |
| v4.0.1 | 100 | 64,469.6 ns | 1,309.01 ns | 1,093.08 ns | 114.7461 | 234.38 KB |
| v5.0.0 | 100 | 92,437.0 ns | 1,766.38 ns | 1,963.33 ns | 111.3281 | 227.34 KB |

While the memory allocation decrease was... fine, the 50% run-time increase was unacceptable. I couldn't figure out what was going on here, so I left it for about a week and started on some updates for _JsonSchema.Net_ (post coming soon).

Initially for the pointer math, I was just creating a new string and then parsing that. The memory usage was a bit higher than what's shown above, but the run-time was almost double. After a bit of thought, I realized I can explicitly build the string _and_ the range array, which cut down on both the run time and the memory, but only so far as what's shown above.

## Eureka!

After a couple days, I finally figured out that by storing each segment, the old way could re-use segments between pointers. Sharing segments helps with pointer math where we're chopping up and combining pointers.

For example, let's combine `/foo/bar` and `/baz`. Under the old way, the pointers for those hold the arrays `['foo', 'bar']` and `['baz']`. When combining them, I'd just merge the arrays: `['foo', 'bar', 'baz']`. It's allocating a new array, but not new strings. All of the segment strings stayed the same.

Under the new way, I'd actually build a new string `/foo/bar/baz` and then build a new array of `Range`s to point to the substrings.

So this new architecture isn't better after all.

## A hybrid design

I thought some more about the two approaches. The old approach does pointer math really well, but I don't like that I have an object (`JsonPointer`) that contains more objects (`PointerSegment`) that each contain strings. That seems wasteful.

Also, why did I make it a struct? Structs should be a fixed size, and strings are never a fixed size (which is a major reason `string` is a class). Secondly, the memory of a struct should also live on the stack, and strings and arrays (even arrays of structs) are stored on the heap; so really it's only the container that's on the stack. A struct just isn't the right choice for this type, so I should change it back to a class.

What if the pointer just held the strings directly instead of having a secondary `PointerSegment` class? In the old design, `PointerSegment` handled all of the decoding/encoding logic, so that would have to live somewhere else, but that's fine. So I don't need a model for the segments; plain strings will do.

Lastly, I could make it implement `IReadOnlyList<string>`. That would give users a `.Count` property, an indexer to access segments, and allow them to iterate over segments directly.

## A new implementation

Taking in all of this analysis, I updated `JsonPointer` again:

- It's a class again.
- It holds an array of (decoded) strings for the segments.
- It will cache its string representation.
- Parsing a pointer already has the string; just store it.
- Constructing a pointer and calling `.ToString()` builds on the fly and caches.

`PointerSegment`, which had also been changed to a struct in the first set of changes, remains a struct and acts as an intermediate type so that building pointers in code can mix strings and integer indices. (See the `.Create()` method used in the code samples below.) Keeping this as a struct means no allocations.

I fixed all of my tests and ran the benchmarks again:

| Parsing | Count | Mean | Error | StdDev | Gen0 | Allocated |
|------- |------ |-----------:|----------:|----------:|---------:|----------:|
| 5.0.0 | 1 | 3.825 us | 0.0760 us | 0.0961 us | 3.0823 | 6.3 KB |
| 5.0.0 | 10 | 36.155 us | 0.6979 us | 0.9074 us | 30.8228 | 62.97 KB |
| 5.0.0 | 100 | 362.064 us | 6.7056 us | 6.2724 us | 308.1055 | 629.69 KB |

| Math | Count | Mean | Error | StdDev | Gen0 | Allocated |
|------- |------ |------------:|----------:|----------:|--------:|----------:|
| 5.0.0 | 1 | 538.2 ns | 10.12 ns | 10.83 ns | 0.9794 | 2 KB |
| 5.0.0 | 10 | 5,188.1 ns | 97.80 ns | 104.65 ns | 9.7885 | 20 KB |
| 5.0.0 | 100 | 58,245.0 ns | 646.43 ns | 539.80 ns | 97.9004 | 200 KB |

For parsing, run time is higher, generally about 30%, but allocations are down 26%.

For pointer math, run time and allocations are both down, about 20% and 15%, respectively.

I'm comfortable with the parsing time being a bit higher since I expect more usage of the pointer math.

## Some new toys

In addition to the simple indexer you get from `IReadOnlyList<string>`, if you're working in .Net 8, you also get a `Range` indexer which allows you to create a pointer using a subset of the segments. This is really handy when you want to get the parent of a pointer

```c#
var pointer = JsonPointer.Create("foo", "bar", 5, "baz");
var parent = pointer[..^1]; // /foo/bar/5
```

or maybe the relative local pointer (i.e. the last segment)

```c#
var pointer = JsonPointer.Create("foo", "bar", 5, "baz");
var local = pointer[^1..]; // /baz
```

These operations are pretty common in _JsonSchema.Net_.

For those of you who haven't made it to .Net 8 just yet, this functionality is also available as methods:

```c#
var pointer = JsonPointer.Create("foo", "bar", 5, "baz");
var parent = pointer.GetAncestor(1); // /foo/bar/5
var local = pointer.GetLocal(1); // /baz
```

Personally, I like the indexer syntax. I was concerned at first that having an indexer return a new object might feel unorthodox to some developers, but that's exactly what `string` does when you use a `Range` index to get a substring, so I'm fine with it.

## Wrap up

I like where this landed a lot more than where it was in the middle. Something just felt off with the design, and I was having trouble isolating what the issue was. I like that `PointerSegment` isn't part of the model anymore, and it's just "syntax candy" to help build pointers. I really like the performance.

I learned a lot about memory management, which will be the subject of the next post. But more than that, I learned that sometimes inaction is the right action. I hesitated, and the library is better for it.
4 changes: 4 additions & 0 deletions assets/css/style.scss
Original file line number Diff line number Diff line change
Expand Up @@ -59,3 +59,7 @@ img {
code.highlighter-rouge {
font-size: .85em !important;
}

table {
width: -webkit-fill-available;
}

0 comments on commit 022557a

Please sign in to comment.