Skip to content

Commit

Permalink
Faster to_list: 20X for small Vectors and 8X for large ones
Browse files Browse the repository at this point in the history
Custom implementation of to_list.
Recursively walk the trie in reverse order (to build the resulting list in the right order).
Adaptive strategy for prepending leaf blocks:
* for Vectors up to 3 levels deep (up to 32,768 elements) use Tuple.to_list and list concatenation
  (up to 1,024 temp lists created)
* for bigger Vectors use custom prepend one by one function as the overhead of creating and reverting
  lists becomes significant

More informative description.
Ability to run benchmark in `full quick` mode.
  • Loading branch information
Dmitry Kakurin committed Oct 5, 2017
1 parent e640956 commit c9f6dd6
Show file tree
Hide file tree
Showing 7 changed files with 68 additions and 35 deletions.
1 change: 1 addition & 0 deletions PerfComp.cmd
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ call mix run bench\bench.exs %* >b

if exist a (
call %diff% a b
echo.
set /P better=Is 'b' better [y/N]?
if /I "!better!" == "y" (
echo New base 'a'
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@

## Description

`PersistentVector` is an array-like collection of values indexed by contiguous 0-based integer index.
`PersistentVector` is an array-like collection of values indexed by contiguous 0-based integer index
and optimized for growing/shrinking at the end.

`PersistentVector` optimizes the following operations:
* Get element count
Expand All @@ -27,7 +28,7 @@ Add `persistent_vector` to your list of dependencies in `mix.exs`:
```elixir
def deps do
[
{:persistent_vector, "~> 0.1.3"}
{:persistent_vector, "~> 0.1.4"}
]
end
```
Expand Down
4 changes: 2 additions & 2 deletions bench/bench.exs
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ only =
_ -> nil
end

full = System.argv == ["full"]
quick = System.argv == ["quick"]
full = Enum.member?(System.argv, "full")
quick = Enum.member?(System.argv, "quick")

defmodule Runner do
@print_opts [benchmarking: false, fast_warning: false, configuration: false]
Expand Down
42 changes: 21 additions & 21 deletions benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ If building speed is not important, but removal from the end happens often, then
This comparison is *not fair* to `Map` as it has much richer capabilities.
The fact that `Map` performs worse for bigger collections is not surprising and is not `Map`'s fault :-).

## Raw Benchmarking results for v0.1.3
## Raw Benchmarking results for v0.1.4

```none
Using 2 inputs
Expand Down Expand Up @@ -162,31 +162,31 @@ Map Enumerate 8.07 - 1.93x slower
##### With input 1'000 #####
Name ips average deviation median
Array to_list 65.50 K 15.27 μs ±18.44% 16.00 μs
Map to_list 55.73 K 17.94 μs ±31.20% 16.00 μs
Vector into 17.06 K 58.61 μs ±129.18% 0.0 μs
Map into 13.31 K 75.14 μs ±104.00% 0.0 μs
Vector to_list 9.26 K 108.02 μs ±4.11% 109.00 μs
Vector to_list 190.39 K 5.25 μs ±14.27% 4.70 μs
Array to_list 67.39 K 14.84 μs ±23.25% 16.00 μs
Map to_list 49.62 K 20.15 μs ±35.27% 16.00 μs
Vector into 17.55 K 56.99 μs ±13.19% 62.00 μs
Map into 13.66 K 73.23 μs ±9.91% 78.00 μs
Comparison:
Array to_list 65.50 K
Map to_list 55.73 K - 1.18x slower
Vector into 17.06 K - 3.84x slower
Map into 13.31 K - 4.92x slower
Vector to_list 9.26 K - 7.08x slower
Vector to_list 190.39 K
Array to_list 67.39 K - 2.83x slower
Map to_list 49.62 K - 3.84x slower
Vector into 17.55 K - 10.85x slower
Map into 13.66 K - 13.94x slower
##### With input 1'000'000 #####
Name ips average deviation median
Array to_list 53.27 18.77 ms ±36.93% 16.00 ms
Map to_list 26.62 37.57 ms ±57.36% 31.00 ms
Vector into 10.97 91.19 ms ±13.77% 94.00 ms
Map into 8.33 119.98 ms ±6.39% 125.00 ms
Vector to_list 6.31 158.49 ms ±3.92% 156.00 ms
Array to_list 53.20 18.80 ms ±36.19% 16.00 ms
Vector to_list 49.58 20.17 ms ±38.37% 16.00 ms
Map to_list 26.54 37.68 ms ±55.39% 31.00 ms
Vector into 10.98 91.05 ms ±15.50% 93.00 ms
Map into 8.29 120.67 ms ±7.02% 125.00 ms
Comparison:
Array to_list 53.27
Map to_list 26.62 - 2.00x slower
Vector into 10.97 - 4.86x slower
Map into 8.33 - 6.39x slower
Vector to_list 6.31 - 8.44x slower
Array to_list 53.20
Vector to_list 49.58 - 1.07x slower
Map to_list 26.54 - 2.00x slower
Vector into 10.98 - 4.84x slower
Map into 8.29 - 6.42x slower
```
45 changes: 38 additions & 7 deletions lib/PersistentVector.ex
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
defmodule PersistentVector do
@moduledoc """
`PersistentVector` is an array-like collection of values indexed by contiguous `0`-based integer index.
`PersistentVector` is an array-like collection of values indexed by contiguous `0`-based integer index
and optimized for growing/shrinking at the end.
`PersistentVector` optimizes the following operations:
* Get element count
Expand Down Expand Up @@ -454,16 +455,46 @@ defmodule PersistentVector do
[]
"""
@spec to_list(t) :: [value]
def to_list(v = %@state{count: count})
when count > 0
def to_list(v = %@state{root: root, shift: shift})
do
Enum.reduce(count-1 .. 0, [], &([v |> fast_get(&1) | &2]))
acc = Tuple.to_list(v.tail)
if root == {} do
acc
else
big? = shift > 2 * @shift
to_list(root, shift, tuple_size(root)-1, big?, acc)
end
end

def to_list(%@state{})
# when count == 0
defp to_list(arr, level, i, big?, acc)
when level > 0
do
[]
child = elem(arr, i)
acc = to_list(child, level - @shift, tuple_size(child)-1, big?, acc)
if i > 0 do
to_list(arr, level, i-1, big?, acc)
else
acc
end
end

defp to_list(arr, _level, i, big?, acc)
# when level == 0
do
if big? do
to_list_leaf(arr, i-1, [elem(arr, i) | acc])
else
Tuple.to_list(arr) ++ acc
end
end

defp to_list_leaf(arr, i, acc) do
acc = [elem(arr, i) | acc]
if i > 0 do
to_list_leaf(arr, i-1, acc)
else
acc
end
end

@doc false # "See `Enumerable.reduce/3`"
Expand Down
4 changes: 2 additions & 2 deletions mix.exs
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
defmodule PersistentVector.Mixfile do
use Mix.Project

@version "0.1.3"
@version "0.1.4"
@github "https://github.com/dimagog/persistent_vector"

def project do
[
app: :persistent_vector,
name: "Persistent Vector",
version: @version,
description: "PersistentVector is an array-like collection of values indexed by contiguous 0-based integer index.",
description: "PersistentVector is an array-like collection of values indexed by contiguous 0-based integer index and optimized for growing/shrinking at the end.",
elixir: "~> 1.4",
start_permanent: Mix.env == :prod,
elixirc_paths: elixirc_paths(Mix.env),
Expand Down
2 changes: 1 addition & 1 deletion test/PersistentVector_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ defmodule PersistentVectorTest do
end

property "Collectable" do
forall n <- nat() do
forall n <- choose(0, 1500) do
v = erange(0, n) |> Enum.into(empty()) |> assert_element_identity()
assert v |> count() == n
assert (v |> Enum.into([])) == (erange(0, n) |> Enum.into([]))
Expand Down

0 comments on commit c9f6dd6

Please sign in to comment.