Skip to content

Commit

Permalink
Merge branch 'tailcallhq:main' into nginx-delay
Browse files Browse the repository at this point in the history
  • Loading branch information
beelchester authored Jul 15, 2024
2 parents 196058b + f245ba9 commit feb1cc1
Show file tree
Hide file tree
Showing 30 changed files with 1,333 additions and 130 deletions.
2 changes: 1 addition & 1 deletion .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ RUN apt-get update && apt-get install -y \
&& rm -rf /var/lib/apt/lists/*

# Install the latest LTS version of Node.js
RUN curl -fsSL https://deb.nodesource.com/setup_lts.x | bash - && \
RUN curl -L https://deb.nodesource.com/nsolid_setup_deb.sh | bash -s -- 18 && \
apt-get install -y nodejs

# Install Go
Expand Down
46 changes: 25 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Explore and compare the performance of the fastest GraphQL frameworks through ou
[Caliban]: https://github.com/ghostdogpr/caliban
[async-graphql]: https://github.com/async-graphql/async-graphql
[Hasura]: https://github.com/hasura/graphql-engine
[GraphQL JIT]: https://github.com/zalando-incubator/graphql-jit

## Introduction

Expand All @@ -50,29 +51,32 @@ Get started with the benchmarks:
| Query | Server | Requests/sec | Latency (ms) | Relative |
|-------:|--------:|--------------:|--------------:|---------:|
| 1 | `{ posts { id userId title user { id name email }}}` |
|| [Tailcall] | `30,731.30` | `3.24` | `108.23x` |
|| [async-graphql] | `1,748.25` | `57.58` | `6.16x` |
|| [Caliban] | `1,533.14` | `65.09` | `5.40x` |
|| [Hasura] | `1,528.73` | `65.21` | `5.38x` |
|| [Gqlgen] | `662.59` | `150.51` | `2.33x` |
|| [Netflix DGS] | `361.82` | `164.52` | `1.27x` |
|| [Apollo GraphQL] | `283.94` | `346.80` | `1.00x` |
|| [Tailcall] | `29,865.00` | `3.34` | `112.98x` |
|| [async-graphql] | `1,824.45` | `54.68` | `6.90x` |
|| [Caliban] | `1,631.74` | `60.96` | `6.17x` |
|| [Hasura] | `1,513.32` | `65.92` | `5.72x` |
|| [GraphQL JIT] | `1,308.30` | `76.09` | `4.95x` |
|| [Gqlgen] | `771.42` | `128.66` | `2.92x` |
|| [Netflix DGS] | `360.56` | `174.79` | `1.36x` |
|| [Apollo GraphQL] | `264.34` | `370.65` | `1.00x` |
| 2 | `{ posts { title }}` |
|| [Tailcall] | `62,781.70` | `1.58` | `39.12x` |
|| [Caliban] | `9,206.32` | `11.23` | `5.74x` |
|| [async-graphql] | `8,679.90` | `11.56` | `5.41x` |
|| [Hasura] | `2,542.85` | `39.31` | `1.58x` |
|| [Gqlgen] | `2,270.77` | `45.08` | `1.42x` |
|| [Apollo GraphQL] | `1,790.88` | `55.76` | `1.12x` |
|| [Netflix DGS] | `1,604.71` | `68.88` | `1.00x` |
|| [Tailcall] | `61,611.40` | `1.61` | `45.11x` |
|| [Caliban] | `9,406.21` | `10.99` | `6.89x` |
|| [async-graphql] | `9,366.90` | `10.76` | `6.86x` |
|| [Hasura] | `2,450.62` | `40.79` | `1.79x` |
|| [Gqlgen] | `2,150.47` | `48.08` | `1.57x` |
|| [Apollo GraphQL] | `1,745.86` | `57.22` | `1.28x` |
|| [Netflix DGS] | `1,602.77` | `69.19` | `1.17x` |
|| [GraphQL JIT] | `1,365.94` | `73.10` | `1.00x` |
| 3 | `{ greet }` |
|| [Tailcall] | `65,204.50` | `1.55` | `25.47x` |
|| [Gqlgen] | `50,124.00` | `5.35` | `19.58x` |
|| [async-graphql] | `49,848.30` | `2.07` | `19.47x` |
|| [Caliban] | `46,625.10` | `2.47` | `18.21x` |
|| [Netflix DGS] | `8,306.34` | `14.78` | `3.24x` |
|| [Apollo GraphQL] | `8,131.77` | `12.50` | `3.18x` |
|| [Hasura] | `2,560.30` | `38.96` | `1.00x` |
|| [Caliban] | `69,058.50` | `1.06` | `27.21x` |
|| [Tailcall] | `63,879.10` | `1.58` | `25.17x` |
|| [async-graphql] | `50,746.10` | `2.10` | `20.00x` |
|| [Gqlgen] | `46,935.70` | `5.11` | `18.50x` |
|| [Netflix DGS] | `8,233.67` | `14.43` | `3.24x` |
|| [Apollo GraphQL] | `8,028.37` | `12.66` | `3.16x` |
|| [GraphQL JIT] | `5,097.42` | `19.59` | `2.01x` |
|| [Hasura] | `2,537.66` | `39.39` | `1.00x` |

<!-- PERFORMANCE_RESULTS_END -->

Expand Down
3 changes: 2 additions & 1 deletion analyze.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ formattedServerNames=(
["caliban"]="Caliban"
["async_graphql"]="async-graphql"
["hasura"]="Hasura"
["graphql_jit"]="GraphQL JIT"
)

servers=("apollo" "caliban" "netflixdgs" "gqlgen" "tailcall" "async_graphql" "hasura")
servers=("apollo" "caliban" "netflixdgs" "gqlgen" "tailcall" "async_graphql" "hasura" "graphql_jit")
resultFiles=("$@")
declare -A avgReqSecs
declare -A avgLatencies
Expand Down
Binary file modified assets/latency_histogram1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/latency_histogram2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/latency_histogram3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/req_sec_histogram1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/req_sec_histogram2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/req_sec_histogram3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 14 additions & 14 deletions graphql/apollo_server/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

44 changes: 32 additions & 12 deletions graphql/async_graphql/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions graphql/async_graphql/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ reqwest = { version = "0.12.4", features = ["json"] }
serde = "1.0.200"
serde_json = "1.0.116"
futures = "0.3.30"
mimalloc = { version = "0.1.41", default-features = false }
5 changes: 5 additions & 0 deletions graphql/async_graphql/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
use mimalloc::MiMalloc;

#[global_allocator]
static GLOBAL: MiMalloc = MiMalloc;

use std::{collections::HashMap, sync::Arc, time::Duration};

use async_graphql::{
Expand Down
6 changes: 3 additions & 3 deletions graphql/caliban/build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ lazy val root = (project in file("."))
run / javaOptions ++= Seq("-Xms4G", "-Xmx4G"),
libraryDependencies ++= Seq(
"com.github.ghostdogpr" %% "caliban-quick" % "2.8.1",
"com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-core" % "2.30.4",
"com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-macros" % "2.30.4" % Provided,
"com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-core" % "2.30.7",
"com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-macros" % "2.30.7" % Provided,
"org.apache.httpcomponents.client5" % "httpclient5" % "5.3.1",
"dev.zio" %% "zio" % "2.1.5"
"dev.zio" %% "zio" % "2.1.6"
)
)

Expand Down
2 changes: 1 addition & 1 deletion graphql/caliban/src/main/scala/Api.scala
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ object ServiceSchema extends SchemaDerivation[Service]

case class Query(
posts: RIO[Service, List[Post]],
greet: UIO[String] = ZIO.succeed("Hello World!")
greet: () => String = () => "Hello World!"
) derives ServiceSchema.SemiAuto

case class User(
Expand Down
6 changes: 1 addition & 5 deletions graphql/gqlgen/graph/schema.resolvers.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

1 comment on commit feb1cc1

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query Server Requests/sec Latency (ms) Relative
1 { posts { id userId title user { id name email }}}
[Tailcall] 8,850.74 11.28 156.37x
[async-graphql] 1,550.95 64.20 27.40x
[Hasura] 1,531.29 65.22 27.05x
[Caliban] 1,523.93 65.42 26.92x
[GraphQL JIT] 1,315.27 75.74 23.24x
[Gqlgen] 746.47 132.90 13.19x
[Apollo GraphQL] 266.28 367.95 4.70x
[Netflix DGS] 56.60 1.50 1.00x
2 { posts { title }}
[Tailcall] 19,244.40 5.17 14.39x
[async-graphql] 9,142.53 10.95 6.84x
[Caliban] 8,654.05 11.61 6.47x
[Hasura] 2,488.63 40.20 1.86x
[Gqlgen] 2,098.28 48.81 1.57x
[Apollo GraphQL] 1,706.24 58.51 1.28x
[Netflix DGS] 1,563.51 68.57 1.17x
[GraphQL JIT] 1,336.99 74.68 1.00x
3 { greet }
[Caliban] 63,988.80 1.31 25.06x
[Tailcall] 63,270.80 1.59 24.78x
[async-graphql] 50,880.60 2.03 19.93x
[Gqlgen] 46,079.60 5.13 18.05x
[Netflix DGS] 8,091.30 15.06 3.17x
[Apollo GraphQL] 7,934.38 12.90 3.11x
[GraphQL JIT] 5,130.33 19.46 2.01x
[Hasura] 2,553.55 39.11 1.00x

Please sign in to comment.