Skip to content

Latest commit

 

History

History
103 lines (65 loc) · 2.78 KB

README.md

File metadata and controls

103 lines (65 loc) · 2.78 KB
Raw

Benchmark for Istio & AWS App Mesh

Goals

  • Only focus on data plane (Envoy) performance

  • Focus on latency, throughput and sytem resouce cost (CPU and memory)

  • Investigate how they perform under different data volumes

Prerequisites

  • Environments

    • AWS EKS
      • Kubernetes v1.18,
      • 1 Node - AL2_x86_64, c5.xlarge(4cpu,8g mem)
    • App Mesh
      • controller v1.1.1
      • Envoy: 840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.15.1.0-prod
    • Istio
      • v1.7.0
      • Envoy: docker.io/istio/proxyv2:1.7.0
    • AWS CloudWatch

  • Application

    • Client side: HTTP client with Golang v1.15.3, native package

    • Server side: httpbin (docker.io/kennethreitz/httpbin)

    • Test case:

      • the client as a NodePort service and expose port 8888
      • curl the client on the host with parametes (c1 t1 ~ c128 t128)
      • the client will send a request to call the server's APIs
    • NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
goclient     NodePort    10.100.180.134   <none>        8888:31952/TCP   28h
httpbin      ClusterIP   10.100.159.150   <none>        8000/TCP         5d14h

  • Benchmark tool:

    • wrk 4.1.0-4-g0896020

Benchmark

Latency

  • For a few data volumes, App Mesh is a little bit better than Istio.
  • For the large data volumes, they are almost the same as baseline.

latency

Throughput

  • For a few data volumes, App Mesh is a little bit better than Istio.
  • For the large data volumes, they are almost the same.

throughput

Timeout

For large data volumes, the timeout count is almost the same.

timeout

System resource cost

  • Transfer data volumes is 100 Bytes: App Mesh's costs (both cpu & mem) is better than Istio's

    Istio: byte100istio

    App Mesh: byte100am

  • Transfer data volumes is 1 MB: App Mesh's mem cost is better than Istio's; The cpu costs the same as baseline.

    Istio: byte1mistio

    App Mesh: byte1mam

Namespace cost

This is not to say that App Mesh's control plane is better than Istio's. App Mesh has only a Controller under the appmesh-system namespace. Because it's transparent to the user and cannot be sure that its control plane is better than Istiod.

namespace-overall

Conclusion

  • For transfering small data volumes, App Mesh is a little better than Istio.
  • For transfering big data volumes, they have the same performance.
  • App Mesh has a lower system resource cost.

Further investigation

  • Both using Envoy, which part is different that makes the difference?
  • Whether the same result for gRPC/TCP or not?
  • Investigation with TLS