Skip to content

Home

Jump to bottom
Ábel Hegedüs edited this page Jun 17, 2016 · 4 revisions

Welcome to the VIATRA CPS Benchmark wiki!

CPS Demonstrator

See details on https://github.com/viatra/viatra-docs/blob/master/cps/Home.adoc

Performance evaluation

The main goal of the demonstrator is to provide a good case study for the performance evaluation of different implementations of the same transformation specification. Our measurement strategy is based on long years of experience with benchmarking model-driven applications. The measurements are performed as a series of steps, with the runtime and post-step memory consumption recorded for each step. In some test-cases MONDO-SAM can be used for automated performance-reporting. For details see the Performance reporting with MONDO-SAM page.

Main findings based on the evaluation:

  • The simple Xtend batch transformation is faster on small models, but even its optimized variant cannot handle larger models effectively.
  • The Xtend and EMF-IncQuery batch transformation is the fastest as a batch transformation and requires less memory than other EMF-IncQuery based variants, since it only matches on the CPS model which does not change during the transformation.
  • While both the explicit traceability and query result bookmarking incremental transformations take longer time to execute the transformation the first time, they are able to follow CPS model changes and synchronize the deployment model incrementally.
  • The explicit traceability variant consumes more memory than the other variants, since it works with complex patterns that are evaluated on the traceability and deployment model as well. In addition, this variant can be started with an existing partially transformed CPS model and only the required changes will be synchronized.
  • The query result bookmarking variant consumes similar amount of memory as the batch EMF-IncQuery variant, with the added advantage that it works incrementally by watching query result changes. However, it can only work correctly if the CPS models are only modified when the transformation is running and the deployment and traceability models are not modified.
Clone this wiki locally