[MacOS]: Performance WindUp (~28 seconds) vs Kantra (~237 seconds) on sample app jakartaee-duke with target jakarta-ee #96

jwmatthews · 2023-10-21T12:02:28Z

I am seeing a noticeable performance difference between running Kantra and WindUp on the same source application and target on my Mac M1 Max (64GB)

Sample app: https://github.com/ivargrimstad/jakartaee-duke/tree/start-tutorial

Using the 'start-tutorial' branch

Environment: Mac M1 Max 64GB

WindUp

Run #1

time ./windup-cli --input /Users/jmatthews/git/jwmatthews/kantra/hack/tmp/jakartaee-duke --target jakarta-ee --output ./out --
52.59s user 2.58s system 206% CPU 
26.769 total

Run #2

time ./windup-cli --input /Users/jmatthews/git/jwmatthews/kantra/hack/tmp/jakartaee-duke --target jakarta-ee --output ./out --overwrite
61.67s user 3.10s system 232% CPU 
27.884 total

Kantra

Run #1

time kantra analyze -i $PWD/tmp/jakartaee-duke -t "jakarta-ee9+" -o $PWD/tmp/out

real	3m50.204s
user	0m0.257s
sys	0m0.387s
./sample_run.sh  
0.30s user 0.46s system 0% CPU 
3:50.65 total

Run #2

time kantra analyze -i $PWD/tmp/jakartaee-duke -t "jakarta-ee9+" -o $PWD/tmp/out

real	3m57.866s
user	0m0.252s
sys	0m0.366s

Notes

I ran Kantra twice, thinking perhaps the first run had some delays pulling images, so second run was invoked soon after with the assumption images have been pulled (running on same podman machine VM)

$ podman machine info
Host:
  Arch: arm64
  CurrentMachine: kantra
  DefaultMachine: kantra
  EventsDir: /var/folders/l1/1b7gzz8n02b5nrtdq51bnj4r0000gn/T/podman-run--1/podman
  MachineConfigDir: /Users/jmatthews/.config/containers/podman/machine/qemu
  MachineImageDir: /Users/jmatthews/.local/share/containers/podman/machine/qemu
  MachineState: Running
  NumberOfMachines: 1
  OS: darwin
  VMType: qemu
Version:
  APIVersion: 4.7.1
  Built: 1696512066
  BuiltTime: Thu Oct  5 09:21:06 2023
  GitCommit: ef83eeb9c7482826672f3efa12db3d61c88df6c4
  GoVersion: go1.21.3
  Os: darwin
  OsArch: darwin/arm64
  Version: 4.7.1

$ podman machine inspect kantra
[
     {
          "ConfigPath": {
               "Path": "/Users/jmatthews/.config/containers/podman/machine/qemu/kantra.json"
          },
          "ConnectionInfo": {
               "PodmanSocket": {
                    "Path": "/Users/jmatthews/.local/share/containers/podman/machine/qemu/podman.sock"
               },
               "PodmanPipe": null
          },
          "Created": "2023-10-21T07:26:57.879808-04:00",
          "Image": {
               "IgnitionFilePath": {
                    "Path": "/Users/jmatthews/.config/containers/podman/machine/qemu/kantra.ign"
               },
               "ImageStream": "testing",
               "ImagePath": {
                    "Path": "/Users/jmatthews/.local/share/containers/podman/machine/qemu/kantra_fedora-coreos-38.20231014.2.0-qemu.aarch64.qcow2"
               }
          },
          "LastUp": "2023-10-21T07:26:57.879808-04:00",
          "Name": "kantra",
          "Resources": {
               "CPUs": 4,
               "DiskSize": 100,
               "Memory": 4096
          },
          "SSHConfig": {
               "IdentityPath": "/Users/jmatthews/.ssh/kantra",
               "Port": 55239,
               "RemoteUsername": "core"
          },
          "State": "running",
          "UserModeNetworking": true,
          "Rootful": false
     }
]

The text was updated successfully, but these errors were encountered:

jwmatthews · 2023-10-21T12:28:31Z

Another run with Kantra on Mac M1 Max after bumping podman machine to have more resources

"Resources": {
               "CPUs": 8,
               "DiskSize": 100,
               "Memory": 16384
          },

Run #1 (assume some time for fetching images as the VM was destroyed and recreated)

./kantra analyze -i $PWD/tmp/jakartaee-duke -t "jakarta-ee9+" -o $PWD/tmp/out  
0.24s user 0.30s system 0% CPU 
4:21.06 total

Run #2 (in theory images have been pulled this should be a quicker run)

./kantra analyze -i $PWD/tmp/jakartaee-duke -t "jakarta-ee9+" -o $PWD/tmp/out 0.23s user 0.31s system 0% CPU 4:07.99 total

shawn-hurley · 2023-10-22T19:15:32Z

Hey, I notice the postman machine only has 4gb of memory, can we run with that a little higher? I also notice the CPUs is 4 is that the max that windup was running as?

…

On Oct 21, 2023, at 8:03 AM, John Matthews ***@***.***> wrote: DiskSize": 100, "Memory": 4096

jwmatthews · 2023-10-22T23:45:08Z

Yeah, I posted another run with higher CPU's and memory after I did the initial, thinking maybe it was due to lower resources but I didn't see an improvement. See the prior comment and I shared the specs and output

shawn-hurley · 2023-10-23T14:38:02Z

Yeah, I saw that after read the first email 😭

I wonder if we are having an issue with volume mounts file I/o getting code snips. I have a sneaky suspicion that this could be pretty bad performance-wise, I don't think we are trying to do anything smart, batching up file reads, etc.

@pranavgaikwad @eemcmullan let me know what y'all think?

pranavgaikwad · 2023-10-23T14:59:34Z

@shawn-hurley I would like to get to the bottom of what exactly happening in John's case here. While I don't deny the I/O bit, there are other several areas I think account for majority of time spent. I/O is a small percentage compared to that. I use Jaeger very often when I run analysis, we have the tracer in analyzer and common thing I have observed in most testing I have done so far is that there are two major factors:

Pulling deps via maven + decompiling
Each condition takes several seconds to complete. (Now this could in turn be due to I/O but wonder if there's anything else here)

Anyway, I am curious to know what exactly is happening in this case.

shawn-hurley · 2023-10-23T15:53:16Z

That is good to know; where is the bulk of the time spent in the conditions?

pranavgaikwad · 2023-10-24T16:43:29Z

@shawn-hurley See two traces - one on Linux, the other on Mac:

On Mac:

On Linux:

Notice that on average, the file and filecontent conditions are 100 times slower on Mac. I do think that analyzer itself isn't causing these issues, and there's very little we could do to solve this specific problem of grep or find running slower by making changes in the analyzer file handling.

The main reason seems to be the overhead in mounting the volumes via podman machine.

shawn-hurley · 2023-10-25T12:30:08Z

A quick test to be sure would be to build an image so that you don't have to mount things and run the analysis.

jwmatthews · 2023-11-07T19:01:24Z

Below was run on MacOS using Kantra (Macbook Pro M1Max)
CPU=4, MEM=8192

git clone https://github.com/deewhyweb/eap-coolstore-monolith.git coolstuff-javaee
time ./kantra analyze -i coolstuff-javaee -t "quarkus" -t "jakarta-ee" -t "cloud-readiness" -o ./out
real	33m41.188s

shawn-hurley · 2023-11-13T17:11:03Z

containers/podman#16994 (comment)

just to note, it looks like Podman machine on Mac is a problem w/ mounting and file I/o.

Do we think we can wait for this to be fixed in the underlying thing or not?

ascheman · 2023-11-13T17:44:49Z

FYI: I have similar (performance) problems with Docker (on Apple Silicon).

shawn-hurley · 2023-11-14T18:29:19Z

Tracking issue: coreos/fedora-coreos-tracker#1548

I feel that we should wait until the above is done to further research

shawn-hurley · 2023-11-29T19:12:16Z

Closing this issues by #127

While the performance may not be one-to-one, this drastically reduces the time we take in file I/o.

If we are still seeing drastic issues, can we add to #121 as I have a feeling we are going to see a difference in the network load on subsquent load because we don't have a shared .m2 while wind up will.

Let me know if we feel this should be re-opened if the explanation is not sufficient

jwmatthews · 2023-11-29T23:46:58Z

Below was run on MacOS using Kantra (Macbook Pro M1Max) CPU=4, MEM=8192

git clone https://github.com/deewhyweb/eap-coolstore-monolith.git coolstuff-javaee
time ./kantra analyze -i coolstuff-javaee -t "quarkus" -t "jakarta-ee" -t "cloud-readiness" -o ./out
real	33m41.188s

I re-ran today using :latest with #127.
Huge improvement: went from ~33mins to ~3.5 mins

$ time ./kantra analyze -i coolstuff-javaee -t "quarkus" -t "jakarta-ee" -t "cloud-readiness" -o ./out2
INFO[0000] running source code analysis args="--provider-settings=/opt/input/config/settings.json --rules=/opt/rulesets/ --output-file=/opt/output/output.yaml --context-lines=100 --dep-label-selector=(!konveyor.io/dep-source=open-source) --verbose=4 --label-selector=((konveyor.io/target=quarkus || konveyor.io/target=jakarta-ee || konveyor.io/target=cloud-readiness) && konveyor.io/source) || (discovery)" input=/Users/jmatthews/git/jwmatthews/kyma_poc/data/coolstuff-javaee log=/Users/jmatthews/git/jwmatthews/kyma_poc/data/out2/analysis.log output=/Users/jmatthews/git/jwmatthews/kyma_poc/data/out2 volumes="{"/Users/jmatthews/git/jwmatthews/kyma_poc/data/coolstuff-javaee":"/opt/input/source","/Users/jmatthews/git/jwmatthews/kyma_poc/data/out2":"/opt/output","/var/folders/l1/1b7gzz8n02b5nrtdq51bnj4r0000gn/T/analyze-config-4113444191":"/opt/input/config"}"
INFO[0000] generating analysis log in file file=/Users/jmatthews/git/jwmatthews/kyma_poc/data/out2/analysis.log
INFO[0126] running dependency analysis args="--provider-settings=/opt/input/config/settings.json --rules=/opt/rulesets/ --output-file=/opt/output/output.yaml --context-lines=100 --dep-label-selector=(!konveyor.io/dep-source=open-source) --verbose=4 --label-selector=((konveyor.io/target=quarkus || konveyor.io/target=jakarta-ee || konveyor.io/target=cloud-readiness) && konveyor.io/source) || (discovery)" input=/Users/jmatthews/git/jwmatthews/kyma_poc/data/coolstuff-javaee log=/Users/jmatthews/git/jwmatthews/kyma_poc/data/out2/dependency.log output=/Users/jmatthews/git/jwmatthews/kyma_poc/data/out2
INFO[0126] generating dependency log in file file=/Users/jmatthews/git/jwmatthews/kyma_poc/data/out2/dependency.log
INFO[0197] generating static report args="/usr/local/bin/js-bundle-generator --analysis-output-list=/opt/output/output.yaml --deps-output-list=/opt/output/dependencies.yaml --output-path=/usr/local/static-report/output.js --application-name-list=coolstuff-javaee && cp -r /usr/local/static-report /opt/output" output=/Users/jmatthews/git/jwmatthews/kyma_poc/data/out2
INFO[0198] Static report created. Access it at this URL: URL="file:/Users/jmatthews/git/jwmatthews/kyma_poc/data/out2/static-report/index.html"
./kantra analyze -i coolstuff-javaee -t "quarkus" -t "jakarta-ee" -t -o

0.13s user 0.18s system 0% cpu 3:19.08 total

jwmatthews changed the title ~~Performance WindUp (~28 seconds) vs Kantra (~237 seconds) on sample app jakartaee-duke with target jakarta-ee~~ [MacOS]: Performance WindUp (~28 seconds) vs Kantra (~237 seconds) on sample app jakartaee-duke with target jakarta-ee Oct 24, 2023

jwmatthews mentioned this issue Nov 16, 2023

[MacOS] Performance observation: Kantra is ~144x slower than WindUp on jboss-eap-quickstarts #121

Open

shawn-hurley closed this as completed Nov 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MacOS]: Performance WindUp (~28 seconds) vs Kantra (~237 seconds) on sample app jakartaee-duke with target jakarta-ee #96

[MacOS]: Performance WindUp (~28 seconds) vs Kantra (~237 seconds) on sample app jakartaee-duke with target jakarta-ee #96

jwmatthews commented Oct 21, 2023

jwmatthews commented Oct 21, 2023

shawn-hurley commented Oct 22, 2023 via email

jwmatthews commented Oct 22, 2023

shawn-hurley commented Oct 23, 2023

pranavgaikwad commented Oct 23, 2023 •

edited

Loading

shawn-hurley commented Oct 23, 2023

pranavgaikwad commented Oct 24, 2023 •

edited

Loading

shawn-hurley commented Oct 25, 2023

jwmatthews commented Nov 7, 2023

shawn-hurley commented Nov 13, 2023

ascheman commented Nov 13, 2023

shawn-hurley commented Nov 14, 2023

shawn-hurley commented Nov 29, 2023

jwmatthews commented Nov 29, 2023

[MacOS]: Performance WindUp (~28 seconds) vs Kantra (~237 seconds) on sample app jakartaee-duke with target jakarta-ee #96

[MacOS]: Performance WindUp (~28 seconds) vs Kantra (~237 seconds) on sample app jakartaee-duke with target jakarta-ee #96

Comments

jwmatthews commented Oct 21, 2023

WindUp

Run #1

Run #2

Kantra

Run #1

Run #2

Notes

jwmatthews commented Oct 21, 2023

Run #1 (assume some time for fetching images as the VM was destroyed and recreated)

Run #2 (in theory images have been pulled this should be a quicker run)

shawn-hurley commented Oct 22, 2023 via email

jwmatthews commented Oct 22, 2023

shawn-hurley commented Oct 23, 2023

pranavgaikwad commented Oct 23, 2023 • edited Loading

shawn-hurley commented Oct 23, 2023

pranavgaikwad commented Oct 24, 2023 • edited Loading

shawn-hurley commented Oct 25, 2023

jwmatthews commented Nov 7, 2023

shawn-hurley commented Nov 13, 2023

ascheman commented Nov 13, 2023

shawn-hurley commented Nov 14, 2023

shawn-hurley commented Nov 29, 2023

jwmatthews commented Nov 29, 2023

pranavgaikwad commented Oct 23, 2023 •

edited

Loading

pranavgaikwad commented Oct 24, 2023 •

edited

Loading