[Explained] 7.16.3 CPU benchmarks are incorrect

[original-birdman]

Describe the bug
The CPU benchmark reported by 7.16.3 are significantly lower figures than 7.14.2

Steps To Reproduce

1. Start 7.16.3
2. Run CPU benchmarks (look in log)
3. Start 7.14.2
4. Run CPU benchmarks (look in log)

Expected behavior
I'd expect the CPU benchmarks to be essentially the same between releases.

Screenshots

Start-up with 7.16.3:

Oct 26 12:01:49 benuc boinc[16216]: 26-Oct-2019 12:01:49 [---] Starting BOINC client version 7.16.3 for x86_64-pc-linux-gnu
   ...
Oct 26 12:01:49 benuc boinc[16216]: 26-Oct-2019 12:01:49 [---] Processor: 4 GenuineIntel Intel(R) Core(TM) i3-8109U CPU @ 3.00GHz [Family 6 Model 142 Stepping 10]
   ...
Oct 26 12:01:49 benuc boinc[16216]: 26-Oct-2019 12:01:49 [---] OS: Linux Ubuntu: Ubuntu 19.10 [5.3.0-19-generic|libc 2.30 (Ubuntu GLIBC 2.30-0ubuntu2)]
   ...
Oct 26 12:01:49 benuc boinc[16216]: 26-Oct-2019 12:01:49 [---] Running CPU benchmarks
Oct 26 12:01:49 benuc boinc[16216]: 26-Oct-2019 12:01:49 [---] Suspending computation - CPU benchmarks in progress
Oct 26 12:02:20 benuc boinc[16216]: 26-Oct-2019 12:02:20 [---] Benchmark results:
Oct 26 12:02:20 benuc boinc[16216]: 26-Oct-2019 12:02:20 [---]    Number of CPUs: 2
Oct 26 12:02:20 benuc boinc[16216]: 26-Oct-2019 12:02:20 [---]    3307 floating point MIPS (Whetstone) per CPU
Oct 26 12:02:20 benuc boinc[16216]: 26-Oct-2019 12:02:20 [---]    9960 integer MIPS (Dhrystone) per CPU
==========

Start-up with 7.14.2:

Oct 26 12:18:35 benuc boinc[17200]: 26-Oct-2019 12:18:35 [---] Starting BOINC client version 7.14.2 for x86_64-pc-linux-gnu
   ...
Oct 26 12:18:35 benuc boinc[17200]: 26-Oct-2019 12:18:35 [---] Processor: 4 GenuineIntel Intel(R) Core(TM) i3-8109U CPU @ 3.00GHz [Family 6 Model 142 Stepping 10]
   ...
Oct 26 12:18:35 benuc boinc[17200]: 26-Oct-2019 12:18:35 [---] OS: Linux Ubuntu: Ubuntu 19.10 [5.3.0-19-generic|libc 2.30 (Ubuntu GLIBC 2.30-0ubuntu2)]
   ...
Oct 26 12:18:35 benuc boinc[17200]: 26-Oct-2019 12:18:35 [---] Running CPU benchmarks
Oct 26 12:18:35 benuc boinc[17200]: 26-Oct-2019 12:18:35 [---] Suspending computation - CPU benchmarks in progress
Oct 26 12:19:06 benuc boinc[17200]: 26-Oct-2019 12:19:06 [---] Benchmark results:
Oct 26 12:19:06 benuc boinc[17200]: 26-Oct-2019 12:19:06 [---]    Number of CPUs: 2
Oct 26 12:19:06 benuc boinc[17200]: 26-Oct-2019 12:19:06 [---]    5659 floating point MIPS (Whetstone) per CPU
Oct 26 12:19:06 benuc boinc[17200]: 26-Oct-2019 12:19:06 [---]    26590 integer MIPS (Dhrystone) per CPU
==========

System Information

• OS: Kubuntuu Linux 19.10
• BOINC Version: 7.16.3 (and 7.14.2)

Additional context
In the past updates to BOINC have resulted in similar CPU benchmarks on a system.

[AenBleidd]

@original-birdman, if you manually rerun Benchmarks, does it show similar values?

[original-birdman]

Yes.
If I click on Tools>Run CPU benchmarks it produces the same values as it does when running a calculation because the version has changed.

[original-birdman]

OK. I now know what is happening and hence have "fixed" it.
Although I'm not sure that the result, or rather the reason, is a good one.

I build my own BOINC code (so that I can have the same version across OS distributions, and also because several years ago I needed(wanted) a feature that was in the latest code, but not the standard OS version).

My 7.14.2 configure script included CXXFLAGS="-O3".

My 7.16.3 configure script included various options I found on the web site (--disable-silent-rules --enable-dynamic-client-linkage ...) and I omitted the -O3 option.

I've just rebuilt with the -O3 compilation option in place and:

• The executables and libraries are significantly smaller.
• The benchmarks are now back to what I normally see.

Thu 31 Oct 2019 11:34:12 GMT |  | Version change (7.14.2 -> 7.16.3)
...
Thu 31 Oct 2019 11:34:12 GMT |  | Running CPU benchmarks
Thu 31 Oct 2019 11:34:12 GMT |  | Suspending computation - CPU benchmarks in progress
Thu 31 Oct 2019 11:34:43 GMT |  | Benchmark results:
Thu 31 Oct 2019 11:34:43 GMT |  | Number of CPUs: 2
Thu 31 Oct 2019 11:34:43 GMT |  | 5710 floating point MIPS (Whetstone) per CPU
Thu 31 Oct 2019 11:34:43 GMT |  | 26614 integer MIPS (Dhrystone) per CPU

Whether CPU benchmarks should be so dependent on the compilation options of the Manager code is something to wonder about...

[Aurum420]

Was the CPU Benchmark always wrong or did it just break going from Linux 7.9.3 to 7.16.6??? Examples show before upgrading from Linux Mint 19.3 (Ubuntu 18.04) BOINC 7.8.3 to Linux Mint 20 (Ubuntu 20.04) BOINC 7.16.6,:

CPU	FP MIPS Whetstone Before	FP MIPS Whetstone After	Integer MIPS Drystone Before	Integer MIPS Drystone After
Xeon E5-2683 v3	3101	3625	78603	43826
Core i7-5960X	4130	4782	103331	58137
Xeon E5-2699 v4	4046	4759	86893	49012
Core i9-9960X	5153	5913	121251	71281
Xeon E5-2673 v4	3315	3912	72160	41076

[AenBleidd]

Table above updated to be more pretty

[original-birdman]

If you read my comment from 31 Oct 2019 you'll see that it wasn't anything to do with the BOINC version (I didn't change the OS version).

Rather it appears to be very dependent on the optimization setting used in the compilation stage.

[Aurum420]

Maybe a new Issue needs to be started. I do not compile my own BOINC client as I indicated. I was trying to use the CPU BM to decide which CPUs work on which projects. But when the results are trash.

[AenBleidd]

This was compiled by package distributor. It's better to ask on their forum somewhere did they do any changes in compilation flag that could potentially lead to this issue. We have no our own official package binary for linux

[original-birdman]

It would, however, make more sense for BOINC to have a benchmarking method which was not so dependant on the compilation options used. It's meant to be benchmarking the processor, not the compiler.

[RichardHaselgrove]

Whetstone was originally intended to be used with the deliberate exclusion of compiler optimisation, to provide an industry-wide comparison without cheating. Our needs are different, and it's stretching a point to go on calling it 'Whetstone'.

Reference: http://www.roylongbottom.org.uk/whetstone.htm#anchorOpt

[AenBleidd]

Converting this to discussion