Possible irregulartity in memory consumption in simple matrix-matrix multiplictation

[alejandrogallo]

Hello Edgar!

We profiled the simple matrix multiplication

(*Cab)["ab"] = (*Aac)["ac"] * (*Bcb)["cb"];

on a 48 core machine. For different matrix dimensions, we found the
following operation speed and memory consumption:

#	a	b	c	mem[storage]	mem[HWM]	flops[GFLOP/s/core]
1	30000	30000	6000	10.27	14.03	49.67
2	30000	30000	8000	11.16	17.93	50.55
3	30000	30000	10000	12.06	64.74	32.41
4	30000	30000	15000	14.30	69.48	39.01
5	30000	30000	22500	17.64	76.45	44.04
6	30000	30000	24000	18.32	77.84	43.53
7	30000	30000	27000	19.63	80.63	44.49
8	30000	30000	30000	21.00	83.48	46.05
9	30000	30000	31000	21.45	40.14	50.61
10	30000	30000	33000	22.34	41.41	46.91
11	30000	30000	36000	23.68	43.51	50.11
12	30000	30000	37500	24.35	44.50	51.48
13	30000	30000	45000	27.71	49.47	50.06
14	30000	30000	52500	31.06	54.49	50.44
15	30000	30000	60000	34.41	59.52	49.66

We find an irregularity in the FLOP/s/core ranging for
c ∈ [8000, 10000], i.e., row 2 to 3 in the table above.
There, the maximal memory usage (HWM) increases drastically.

A further memory irregularity we detect for  c ∈ [30000, 31000], i.e.,
from row 8 to row 9;
The FLOP/s/core count is roughly the same with only 50% of the maximal
memory consumption.

Are these findings reproducible and are the effects understandable?

Thanks!