Realm: core->get_resource("cpu", ...) returns 0 on ARM

[manopapad]

Apparently cat /proc/cpuinfo has no physical id and cpu cores, which is what Realm checks.

Note that this works fine on Apple Silicon.

[eddy16112]

Should be fixed by https://gitlab.com/StanfordLegion/legion/-/merge_requests/1511. The reason it works on Mac is because it provides its own API sysctlbyname for querying resources.

[muraj]

@manopapad What ARM system are you running on that doesn't have physical id or cpu cores?

We should really just move everything over to hwloc...

[eddy16112]

@muraj Do you think if we can assume hwloc is always available if we move to it? I just do not want to introduce an extra dependency with a fallback path.

[muraj]

we can't assume hwloc is always available I don't think. It's not default available on all distros.

[eddy16112]

hwloc supports all systems people usually use, I think we can make it a must have dependency.

[muraj]

Not without discussing it with the general community. We can ask in the next legion meeting. In the meantime -- @manopapad can you compile and run the following snippet on your target system and verify if it returns the correct number of processors?

https://godbolt.org/z/1vTdGxq94

it would return everything that reading /proc/cpuinfo and grepping for processors would return.

[eddy16112]

Not without discussing it with the general community.

Agree

can you compile and run the following snippet on your target system and verify if it returns the correct number of processors?

It returns 72, which is the number of cpu cores on that ARM machine.

[muraj]

Then instead of parsing /proc/cpuinfo, we should use sysconf() instead. We'll still need /proc/cpuinfo for parsing out which cores are physical and which are hyper-threads, but the total cores should probably come from this sysconf instead.

[eddy16112]

I think people usually do not interested in hyper-threads. We can use sysconf to read the total cores, and if we can parse /proc/cpuinfo correctly, we will return the total physical cores, otherwise, we will just return the value of sysconf.

[manopapad]

Fix is WIP on https://gitlab.com/StanfordLegion/legion/-/merge_requests/1511

[manopapad]

This work now, but still sometimes runs into trouble:

[0 - fffde36709c0] 0.000292 {5}{numa}: can't read '/sys/devices/system/node/node-1/distance': No such file or directory

Hopefully hwloc would be able to figure things out even in this case.

[eddy16112]

This error comes from https://gitlab.com/StanfordLegion/legion/-/blob/master/runtime/realm/numa/numasysif.cc#L329. Before adding the hwloc, I will need to test it to make sure hwloc support this linux.