rtdl: Return null when non-elf file is dlopened

[Qwinci]

Right now it just fails at an assert in _fetchFromFile, it should just return null according to the man page.

On success, dlopen() and dlmopen() return a non-NULL handle for
the loaded object. On error (file could not be found, was not
readable, had the wrong format, or caused errors during loading),
these functions return NULL.

[no92]

LGTM

[Dennisbonke]

LGTM

[qookei]

Why not make _fetchFromFile return some value to indicate failure instead of duplicating the checks

[Qwinci]

Why not make _fetchFromFile return some value to indicate failure instead of duplicating the checks

Actually thats a better idea, Ill do that

[Geertiebear]

Wouldn't it be a lot cleaner to use frg::expected in this case?

[Qwinci]

Wouldn't it be a lot cleaner to use frg::expected in this case?

Instead of the bools? Not sure how would that make it cleaner though.

[Geertiebear]

Instead of lastError being a global accross files it could stay private to main.cpp. The functions can then return a frg::expected<void, LinkerError> where LinkerError is some enum. This way we keep the error handling out of linker.cpp.

[Qwinci]

Fixed and also added more checks to the elf to make sure that it also has the correct machine and bitness.

[qookei]

Looks good apart from the comments.

[no92]

LGTM