You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we are overwriting the parameters for the benchmark passed in BenchmarkRunner.run() with the params slot saved on the benchmark. This is not thought through entirely, and needs changing.
Points to consider:
Removeparams on the Benchmark class as a separate slot. All default values can be given either as variable defaults in the benchmark signature, or via params in BenchmarkRunner.run(). (Remove benchmark params slot, type core decorators #15)
We should think of validating the interface typing against the passed parameters, and warn/error on mismatches. (Add Interface-checks #12)
In this sense, we can also warn if the benchmark function interface is partially/entirely untyped. (Debug log prints in _check if a member of any benchmark interface is untyped, i.e. param.annotation == inspect.Parameter.empty.) (Handle untyped interfaces in _check #18)
Save benchmark function interface as a member of the Benchmark class. This should be a nnbench class called Interface etc., which is immutable/frozen, and provides easy access to function interface properties such as varnames (->inspect.signature.parameters.keys()), vartypes (->inspect.signature.parameters.values()), tuples of the two previous (->inspect.signature.parameters.items()), and default values (->p.default for p in inspect.signature.parameters). (Add interface class member to Benchmark class #20)
The text was updated successfully, but these errors were encountered:
Currently we are overwriting the parameters for the benchmark passed in
BenchmarkRunner.run()
with theparams
slot saved on the benchmark. This is not thought through entirely, and needs changing.Points to consider:
params
on the Benchmark class as a separate slot. All default values can be given either as variable defaults in the benchmark signature, or viaparams
inBenchmarkRunner.run()
. (Remove benchmarkparams
slot, type core decorators #15)_check
if a member of any benchmark interface is untyped, i.e.param.annotation == inspect.Parameter.empty
.) (Handle untyped interfaces in_check
#18)Benchmark
class. This should be a nnbench class calledInterface
etc., which is immutable/frozen, and provides easy access to function interface properties such as varnames (->inspect.signature.parameters.keys()
), vartypes (->inspect.signature.parameters.values()
), tuples of the two previous (->inspect.signature.parameters.items()
), and default values (->p.default for p in inspect.signature.parameters
). (Add interface class member to Benchmark class #20)The text was updated successfully, but these errors were encountered: