kpredict: evaluation, scripts and API

[acherm]

Hi,

related to #17...
The goal is to program a Python script that would assess kpredict accuracy over the data we have and using the Web API.

By accuracy, I mean the difference between the predicted size and the actual size.
We can compute such differences for many configurations...
See this metric: https://en.wikipedia.org/wiki/Mean_absolute_percentage_error

Right now, it's possible to do something like

wget https://tuxmlweb.istic.univ-rennes1.fr/data/configuration/167950/config; mv config .config; kpredict .config

wget https://tuxmlweb.istic.univ-rennes1.fr/data/configuration/167950/config; mv config .config; kpredict .config
--2021-02-17 11:53:08--  https://tuxmlweb.istic.univ-rennes1.fr/data/configuration/167950/config
Resolving tuxmlweb.istic.univ-rennes1.fr (tuxmlweb.istic.univ-rennes1.fr)... 148.60.11.207
Connecting to tuxmlweb.istic.univ-rennes1.fr (tuxmlweb.istic.univ-rennes1.fr)|148.60.11.207|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 134407 (131K) [application/octet-stream]
Saving to: ‘config’

config                                                                                                                100%[=========================================================================================================================================================================================================================================================================================================================>] 131.26K   653KB/s    in 0.2s    

2021-02-17 11:53:09 (653 KB/s) - ‘config’ saved [134407/134407]

Predicted size : 68.1MiB

and kpredict returns a number (68.1).

What's missing to realize the evaluation scenario is to:

• do it over several configurations: here a challenge would be to request only configurations with kernel versions AND compiler version supported by kpredict (see https://pypi.org/project/kpredict/ for the details)
• get the actual size out of a configuration id

So basically there are two things to do:

• write the whole script (in Python) by calling the API and process the data
• modify a bit the API to ease/enable the requests we want

How does it sound?