The Model Openness Tool (MOT) is designed to facilitate the evaluation and classification of machine learning models based on the Model Openness Framework (MOF). This tool provides a comprehensive platform for model producers to assess their models against the 16 components of the MOF, ensuring transparency, reproducibility, and usability. MOT not only evaluates the openness of the license for each component but also ranks the models, helping the community identify models that adhere to the principles of open science. Features
- Model Evaluation: Assess machine learning models against the MOF's 16 components.
- License Evaluation: Analyze and validate the openness of licenses used for each model component.
- Model Submission: Enable producers to submit their models for classification and listing.
- Ranking and Listing: Display all submitted models along with their rankings and adherence to each MOF class.
- Prepare your model's artifacts according to the guidelines specified in the MOF.
- Submit your model through the 'Evaluate Model' function of the MOT interface, providing details and licenses for each component.
- Receive feedback on the classification and suggestions for achieving higher openness levels.
- At the end of the evaluation you have the option to submit the model for addition to the list of models displayed on the MOT.
While it is recommended to start with the 'Evaluate Model' function of the MOT described above, you can use the "Submit Model' function to submit your model directly.
Access the MOT's web interface to view a detailed list of all models submitted, their openness classification, and how they rank against each MOF class.
The MOT's web interface only allows user to modify model records they created. However, anyone can submit changes to other records through GitHub. New models can also be submitted for addition this way. For details on how to do that see the CONTRIBUTING.md file.
Contributions to the Model Openness Tool are welcome! To contribute to the software or the models, please see the CONTRIBUTING.md file.
The Model Openness Tool is open-sourced under the MIT license. See the LICENSE file for more details.
For support, feature requests, or bug reports, please file an issue through the GitHub issue tracker associated with this repository.
This tool was developed in collaboration with the authors of the Model Openness Framework and the Linux Foundation. Special thanks to Matt White, Ibrahim Haddad, Cailean Osborne, Xiao-Yang Liu, Ahmed Abdelmonsef, Sachin Mathew Varghese, and Arnaud J Le Hors for their invaluable input and guidance.