Skip to content

lfai/model_openness_tool

Repository files navigation

Model Openness Tool

The Model Openness Tool (MOT) is designed to facilitate the evaluation and classification of machine learning models based on the Model Openness Framework (MOF). This tool provides a comprehensive platform for model producers to assess their models against the 16 components of the MOF, ensuring transparency, reproducibility, and usability. MOT not only evaluates the openness of the license for each component but also ranks the models, helping the community identify models that adhere to the principles of open science. Features

  • Model Evaluation: Assess machine learning models against the MOF's 16 components.
  • License Evaluation: Analyze and validate the openness of licenses used for each model component.
  • Model Submission: Enable producers to submit their models for classification and listing.
  • Ranking and Listing: Display all submitted models along with their rankings and adherence to each MOF class.

Evaluating a Model

  • Prepare your model's artifacts according to the guidelines specified in the MOF.
  • Submit your model through the 'Evaluate Model' function of the MOT interface, providing details and licenses for each component.
  • Receive feedback on the classification and suggestions for achieving higher openness levels.
  • At the end of the evaluation you have the option to submit the model for addition to the list of models displayed on the MOT.

Submitting a Model

While it is recommended to start with the 'Evaluate Model' function of the MOT described above, you can use the "Submit Model' function to submit your model directly.

Viewing Model Rankings

Access the MOT's web interface to view a detailed list of all models submitted, their openness classification, and how they rank against each MOF class.

Contributing new models or changes to existing models using GitHub

The MOT's web interface only allows user to modify model records they created. However, anyone can submit changes to other records through GitHub. New models can also be submitted for addition this way. For details on how to do that see the CONTRIBUTING.md file.

Contributing to the MOT

Contributions to the Model Openness Tool are welcome! To contribute to the software or the models, please see the CONTRIBUTING.md file.

License

The Model Openness Tool is open-sourced under the MIT license. See the LICENSE file for more details.

Support

For support, feature requests, or bug reports, please file an issue through the GitHub issue tracker associated with this repository.

Acknowledgements

This tool was developed in collaboration with the authors of the Model Openness Framework and the Linux Foundation. Special thanks to Matt White, Ibrahim Haddad, Cailean Osborne, Xiao-Yang Liu, Ahmed Abdelmonsef, Sachin Mathew Varghese, and Arnaud J Le Hors for their invaluable input and guidance.