MLPerf Inference v1.1

GitHub Submission HOWTO

Clone the MLPerf Inference v1.1 submission tree

Clone the submission tree e.g. under your home directory:

$ export SUBMISSION_ROOT=$HOME/submissions_inference_1_1
$ git clone [email protected]:mlcommons/submissions_inference_1_1.git $SUBMISSION_ROOT
$ cd $SUBMISSION_ROOT

Create a branch

We recommend creating a new branch for every logically connected group of results e.g. all results from your System-Under-Test (SUT) or only relating to a particular benchmark. Prefix your branch name with your organization's name. Feel free to include the SUT name, implementation name, benchmark name, etc.

For example:

$ git checkout master && git pull
$ git checkout -b dividiti-closed-aws-g4dn.4xlarge-openvino

Populate your branch according to the submission rules.

You can inspect your changes:

$ git status
On branch dividiti-closed-aws-g4dn.4xlarge-openvino
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        closed/dividiti/code/
        closed/dividiti/compliance/
        closed/dividiti/measurements/
        closed/dividiti/results/
        closed/dividiti/systems/

nothing added to commit but untracked files present (use "git add" to track)

and make intermediate commits as usual:

$ git add closed/dividiti
$ git commit -m "Dump repo:mlperf-closed-aws-g4dn.4xlarge-openvino."

Run the submission checker

Once you are happy with the tree structure, truncate the accuracy logs and run the submission checker, culminating in e.g.:

      INFO:main:Results=2, NoResults=0
      INFO:main:SUMMARY: submission looks OK

Push the changes

Once you and the submission checker are happy with the tree structure, you can push the changes:

$ git push

fatal: The current branch dividiti-closed-aws-g4dn.4xlarge-openvino has no upstream branch.
To push the current branch and set the remote as upstream, use

    git push --set-upstream origin dividiti-closed-aws-g4dn.4xlarge-openvino

Do exactly as suggested:

$ git push --set-upstream origin dividiti-closed-aws-g4dn.4xlarge-openvino

Create a pull request

If you now go to https://github.com/mlcommons/submissions_inference_1_1/, you should see a notification about your branch being recently pushed and can immediately create a pull request (PR). You can also select your branch from the dropdown menu under <> Code. (Aren't you happy you prefixed your branch's name with the submitter's name?)

As usual, you can continue committing to the branch until the PR is merged, with any changes being reflected in the PR.