Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Inception Score #82

Closed
wants to merge 3 commits into from
Closed

Implement Inception Score #82

wants to merge 3 commits into from

Conversation

GiangHLe
Copy link

Hi,
Recently, there is some research still evaluating their model by Inception Score, and issue #20 also mentions it. I create a pull request to add the implementation of computing the inception score.

Change

  • implement the calculate_inception_score . In this implementation, I used the weight of Inception-V3 from the Pytorch team to predict the probs of images (to validate my implementation in the test phase).
  • add --inception-score as store_true argument and --splits with default is 10 (just like the original code).

Test

I tested it with CIFAR-10 (pytorch-fid/src/pytorch-fid/test.py), in this paper, the mean and std of CIFAR-10 are 9.737 and 0.1489. In this PR, the test result return inception score 9.6728±0.1499. (the difference may come from Torch or NumPy version).

Problems

  • While testing, I found out that IS is really hard to use, not only because of the issues from the paper above. Because the predicted results then have to be split into groups, the arrangement of data is also a problem (if shuffle=True in dataloader, the result will be different).
  • Different from FID, IS is computed individually for each dataset, so the input only needs one path. The function is taking the first path as input and ignoring the second, I wonder is there any more flexible way to fix this.

@GiangHLe GiangHLe closed this Nov 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant