How to generate tags like civitai?

[aisensiy]

This is a question about the tag api when I upload a image. Civitai will automatically add tags for the uploaded images. And if there is some NSFW tags the image will be filtered out based on user options. I think this is a much better rule than the black boxed rule in https://huggingface.co/CompVis/stable-diffusion-safety-checker.

I want to implement a similary api which can add tags for my own images, and do I need to retrain the data myself? (I know a project https://github.com/LAION-AI/CLIP-based-NSFW-Detector). Or if there is some model which is already exists for this function?

I know CLIP can be used for zero-shot classification. But the distance between the text is quite hard to design, and the classification result is not as good as the stable-diffusion-safety-checker.

[aisensiy]

Any one give some help? Thanks a lot.

[aisensiy]

Is anyone can give some tips about this?

[RoccoFortuna]

@aisensiy
My guess is they have a predefined list of tags and they apply tags on images based on cos-similarity (or analogous) between CLIP image embedding and tag. The threshold that defines whether the tag should be applied is a tricky one to set.

[JustMaier]

We'd like to do it this way. If anyone builds this in an open-source way, let me know, we'd love to use it.
In reality we're currently using Amazon Rekognition.

[RoccoFortuna]

I would love to contribute, big fan of this project. I see contributions are appreciated, so I'll look to make some time and work on this. I'll dig deeper on how it's currently being done, do you make use of Amazon Rekognition Custom Labels feature?

[JustMaier]

We aren't using their custom labels feature, just their standard content labeling and moderated content labeling.

[aisensiy]

That will be so great. I want to have a similar model to tag my own images.

[JustMaier]

@RoccoFortuna

do you make use of Amazon Rekognition Custom Labels feature?

No, we haven't. We met with Amazon for the first time this week and mentioned that we'd like to be able to customize our labeling, they're going to get back to us.

[aisensiy]

Thanks sooo much for these replies.

[AddictiveFuture]

I think currently you can create only tags for Models and then use it to tag images. I created tag "visualnovel".