Combine Segment Anything and CLIP and control the generating mask with text has been added!
Segmentation can also be performed from the PBRemTools tab.
However, to use it, the Segment Anything Model must be added manually.
Download the model from here (https://github.com/facebookresearch/segment-anything#model-checkpoints) and place it under "models"(stable-diffusion-webui/extensions/PBRemTools/models).
This program is prepared with reference to (https://github.com/facebookresearch/segment-anything) and (https://github.com/Curt-Park/segment-anything-with-clip)
PBRemTools(Precise background remover tools) is a collection of tools to crop backgrounds from a single picture with high accuracy.
- Base Image
- PBRemTools(Tile division ABG Remover)
- PBRemTools(CascadePSP)
- ABG Remover
- RemBG
This tool is based on Anime Remove Background(https://huggingface.co/spaces/skytnt/anime-remove-background) and ABG_extension(https://github.com/KutsuyaYuki/ABG_extension).
Post-processing is added for more precise cropping based on the mask image generated by Anime Remove Background.
In this post-processing step, the input image is divided into a specified number of tiles, and the pixels in each tile are clustered based on color information.
Extract clusters whose mask image content exceeds a threshold value as foreground.
- horizontal split num: Number of horizontal tile divisions.
- vertical split num: Number of vertical tile segments.
- n_cluster: Number of clusters based on color information.
- alpha threshold: Transparency of the mask considered as foreground.
- mask content ratio: Threshold for how much mask a cluster should contain to be considered foreground.
This tool is based on CascadePSP(https://github.com/hkchengrex/CascadePSP).
- Stable diffusion web ui. Install from webui's Extensions tab.
This program uses code that contains the Apache License 2.0
Currently, there are two APIs available:
- To get the currently available SAM model:
GET http://localhost:7861/pbrem/sam-model - To process an image:
POST http://localhost:7861/pbrem/predict
After launching the web UI API, you can visit the detailed API documentation at http://localhost:7861/redoc
