Merged model relationships?

[Jonseed]

I wonder if it is possible to tell from a model itself how closely related it is to other models. Can people do a merge of their favorite model with another, at like 99% weighted sum, so it generates a whole new hash, and then upload it to Civitai and call it their own? Is that preventable somehow, or at least will show that 99% of the model is similar to another model already on Civitai? Is there a way to compare models, essentially, in their internal weightings, to see how similar/different they are from each other?

[JustMaier]

There isn't a way to know how much a model has changed or what's in a merge in any verifiable way at this time. So yes, you're scenario of someone using 99% of a model is totally possible.

[Jonseed]

@JustMaier I asked ChatGPT, and it sounds like there is a way to compare the weights between two models to see how similar they are. It recommended two methods, MSE and MAE. It sounds like MAE might give a better general idea of how similar the weights are overall:

Yes, Mean Absolute Error (MAE) can be used to get a general sense of how different the weights are between two model files. By calculating the average of the absolute differences between the weights of the two models, you can obtain a value that represents the overall difference between the weights, regardless of their magnitude.

You can then use this value to determine the percentage of similarity between the two models by taking the ratio of the sum of the absolute weights difference to the total weights of the model.

For example, if the MAE between the weights of two models is 0.1, and the total weights of the model is 1, the percentage of similarity is (1 - 0.1) * 100 = 90%

MAE is a good method for this kind of comparison because it gives equal weight to all differences, so it gives you a general sense of how different the weights are overall. This is useful when you want a quick and easy way to compare two models and get a sense of how similar or different they are.

I wonder if this method could be used to see how similar or different two model files are. I tried using ChatGPT to whip up some Python code to check two ckpt files using the MAE method, but my programming skills are insufficient to know how these ckpt files are structured, and how to extract the data for comparison. It recommended extracting the state_dict key and then use this code to compare the weights:

# Compare the weights
mae = torch.nn.L1Loss()
loss = mae(weights1, weights2)

# Determine the percentage of similarity
similarity = (1 - loss.item()) * 100
print("Percentage of similarity between the two models: {:.2f}%".format(similarity))

[TheBarret]

MSE is an image comparison method, not sure how you would apply that on a multi layer model?!

[Jonseed]

@JustMaier Here's a Python tool I just discovered which claims to do the kind of thing I was talking about, compare the similarity of models by percentage:

https://huggingface.co/JosephusCheung/ASimilarityCalculatior

Civitai could use this tool (or something similar) to help users see how models are related to each other, or show users which models are most similar to any particular model.

For example, someone on Reddit used this script to get the similarity of all models in a folder compared to a "base model":

https://www.reddit.com/r/StableDiffusion/comments/10kuzmh/how_to_extract_small_lora_file_from_custom/j8350uc?utm_medium=android_app&utm_source=share&context=3