UC: Discourage Transparency

[jawache]

You receive a low rating compared to your competitor and don't want to disclose since no rating is better than a worse rating.

With a percentile based rating system a lack of disclosure is deeply problematic, since only through disclosure and transparency are the boundaries between A - D set.

Equivalence

• Taking CDP as an example, the ratings are private and not disclosed - only the A lists are disclosed, it's used more as a signal to organizations the work they need to do in order to receive and A rating.

Counter

• Remove ratings and focus on just being transparent regarding the score, leave it to the user to decide if the trade-offs are worth it. For example, food has had labelling for decades prior to nutri-score, we read the labels and decided if the product was healthy.
• Make it a regulatory requirements to disclose.

[chrisxie-fw]

1. Percentile based system is a default implementation of the base class (SCER base standard framework), unless the derived class (implementation of the SCER framework) defines a different rating algorithm and rating range.
2. If SCER is used internally by an organization, the org decides to disclose it or not publicly. If the SCER is used externally, or in a public facing service, it's the service owner's decision to disclose the rating.

By the way, the detailed disclosure of how the rating was calculated is a requirement, which is specified in the base SCER specification in the "User Access and Transparency" part under the Visualization and Labeling section:

"Ensure that the labels are easily accessible and understandable to users, providing detailed explanations of the ratings through tooltips or supplementary guides."

Here is the sample illustration of the rating label with the absolute and relative ratings along with QR code:

[chrisxie-fw]

Another way to understand this "UC" is that: should SCER remove the rating portion of it because it could cause potential "tension" by the LLM providers? And leave the rating part to the end users? Think youtube videos have the number of views, and the products sold on Amazon has the rating scores by the customers.