Add announcement for metrics initiative

[yaahc]

Related: rust-lang/rust#128914

Rendered

[jieyouxu]

Discussion: would we want to explicitly mention here that the initiative fully intends to respect user privacy (i.e. no automated uploads / networking)

NO TELEMETRY, NO NETWORK CONNECTIONS

as mentioned in the tracking issue?

[yaahc]

My preference is to direct people to the tracking issue and keep the blog post as concise as possible.

[Noratrieb]

I think it's incredibly important to mention this in the first sentence in the blog post. If we don't say this clearly, people will get the wrong impression. Most people won't care enough to read the tracking issue or anything other than the blog post, and they shouldn't get wrong impressions from it.

[Noratrieb]

For example,

to announce the start of the Metrics initiative, a privacy-respecting way to get user feedback.

"metrics" sounds very ambiguous

[yaahc]

I've applied the suggested wording from @Kobzol below which I believe resolves this concern

[Kobzol]

I'm very much looking forward to the initiative and getting more information about the usage of Rust in the wild, however I think that this thing should really not be presented in a concise manner, otherwise it might generate a lot of questions and confusion; we already saw that happen with Go and other open-source projects that tried to introduce telemetry. If we present this initiative to the world, we should already have prepared very explicit answers to questions like "Is this opt-in or opt-out?", "What information exactly will be recorded?", "How will the data get to Rust maintainers?", "Who will be the gatekeeper of the data and how it will be secured? Will it be in the hands of the Rust Foundation?", "What about GDPR?", "Will my IP address be stored?" etc.

I assume that we don't yet know all the answers to these questions. I wonder if it might be a better idea to postpone the official announcement (which a blog post on the official Rust blog very much is, even though it's "just" Inside Rust) about the initiative until we have this figured out. I know that you probably meant this blog post to be just a brief announcement to get some ideas from people, but this is a very delicate topic and based on historical experience, I'm pretty sure that many people would take this blog post as a "done deal", and start asking questions and possibly be infuriated.

If you want to present the initiative right now, I think that we should at least add more details to the blog post. This is how the Go telemetry initiative was presented. It contained a lot of technical details about the collection of data, and also the justifications and motivation for the feature, and it still generated a lot of heat. Something like this is missing from the blog post, I would add a link to https://estebank.github.io/rustc-metrics.html at the very least.

I don't want this post to look like "We are going to gather some metrics from you. Btw, we don't know how will it look like yet, if you want to find out more, check this tracking issue with a bunch of links.". That wouldn't be a good impression.

[jieyouxu]

If you want to present the initiative right now, I think that we should at least add more details to the blog post. This is how the Go telemetry initiative was presented. It contained a lot of technical details about the collection of data, and also the justifications and motivation for the feature, and it still generated a lot of heat. Something like this is missing from the blog post, I would add a link to estebank.github.io/rustc-metrics.html at the very least.

I don't want this post to look like "We are going to gather some metrics from you. Btw, we don't know how will it look like yet, if you want to find out more, check this tracking issue with a bunch of links.". That wouldn't be a good impression.

Good points. I appreciate that this initiative is very new and much details or even key ideas have not been figured out yet, but I feel like we should at least have some concrete principles that can be perhaps presented in some kind of Q&A format that's easy to digest, something like:

---

What is the Metrics Initiative?

• Opt-in, privacy-preserving infrastructure to provide anonymized data to aid Rust project contributors in analyzing usage patterns, pain points, errors, internal compiler errors, etc.

What the Metrics Initiative is not

• Auto-upload of user data
• Requires internet connection

---

or something to that effect.

[jieyouxu]

A TL;DR could be super helpful to mitigate confusion as well.

[jieyouxu]

Suggestion: we probably want to make clear that we actively want and appreciate users to register their concerns, feedback and suggestions, and not just what the metrics infra could be used for.

[Kobzol]

Thank you for making these changes! I still think that a big loud TLDR or FAQ should be present at least at the top of the tracking issue, if not directly in the blog post, but now at least the most pressing questions (opt-in, privacy-preserving, manual upload) are at least briefly answered at the beginning of the post.

[yaahc]

I assume that we don't yet know all the answers to these questions. I wonder if it might be a better idea to postpone the official announcement (which a blog post on the official Rust blog very much is, even though it's "just" Inside Rust) about the initiative until we have this figured out. I know that you probably meant this blog post to be just a brief announcement to get some ideas from people, but this is a very delicate topic and based on historical experience, I'm pretty sure that many people would take this blog post as a "done deal", and start asking questions and possibly be infuriated.

I'm not against postponing the blog post. My goal here is to communicate with Rust Project Contributors, not users. Public communication is neither an expertise nor a passion of mine.

Just to give you an idea of where my head is at here are some notes I took when planning how to announce the metrics initiative. I'm open to alternatives to a blog post but would like to inform contributors to the project as well as I possibly can.

---

• Communicating in Open Source

  • Questions

    • How do we reliably inform project members of relevant work within the project?
    • How do we test the reliability of existing communication tools?
      • What qualifies as effective communication (how do we quantify this?), what percent of project members do we expect to reach?
        • I want to hit all of the active team members

  • Current Tools

    • Async
      • Issues
        • reach:
        • open conversations
        • topic oriented
      • Blog announcements
        • reach: this-week-in-rust readers, rss subscribers
        • Open announcements (e.g. seen by users)
        • Out of band notification strategies
        • how many people in the project actually see these?
      • all@
        • reach: all project members
        • Closed announcements
        • often goes into spam
      • x.py announcements
        • reach: contributors
        • used for communicating changes in the rust repo tooling and config
    • Sync
      • team meetings
        • reach: majority members of the team, small subset of contributors
        • personal feelings poor communication between teams, no officially supported structures to propagate information across the project teams other than individuals reaching out to every team personally
    • Both
      • Zulip
        • reach: subscribed users per stream (plus or minus users who mute streams)
        • long running conversations
        • topic oriented
        • more casual than github issues
        • also heavily used for DMs
        • heavily fragmented
          • personal feelings I find it really annoying having to create a bunch of identical topics in a ton of different streams

  • Motivation

    • I want to inform project members of the metrics initiative so that they can begin to leverage it and we can build the infrastructure around real world needs
    • I want to try leveraging the existing tools and somehow test how effectively

  • Progress

    • DONE Zulip council
    • DONE zulip per team
    • DONE Github
    • TODO Blog
    • TODO meetings
    • TODO test effectiveness 2024/08/19

[Kobzol]

I'm not against postponing the blog post. My goal here is to communicate with Rust Project Contributors, not users. Public communication is neither an expertise nor a passion of mine.

I see. In that case, I would probably suggest to indeed postpone this blog post. As I stated before, this is a very delicate matter and it should be communicated carefully, to avoid misunderstanding and drama.

The problem with the Inside Rust blog is that it's not really a blog aimed at Rust contributors. In fact, I would claim that to most Rust users, it's exactly the same as the main Rust blog. People don't go to the blog by typing its address in the browser, in the majority of cases they are redirected to specific blog posts from RSS readers, Reddit, This Week in Rust, Twitter etc. When they click the link, they arrive at a website that looks like the Rust blog, and they start reading. I doubt that many people even recognize or care about which one of those two blogs it is; as long as a post appears on our website, it's official, and people will react to it with that in mind. That is why I want to be very careful about posting about such a delicate topic with a post that is too brief. If your goal is to reach Rust contributors, rather than Rust users, then I would suggest to use a different medium and postpone a blog post such as this one for a time when we have more answers prepared. Otherwise we risk that an innocent post aimed at Rust contributors will get interpreted as an official message towards Rust users, that might be however seen as inadequate, given the amount of backlash any form of telemetry usually produces.

Now, on the topic of how to inform a lot of Rust contributors, that's indeed not easy. But I think that the existing channels, such as Zulip, or if needed the @all address, should be enough in this case. I'm not sure if most Rust contributors even read all posts on the Inside Rust blog. Zulip seems like the best bet to me. We can also talk with the council to try to nominate this topic for meetings of individual teams.

[yaahc]

We can also talk with the council to try to nominate this topic for meetings of individual teams.

This was one of the first things I asked. While I only talked to one council member directly about this, they didn't feel like the council was well-positioned to do this or that it should be the council's responsibility, but rather that the council should support another team or group of people who handle this sort of cross-team communication.

For now I agree I should at least start with an all@ message, I'm leaning towards just sending the content of the blog post as is with the hope that the concise communication and directing people to the tracking issue will be appropriate for that venue. Then I'll see if I can somehow evaluate how many people have heard about the metrics initiative at that point and if it feels like there are still major gaps then I'll look into advocating more strongly for proactive cross team communication from the council and ask @eholk to advocate for it as the compiler team rep.

[Kobzol]

Thanks, I think that's a good idea. While posting also to Inside Rust could help you get some more contributor eyes on the topic, I don't think it's worth it in this case because of the potential public backlash.

I'd mark this PR as a draft, just to avoid any surprises :)