API for capturing all screens

[shangl]

こんにちは TAG-さん!

I'm requesting a TAG review of "capture all screens" (getAllScreensMedia).

In some scenarios (e.g. legal compliance or training purposes) it must be ensured that all screens attached to a client are captured. With getDisplayMedia, a web application would need to call the API N-times, asking the user to pick a different screen each time. This is a cumbersome and error-prone process and it can't be guaranteed that all the screens are captured. We propose the getAllScreensMedia API that allows capturing of all screens without user interaction. To protect the users privacy, the API is safe-guarded by an enterprise policy and the user must be notified before capture can happen and while capturing is happening.

• Explainer: Url
• Specification URL: Spec Url, Code
• Tests: No WPT as a policy needs to be set. Other tests
• User research: Developer feedback
• Security and Privacy self-review: TBD (will be posted in a comment below)
• GitHub repo: Url
• Primary contacts (and their relationship to the specification):
  • Simon Hangl (@shangl, Google)
  • Elad Alon (@eladalon1983, Google)
• Organization(s)/project(s) driving the specification: Google
• Key pieces of existing multi-stakeholder review or discussion of this specification: Developer feedback, SCCG presentation, SCCG meeting minutes (SCCG)
• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status): Chrome status

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• Relevant time constraints or deadlines: N/A
• The group where the work on this specification is currently being done: SCCG
• The group where standardization of this work is intended to be done: N/A
• Major unresolved issues with or opposition to this specification: N/A
• This work is being funded by: Google

We'd prefer the TAG provide feedback as (please delete all but the desired option):

💬 leave review feedback as a comment in this issue and @-notify @shangl

[cynthia]

Why does this need to be provided as a web-facing API? All three provided use-cases could likely be done with closed (e.g. extension specific) APIs and you could deploy that as an enterprise policy, which would alleviate the risks of abuese by having it exposed on the web.

[eladalon1983]

Why does this need to be provided as a web-facing API? All three provided use-cases could likely be done with closed (e.g. extension specific) APIs and you could deploy that as an enterprise policy, which would alleviate the risks of abuese by having it exposed on the web.

An extension API requires the installation of an extension, which implicitly grants permission to the extension to access a myriad other APIs that the admin might not even know about, let alone wish to allow. In some regards the extension path is actually riskier than a Web-exposed path, where the default stance is more security-conscious, and the holes enterprise policies poke through the wall are more deliberate.

[torgo]

Hi @eladalon1983 I'm not sure we entirely follow your argument. As @cynthia said, there are risks for abuse by exposing this to the entire web. From a safe to browse point of view, it should be difficult for the user to get into a scenario where they are sharing their screens. A risk I see that is not discussed in the explainer or the spec is users' responses to permission prompts being gamed – e.g. by phishing attacks claiming to be customer service for a financial provider. At the very least, it feels like there needs to be more in the spec about the potential abuse scenarios and mitigations, beyond presence of a "privacy indicator".

[eladalon1983]

A risk I see that is not discussed in the explainer or the spec is users' responses to permission prompts being gamed

@torgo, a user cannot chance upon an arbitrary malicious site and be shown a dialog to share all their screens with that site. This API is restricted to origins allowlisted by the admin. That configuration should even be enough to forego a user-prompt altogether, if the user was warned earlier that the device might be subject to surveillance at any time, allowing them to walk away from the device rather than use it if they don't accept that.

That said, I believe other concerns (XSS) have prompted discussion of potentially moving this API off of the Web. @shangl would know the status of these and whether this TAG review request is still relevant.

[torgo]

Hi @eladalon1983 we've just been discussing in our TAG f2f today. We're of the mind that, for many of the same reasons articulated in our review of the managed device API we don't think this should be part of the web platform. The kind of surveillance you're talking about (where, by the way, people might not be able to make the decision to "walk away" without also losing their job or incurring other harms) is not something we're comfortable signing off on.

[eladalon1983]

The kind of surveillance you're talking about (where, by the way, people might not be able to make the decision to "walk away" without also losing their job or incurring other harms) is not something we're comfortable signing off on.

I understand the discomfort and even share it, especially after seeing some public backlash to other controversial APIs. So I'd like to mention[*] for record that this very unfortunate dilemma - "accept or quit" - is already present in the lives of countless people, whose employers provide a machine running Windows/Mac/Linux with a native application installed that surveils the employee. Such existing native applications can even surveil the user without their knowledge. The API proposed by Simon here would take steps to improve our world, because it would at least ensure that the user knows the machine is subject to potential surveillance.

Creating a world completely free of surveillance is beyond our powers; we take pride in the little strides forward we can make.

--
[*] Also mentioned here.