Chrome stops feeding camera stream and shows gray background

[jsalli]

Here is a video showing a problem I have with these demos:
https://youtu.be/htXAzbYrAkU

When starting any of the demos the background of the webpage turns gray and camera live feed stops. The tracking seems to still work when moving the phone.

Device: Samsung S8 (SM-G950F) and Samsung S7 Edge
Both devices have:

• Android version 8.0.0
• Chrome Canary version 70.0.3501.0
• ARCore version 1.3.180604066
• I have turned on the #webxr and #webxr-hit-test flags and nothing else.

The "phone-ar-hit-test.html" and "phone-ar.html" demos work nicely on the immersive-web github page:
https://github.com/immersive-web/webxr-samples/tree/master/proposals
Also the Chacmool demo works at: https://web-education-ar-demo.appspot.com/

Any idea what is causing the problem?

[vziatkov]

something wrong with shadow. I removed shadow and it started to work

[jsantell]

You need three.js >=94 where setFramebuffer was added in order to use render targets with three.js, since before that revision, it assumes the default framebuffer, which is not the case when using AR. This is my guess since it works once removing shadows @vziatkov.

Does that fix your issue as well @jsalli?

[jsantell]

Actually, this demo does not yet use three 94 with setFramebuffer yet, hence #2 this outstanding issue, but there is a workaround for older versions as seen in the demo code of DemoUtils.fixFramebuffer()

[jsalli]

I commented out all lines on final/app.js and shared/utils.js which have "shadow" on them and made function createLitScene() in shared/utils.js to return only an empty scene. But still I get the gray background.

@vziatkov Did you do something else to remove the shadows?

I also tried to use the three 94 instead of the three 89 but couldn't make things work. I replaced the line DemoUtils.fixFramebuffer(this); with this.renderer.setFramebuffer(this.session.baseLayer.framebuffer); but that didn't help at all. I also tried couple of other ways but nothing worked.

@jsantell How should I setup the setFramebuffer() function when using three 94?

[jsantell]

This sample here uses three 94 + setFramebuffer if you want to see it in action.

Do these demos work for you unaltered? https://googlecodelabs.github.io/ar-with-webxr/final/

Also, you must be using https or localhost since the WebXR API requires those

[jsalli]

None of the samples (final, work, step-05, step-06) work from https://googlecodelabs.github.io/ar-with-webxr/ . The same problem with gray background is still there.

I can't make your repo work either on my S8.

I modified your repo's code so that every second tap on the screen removes the fox-object from the scene following tap adds it again. I also removed the reticle-object and the directional light. The only light in the scene is the ambient light which can't cast shadow. I removed also the the fox-object's shadow casting. Also set the the this.renderer.shadowMap.enabled = false

After this when the fox is removed from the scene the camera feed comes back and when the fox is added again the camera live feed disappears.

My dev server is using https (self-generated certificate on local dev machine which does work with other things needing https like getUserMedia() ) so that should not be causing a problem.

Any idea how to get this working? WebXR is a really usuful and we have a use for it in our company so would be really good to get this working. Thanks for all the help :)

[vziatkov]

try comment //directionalLight.castShadow = true;
How it works for me https://www.youtube.com/watch?v=w9MIDWK175E&feature=youtu.be
https://www.youtube.com/watch?v=S-0fl-Eykbk&feature=youtu.be

[jsalli]

@vziatkov Which versions of Chrome Canary and ARCore you have on the S9+?

Have you made other changes to the code than commenting directionalLight.castShadow = true; out ? You are using three 94dev but the original is using three 89. Have you taken the setFramebuffer into use?

I took the original repo of https://googlecodelabs.github.io/ar-with-webxr/final/ in which I commented out the line directionalLight.castShadow = true; in the shared/utils.js. I tried with THREE revisions 89, 91, 94, 94dev and 95dev but none of them fix this.

Also tried this.renderer.setFramebuffer(this.session.baseLayer.framebuffer); in place of DemoUtils.fixFramebuffer(this); in app.js while using THREE 94dev but it didn't work either.

[jsantell]

What about https://googlecodelabs.github.io/ar-with-webxr/work/ ? This is a more simple scene (no shadows, shouldn't be prone to issues with setting framebuffers) that should result in a sea of cubes on top of a camera feed

[jsalli]

The https://googlecodelabs.github.io/ar-with-webxr/work/ also has the gray background as shown in the image below

The Chacmool demo at https://web-education-ar-demo.appspot.com/ works well as in the image below. It seems to use THREE revision 91

Can anyone else reproduce this? Can it be some combination of ARCore, Chrome Canary and THREE versions that cause this?

[billorr-google]

Do you see any webgl errors in the console, or anything similarly interesting in adb logcat? I've heard a few reports of issues like this, but I've been unable to repro.

[jsalli]

This is what I get when trying the https://googlecodelabs.github.io/ar-with-webxr/final/

An image of the Chrome console:

Two adb logcat files of 2 different occasions entering the site, clicking the button to enter AR and placing the fox.
logfile_1.txt
logfile_2.txt

Some lines seem interesting like:

1 W/native: data_synchro.cc:69 Can not add image for camera 0: image-valid status is 1 and image-expiry status is 1.

2 E/chromium: [ERROR:gles2_cmd_decoder.cc(18066)] [.RendererMainThread-0xcb195200]GL ERROR :GL_INVALID_OPERATION : glCreateAndConsumeTextureCHROMIUM: invalid mailbox name

Chrome Canary version 70.0.3503.0
ARCore version 1.3.180604066

@billorr-google Is this of any help? Do you need something else?

[ybinstock]

This is happening to me as well with the same error on utils.js

And I've also tried the same changes @jsalli tried.

[billorr-google]

@jsalli , yes that is helpful to understand at a very high level what is happening. Unfortunately, I think I'll need a repro to really investigate.

We're thinking this may potentially be a hardware-specific problem. For those hitting it, what type of phone are you using? In particular, what GPU in the phone?

I filed Chrome bug http://crbug.com/867118 to investigate on the Chrome side.

[jsalli]

@billorr-google These are our findings:

S8 SM-G950F with GPU: Mali-G71 MP20 Has this error.
S7 Edge: SM-G935F with GPU Mali-T880 MP12 Has this error.
S7 SM-G930F with GPU Mali-T880 MP12 Works well.

[jsantell]

@jsalli and to confirm, your S8 and S7 Edge can successfully run the Chacmool demo?

[jsalli]

@jsantell I confirm all my three devices, including the S8 and the S7 Edge can successfully run the Chacmool demo.

[ybinstock]

I have Galaxy S9 SM-G960F and the Chacmool demo works successfully, but the ar-with-webxr demo is leading to the grey background screen.

[jsantell]

So the differences between the 3 projects that I can see (and mentioning the framebuffer work arounds, although I don't think that's related if this demo from the codelab doesn't require it and still does not work for some folks):

• Codelab ❌: three.js r89, uses framebuffer hack in step-05, step-06 and final (not work directory)
• Chacmool ✅: three.js r91, puts canvas in fullscreen, uses similar FB workaround used in codelab
• web-ar-samples ❌: three.js r94dev, uses three's framebuffer handling

I wonder if using three.js r91 in the codelab/web-ar-samples changes anything, but not optimistic about that.

[jsalli]

Changing codelab/web-ar-samples to use three.js r91 causes the this.renderer.setFramebuffer at line 154 of the reticle.html to be undefined. No problem, I comment that and all the shadow stuff out. After that the same problem appears, gray background. The environmental tracking of ARCore works well like every time before, there is just no camera feed in the background of the page.

Using the codelab/web-ar-samples with three.js r91 above and setting the page fullscreen when pressing the button and then after the "webkitfullscreenchange"-event running the onEnterAR() makes no difference. Still gray background.

TheChacmool seems to have also two canvases: one with id="gl-canvas" and the other with id="xr-canvas". The "xr-canvas"-canvas is used to get the ctx for the "this.device.requestSession"-function. The "gl-canvas"-canvas is given to the "this.renderer = new THREE.WebGLRenderer" which then handles the rendering of the scene. I don't quite understand why this is done. The "xr-canvas"-canvas is never used for rendering anything it seems.

[jsantell]

Some background info on three.js features and WebXR multi canvases:

renderer.setFramebuffer was added to three r94 for WebXR I believe, so that's why there is the fixFramebuffer hack in the demos using r89 and r91, but like you said, not using WebGLRenderTargets (like the shadows in three.js) will alleviate the need for this.

The double canvas technique in Chacmool is necessary for all magic window or mirroring use cases of WebXR, and is a bit confusing. There's a WebGL canvas that you render to (wrap up in three's WebGLRenderer) that doesn't/shouldn't be in the DOM, and gets copied to the XRPresentationContext defined in the device.requestSession({ outputContext: xrPresentCtx }) call -- you can see that in the codelab code as well, with creating an XRPresentationContext, and when we create the WebGLRenderer, that creates its own canvas that we write/render to that never is injected in the DOM

---

That being said, thanks for checking out those alternatives! Unfortunately it still doesn't seem obvious what the issue is and why Chacmool works. I wonder if there's something in permissions in Chrome that prevents it from displaying the camera feed, since the AR integration is still obviously working, but either way sounds like an issue in Chrome.

[PaulTVungle]

This sounds like an issue I am having on my Moto G6 Plus. I think I'm right in saying that versions up to 69.0.3494.0 work fine, then the grey background appears on later versions.

Maybe this might give a clue as to what change caused the breakage?

[sarathsi]

I have observed the same gray background problem (with demos step-05, step-06 and work/) on Huawei P20 Android 8.1.0. Chrome canary 70.0.3530.0 and 69.0.3494.0.
After some investigation and experimentation, I commented out "this.renderer.clearDepth()" in onXRFrame(). Then problem has been solved. This also works fine on Google Pixel device.

Is it something to do with THREE.js or may be hardware specific.

[freinbichler]

I can confirm that commenting out this.renderer.clearDepth() makes it work on my Samsung Galaxy S8 too. Thank you very much for finding this, @sarathsi!

[jsalli]

I can confirm this also that commenting out this.renderer.clearDepth() works with demos step-05, step-06 and work on my Samsung Galaxy S8. With final-demo it does not work and the gray background is still there.

[ranbuch]

In ar-with-webxr example I can see that they have this.renderer.clearDepth(); on onXRFrame (probably added this line after you guys wrote your comments), however, I can still see the gray background.

Chacmool still works fine thought . . .

Chrome Canary (71.0.3562.0)
Samsung Galaxy S9+

Any ideas?

[ranbuch]

For me removing this.renderer.clearDepth(); did the trick 🤔

[alon-grinshpoon]

Same here on Samsung Galaxy S7, commenting out this.renderer.clearDepth(); fixed the issue.
Works either way on a Nexus 5X.

[unarmedpuppy]

On the Pixel 3, the camera feed doesn't seem to work properly for any of these demos (chacmool, cubes), the background color changes to a solid color that seems to be a random sampling of the camera feed. (If I point the camera at something red, then entire camera feed background turns red) Has anyone else experienced this?

[ranbuch]

@dangerbahn that seems to be the case for the last couple of days.
Looks like another Chrome issue.

[jsantell]

I've updated the framebuffer fix and removed clear depth. I'm seeing a session failure in Chrome 70 currently (before and after landing #11, will file a chrome bug), but working in 72

[klausw]

@dangerbahn, the single-pixel camera feed issue is https://crbug.com/897525 and should be fixed as of Canary 72.0.3590.0. There was also a crash bug on exiting AR, that should be fixed as of Canary 72.0.3592.0. Sorry about that.

[alon-grinshpoon]

Only seeing gray screen (no tracking, no camera) with latest Chrome Canary 73.0.3660.2 (and latest ARCore, both through Google Play installations).
Reverted back to Canary 72.0.3592.2 (with latest ARCore through Google Play) to make it work again.
Tested on both Nexus 5X (Android 8.1) and Samsung S7 (Android 8.0).

[klausw]

@alon-grinshpoon that's likely to be a different issue. The webxr API is currently being heavily modified, including incompatible renames, so please check if you're getting JavaScript errors about unknown attributes.