Support for chrome/chrome_extensions externs

[sgammon]

Hey there,

We have some interest in building a Chrome extension with the elemental2/J2CL stack. However, we can't yet use elemental2 to great effect because there isn't a chrome and/or chrome_extension package that brings those extern files into Java.

I know this project is quite early, but we wanted to file this to track progress against this goal. We are also happy to propose PRs to introduce this functionality.

Thank you for this awesome project and we're excited to help make it better how we can

[jDramaix]

This is a good idea but I think this should not be part of Elemental2 as Elemental aims to provide standardized javascript apis.

[realityforge]

I wonder about that. The "chrome" extension API is supported by Chrome, Opera and Firefox and presumably the new chromium based edge (although I have not checked this). While it is true that the "cross-platform" standardized API is meant to have a symbol named "browser" rather than chrome (and a few other changes IIRC), I don't believe there is any browser except Firefox that supports it? Presumably other browsers will in the future although I don't know when that will be.

I say this as I have the need to write a browser extension in closeish the future and would love to see it somewhere easy to collaborate. Anyhoo I would love to see mapping worked out and a simple example of an app if/when you get it going ;)

[sgammon]

@realityforge great point. in fact, the Chrome APIs have support outside of the traditional web platform. there are also Chromebooks, and so on.

it would be really cool to architect and distribute apps in Java for those platforms. i feel like elemental2 is basically a perfect container for it, especially since it is authored in a modular way that would require a user to opt-in to the Chrome APIs.

i may investigate forking elemental2, but I want to be careful to keep things easy to maintain. if anybody here has ideas about a good way to approach this, i'm all ears

[realityforge]

I had a poke around the API docs and it seems that the set of browser specific APIs is not as simple as those in one particular namespace or not.

I believe it would be possible that rather than forking elemental2 you could use the targets from the elemental2 project and then reference them from within a new elemental2-chrome project (or whatever) so you could reference the original types in elemental2 and the new project. It may require a little more bazel-fu than I have at the moment but it looks doable.

[jDramaix]

You don't need to fork elemental2. Elemental2 is built on top of JsInterop-generator that is public.
You just need to move your pr #120 in your own repository and it should work. If something is missing (like the patch process maybe), ping me.

[gkdn]

Maybe we can have elemental2-contrib-xxx. As Julien pointed, it is not technically necessary but it might be worthwhile since it simplifies discoverability and avoid any duplications.

[sgammon]

@gkdn i would love that, if i can help make it a reality let me know how

[sgammon]

@realityforge, do you happen to know the RFC that defines the browser symbol? it is very hard to find that on google

[realityforge]

Hmm ... poking about it looked like the standardization process kinda stalled from the post [1]. I tried to google around but ran into the same sorts of issues ;) The best I found was Mozilla's [2] and Microsoft's [3] browser extension docs.

HTH

[1] https://lists.w3.org/Archives/Public/public-browserext/2017Jul/0001.html
[2] https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions
[3] https://docs.microsoft.com/en-us/microsoft-edge/extensions

[tbroyer]

https://browserext.github.io/browserext/ ? (linked from the Mozilla doc linked above)

[jDramaix]

We may consider to create a contrib directory that will contain project like this one that is created by the open source community but not officially maintained by the Google J2CL team. I created #124 for that purpose.

Now I recommend you first to create you own repo containing your code and get ride of the extern diff file (fix directly the extern file in closure compiler repo).
Then if we get more request like this one, I'll create the contrib directory

[sgammon]

@jDramaix this had the effect of stopping our use of elemental2, which is sad, because chrome extensions are perhaps a perfect place to deliver Java-authored JavaScript

i understand you are wanting to wait until there are other requests, but that doesn't really help us. yes, we could avoid forking the project and just make our own project, but that's a little ridiculous when Bazel, Chrome, Chrome Extensions, Closure Compiler, and indeed all the glue between them are authored and kept up to date by Google

we wish elemental2 the best of luck and we'll try to use it if we can find a case for it in the future

[sgammon]

i am not entirely sure the internal need or direction behind elemental2. if the idea is to help add bricks to the open source community, which we can use, then why say no when we want to use them?

yes, browser/chrome isn't yet a standard. that never stopped google from shipping code before when it served google3

look, with all due respect, why even release this open source, if you aren't going to accept contributions? so you don't want people writing high-quality, high-performance Chrome extensions? it just strikes me as confusing.

[niloc132]

Sam, I started on this a while ago because I agree it seems like the perfect use case for j2cl/gwt. If memory serves, you're mostly interested in it working bazel (rather than maven, where I do most of my work), if so I think we could collaborate on that?

[gkdn]

@sgammon, why is this had the effect of stopping your use of elemental2? Our recommendation was just to put the chrome target to somewhere else for now.

[sgammon]

@niloc132 sweet, i think it would be really awesome. i am indeed working from Bazel in this case. i'd be happy to collaborate with you on it.

@gkdn because while i understand the JS generator is open source, we would first have to figure out how to use it, then we'd have to inline the externs in our own codebase, or create some external add-on contrib project which we would by definition be responsible for maintaining. i've tried, and merging it with our project's structure, or creating a fork, or creating a small container project that works from this one, inevitably creates surface area with tight coupling (that is, if we want to use Bazel's pin-to-commit and hash features). so, it just occurs to me, that it would be easiest to place it in the automated extern gen process that seems to take place in elemental2, since after all, Google is the one authoring Chrome, the Chrome Extension APIs, Bazel, J2CL, GWT, elemental2, rules_closure and all the components that go with them.

it has been awhile since i filed this issue, but i believe a PR came along with it. i'd be happy to try again, but there's no sense in us adopting elemental2 as a tiny startup if we'd have to accept the burden of maintaining Chrome Extension externs along with it. it is a surefire way for our code to break silently as Google pushes updates to any one of (1) Bazel, (2) J2CL, (3) elemental2, (4) Chrome or (5) Chrome Extensions. some of that is handled by the pin-and-hash technique, but a good deal of it is not.

it is just strange to me, all-around, that elemental2, J2CL, and other tools like it inside Google don't work or interoperate with other Google products or platforms. here we are, a small developer, who wants to build a high-performance extension. but you are asking us, rather than the $300bn/yr company you work for, to accept the maintenance burden it incurs. that's what prompted my question: is it adoption and an ecosystem you want, or not? it leaves us confused.

re/it stopping our use of elemental2, as i mentioned, our Chrome Extension is the only place we're able to use this stack for now. other than these issues getting suggestions merged (here, also here), it has been great, and we really appreciate the hard work Google puts into releasing this stuff. we just wish we could use it effectively.

[sgammon]

@gkdn / @jDramaix i am going to assume that you guys have seen this comment and remain unchanged in your opinion here. i understand your motivations and can see why you made the decision you did, but it is sad and unfortunate for devs like us, nonetheless.

we are hoping over here that elemental2 gets to a point of maturity where you can accept contributions. until then, good luck, and thank you for the code and effort behind this and all of google's open source work.

[jDramaix]

@sgammon As we said, we would be happy to review and submit an external contribution for that.
Just fork this repository and create a directory named contrib/chrome_extension and create the build file needed for invoking the jsinterop generator (you can use this BUILD file as an example ). Then create a PR in this repo.

as a tiny startup if we'd have to accept the burden of maintaining Chrome Extension externs along

You will not need to. You just need to refer to the extern file defined in the closure compiler project. They (closure compiler guys) are maintaining this files if something change on the chrome side.

Most of the rest of the maintenance you mentioned (Bazel, J2CL) will occur in JsInterop Generator project and will be done by us as we maintain the core of Elemental2.

[sgammon]

@jDramaix no worries. i'd be happy to do that.

thank you for pointing me in the right direction with these links as well. my concern was keeping the contrib targets in a separate repo, which is where that additional surface area of maintenance might be introduced. with your comment here, you've addressed it, and i very much appreciate your willingness to do that.

i'll tag this issue from the PR when it is pushed, thank you again

[niloc132]

Following up on this: I spent a few hours a week ago and got a couple of "contrib" sorts of externs building into apparently usable jars - though I haven't done more than a very simple smoke test. These include:

• google maps api v3 - several small changes required, enums+unions don't play nice, and something isn't right about typedefs, but I'm still debugging that.
• ace editor v1.2.3 - a @const namespace object is required by jsinterop-generator (but not by closure) to actually be initialized, just change var ace; to var ace = {};
• chrome, chrome-extensions api - this requires the "nonstandard_fileapi.js" and "w3c_webcrypto.js", and as above you have to hide away all of the typedefs. There is also another issue around putting a function(...):... in a template param when extending something, my workaround for that isn't as workable, but likewise I'm hoping to dig a bit deeper before filing something. Only the chrome.js part is usable, I'm still iterating on extensions.

[sgammon]

@niloc132 that's awesome and honestly i would totally use it as soon as chrome becomes a priority for us again (everything is on pause as you might imagine)

[niloc132]

• https://github.com/Vertispan/gwt-ace/
• https://github.com/Vertispan/gwt-googlemaps-api/ (note that this does not build presently since elemental2 master doesn't either, and that's how bazel plays. the gwt-ace project has a workaround that I'll move over here once i finish getting gwt-ace's CI going)
  The idea seems sound, just need to iterate on a few setup issues. Once we've got those in better shape I'll move to chrome+extensions, but as you say, other things are presently taking priority when I do happen to find free time...

[jDramaix]

https://github.com/Vertispan/gwt-googlemaps-api/ (note that this does not build presently since elemental2 master doesn't either, and that's how bazel plays. the gwt-ace project has a workaround that I'll move over here once i finish getting gwt-ace's CI going)

Why don't you depend on 1.0.0 branch of elemental2 instead of master? Do you need something that is not in the 1.0.0 branch ?

[jDramaix]

https://github.com/Vertispan/gwt-googlemaps-api/ (note that this does not build presently since elemental2 master doesn't either, and that's how bazel plays. the gwt-ace project has a >workaround that I'll move over here once i finish getting gwt-ace's CI going)

Why don't you depend on 1.0.0 branch of elemental2 instead of master? Do you need something that is not in the 1.0.0 branch ?

Note that in Bazel, you can also directly depend to a commit hash of an external repo and not necessarily wait to have a tag. Look at what we do in j2cl: https://github.com/google/j2cl/blob/master/build_defs/repository.bzl

[niloc132]

Sorry, lost track of this.

Depending on releases is a non-starter, since j2cl "release" would need to be a specific sha (better to "just use latest" until there is a specific release that you tell us to use, right?). Updating j2cl means keeping to latest closure-compiler, latest closure-library, and in turn that means using latest elemental2 builds, since closure-compiler keeps updating and releasing with externs removed, while elemental2 releases are less frequent (and sometimes don't correctly pick up changes?).

Plus, #141, just filed now after making sure the dates do seem to be wrong.

So, we've tried to just sit on top of latest of each repo and hope that these build - and when they fail, we incrementally have been moving to our own forks that build in open source consistently, though we do need to keep them rebased so we have upstream changes.

Using a sha instead of a tag is doable, but as in #141, it does mean some guesswork/bisecting to find the "right" combination, and means that in updating we have to incrementally move all commits up to a certain point and then walk them backward until everything builds again. If we get to the point where that is easier than forking, we'll probably switch to that.

[gkdn]

Updating j2cl means keeping to latest closure-compiler, latest closure-library

Not usually true. J2CL is generally backward compatible for a while with closure releases unless we need modification in both but that's not common.