Load/mount components and data via URL?

[zeluspudding]

(If nothing else, please see my screen recording at bottom)

According to the idyll-embed readme, adding components to Idyll requires:

1. Cloning this repo
2. Running npm install to install the dependencies.
3. Edit index.js according to your own use-case. See the comments in index.js for more info.
4. Rebuild the dist file: npm run build
5. Include the newly created JavaScript file (dist/idyll-embed.min.js) in your HTML.

So extending idyll-embed requires having knowledge of Node, git and of course javascript. Then, to publish an Idyll article, authors also need to know how to host their article. That means that authors need be quite technical before they can write a sentence with an Idyll pen. This was to be expected, I suppose, as writing javascript alone requires more technical know how than most of my journalist friends have (no shame on them). And Idyll's developers have recognized as much:

While the results of the classroom study indicate that technical users are able to use Idyll to create a range of designs, the study does not speak to usage by non-technical editorial users... Custom Idyll components can grow to become arbitrarily complex, and beyond providing reactivity, Idyll doesn’t do much to decrease the complexity of these custom graphics. A closer integration with tools that focus on the creation of custom graphics may be needed in order to lower the threshold for creating articles that include custom components

That said, I am very excited about Idyll's potential to lower the obstacle of writing rich, compelling, interactive stories. It is in that excitement that I ask the following question:

Can we make Idyll load components via URL?

I'm not sure if this is even technically possible without an overhaul of the way idyll-embed works. But if it were, I could see this saving Idyllers a lot of time. As follows:

• Authors could interactively develop new components in observablehq.com or local text editor

• In idyll-embed, we would then feed in a list of urls to where the components are hosted. In the case of having developed in observable, this could be the notebook-cell url. Otherwise, these scripts could be hosted on github, S3, or on Bubble which makes it almost trivial to stand up an Idyll backed publication (once this issue is solved).

  // Get your markup.
  var idyllMarkup = '# Hello World';
  
  // All standard components are available by default.
  // To add additional components, register them.
  
  const customComponents = {
       "incrementer": "https//s3.amazonaws.com/appforest_uf/f1582258792211x368659424917972000/incrementer.js",
        "otherComponent": "urlToComponent"
  }
  Idyll.registerComponent(customComponents);
  
  // Instantiate the Idyll runtime.
  Idyll.render(idyllMarkup, div);
  

• When it comes to publishing, all we'd have to do is store our idyll-markup, and customComponents in our database (or simply links to it if those resources are hosted elsewhere). Then when the client fetchs an article, idyll-embed will mount and render customComponent and idyll-markup.

In this way, I believe we avoid needing a local node environment. We wouldn't need to re-build idyll-embed each time one wants to author an article with new custom components. We could use intuitive tools like observable to create node components. So adoption is easier, even among technical users, since there are fewer hurdles to get started

Ideally, we could mount not only components, but also data via URL. That would be fantastic! To convey the awesomeness of this idea, I went ahead and made a personal "Idyll Studio" type backend to show what type of publishing workflow this would enable. I put this together in about 2 or 3 hours in Bubble. ..Obviously I'm not actually mounting components or data like this, but everything else is real / shows how nice and quick Idyll and Bubble can be stitched together to produce an Idyll driven publication.

This is the dream :)

... and then the last 2 steps in that publishing flow would be to add the component and load the data within the idyll markup

[mathisonian]

Hey @zeluspudding, I totally agree with you that there are too many steps involved in updating this library now and I'd love to see this get streamlined.

For what its worth we expect that most users will come to Idyll through the CLI rather than idyll-embed, and the CLI handles things like deployment and bundling new components on behalf of the users. This is why there hasn't been much effort to polish idyll-embed as its usually been more power users who are interacting with this module.

Your example looks super promising and very cool. I have to admit that I haven't stayed on top of the latest developments in the JavaScript module bundling world. Do you have a sense of the infrastructure that would be required to get components loading from a URL to actually work?

I'm in general pro anything that makes it easier to integrate Idyll with libraries that make it easier to author components (observable, p5js, apparatus, and lyra being the big ones I'd want to support).

[mathisonian]

@zeluspudding if it would be helpful we could get on a video chat to talk this through. It seems like you've been thinking a lot about improvements here

[zeluspudding]

Would be glad to discuss on call! Though I'm not sure how much technical guidance I can add as I don't currently have a great sense of the Javascript it would take to get components loading from URL (my background is actually in mechanical engineering... I'm strongest in Python but more of a hatchling in Javascript).

I suppose that being a novice in this space is truly a testament to all the hard work and nice design choices that creators such as yourself have built into Idyll and those who built Bubble... that despite my lack of skill, I was able to setup the demonstration above in just a few hours. There is great power in these tools and I would love to see them in more hands.

That said, if you think there's something I could add to the conversation - or if you just want to better understand what I've implemented on Bubble, then I'd be glad to share! I've also found Mike Bostock, the creator of Observable and D3, to be quite helpful debugging issues at the interface of Observable and programs that try to work with it. But if he's too busy perhaps we can solicit advice from Tom MacWright, who appears to have mastered the art of loading stubborn modules into Observable/Javascript in general. As an avid writer himself, he may be interested in sharing very useful pointers. Let me know :)