Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blog post: Announcing MyST-JS #936

Merged
merged 3 commits into from
Feb 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@ python:
- requirements: requirements.txt

sphinx:
builder: html
builder: dirhtml
fail_on_warning: true
16 changes: 16 additions & 0 deletions docs/_static/python-stack.excalidraw.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes.
File renamed without changes.
140 changes: 140 additions & 0 deletions docs/blog/2023/announce-mystjs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
---
myst:
substitutions:
cta: |
:::{admonition} Learn more and get involved with MyST
:class: tip

- See our new website at [myst-tools.org](https://myst-tools.org).
- Try the [MyST quickstart tutorial](https://myst-tools.org/docs/mystjs/quickstart).
- Follow the Twitter handle [@myst_tools](https://twitter.com/myst_tools).
- [Get involved at our GitHub organization](https://github.com/executablebooks).
- [See the Executable Books team compass](https://compass.executablebooks.org) for more about out organization.
:::
---
# Announcing MyST-JS: Bringing MyST to JavaScript and Jupyter

```{post} 2023-02-09
:author: EBP
```

MyST Markdown is an extensible, semantic, and community-driven flavor of markdown designed for scientific and computational narratives.
It was created for [Jupyter Book](https://jupyterbook.org), and has gained wide adoption in Python's [Sphinx documentation ecosystem](https://myst-parser.readthedocs.io/en/latest/sphinx/intro.html).

Over the years we have come to believe that MyST has the right technical and social foundation to serve as a truly community-driven, cross-platform standard for authoring scientific content.
In this post, we're happy to announce **MyST Markdown as a standalone project and standard from the Executable Books community**.

The rest of this post provides ideas on where this project is heading and fleshes out some details.
We are still exploring what is possible, and would love feedback about what you're excited about.

{{ cta }}

## What is MyST Markdown?

Two years ago, the Executable Books project was [awarded a grant from the Sloan Foundation](../2020-02-25-hello-world) to build open source tooling for authoring and reading computational narratives.
We focused our efforts around [Jupyter Book](https://jupyterbook.org), a tool for writing and sharing computational narratives as websites and PDFs.
This project was built on the [Sphinx ecosystem](https://sphinx-doc.org), an open source community that is at the foundation of documentation tools with Python.

MyST, short for _Markedly Structured Text_, was designed to combine the fluid experience of writing [Markdown](https://commonmark.org/) with the programmable extensibility of [reStructuredText](https://docutils.sourceforge.io/rst.html).
We needed these properties in order to meet the needs of Jupyter users in research and scholarly communication.
From this starting point, MyST was designed with the following goals in mind:

- **Be extensible**: We wanted a flavor of markdown that had extensibility built in from the start. This way users could extend its functionality in an intentional and structured way.
- **Be easy to read**: This extra extensibility should come without sacrificing the simplicity and readability of CommonMark markdown syntax.
- **Be semantic and structured**: We believe that the future of scholarly communication involves semantic and well-structured data, and wanted MyST to have a clearly defined specification and underlying data structure that authors and readers could leverage to connect MyST documents with one another.
- **Be community driven**: We believe that standards for scholarly communication require community governance and leadership, rather than being driven by one stakeholder. We framed MyST and the Jupyter Book stack as a collaboration between many organizations and will develop open governance that follows [the principles of open scholarly infrastructure](https://openscholarlyinfrastructure.org/).

The first implementation of MyST was [the MyST Markdown Python parser](https://myst-parser.readthedocs.io), a parser that allows users to take advantage of Sphinx's powerful **directives** and **roles** syntax in building their documentation.

Over the years we have seen interest in MyST raise considerably - it allows authors and researchers to express themselves more efficiently than traditional source text languages like LaTeX.
MyST has also become [a staple in the Python documentation ecosystem](https://www.sphinx-doc.org/en/master/usage/markdown.html), and is the backbone of authoring content in [Jupyter Book](https://jupyterbook.org).

## Bringing MyST to JavaScript with a new partnership
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀


Because the original parser for MyST is written entirely in Python (and strongly tied to Sphinx), we realized that it would be difficult for the impact of this implementation of MyST to make its way out of the Jupyter Book and Python communities.

As a result, we've been exploring how to _extend MyST markdown into JavaScript_, with the goal of integrating with new technical ecosystems and communities.
We believe this will grow the impact of MyST, and allow for new kinds of workflows around computational narratives.
The project is called [MyST-JS](https://myst-tools.org/docs/mystjs).

To drive this effort forward, we have [partnered with Curvenote](https://curvenote.com) to upstream their open-source documentation engine into the Executable Books community.
This effort is based [on their command line interface](https://curvenote.com/docs/cli), which was designed for building technical documentation and computational narratives with JavaScript.
It will become the backbone of **MyST-JS**, and moving forward will be maintained by the Executable Books community.
There is still much to do, but you can preview this work already at [js.myst-tools.org](https://myst-tools.org/docs/mystjs).

Here are a few major new steps summarizing this work over the coming months:

### MyST is now a top-level project in Executable Books

We'll treat MyST as another major effort of the Executable Books project, similar to Jupyter Book.
It will have its own strategy and goals.
Jupyter Book will remain a consumer of MyST markdown, and it will remain one of many stakeholders in the MyST ecosystem.

:::{admonition} A new website!
We have set up a dedicated site to describe the MyST project at [myst-tools.org](https://myst-tools.org).
:::

### MyST will have first-class support in JavaScript

The [MyST-JS project](https://github.com/executablebooks/mystjs) is written entirely in TypeScript, with the goal of being re-usable across more applications and interfaces.
It also allows us to leverage the vast ecosystem of JavaScript libraries and frameworks for communication.
For example, we can use [templating frameworks to export MyST documents to a variety of LaTeX publication formats](https://github.com/myst-templates), and leverage the [unifiedjs project](https://unifiedjs.com/) to make MyST documents even more structured and consumable as data.

:::{admonition} Learn more
You can find the documentation for MyST JS at [js.myst-tools.org](https://myst-tools.org/docs/mystjs). You can also find an [issue from Rowan discussing major new planned functionality here](https://github.com/executablebooks/meta/issues/838).
:::

### MyST will have a JupyterLab extension

By bringing MyST into JavaScript, we can integrate it directly with JupyterLab and other web-based Jupyter interfaces.
For example, the [`jupyterlab-myst`](https://github.com/executablebooks/jupyterlab-myst) extension brings native MyST rendering into JupyterLab including admonitions, cross-references, and figure numbering using `mystjs` as the document engine.

By using a single core engine in JavaScript, we can ensure that the live interactive experience in web clients (such as JupyterLab) is identical to the HTML generated for static viewing.
This goal would be very difficult to reach with a combination of Sphinx and JavaScript separately rendering static and live content.

:::{admonition} Learn more
Learn more about the `jupyterlab-myst` project at [`executablebooks/jupyterlab-myst`](https://github.com/executablebooks/jupyterlab-myst).
:::

### MyST will have an implementation-agnostic specification

We are creating a formal definition for MyST at [spec.myst.tools](https://myst-tools.org/docs/spec).
This will allow us to extend MyST's syntax through a community-driven enhancement proposal process, and allow implementations of MyST to bring in new syntax as they wish.
Modifications to this specification will follow a [MyST Enhancement Proposals process](https://github.com/executablebooks/myst-enhancement-proposals) (MEPs for short).
We are still working out the details of how this spec will be structured, and what the process looks like, so please provide feedback if you have ideas for how to build a healthy and inclusive process around the MyST language.

:::{admonition} Learn more
The [MyST Enhancement Proposals repository](https://github.com/executablebooks/myst-enhancement-proposals) has the latest information about the MEP process and any active MEPs.
:::

### MyST-JS will complement Jupyter Book and focus on articles

We think that MyST-JS and Jupyter Book will be able to complement one another in the use-cases they focus on.
The Jupyter Book project (and its Sphinx stack) are focused on building multi-page books and project documentation.
It will continue to lean heavily into the use-cases that [Sphinx](https://sphinx-doc.org) was designed for.
It also has major functionality that is missing in MyST-JS (for example, full internationalization support and extensions).
We will continue to improve and maintain this stack (both as individual projects in the Sphinx ecosystem and as the Jupyter Book distribution).

The MyST-JS project will initially focus its efforts on a slightly different use-case: **scientific publishing usecases**, such as reproducible analyses meant for publication, and integration with publisher workflows like [JATS](https://en.wikipedia.org/wiki/Journal_Article_Tag_Suite) and [LaTeX](https://www.latex-project.org/).
It will likely grow into new kinds of published content as well, and we will see how far the "MyST in JavaScript" experience can get.
We also hope to see more third-party interfaces and services take advantage of MyST-JS for their own use-cases.

Over time, we will continue to explore the potential of both of these ecosystems.
As an implementation-agnostic markdown standard, users should be able to switch back and forth between MyST parsers with low friction.

## Formalizing organizational structure for the Executable Books project

Finally, we have begun [formally defining the organizational structure and governance](https://compass.executablebooks.org/) of the Executable Books project.
Our goal is to transition this from a small grant-funded team into a community-driven project that has multiple stakeholders as well as clear pathways for participation and leadership.
We will continue to refine these organizational practices moving forward, and invite you to join our efforts!
More on this effort in a subsequent update!

:::{admonition} Learn more
See [this GitHub issue](https://github.com/executablebooks/meta/issues/493) for a long list of ideas and suggested improvements to make as we formalize our processes and structure.
:::

## Get involved

We are incredibly excited about the impact that MyST and Jupyter Book have had, and equally excited about the new things we'll enable with MyST's new foray into the JavaScript community. If you are interested in these efforts, or want to share your ideas, [please reach out](https://github.com/executablebooks/meta/discussions). We look forward to what the coming years will bring.

{{ cta }}
File renamed without changes
17 changes: 9 additions & 8 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,11 @@
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ["myst_nb", "sphinx_design", "ablog", "sphinx.ext.intersphinx"]

# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
extensions = ["myst_nb", "sphinx_design", "ablog", "sphinx.ext.intersphinx", "sphinxext.opengraph"]

fontawesome_included = True
blog_path = "updates"
blog_title = "Executable Books Updates"
blog_path = "blog"
blog_title = "Executable Books Blog"
blog_baseurl = "https://executablebooks.org"
blog_feed_archives = True

Expand All @@ -44,7 +41,7 @@
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

# MyST Configuration
myst_enable_extensions = ["colon_fence", "linkify"]
myst_enable_extensions = ["colon_fence", "linkify", "substitution"]
myst_heading_anchors = 3

# -- Options for HTML output -------------------------------------------------
Expand Down Expand Up @@ -73,8 +70,12 @@
# Intersphinx
intersphinx_mapping = {"jb": ("https://jupyterbook.org/en/latest", None), "tc": ("https://compass.executablebooks.org/en/latest/", None)}

# -- Custom scripts ----------------------------------------------------------
# Opengraph social cards
ogp_social_cards = {
"image": "_static/logo-square.png",
}

# -- Custom scripts ----------------------------------------------------------
import os
from pathlib import Path
import random
Expand Down
77 changes: 69 additions & 8 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,63 @@ This is the team documentation for the
[ExecutableBooksProject](https://github.com/executablebooks), an international
collaboration to build open source tools that facilitate publishing
computational narratives using the Jupyter ecosystem.
Below are two marquee projects that we steward.

We are the community behind [**Jupyter Book**](https://jupyterbook.org)--an open source project for building beautiful, publication-quality books and documents from computational material--as well as [**MyST Markdown**](https://myst.tools)--a rich and extensible flavor of Markdown meant for technical documentation and publishing.
<style>
.marquee-projects .sd-card-header img {
max-width: 1.5rem;
}

.marquee-projects .sd-card-body img {
margin: auto;
}

.marquee-projects .jb-logo img {
width: 100%;
}

.marquee-projects .myst-logo img {
width: 75%;
}

.myst-logo-sq img {
width: 1rem;
}
</style>

::::{grid} 1 2 2 2
:gutter: 2
:class-container: marquee-projects

:::{grid-item-card}
:link: tools:jupyter-book
:link-type: ref
:class-body: jb-logo sd-d-flex-row sd-align-minor-center sd-text-center sd-mx-auto

Jupyter Book ![](https://jupyterbook.org/en/stable/_images/logo-square.svg)
^^^
![](https://jupyterbook.org/en/stable/_static/logo-wide.svg)

+++
A command-line interface for building beautiful, publication-quality books and documents from computational content.
:::

:::{grid-item-card}
:link: tools:myst
:link-type: ref
:class-body: myst-logo sd-d-flex-row sd-align-minor-center sd-text-center sd-mx-auto

MyST Markdown ![](https://myst-parser.readthedocs.io/en/latest/_images/logo-square.svg)
^^^
![](https://myst-parser.readthedocs.io/en/latest/_static/logo-wide.svg)

+++
An extensible, semantic, and community-driven flavor of markdown designed for scientific and computational narratives.

:::
::::

## Community links

::::{grid} 1 2 3 3
:class-container: landing-grid
Expand Down Expand Up @@ -62,18 +117,17 @@ Our team policies, structure, practices, and contributing guides.
:::

:::{grid-item-card}
:link: gallery
:link-type: doc
:link: https://www.myst.tools/
:class-header: myst-logo-sq

Gallery of books 🖼️
MyST tools overview ![](https://myst-parser.readthedocs.io/en/latest/_images/logo-square.svg)
^^^
A community-maintained list of books that others have built with Jupyter Book.
An overview of MyST Markdown and the MyST ecosystem of tools.

:::

::::


```{toctree}
:hidden:
:caption: Our tools and standards
Expand All @@ -85,9 +139,16 @@ gallery.md
```{toctree}
:caption: About our team
:hidden:
contribute.md
contribute
Team compass 🧭 <https://compass.executablebooks.org>
updates.md
blog
```

## Blog posts

_Below is a list of recent blog posts from [our blog](blog.md)_

```{postlist}
```

## Acknowledgements
Expand Down
Loading