Proposal: Web Charts #9295
Comments
|
It's worth keeping in mind, there are quite a few solutions for this available already, albeit not web-standards, but widely used, built on open-standards, and open-source. For now, people can already use these with the desired interoperability. At the very least, it may be a good reference for further discussion. Apache ECharts, released under the Apache 2.0 license, is a JavaScript library for creating graphs in JavaScript. This can create many diagrams like pie, doughnut, candlestick (stock markets), radar, funnel, etc. The result can be exported as an SVG, or drawn to an HTML
Code to generate the above.const echarts = require('echarts/lib/echarts');
require('echarts/lib/component/title');
require('echarts/lib/component/legend');
require('echarts/lib/chart/pie');
require('zrender/lib/svg/svg');
var chartDom = document.getElementById('main');
var myChart = echarts.init(chartDom, null, {
renderer: 'svg'
});
var option;
option = {
title: {
text: 'Pets adopted by volunteers',
left: 'center'
},
legend: {
orient: 'vertical',
left: 'left'
},
series: [
{
type: 'pie',
data: [
{ value: 386, name: 'Dogs' },
{ value: 85, name: 'Cats' },
{ value: 15, name: 'Rats' }
]
}
]
};
option && myChart.setOption(option);Mermaid, released under the MIT license, is a markup language that builds various types of charts like a flowchart, state diagram, pie chart, etc. These can be exported as SVGs and used on the web. It's supported on many websites such as GitHub, GitLab, and Docusaurus (via an official plugin). pie title Pets adopted by volunteers
"Dogs" : 386
"Cats" : 85
"Rats" : 15
Code to generate the above. |
|
More prior art from 2015. Note: Three of the roles mentioned here are now w3c recommendations as part of Graphics ARIA. It is absolutely crucial, that any charting semantics that appear in HTML are accessible. Most of the "solutions that are available already" have severe shortcomings in this area. |
|
Also very notable in this area is d3, which is widely used by news organizations - see examples here - https://d3-graph-gallery.com/ |
|
@brennanyoung, absolutely, accessibility would be a priority moving forward for Web Charts. Reusable scripting logic should be able to generate accessible visualizations, e.g., charts and infographics, for any valid input data. Atop that, there are R&D topics which can be explored such as how chart semantics and metadata could enhance accessibility. |
|
How would displaying a chart for a data table in the same document look like? <table id=foo>…</table>
<chart>
<data src=#foo />
<script src="piechart.js" />
<style src="custom.css" />
</chart><chart>
<data>
<table>…</table>
</data>
<script src="piechart.js" />
<style src="custom.css" />
</chart> |
|
@Crissov, thank you. I hadn't thought of that scenario of using "text/html" content from elsewhere in the same document. For creating a chart from a table which is intended to be displayed elsewhere in the document, I would be inclined towards the first possibility that you shared: <table id="foo">...</table>
<chart>
<data type="text/html" src="#foo" />
<script src="piechart.js" />
<style src="custom.css" />
</chart>For creating a chart from an inline table which is not intended to be displayed elsewhere in the document, I would be inclined towards the second possibility that you shared: <chart>
<data type="text/html">
<table>...</table>
</data>
<script src="piechart.js" />
<style src="custom.css" />
</chart>Also, in addition to those two possibilities, there is a third possibility: tables could be considered to be a type of data visualization. <chart>
<data src="file.csv" />
<script src="table.js" />
<style src="custom1.css" />
</chart>
<chart>
<data src="file.csv" />
<script src="piechart.js" />
<style src="custom2.css" />
</chart> |
|
Based on the SVG charting semantics mentioned above, I suggest distinguishing three main
All three types are likely to benefit from some kind of legend. (Unfortunately And if we are going to have some kind of The spec should require a non-null text alternative for all three types. A title or heading is particularly important. (Validator should flag an error if omitted). Beyond this, I suggest coining elements for distinct and meaningful sub-elements, mapping to ARIA's graphics-object (nestable) or graphics-symbol (non-nestable). Graphics ARIA recommends overriding the names of these roles, since they are rather abstract. Perhaps if we think the "legend" idea through properly, we can allow the legend to supply the (author-defined) name - e.g. (on a map) authors might provide the user-facing name "Line" for a graphics-object representing a railway line in the legend, with the actual mapped object having the name "Bakerloo". Screen reader might then announce "Bakerloo Line" rather than "Bakerloo graphics-object". Including such nomenclature in the legend text nodes will make translation easier (and more reliable) than hiding such data in attributes (as ARIA does with aria-roledescription). Should the elements of a chart be presented (e.g. to assistive tech) in the order that they appear in the DOM, or should other orders/sequences be supported? Do I really need to steer my screen reader all the way through the Bakerloo Line before I can "read" the stations on the Piccadilly Line? What about stations that appear on more than one line? Simple nested containment is not going to be enough. We need something closer to IMO the real value of charts is that multiple interpretations may be explored, with multiple starting points and directions. Plenty to think about. Finally, some thought should be given to navigating charts with a keyboard. Should we adopt the tab sequence used by operable elements (such as buttons or links), or imagine something new. |
|
@brennanyoung, some initial thoughts about navigating with a keyboard through various kinds of charts are that instead of, or in addition to, using the tab key (which enables a sequence-based navigation), using the arrow keys (which enables a tree-based navigation). With arrow keys, one could use the left and right arrow keys to navigate between a "Bakerloo Line" and a "Piccadilly Line". One could use the down arrow key with a line selected to enter into a line to subsequently navigate its contents, e.g., stops on the line. One could then also use the up arrow to navigate back from the granularity of stops to the granularity of lines. With arrow keys for tree-based navigations, I would tend to think that backwards-compatible tab-based navigations would traverse the most granular nodes, the leaf nodes. What do you think about uses of arrow keys, beyond the tab key, for keyboard-based navigation and selection?
I will put some thought into keyboard-based approaches for more complex navigations and also take a closer look at the Brainstorming on accessibility,
What do you think of these ideas? |
|
i am not sure about that at present the most near that i use is mermaid https://mermaid.js.org/ it has markdown syntax support. |
|
@lemanschik, thanks for sharing the hyperlink to Mermaid. As envisioned, Web Charts would be able to support a broad set of data formats including markdown, e.g., <chart>
<data type="text/markdown" src="file.md" />
<script src="flowchart.js" />
<style src="custom.css" />
</chart> |
|
@AdamSobieski <chart-diagram>
<data type="text/markdown" src="file.md" />
<script src="flowchart.js" />
<style src="custom.css" scoped>
</chart-diagram>globalThis.customElements.define('chart-diagram',class extends HTMLElement {}) |
|
@lemanschik, thank you. I was also thinking about ways that Web Components, the Shadow DOM, and Custom Elements could be useful for prototyping and demonstrating Web Charts, ideally towards a new standard. It appears that some work would be required to create interoperating elements and logic utilized by Also, there are still to discuss dynamic charts, charts which connect to and visualize streaming, potentially real-time, data sources like those pertaining to the weather or stock markets. |
|
@AdamSobieski i guess this belongs into a framework and you do not need even components for that as you got svg and scoped as property on css i am browser and operating system engineer i see not much that we could do to make that better at present. everything that is left open depends already on users choice. |
|
maybe you should look into the new independent animation parameters before this year css animations needed to be a single line now you can compute the properties individual. also the css container queries cqw cqh could help you as they allow you to design elements that depend on the size of a other element that allows a lot of chart magic |
|
Thanks @lemanschik. Users do enjoy smoothly animated, resizable, interactive visualizations, e.g., like those produced with D3 which @AramZS indicated. @brennanyoung noted the importance of accessibility and I'm thinking about the accessibility of resultant visualizations (e.g., HTML, SVG, Canvas2D, WebGL, WebGPU), the content output by the script logic, to enable features like:
towards large language models (LLMs) being able to answer questions about and to discuss both data visualizations and documents which include such visualizations. On the topic of AI, LLMs, and data visualizations, you might find interesting the LIDA project [1][2] which enables end-users to create and style data visualizations using natural-language dialogue. Combining these ideas, it could be, in the near future, made possible for end-users to be able to conversationally search for, create, and style data visualizations, and to be able to clipboard and drag-and-drop these between applications, such that resultant visualizations could subsequently be conversationally explored between end-users and AI systems, including in the contexts of accessibility (see also: [3][4]). [1] https://microsoft.github.io/lida |
|
@AramZS we got image Shape recognition already and ocr you will gain not much by adding a human method to annotate charts to annotate chart content put in good alternate text that explains the content is more worth. |
|
If AI systems, e.g., dialogue systems, chatbots, LLMs, could better process and understand data visualizations (or documents containing these), they would be better able to answer questions involving or about these visualizations and to engage in dialogues involving or about these visualizations with end-users including in accessibility contexts. How might AI systems better understand data visualizations to be able to answer questions about them and engage in dialogues about them? What the Web Charts approach brings includes:
One means of delivering far more featureful and far more accessible data visualizations involves abundant, granular, semantic annotations and metadata. Some of these metadata would be about the parts of charts, the things in charts (e.g., points, slices, and bars), and other of these metadata would be about the charts in their entireties (overall descriptions or summaries, creators, date of creation). These metadata could be generated by machines, processed by machines, and could, in particular nowadays, include natural language. Human designers haven't tended to annotate the parts of charts, e.g., points, slices, or bars, with natural-language metadata. AI-enhanced design tools could thusly annotate data visualizations for AI-related usages. The entirety of lengthier natural-language descriptions for charts could be auto-generated, aggregated, including from those descriptions of their parts. These natural-language descriptions could be consumed by AI systems such as LLMs. As envisioned, metadata could be:
Metadata could enable interoperability with AI systems, chatbots, LLMs, and other accessibility technologies in:
|
|
Brainstorming, with respect to automatically generating natural-language descriptions of interactive data visualizations from combinations of:
we can look to solutions involving templates. Using templates, e.g., those based on Handlebars, developers and AI systems could annotate parts of charts, e.g., points, slices, or bars, with natural-language descriptions which include to-be-determined styling information, such as background color, to be computed at runtime, e.g., using separate stylesheets. Templated descriptions of the parts of charts could be composited and assembled together into descriptions of charts in their entireties. Towards concrete examples, a bar in a chart could be annotated with a natural-language templated description:
Or, using templates in a fuller manner:
It might be reasonable to have two separate templates, one for semantics and one for style:
Note that templates pertaining to style alone would be highly reusable across chart instances. Both data and style could be considered to be potentially dynamic at runtime and, by using templates, natural-language descriptions could react and synchronize accordingly. Were developers or designers to swap a chart's input data or stylesheet, its natural-language description would react automatically. Natural-language descriptions of interactive data visualizations could be provided to LLMs as input, as parts of input prompts, accompanying users' questions and dialogue moves. Templates are also being explored with respect to LLMs [1] with "simple, intuitive syntax, based on Handlebars templating." |
|
i am not sure about your knowledge but all that is done already no homan needed for that you got no ai that answers such questions because no one did train and deploy one in public but it is easy do able i can do it even as single person today. You need image detection and then you need some connected systems. I can even do most of that stuff without any machine learning image detection in case of charts is not hard. as that are clear shapes. |
|
for example that guidance lang is again a lang that A LLM can learn and write you can use it as interface but i would not do so it is all not needed overall i guess if you want to have some AI Stuff you should simple hire a AI Expert today most can tell you how to use existing stuff and will even be able to hack some image filtering and ocr together. |
|
@lemanschik, based on your indications of the applicability of computer vision and optical character recognition for enhancing the accessibility of interactive data visualizations, you might find interesting: https://ai.facebook.com/blog/imagebind-six-modalities-binding-ai/ . With respect to exploring uses of templates, e.g., Handlebars-based, for generating natural-language descriptions of the parts of charts and of charts in their entireties to enable dialogue systems, e.g., chatbots, to be able to answer questions about and to engage in dialogues about interactive data visualizations, including in accessibility contexts, Brainstorming, I am presently considering there to be five categories of descriptive natural-language content: 1. Metadata
2. Structure
3. Data
4. Analysis
5. Style
A solid taxonomy of types of descriptive content could be useful including to inform the eventual design of APIs. For instance, document.getElementById('chart123').getDescription('en')
document.getElementById('chart123').getDescription('en', { filter: 'metadata' })
document.getElementById('chart123').getDescription('en', { filter: 'structure' })
document.getElementById('chart123').getDescription('en', { filter: 'data' })
document.getElementById('chart123').getDescription('en', { filter: 'analysis' })
document.getElementById('chart123').getDescription('en', { filter: 'style' })
That is, if or when a chart's input data or style were changed by a Web developer or as the result of a data streaming scenario or Web animation scenario, the descriptive contents of the chart would automatically update. Any thoughts on these preliminary ideas involving dynamic and reactive descriptive content? |
|
Oh i am also language implementer i should mention that handlebars and all Template Engines got replaced with nativ ECMAScript tagged Template String Literals that looks like const myChartTag = (strArray, ...props) => strArray.reduce(
(acc, curr, ind) => acc.concat([curr, props[ind]]), []
).join('');
const newChart = welcome => myChartTag`Hello ${Welcome}`
const templateForChart = (data)=>`<i-am>${data}</i-am>`
document.body.append(templateForChart('chart'))i am inventor of Stealify Lang a System to implement whole languages via tagged template strings :D Important concepts taggedTemplateStringLiterals do not need to return a string they can return even html elements and so on and so on like lit-html does it or like many did before event and data binding simplification i see only that you would like to see a fixed api while there exist fundamental concepts to implement such apis. If you would be aware of the concepts you would need to api for example getDescription as a api is a nightmare. HTML is a Text based lang the way to supply custom data that is readable as text you add data-my-name propertys to the html tags. the data- propertys are special because they do not modify the internal shape of the element and they are readable in text form and programatically in the dom tree |
|
for me it looks strong like you would be even only needing querySelectors and dataset properties to get your desired AI Description properties. as a current LLM even is able to interpret the line my-chart data-from-new-york="20" data-from-austin="50" |
|
@lemanschik, thank you for indicating template literals. I am interested in how convenient it can be made for Web developers, AI systems, and accessibility technologies to obtain extended natural-language document content, content including dynamic descriptions of documents' interactive data visualizations. Although we are discussing models which utilize scripting logic to process input data – either inline data or query-response data – into various types of content in the shadow DOM (e.g., HTML, SVG, Canvas2D, WebGL, or WebGPU), there could be an interface, a fixed API, for Implementations of any |
|
the linking of the custom propertys gets done in the so called customElementDeclaration there is a observed attributes setting where you put the logic for the data attributes. then you can listen internal and external to data changes. for a LLM only a descriptive name for the data properties is needed |
|
As interesting to others reading along, I found some Web documents:
which showcase the technical topics of callbacks and observed attributes. My thinking with respect to a Also, beyond dynamic natural-language descriptions and related events, there are other topics, including some other accessibility-related topics (see: #9295 (comment)), which might be relevant to the design of a standard interface and/or base class, perhaps P.S.: If only it were possible (a) to specify elements in the shadow DOM by ID, or (b) to specify JavaScript instead of an element ID, for <chart id="chart123" alt="javascript:desc('chart123')" aria-describedby="javascript:descElement('chart123')">
<data src="file.csv" />
<script src="barchart.js" />
<style src="custom.css" />
</chart> |
|
@AdamSobieski thats the wrong way to think about it current models like BLOOM are multilang already so the chart can be in any lang and even displayed in any lang already without human intervention or code. |
|
@lemanschik, thank you for indicating BLOOM and machine translation. BLOOM works with 46 different languages. Modern AI systems could, then, translate templates and resultant natural-language descriptive content between at least these popular languages. However, on the topics of internationalization, there are also low-resource and endangered languages to consider. According to Ethnologue, there are currently 7,168 languages spoken by people around the world. Of these languages, 90% are spoken by fewer than 100,000 people. In these regards, one can see this list of languages by total number of speakers. |
|
Now that we have discussed some of the accessibility- and internationalization-related topics relevant to charts, and how charts can be searched for, created, styled, discussed, and analyzed using conversational AI, I would like to also broach, for discussion, the visualization of streaming data and the related matters of (real-time) data-streaming formats, queries, and protocols. Charts could also display and provide interactions for dynamic forms of data, such as streaming data for: stock-market tickers, business intelligence and analytics, the weather, and various kinds of sensors. Some preliminary ideas:
Any thoughts on this initial brainstorming pertaining to (real-time) data streaming and visualization for charts? |
|
@AdamSobieski i added you on linkedin we could chat there if you like because it looks like you want to make a product and that is not standard related. The Normal Cycle is that first a Product exists and then a standard gets driven out of that not the other way around. |
|
And the update sync and interactivity api got also shipped generic we call that Streams But you need access to some one who knows how to use such api's to create your desired behavior. or you need to learn the browser api's your self and keep you updated. |
|
Thank you @lemanschik for indicating the Streams API. I suppose that there are a variety of opinions about the cyclical interrelationships between industry, standards, and innovation. Standardization and related efforts have enabled industrial innovation in the form of new products and services for developers and end-users. Perhaps, in particular after versioning the markup examples to include support for (real-time) data streaming and visualization, the W3C Web Incubator Community Group would be a next logical venue for this Web Charts proposal. There, anyone interested could contribute to incubating these ideas until they are more ready for a potential integration into the HTML living standard. Are there any other opinions, here, on any of these Web Charts ideas? Update: Here are updated sketches, towards a next edition of the Web Charts markup concepts, which include support for (real-time) streaming data. In these examples, <chart>
<data>
<source type="text/csv" src="data:text/csv;charset=UTF-8,..." />
</data>
<script src="https://cdn.webcache.org/chart.js" />
<style src="custom.css" />
</chart><chart>
<data>
<source type="text/csv" src="data.csv" />
<source type="text/tsv" src="data.tsv" />
</data>
<script src="https://cdn.webcache.org/chart.js" />
<style src="custom.css" />
</chart><chart>
<data>
<source type="video/mp4" src="data-video.mp4" />
<source type="video/webm" src="data-video.webm" />
</data>
<script src="https://cdn.webcache.org/chart.js" />
<style src="custom.css" />
</chart><chart>
<data>
<source type="application/x-mpegurl" src="data-video.m3u8" />
</data>
<script src="https://cdn.webcache.org/chart.js" />
<style src="custom.css" />
</chart>In these examples, <chart>
<source type="text/csv" src="data:text/csv;charset=UTF-8,..." />
<script src="https://cdn.webcache.org/chart.js" />
<style src="custom.css" />
</chart><chart>
<source type="text/csv" src="data.csv" />
<source type="text/tsv" src="data.tsv" />
<script src="https://cdn.webcache.org/chart.js" />
<style src="custom.css" />
</chart><chart>
<source type="video/mp4" src="data-video.mp4" />
<source type="video/webm" src="data-video.webm" />
<script src="https://cdn.webcache.org/chart.js" />
<style src="custom.css" />
</chart><chart>
<source type="application/x-mpegurl" src="data-video.m3u8" />
<script src="https://cdn.webcache.org/chart.js" />
<style src="custom.css" />
</chart> |
|
i guess you will find no one for that in the w3c world i am in it since more then 35 years and even i did leave w3c standards 5 years ago because i needed to make faster progress the standard is really in this case a standard for things that are defined already. In your case all is defined and even integrated shipped in the web platform already the missing parts are all scheduled even already but you need some one with insights for example. the w3c only makes things a standard when a browser vendor eg me: "Chromium" supports your effort and has interest to integrate your proposal. But as said there are groups that are highly specialised in technical interfaces and interOp. And as Principal Engineer i know the current efforts and understand the upcoming results of them. For Example Project Fugu https://fugu-tracker.web.app/ Overall you need to know that the w3x while shaping a HTML standard for Browser Vendors they come far behind the browser vendor efforts them self. And the Code base of Chromium and Firefox as also Safari are Millions of Lines Hugh. There are not much people out there that have a near complete view. |
WICG does seem like a suitable place to explore this. Novel features — especially those providing high level abstractions, introducing new capabilities, or which would entail many moving parts — are normally developed as standards outside HTML and then integrated upon becoming mature given consensus from editors and interest from implementors. A proposal — especially one as ambitious as this would seem to be — should probably be initially focused on articulating:
The concrete solution proposed then is initially illustrative. It need not be exhaustive because it’s only a possible solution. Initially it might only be a sketch meant to promote and direct discussion: the true initial “proposal” is the problem and a demonstration of its potential to be solved, not the specific solution. I mention that because jumping too far ahead with a specific solution is a common pitfall, not because I think you’re doing so (my guess is that most likely what I’m saying is already understood or at least intuitive to you). It’s also good early on to think about what the most minimal form of a useful solution could look like. Ambitious proposals seem to be most successful if they can be built up one (useful) part at a time, so an initial broad proposal might end up becoming a “core” proposal whose scope makes it easier to get buy-in and consensus alongside secondary proposals aiming to build on top of that core. Another advantage of this is that it lets real world experience with the baseline inform where things go next. |
|
@bathos thank you so much for jumping in i am not a native speaker and not a good community manager you pinned it really well while i saw clear that all that he posted here is already do able even as fully working MVP in weeks i offered him as implementer to bypass the proposal part and create a MVP first that he then can iterate over as Proposal to get a standard. |
|
Great and useful advice from @bathos |
zcorpan
added
addition/proposal
Note that there is an existing issue proposing capabilities and features to support maps: #6380. |
Introduction
Web Charts is a markup concept for data visualizations such as charts and infographics.
Examples
Example 1
This example shows a basic chart comprised of inline data and a script which processes that data into document structure.
Example 2
This example shows a basic chart comprised of inline data, a script which processes that data into document structure, and custom styling.
Example 3
This example shows a chart which, instead of providing inline data, provides a SPARQL query and service from which to retrieve data.
Example 4
This example shows a means of providing extensible chart semantics or metadata.
Example 5
These examples show possibilities for adding a text description to a chart.
Example 6
This example provides alternative versions of a chart, in this case multiple PNGs, in a fallback mechanism.
Example 7
These examples show that markup for reusable charts could also be stored in separate resources from those documents including the charts.
Notes
Questions
<chart>element and/or protocol response headers?<options>element, e.g., for JSON data, be useful?Conclusion
Ideally, a team of interested participants would assemble to discuss and develop Web Charts towards eventual interoperability with HTML.
Thank you. I look forward to discussing these ideas with you.
The text was updated successfully, but these errors were encountered: