Extension:Chart/Project

Wikimedia Foundation projects Chart Secure and extensible tool for data visualization Start: 2024-07 End: 2024-12 Team members: People Backlog: #chart Lead: Chris Ciufo Updates: Updates

Between 2015 and 2023, editors could use the Graph extension to visualize data across Wikimedia projects. Hundreds of thousands of Wikipedia articles contained these graphs as a way to enrich the text-heavy content of the wikis. In 2023, the extension was disabled for security reasons. This left editors frustrated that readers were unable to view graphs that they have invested time and energy into creating.

The Wikimedia Foundation discussed the future of graphs with community members, and this project is the outcome of those discussions. We are building a new secure tool, which we are calling Chart, as a successor to the Graph extension. It will handle the majority of simple data visualization uses on Wikipedia articles. The new tool will be built in a way that will make it easy to add support for more use cases in the future if the Foundation or community developers choose to do so. We will know we’ve achieved success when editors are converting old graphs using the new tool.

In the 2024–2025 Annual Plan, the Charts project is documented as the key result Wiki Experiences 3.3.

Strategy

• Audience. We are guided by the needs of Wikipedia readers. We will focus on the most common uses of data visualization for enhancing encyclopedic content. This means that uses outside of articles, such as for PageViews, are not a priority now. However, although the primary focus will be on Wikipedia articles, charts will not be limited to pages in the main namespace and will be available on non-Wikipedia wikis.
• Ease of use. We want to make it easy for any editor to be able to create a chart intuitively. Previously, it was fairly difficult to add and edit graphs. Editors needed to be comfortable interacting with Vega, the underlying software powering graphs. This is why editors created most graphs using templates that contained pre-defined Vega code. Templates are more user-friendly, but cannot be easily reused across wikis. This time, different types of charts will be defined by the tool and will be available across all wikis. Editors will be able to add charts using a familiar wikitext format and will not need technical knowledge about the software underneath. These changes will reduce the need to rely on templates. They will also make it possible to upgrade the underlying library without disrupting existing charts.
• Architecture. Charts will be rendered (turned from code into visual content) on the Wikimedia Foundation servers using an open-source data visualization library, generated from data pulled from defined sources. More specifically:
  • Rendering charts on our servers will allow for pages to be loaded quickly. It also avoids the security issues that we faced with the graph extension. It will also enable charts to appear on the mobile apps, in exports, and in other reuse/offline scenarios.
  • For data visualization, charts will be generated using an open-source library. This library must meet our requirements like support for different languages, some level of interactivity, color customization, and more.
  • At first, there will be a single data source for charts: pages in the Data namespace on Wikimedia Commons. This will make it easy to reuse data between charts, avoiding the clutter of having the data inlined in wikitext. We will also design the system to support adding more data sources in the future.
• Editor support. This project will only succeed if editors convert existing graph code to charts and start creating new charts. We will find ways to support editors with converting existing graphs through automated and manual processes. We will also look for ways to preserve existing graphs that are unlikely to be converted to charts. This may mean replacing them with static images or transitioning their use case to other tools.

Updates

Main page: Extension:Chart/Project/Updates

October 2024: Working towards production deployment

Throughout September and October, the team has been working on taking the prototype that is currently deployed in the beta cluster and getting it into a production-ready state. The next major milestone we are working towards is deploying the Chart extension to testwiki and testcommons, which we are aiming to do in November. We are simultaneously working towards a security review, which is an important step for the project given security concerns were the reason the Graph extension was ultimately disabled. Following production deployment and security approval, we will begin rolling out to real wikis.

What to expect

The initial version of the Chart extension released to testwiki and testcommons in production will be fully functional, but not yet feature-complete. We've simplified the prototype to focus on the core workflows of creating and embedding charts, in addition to designing a visually accessible reading experience on mobile and desktop devices. Editors will be able to define basic line, area (stacked), bar, and pie charts on testwiki using tabular data stored on testcommons. Documentation for the extension is now available on the extension page, with a working version on beta.

Once we've deployed to production, we'll be adding more features and functionality. Notably, we want to improve the experience of working with tabular datasets and support more customization of the visual presentation of charts (e.g. customizing the colors). We will also be turning our attention towards supporting editors with migrating legacy graphs to charts.

Deployment to real wikis

Soon after deploying to the test wikis, we are planning to deploy the extension to production Commons and real wikis so that we can collect feedback about real-world use and quickly address any high priority issues. Let us know if you'd like your wiki to be one of the first to receive the new extension.

People

• 
  Chris Ciufo
  Product Manager
• 
  Nat Baca
  Engineering Manager
• 
  Lani Goto
  Program Manager
• 
  Roan Kattouw
  Tech Lead
• 
  Brooke Vibber
  Engineer
• 
  Katie Filbert
  Engineer
• 
  Derek Torsani
  Designer
• 
  Szymon Grabarczuk
  Movement Communications

FAQ

What is the timeline of the project?

We want to have charts ready to use by editors by the end of September 2024. The project will run until the end of calendar year 2024 (half of the fiscal year 2024–2025).

Which library will be used to render charts?

We have decided to use Apache eCharts. However, if we experience issues with eCharts and think Vega (the library previously used for Graphs) might be better, we might switch to it later. We might also build most charts in eCharts but use a different library for certain specific chart types if it turns out to be much better than eCharts.

For details on why we chose these two libraries and prioritized eCharts, go to T368336#9971574.

What types of charts will be supported?

There will initially be a limited set of supported chart types. We will start with line charts. Their parameters should be reusable for other related chart types too, like area, bar, and pie charts. After line charts, we will move on to map-based visualizations.

For the exact list of line chart parameters, see T368478#9978405.

Where will we store chart definitions? Where do I go if I want to edit a particular chart?

Chart definitions will live on their own .chart pages on Commons, under the Data: namespace. We want to treat charts as a standalone content type, rather than just a part of an article. It will be easy to reuse the same chart across wikis, and beyond Wikimedia platforms by making them available as links. Editors who want to embed charts in an article will be able to do so with a short piece of wikitext, similar to including an image from Commons, all without needing to interact with complex templates.

We have heard comments that requiring the data come from Commons tabular data may not address some common data sourcing flows, like from MediaWiki APIs or Wikidata Query Service. While those sources are not the focus for this project, we want to ensure the extension is designed in a way that they can be supported in the future.

Example chart definition (from Commons)

// Data:1993 Canadian federal elections.chart {    "version": 1,    "type": "line",    "width": 350,    "height": 200,    "xAxis": {        "title": "",        "type": "date"    },    "yAxis": {        "title": {            "en": "%support",            "fr": "%soutien"        }    },    "legend": {        "en": "Party",        "fr": "Parti"    },    "interpolate": "basis",    "showSymbols": true,    "source": "1993 Canadian federal election.tab" }

Example invocation (in an article page):

{{#chart:format=1993 Canadian federal election.chart}}

How can volunteers add more chart types?

Once we set up the initial infrastructure to create charts, we will welcome technical volunteers to contribute more chart types. In the meantime, we encourage you to tell us what you think – which types of charts are most important to start with.

What will happen with the existing graphs?

For now, the existing graphs will remain in their current state. Some wikis have chosen to hide graphs entirely while others render a message explaining that graphs are disabled. Once it's possible to create charts that can replace graph uses, we will work with volunteers to start converting them so that readers can start to see them again.

It is unlikely that all graphs will be able to be converted to charts. For graphs that cannot be converted, it may be more beneficial to either: find an alternative tool to recreate the graph, convert the graph to a static image, or remove the graph altogether.

Get involved

• Contact: Chris (talk) or Szymon (see my user page and pick a platform)
• Phabricator tag: #charts