Structuring OpenSpending JS code (and this repo) going forward

[rufuspollock]

I'm opening this issue as a place to discuss how we evolve openspendingjs going forward.

Current Situation

This is my rough understanding and is likely wrong - please correct and amend!

README states

Visualisations and other utils that use data from OpenSpending to help people understand where and how money is being spent.

Note in terms of file structure:

We are currently in a migration stage in order to make contributions easier and more consistent, so the code base might be a little confusing. The best place to add your visualisation is src/. We're in the process of moving all of our visualisations and utils into that folder (to make the build process cleaner and simpler).

External libraries (requirements) are placed in lib/vendor/. There are a lot of files there at the moment. Some of those files are old versions of these libraries. We're in the process of cleaning these files up, removing unused files.

So, basically we have:

• src/utils - common core code
• src/visualisations - visualisations

Sort of deprecated:

• lib Common core code such as JS that wraps the OpenSpending API
• app - various viz / apps (see below)

apps:

bob  # the query builder (?)
browser # data browser (?)
bubblemap # bubblemap viz
dailybread # dailybread viz
data_table # data_table browser
dimensions-editor # dimensions editor (OS import process) (cf http://nickstenning.github.io/openspending.modeleditor/)
faceter # not sure 
spending-map # not sure but maybe the front page map on OS.org
treetable # not sure ...

The Future

Here's a proposal for discussion:

• Core JS library (no visualizations) lives in openspending.js (could keep openspendingjs name but rename might be useful to symbolize change)
• Every visualization gets its own separate repo named e.g. openspending.{vizname}.js

Lastly some process points:

• We should complete clean-up of this repo asap (possibly in tandem with doing the split-out)
• Anyone can develop a new viz but for it to live in openspending orgnanization it needs to "approved" (i.e. checked for quality, documentation, and that there is an ongoing maintainer).
• We document the set of visualizations people can use off-the-shelf somewhere prominent

Why?

• Core code is more clearly demarcated from apps/viz and can be maintained more easily going forward. It also can become a core dependency that people can more easily use in their own work without getting caught up in viz stuff
• By splitting out visualizations we make them easier to work on and more clearly distinct. Dependencies are also clearer etc.

[rufuspollock]

cc @tryggvib @nickstenning @mihi-tr wdyt?

[mihi-tr]

@rgrp I'd argue the other way around. For usability I'd rather have dependencies down.

e.g.

<script src="d3.v3.js"></script>
<script src="underscore.js"></script>
<script src="openspending.js"></script>

gives you all the visualizations with easy to use code. Even better - just depending on openspending.js...

My main reason for this is to make it as simple as possible to build satellite sites. People who know how to write visualizations can already use the openspending API and wrappers to do so. It's the people who can't who'll need a clear and easy way to do this.

[rufuspollock]

@mihi-tr but what happens if i'm building my own visualization and I just want the core openspending.js (I won't want all the other viz stuff e.g. the treemap, bubbletree etc).

Even for people building a site and wanting the viz may not want all viz - just a few specific ones.

If one really wants an all in one js file compiling down would not be that hard and would be part of stuff like creating the wordpress plugin etc. (I definitely agree we want to make it easier for people building satellite sites but I feel part of the way to get there is making this codebase easier to develop and contribute to).

[mihi-tr]

@rgrp honestly : for some of the stuff I don't even use the Openspending JS API. If you are just going for a specific lightweight viz building on top of the plain aggregation API is easier.