Implement a generalized version AmiGO1.x's term tree with annotation counts

[kltm]

This has been requested by a couple of groups, and was apparently a loved feature of AmiGO 1.x.

There are likely two approaches to this. One is to add graph structure (as an option or when available) to accordion filters, by either having a browsable tree or a compact inferred view. This would likely be no small task and involve a more complicated configuration setup, in addition to solving the problems of getting it to work and display in such a small space.

Another approach would be create an entirely new widget/app that combines the accordion (see berkeleybop/bbop-js#17) with a graph or tree view. While skirting some of the issues of the first approach, most of the ways of doing this off the top of my head either run into GET size issues, require solr 4.x (see berkeleybop/bbop-js#16), and/or hammering the GOlr server with requests order the size of the current tree.

Any solution would need to have the base resolved in kltm/bbop-js before moving forward.

[cmungall]

is the difficulty in generating the counts efficiently, or in the UI aspects of integrating the browser with the facets?

(don't need to answer here just now, we can discuss tomorrow and post summary here)

[kltm]

You're too fast--maybe the updated summary captures that info now?
This is also in reference to a skype conversation I'm having with Tanya about this.

[raymond91125]

apologies for being presumptuous. but just in case if this (below link) is the type of counts you're talking about, it's one GOlr query per term in tree, not bad at all for speed.
http://mangolassi.caltech.edu/~azurebrd/cgi-bin/testing/amigo/wobr/amigo.cgi?action=Tree&goid=GO:0006950

[kltm]

Not at all! Let me give you a little background with my experience with it so far.

The one call per node is more or less what I'm trying to avoid (we have a demo around here like that). For example, I think the somehwat extreme (at this point) GO:0086037 would make live filtering using per-node queries a bit of a slog. With deeper terms at the volume of traffic that I expect on the main site, I'm rather worried that it would not scale very well. Since it would be hooked to the filters so people could select arbitrary filter combinations (a more generalized version of what is on AmiGO 1.x, which gets away with it by having only two mutually exclusive filters precalculated in the db and dumped to a static file per term on request: http://amigo.geneontology.org/cgi-bin/amigo/term_details?term=GO:0086037), browsing around could quickly generate hundreds or thousands http requests, slowing the user experience and potentially slowing the available backend pool for other users. For early tests, this was not so bad when the backend was nearby (although, for larger query numbers it still became slow, like in our matrix tool), but a factor when was is some distance. Also, given the responsiveness of the rest of the interface, users became frustrated and started sending even more requests.

Of course, built into that is some speculation about the final use profile and the ability of Solr/Jetty to take a hit--we'll have more idea of what real load looks like when we go live in a few weeks. Also, it looks like you've got your own client there proxying for the Solr server, which would take care of some of the distance concerns.

[cmungall]

Could you not do it in one call on the bioentity document using the
annotation closure facet counts?

[kltm]

Not a general solution--that would work for creating a separate app/widget that had a subset of filters from the bioentity personality/doc_type. But if we just want to get the A1 functionality in quickly, we could add that for 2.1 (along with the accordion widget spin-off). Or that could be 2.1 by itself--a nice short goal.

[kltm]

Think of 2.2, as that's now the A1completion target.

[kltm]

This has now been bumped to 2.3, with the expectation of getting it as part of the transition to Solr 5.x in berkeleybop/bbop-js#16.

[cmungall]

I think this is doable without solr 5.x if we restrict this to a drill down browser. Start with root node. Every expansion on a node N does a single facet count query constrained by R_closure=N which should provide counts for the majority of the child nodes (at times the tree will be imbalanced, can do subsequent queries OR we could simply have a direct_parent facet and query on that)

[kltm]

@cmungall This would no longer be a general solution, but rather a new widget built off of a drill-down (and a different ticket), which is fine.

This is different from above (#51 (comment)), but I'm not sure it would be performant either. Taking the example of locking onto CC and looking at the full facet counts for inferred annotation gave me a response in 15 seconds of full out wire and parsing. That bit is important since it will freeze the interface, and likely the browser, until completed. On slower connections this can be an issue: #62, and can lead to "JS not responding" errors.

If you're getting into the territory of having new fields to support this, I think that it would be easier to spend that time on the general solution (5.x), which would also open up a lot of new possibilities and fixes. Again, this can totally be done with what we have out there now, but I feel the hacks and workarounds necessary to make a nice fast widget for GO end users would be better spent on clearing a task that we need to do anyways to advance what we have.

[kltm]

We're finishing up a performant widget over in #163; I'm not even sure why this is marked as a separate item...