Blank Sigma Visualization (on AUK)

[ianmilligan1]

On some collections, instead of getting a nice interactive Sigma.js diagram, we get a blank graph and a console error claiming that we have some Invalid XML.

This GEXF file works in Gephi but doesn't work in Sigma js (verified by swapping it out with working GEXF files).

From Firefox, here's the whole console error:

uncaught exception: Container not found. turbolinks.self-1d1fddf91adc38ac2045c51f0a3e05ca97d07d24d15a4dcbf705009106489e69.js:6:1121
XML Parsing Error: XML declaration not well-formed
Location: http://localhost:3000/users/2/collections/9864
Line Number 1, Column 16: 9864:1:16
[Show/hide message details.] Error: Invalid XML: <?xml version="1.1" encoding="UTF-8"?>
<gexf xmlns="http://www.gexf.net/1.3draft"
         xmlns:viz="http://www.gexf.net/1.3draft/viz"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://www.gexf.net/1.3draft
         http://www.gexf.net/1.3draft/gexf.xsd"
         version="1.3">
<!-- Created by igraph -->
<meta lastmodifieddate="Thu Aug 23 15:59:47">
<creator>Graphpass filtering using Igraph by Archives Unleashed</creator>
<description> A Filtered Derivative Graph</description>
</meta>
  <graph id="G" defaultedgetype="directed">
  <attributes class="graph">
  </attributes>
  <attributes class="node">
  <attribute id="v_label" title="label" type="string"></attribute>
  <attribute id="v_id" title="id" type="string"></attribute>
  <attribute id="v_Degree" title="Degree" type="double"></attribute>
  <attribute id="v_WalkTrapModularity" title="WalkTrapModularity" type="double"></attribute>
  <attribute id="v_r" title="r" t… jquery3.self-06c43429d1047ce3f355da574d8a9750209971b8b1b8f264f91f5518c5fcc060.js:274:9

How Will This Be Resolved

If we can have the GEXF file linked above work in AUK.

[ruebot]

Weird. Wonder why it is producing <?xml version="1.1" encoding="UTF-8"?> instead of <?xml version="1.0" encoding="UTF-8"?>. The gexf files that display fine are all 1.0.

[ianmilligan1]

Heh I was just looking at that (since the xml linter I have only supports 1.1). That said, the Halifax Explosion GEXF file that I have (the file is here for downloading) and which works is also 1.1.

[ianmilligan1]

OK if you manually change <?xml version="1.1" encoding="UTF-8"?> to <?xml version="1.0" encoding="UTF-8"?> in a text editor, it renders in sigma js.

But in this case, the error documented in #25 appears (see screenshot below) – so these two issues are probably related.

[greebie]

I think the problem is related to #4 in this article -- you have to use \x0A instead of \n for XML 1.1 line breaks.

After my research, my opinion is that 1.1 is not worth supporting. 1.0 covers the vast majority of use cases and does not create problems like 1.1 does.

[ianmilligan1]

OK, agreed, XML 1.0 would be ideal I think – XML 1.1 throws up errors and I think there aren't too many linters/etc. that work out of the box with it (that said, I'm not really an XML guy).

[ruebot]

What's producing the 1.1 files? iGraph or GraphPass?

[greebie]

Looks like GraphPass when it creates the Gexf. Don't know why I added that. Might be when I tried to switch to Gexf 1.3.

[ruebot]

Ok. We need to be very cognizant of breaking changes like this. If there are any major version changes that effect things downstream in the stack, we need to talk about them. Not just implement them.

[ianmilligan1]

Yep the change happened here.

[greebie]

Gexf 1.2 is deprecated. Looking at the traceback on the Gexf 1.3 issue, there were requests to upgrade to 1.1. https://github.com/gephi/gexf/wiki

It's possible switching back may not fit the XML schema, but I'll check on this tomorrow.

[ianmilligan1]

Can you link to the GEXF 1.3 issue that you mentioned above?

Presumably we can revert some of the changes that appeared in this commit?

In any case, if we're doing changes like GEXF 1.2 -> 1.3 or XML 1.0 -> 1.1, in the future we can't have that sort of stuff sneaking into pull requests like that one above (which was solving the 50k node cut off value).

[greebie]

It was not an issue here on Graphpass, but on the switch on the Gephi repo from 1.2 -> 1.3.

It's possible that this was causing headaches for me, since Graphpass originated as a personal project for filtering networks in different ways.

It may make sense in future to take the components necessary for AUK and move them to a separate library.

[ianmilligan1]

Can you still link to issues when you mention them so we know what’s going on? :)

That might make sense as we are trying to use this in a production environment. In any case we need to get this to work with AUK again.

[greebie]

Better yet - when I do personal / research project changes, I will work from a fork starting now. Graphpass can be formalized into a proper AUT project.

[ruebot]

If 1.2 is deprecated, why does the 1.3 primer have a giant "WORK IN PROGRESS" message at the top of it, and there is no mention of 1.3 on the website?

This gexf issue about xml versions doesn't seem to be resolved either.

[greebie]

I don't know - I just know that I get a 1.2 deprecation warning when I use Gephi. I added an issue to the Gexf repo to see if I can get some clarity.

[ianmilligan1]

FWIW I tested @ruebot's branch on the Scarborough collection. Less errors on load so I think it's fixed (I'm testing on some UVic collections right now). But it still displayed a blank canvas.

I think that is because in the case of Scarborough (75/5715), this snuck in to the GEXF:

    <node id="n0" label="wix.com">
    <attvalues>
      <attvalue for="v_label" value="wix.com" />
      <attvalue for="v_id" value="d2a067fb67354846222e462d124233e0" />
      <attvalue for="v_Degree" value="1" />
      <attvalue for="v_WalkTrapModularity" value="0" />
      <attvalue for="v_r" value="35" />
      <attvalue for="v_g" value="217" />
      <attvalue for="v_b" value="211" />
      <attvalue for="v_x" value="-112.597" />
      <attvalue for="v_y" value="86.8064" />
    </attvalues>
      <viz:color r="35" g="217" b="211"></viz:color>
      <viz:size value="nan"></viz:size>
      <viz:position y="86.806363" x="-112.596639" z="0.0"></viz:position>
    </node>
    <node id="n1" label="youtube.com">
    <attvalues>
      <attvalue for="v_label" value="youtube.com" />
      <attvalue for="v_id" value="14dd5266c70789bdc806364df4586335" />
      <attvalue for="v_Degree" value="1" />
      <attvalue for="v_WalkTrapModularity" value="0" />
      <attvalue for="v_r" value="35" />
      <attvalue for="v_g" value="217" />
      <attvalue for="v_b" value="211" />
      <attvalue for="v_x" value="119.755" />
      <attvalue for="v_y" value="20.5159" />
    </attvalues>
      <viz:color r="35" g="217" b="211"></viz:color>
      <viz:size value="nan"></viz:size>
      <viz:position y="20.515895" x="119.755066" z="0.0"></viz:position>
    </node>

Note that both nodes have a size value of nan. Probably related to #25?