Visualisation libraries in JavaScript

Build a Solid App
1

I’m looking for recommendations for visualisation libraries in JavaScript. I’m going to do a search so am looking for your recommendations and warnings! I see Spoggy uses vssjs.js which looks good but there are quite a few others too.

My priorities:

  • I’m displaying RDF graphs (of course :smile:)
  • I’ll want to represent data in multiple ways (entity relationship, timeline, tabular)
  • UI is crucial (flexible, explorable, interactive: easy zoom/pan, ability to configure representation with icons/labels/attributes/styles, provide context menus, handle input events)
  • in time I’d like this to be a UI for exploring large (unlimited size?) datasets but am not expecting these libraries to be that clever - but if you know of any designed for that please say.
  • it would though be good if the library is optimised for treating large amounts of in memory data efficiently when the viewport is scaled out (eg switching to simpler representation of elements when detail cannot be discerned, so large scale icon -> simpler icon -> tiny square -> dot at very small scale)
2

You’ve probably already heard about it, but to be sure: the main library for data visualisation in in-browser Javascript is D3.

2 Likes
3

A D3 force simulation, like in https://github.com/d3/d3-force/blob/master/README.md would work.
Putting it in SVG could be added as a serialization option to rdflib.js. But for very large datasets I think some kind of 3 dimensional rendering will probably be necessary.

1 Like
4

@happybeing D3 is also used by Webvowl . They use a D3 force simulation, which I think would be not too hard to add to rdflib.js. The changes to add an svg serialization option I think can be pretty much localized to two files: serialize.js and serializer.js.

1 Like
5

Thanks everyone, D3 does look promising. I have lots to follow up from here and other places which will keep me busy.

This never occurred to me. I’m not sure if it fits for my purposes yet but I’d like to keep it in mind. Can you explain how this would be used to implement things?

I see that rdflib.js is useful for retrieving and modifying RDF resources over LDP, and that it can consume and generate (serialise) different formats, as well as creating, editing, querying RDF graphs in memory.

I’m not sure how an SVG serialisation would fit into this. I can see you could use it to generate a visualisation of a graph, and display this in an HTML document. But I’m wondering if this would make sense, rather than using D3 to provide the generation plus interaction?

6

The code in the serializer visits the nodes and enumerates the links so you could use that to serialize to SVG, which is an XML format.

The animation itself will be written to the SVG serialization, so the interaction with the animation such as stretching links and moving nodes will be taken care of in the browser.

As for actually editing nodes, onclick events could be added to the SVG that would call some JavaScript to do that. That would maybe be better for the second iteration.

For big datasets like biologists use the graphs would get too big and detailed and slow and not so useful. But if you look at foaf in WebVowl it looks pretty good and is useful, I think.

1 Like
7

Thanks, that helps. I need to understand how these parts fit together better, so it’s time to decide what to start poking.

Feels like I’m standing looking at a large number of bears :bear:, and I have only a stick!

1 Like
8

For Spoggy, I used @jeffz solid-file-client that give me folder/subfolders/files in json and I wrote a parser to retrieve file content, that give me nodes and edges. Checking next if the node with the same label already exist, I use it else I create it

3 Likes
9

Thanks, I haven’t looked at spoggy code yet. I can see it would be simple to start with solid-file-client, so quicker to get something going, but think it will be more flexible to get the graph directly: GET & parse RDF and traverse the graph directly (eg with rdflib.js).

1 Like
10

I’ve used D3 to visualize RDF a lot in the past, and indeed it is super-cool (and yeah, the force-directed graph is always going to be your go-to starting point). But I’d just forewarn you that if you haven’t used it before, it has a steep learning curve. It’s a very low-level library (meaning you can make it do almost anything!), but that also means you have to write a lot of code, and learn how all that code needs to work together. In my opinion, it’s a major time investment, but really worth it if you want a kick-ass UI.

Quick tip (I’d start out using Canvas and not SVG - you can use either, and SVG is slightly easier to work with, but Canvas will perform slightly better for small graphs (e.g. up to a couple of hundred nodes), but much better for big graphs (e.g. up to 10,000 nodes)).

3 Likes
11

Thanks @pmcb55, you’ve confirmed a few things and taken my understanding forward significantly too. I’ve done some basic work on getting d3 to work in a https://svelte.dev app and found some really cool things using it to show me what it can do and how to use it. I’m fine with the learning curve (he says :roll_eyes:) and had an inkling about Canvas from a conversation on the sveltejs chat, so will aim in that direction.

1 Like
12

I think that… if the banner of rdflib.js (or comunica) serialization to graphics formats (like SVG or canvas) falls, then someone must pick it up because like some other recent apps it will open the way for a lot more. For one thing, apps will have a way to show off their rdf and the possibilities of connecting it with other apps, and that its different from just something run from a file server. For another thing, it will be a short hop from there to editing the rdf visually.

13

Still early days but I’m not sure that’s the way I want to go because I’m not sure it will fit with the kind of interactive, multi visualisation UI I have in mind. D3 and sveltejs each have their own way of supporting this, by helping you integrate the graphic elements and the data. I’m not sure that can work with SVG generated by another library - I think this all needs to be done by the code which generates the view.

For example D3 includes animation (they call it simulation) of a graph layout as it adjusts, so I’m not sure how that could work if you generate SVG from rdflib.js

What do you think, could your suggestion produce the interactive animated layout in spoggy for example?

14

I think it should be in rdflib.js (or comunica) itself so many apps could use it. I don’t think I have the juice to do it, but somebody should. What I mean is serializing it in rdflib.js with D3 into SVG or canvas. I believe that when you move a node in the D3 force animation and stretch the links for example, that it will not require a reserialization, that the SVG doesn’t change. I may be wrong on that, it’s been a while.

I think when you serialize it you could include onclick events for a right click editing menu. I was working on this a little about a year ago but I wasn’t thinking about canvas or about editing, just displaying in SVG. A couple of tricky things I recall. When you serialize it there can be cycles, so it’s not as simple as a tree. There is no root to start with. Also there could be unconnected nodes. It might be necessary to follow a canonical way to serialize so that it’s the same way every time. I think in the docs for http signatures they define a canonical serialization.

15

I’m not that familiar with Spoggy. For it’s visualization, does it make assumptions about the container structure in a pod, or can it render any kind of graph?

16

What I create can easily take its RDF from rdflib.js but I thinkcan be more flexible as a self contained component. Still learning though.

I can see what you have in mind would be helpful for some but also limited.

I’m not sure how useful it would be. It would not be that hard to write per application, and could then be customised for the application rather than one size fits all.

Spoggy reads and visualises the RDF.

17

for Spoggy I used this type of data https://visjs.github.io/vis-network/examples/network/basic_usage/standalone.html,

<script type="text/javascript">
  // create an array with nodes
  var nodes = new vis.DataSet([
    { id: 1, label: "Node 1" },
    { id: 2, label: "Node 2" },
    { id: 3, label: "Node 3" },
    { id: 4, label: "Node 4" },
    { id: 5, label: "Node 5" }
  ]);

  // create an array with edges
  var edges = new vis.DataSet([
    { from: 1, to: 3 },
    { from: 1, to: 2 },
    { from: 2, to: 4 },
    { from: 2, to: 5 },
    { from: 3, to: 3 }
  ]);

  // create a network
  var container = document.getElementById("mynetwork");
  var data = {
    nodes: nodes,
    edges: edges
  };
  var options = {};
  var network = new vis.Network(container, data, options);
</script>

with a self made parser to make nodes & edges from triples.

I’ve used $rdf.graph() from rdflib

          let store = $rdf.graph()
          console.log(store)
          try {
            $rdf.parse(result, store, doc.uri, 'text/turtle');
          }
          catch(error) {
            alert(error);
          }

          /*  $rdf.parse(result, store,base, mimeType)
          console.log("STORE",store)*/
          data = statements2vis(store.statements);
          callback({data:data,params:params})
...

and then the statements2vis function https://github.com/scenaristeur/spoggy-simple/blob/8a0ccbe081706c32699234a673c85cbb6eb682b9/js/import-export.js#L663

to build data with nodes & edges that i push to the vis graph.
Feel free to ask if you want more :wink:

2 Likes
18

Thank you! That is a much better and simpler way than I had in mind :grinning:.

1 Like
19

That form of JavaScript object seems common for different visualisation libraries like D3, so it’s a good way to handle this I think.

20

For Solidash, I aim to review the code of Spoggy to build a widget, that connect to POD, or other Rdf endpoint and permit to explore it in a Spoggy way,
as a lib / module / web-components that could be simply embedded in any html page.
That module could also be used to create mindmaps/ graphs , as Spoggy does and write that on the POD we want.

Adding some filters to hide some ‘technicals triples’ , put a search engine to find, highlight and focus a word in a Pod’s graph, and in user’s friends Pods…

That’s what I think a good visualization could start to be … That’s a lot of work and my time is short, so if someone wants to try with me, let’s go!

2 Likes