Live shared Pod Data – making it happen

I’ve been working on m-ld, a software component for live information sharing [1]. It uses RDF as its native data representation, and CRDTs for eventual consistency. So far I’ve been focused on making m-ld generally applicable and useful in many architectures, whether RDF-based or not, by working top-down from a JSON API.

I’m aware of the need for convergence though, and I’d like to start a deliberate project to lean-in to Solid with m-ld: to make interoperation with Solid seamless.

This could have lots of upsides, for example working directly or indirectly on:

  • Multi-collaborator editable Pod data (e.g. for private group data, as an extension of personal data)

  • Local-first offline editing of Pod data (e.g. [2])

  • Reliable caching of Pod data at the edge (e.g. [3])

  • A generally applicable patch distribution model for Pods [4]

  • An authorisation and identity model for m-ld (currently delegated to the owning app [5])

I have in mind to apply for grant funding for this project. But I think the support of the Solid community is even more important.

Is this timely?

Would anyone like to work on this with me?

Would anyone like to work on this with me if we can get funding? :slight_smile:

Any other thoughts?


[1] Topic: A project for live data sharing
[2] Solid FAQ: Is it possible to use Solid offline (at least partially)?
[3] Topic: Constructive criticism from an experienced developer:

[4] GitHub: Directions for patch distribution
[5] m-ld.org/doc/#security

4 Likes

Is your prior work available for review?

Hi Jeff. Sure:

Happy to talk through anything. I’m starting to put together the proposal for this project now.

Thanks to Sarven Capadisli’s prompt on solid/chat, I have scraped more of the Solid community for motivation and prior work on this area. I’m sure the references below are not yet comprehensive, but it’s a start.

Opinion

Support for live collaborative editing of Pod data in the corpus of documented use-cases and requirements is found only in allusion, and not called out as motivating for the Solid specification. However, engineers have made calls for patch-passing data distribution mechanisms, for strong reasons which do include enablement of user features. This is a disconnect.

There is an accelerating trend towards remote collaboration in software, including live collaborative editing becoming table stakes for many kinds of user content; as well as version control with branching and merging, for others. I think engineers are aware of this, but it’s hard to capture feature requirements for new applications that users don’t have in front of them yet. This is especially a problem when other motivations for patch-passing are to do with performance and resilience, things that are only perceived by the user when they’re broken.

Data

solid/user-stories

As a developer, I want to be able to subscribe to a stream of pod events #22 – Open

  • “Event-driven architectures are very powerful in that they allow to create loosely coupled reactive systems.”
  • This is marginal as a user story, but refers to other feature requests:
    • As a user I want to be able to have an audit log of what happened to my data #12
    • As a user I want to be able to restore previous data #10

solid/specification

Standardizing state changes in resources (history, undo, sync) #161 – Open

  • Ruben V: “but then we should probably have collaborative editing as an issue/use case”

Act on the latest version of the resource state #91 – Open

  • “This discussion is re-discovering wiki edit conflicts, and source code change management, and ACID-style databasing, among other things.”

What server-based notification support is required? #49 – Open

solid/data-interoperability-panel

Problems and Goals for Interoperability, Collaboration, and Security in a Solid Pod

  • “data will be manipulated not only by different applications, but also by different people or automated agents.”
  • Does not mention live or realtime manipulations

w3

Collaboration cases in pwp-ucr, but live collaboration is not specified:

  • UC 72: “Andreas is working on his first collaborative research paper with a fellow student.”
  • UC 20: “Tanya and Kelly are collaborating on curriculum for the upcoming school year.”

Annotation cases for ‘live’ data in dpub-annotation-uc:

Open Data collaboration in dwbp-ucr:

Socialwg/Social API/User stories - W3C Wiki

4 Likes