Yeah the ingredients list has no order (I’ll probably sort them by quantities or something, I’m not sure yet). It seems like it’s still kind of flaky, I’ll have to test a lot before releasing the first production version. Thanks for trying it out!
Thanks for trying it out and all the feedback, it’s really useful :D.
I think these two are the same bug, the recipe page doesn’t update properly, I’ll look into that. If you log out from the home screen, recipes should disappear.
All of this is related with one of the concerns I had about this approach (the 2nd one in particular, which I called “interoperability”). I will probably tackle it before release, and I think it should be fairly straightforward (just adding new operations for inconsistencies between the operations and the resource). But I haven’t looked into it yet.
In a nutshell, there are local models (stored in IndexedDB) and remote models (stored in the Solid POD). And when either of them is updated, the operations are sent to the other one. But the model itself is only reconstructed from the operations, so if there is something that has been modified without operations, it may missbehave. It’s also possible that there are bugs :). But I’m not testing in scenarios where changes happen outside of my app, yet.
I think this might be relevant and complementary related work: we are slowly researching our way towards ways to supporting multiple views on top of 1 authoritative write source. This idea resembles the CRDT, CQRS and Event Sourcing ideas. All of them needs something like an append-only container like in the Linked Data Event Streams spec. So that’s what we tried to realize: an LDES in a container-resource structure in LDP.
We published an LDES in LDP NPM library that allows you to do CRUD operations, abstracting away an append-only log of versions of this resource. We could also look into using on top your vocabulary instead of having a version-based approach.
→ https://www.npmjs.com/package/@treecg/versionawareldesinldp
However, we saw some limitations and had to apply quite some work-around to make this work and we’re now going to look at whether adding features in the core Solid spec would help managing append-only logs with less hassle. The paper discussing the limitations can be found here: https://raw.githubusercontent.com/woutslabbinck/papers/main/2022/Linked_Data_Event_Streams_in_Solid_containers.pdf
3 Likes
Hi all! I’m from the Braid group (braid dot org), and was part of the “Faster CRDTs” project with josephg mentioned above.
I’m pleased to meet you guys! We work on making state synchronization interoperable, by connecting with other projects and generalizing our approaches into common protocols.
Noel showed me a demo of this system in the SolidOS meeting this morning. It’s sweet! I have a suggestion on the architecture.
I’m seeing the following stack of abstractions:
State (of the recipe) <-
------------------------------ \
CRDT history + metastate |
------------------------------ notifications
RDF ^
------------------------------ /
HTTP (state transfer protocol) --
There’s a deep mismatch here inherited from HTTP. HTTP is a state transfer protocol (consider that REST stands for REpresentational State Transfer), but we are trying to do state synchronization over it. So we end up expressing the CRDT history itself as state, on top of RDF, and then each client computes its current state from that RDF state. We have recursive state, with state computed from state, and state at both the top and bottom of the stack.
Moreover, since HTTP itself doesn’t provide any subscription notification system, we have to use out-of-band solid notifications, which tell the client to re-fetch the state using the HTTP state transfer protocol. This requires extra roundtrips, and ends up more convoluted and messy than if we solve state synchronization in HTTP directly.
If we add CRDT+Notifications into HTTP directly, we can simplify the architecture quite a bit:
State of the recipe
------------------------
RDF
------------------------
HTTP with CRDT history + Notifications (state sync protocol)
This reduces round trips, simplifies the stack, and also makes it more general, because the synchronization features (versioning, subscriptions, patches, and CRDT/OT semantics) can interoperate beyond Solid, and work for any content-type, not just RDF.
We’re extending HTTP in this way with the Braid-HTTP protocol. I think we could add some powerful features to Solid with Braid-HTTP, and do it in a general way that interoperates with other systems too. Solid is uniquely poised to benefit from this work, because it’s built on top of HTTP, and also seeks interoperable standardized solutions. I think our projects are very complementary.
2 Likes
Hey @toomim, thanks for attending yesterday’s meeting and sharing your knowledge with us.
Your proposal seems very interesting, and it’d be great to see some proof of concept or demo using both Solid and Braid to understand this better; although I think I already get the gist of what you’re proposing.
I want to clarify something though. I think we already touched on this yesterday, but for others who didn’t attend the meeting. In order to have this working, we need to have something installed in the server such that the state sync protocol can be run outside of RDF and the Solid Protocol, right? If that’s the case, I think that could be a problem. The promise of Solid is that Solid Apps can work against any Solid POD implementation. If you add any other requirements, it won’t be a universal Solid App.
There have been already a couple of threads in this forum discussing similar issues:
However, that doesn’t mean this isn’t worth exploring. Eventually, if this proves to be a better solution, it could be included in the Solid Protocol itself.
1 Like
the state sync protocol [would] be run outside of RDF and the Solid Protocol, right?
No, I think we’d put it inside the HTTP part of the Solid Protocol, giving it the ability to communicate in terms of patches of RDF. We would extend the Solid protocol at the HTTP level.
You’re totally right that the next step would be to make a demo! We need a patch format for RDF. I’m not too familiar with RDF. What’s a good way to specify ranges of RDF data? Like, is there an equivalent of xpath or jsonpath for RDF? I want to describe an arbitrary part of RDF, within a resource at a URL, that has changed.
1 Like
Ok, well what I meant with “outside the Solid Protocol” was that it cannot be implemented just using the Solid Protocol (as it stands today). So we would indeed need to implement something in the server side.
I’m very focused on doing apps that can be used today by anyone who just has a Solid Pod, that’s why the solution I came up with is based on moving RDF triples from apps to a POD. But yeah, it would be great if the protocol itself handles this for us.
Although I will also say, there is always some tension on how complex Solid servers should be. Whenever any new feature is added to the protocol, one of the concerns is if it makes life harder for POD implementers. So depending how feasible your proposal is to implement, there could be some push back. Even if you provide a reference implementation, Solid Servers are implemented using a variety of languages.
Again, that doesn’t mean it’s not worth exploring. I think one way to get this moving is creating an issue in the solid/specification repository. If you browse the list of open issues, you’ll notice how many of them are ideas such as this. That way you could also get some feedback from people who are actually working on the spec (I only make apps, so I’m not too familiar with how the spec evolves).
Other than that, I think a demo would indeed go a long way.
I don’t know of any solution for that, but I’m sure someone in the community will know if such a thing exists. I suggest that you ask about it in the chat or opening a new thread in this forum. If you don’t get a reply, I can point you towards some people who I’m pretty sure will have an answer.
1 Like
Thanks a bunch @NoelDeMartin! I get where you’re coming from, and your approach makes total sense if you aren’t looking at modifying HTTP.
So I’m concluding that we can simplify things a lot by braidifying HTTP in Solid, and the next step is to make a demo.
Towards that end, we need a patch format, and Rahul just showed me the N3 Patch in Solid, which looks super viable, and also pointed out that we can issue JSON patches to any RDF expressed in JSON-LD. These sound like great approaches.
The next thing I’ll need is a demo app. Maybe I should take one of your apps and braidify it. I wonder if we have any simple Solid demo apps that let the user edit some data (maybe text) expressed as JSON-LD, with a server written in nodejs.
I made very simple app that only uses the native fetch to communicate with the POD, no external libraries (only one for authentication). So maybe you can look at that one: Solid Hello World (REST API)
It doesn’t “edit” any data, but I think you should be able to illustrate CRDTs with the two actions that are implemented (create a task, and complete a task). Or if you want, implementing renaming tasks shouldn’t be too difficult.
The thing about “expressed as JSON-LD” is not as simple as it seems. Any RDF data can be represented in many formats (turtle, JSON-LD, etc.). But not all Solid PODs support updating data using JSON-LD. In fact, the only format mandated by the spec is N3-patch. Although I can tell you that most PODs also support text/sparql-update as well, which is the format I use in my apps. However, I guess your implementation won’t go through the Solid Protocol as we discussed, so you should be able to translate your RDF into JSON-LD before doing anything with it and viceversa.
As per the server, you can look at the Community Solid Server (often referred to as “CSS”) which is written in NodeJS and is well documented. You can also implement plugins to add custom functionality, so maybe that’s all you need to do. I use it for local development as well, so it should be easy to tinker with it without the need to set up any real servers.
1 Like
Wow! This is super helpful! I’m now facing in the right direction and feel myself making progress. 
Thank you Noel!
1 Like