I’m the developer behind Phojo, a daily photo journal. I currently use Google’s Firestore as the database and Google Storage for storing photo files, but I am considering incorporating Solid as an opt-in feature for users who would prefer to have greater privacy/implicit ownership of their data. (I already provide export options for users so I already consider it ‘their data’.)
I have two questions.
Question 1: Does Solid provide any kind of database feature?
If I would need to, for example, query a Pods data structure for "All daily photo entries with a date > 2020-05-10 && visibility = ‘Public’ ", is there any kind of way to do this?
If no, then I would be forced to keep using Firestore for the DB functionality, but I guess I could also keep a ‘copy’ of the data in the user’s Pod as a sort of continuous backup of the data.
Question 2: Has anyone here had experience of an opt-in Pod feature?
And by that, I mean you offer a normal email/password account setup with user data kept in an app-specific silo for the ‘average’ user who does not have a Pod and wouldn’t go through the ‘hassle’ of setting one up, and then allow ‘Pod-savvy’ users the option of moving their data from the silo to their pod?
It seems like it will double the complexity of the app, but perhaps good guidelines have been established for this?
Many thanks for reading this far, and grateful for answers.
Gustav
Phojo is exactly the type of app I’d like to have Solid support!
That said, Solid puts relatively strict requirements on your data model, requiring them in the end to be stored as RDF. Generally, I expect it to be hard to model that post-facto onto an existing app, especially as additional functionality - though using it as an export target could certainly work relatively painlessly.
So to answer your question 1: no, Solid does not support the ability to run queries like the one you described. Some people would like it to support SPARQL, and who knows, maybe someday it will - but I wouldn’t hold my breath for that to be reliable any time soon. An alternative option is to build your own indexes while adding photos.
As for question 2: I know of two apps that have an opt-in Pod feature, both by @NoelDeMartin: Media Kraken and Focus. For users who do not use a Pod, they store data in browser storage. They use Noel’s Soukai library for that.
Hope that helps, and feel free to ask additional questions
To clarify, I don’t know how it if this helps any particular app. But you don’t have to store data as RDF on Solid. It is built to support that, and some of the gains from Solid that we hope to achieve are built on that assumption, but you can store any raw data (images, binaries, flat files, CSV) as objects within RDF containers.
With regard to SPARQL is true that it’s not pay of the Solid protocol and I understand there were several reasons for that, but you can still use a SPARQL library in the client (eg browser, or app) to give you the same functionality, but without the pros and cons of it running in a severe. I haven’t done this myself yet but I’m assured it can be done.
Solid-Rest provides a plug-in system which can treat whatever backend is plugged in as if it were a pod (i.e. use rdflib and other tools against it). Currently supported backends include the file system and any backend supported by BrowserFS ( in-browser IndexedDB, Native File API, Dropbox, etc.). So any of these backends can serve as a non-pod (but pod-like) experience. The gitter-solid app, in a develpment version, is using this feature for opt-in deployment to either a pod on a server or a file system that has no server.
As Vincent mentioned, I’ve been building my apps with two “databases”: Solid and browser storage (Focus uses localStorage and Media Kraken uses IndexedDB). But I think at this point everything is very experimental. I’m the only one using Soukai that I know of, and I needed to modify it for many use-cases. It’s also true that I’m not an expert in Solid or Semantic Data, I’m still learning. But it’s possible.
In my opinion the only real requirement for an app to be compatible with Solid is that it’s modeled with RDF. Like Mark mentions it’s not strictly required, but if you don’t do it your app won’t be interoperable with other apps. Which is, in my opinion, the point of Solid. I guess if your app only manages a bunch of images you don’t need a lot of modeling, but it’s not usually the case. For example you may have albums of photos and it’d be nice that they are modeled with RDF.
When it comes to querying the data… Yes, that’s a big problem at the moment. It’s the reason why for Media Kraken I cache all data in the browser. I started a discussion about this a while ago in case you want to take a look: State of the art for querying large containers.
So, it seems to me that Solid might be a place where I may ‘store’ information, but that it won’t let me query it very easily. So, I might still need a central database held in my own silo. But, I could let people store 1) their binary photo data and 2) a copy of their DB data, in their Solid, and that way, they know that even if I suddenly go bankrupt that their data is still safe with them and they can take it elsewhere. Also, if only they have read permission on the photo data, then they know that no one else can view it, if they want privacy. If they want to share it, or re-use it, then I need to also model it properly with RDF.
I’m would like to expand on @NoelDeMartin’s definition of a Solid-compatible storage. I would suggest a three-tiered definition of storage spaces:
RDF Storage : a space that can contain RDF (lindked data) resources
Solid-Compatible storage : a space that, via a server or middleware, can be accessed with solid-logic (see below)
Serverless pod a space that can be accessed with solid-logic and has key pod structures like a profile and a preferences file such that it can be used with a standard Solid follow-your-nose principles.
By solid-logic I mean the ability to use the methods and business logic in the solid-logic package. This includes ability to find auxiliary resources (like ACL files) from the header of GET/HEAD responses, the ability to get a turtle representation of a folder, the ability to use any of the solid-logic methods such as loadProfile() and loadIndexes(), the ability for an app to follow-its-nose from a webId to the various preferences, type indexes, and storage spaces it points to.
Middleware that implements a serverless pod should be compatible with the core SolidOS libraries (rdflib, mashlib, solid-ui, solid-panes, solid-logic). In other words, one should be able to run the standard Solid databrowser and most of its panes against the serverless pod.
Solid-rest implements all of solid-logic and has a createServerlessPod() method that supports creating a serverless pod on any of its backends (file, in-browser, or cloud). It has successfully been used with the databrowser against both local file systems and in-browser IndexedDB. When operating in nodejs, the easiest way to use solid-rest, is via solid-node-client. In the browser, it can be run in conjunction with either solid-client-authn-js or solid-auth-client, see the README for details. (or just ask me here if you have questions/problems).
Note : both solid-rest and solid-node-client are in a current phase of rapid development so make sure you have the latest npm version of both (as of 5 minutes ago, that would be solid-rest 1.2.8 and solid-node-client 1.2.4).
