Idea: Using other backends for Solid Projects


#1

I’ve recently thought about how we could make solid apps work with other projects as a backend, like Dropbox, RemoteStorage, the SAFE network, IPFS, browser storage, etc. This would allow people to start using Solid Apps without requiring them to store the data in a Solid Pod.

Here are some thoughts from me, not that structured yet and likely I’m missing some points. If I find some time I will try to put it into practice, but that won’t be in the near future.

Limitations

  1. Features

Most backends will only be able to support a subset of the api spec. In particular, I think that GET/POST/PUT/… requests will work, but features like Websockets and SPARQL requests likely won’t. Access Control would probably be hard to implement, likely not possible for some.

  1. Making apps compatible

This could be done either by modifying the code of the app (e.g. adding different libraries), by making a browser extension which replaces libraries (e.g. solid-auth-client-compatible instead of solid-auth-client) or by running a mini-server on localhost which provides a solid pod endpoint.

  1. Usability

Installing browser extensions or running a mini-server isn’t very user friendly. Making the apps include compatibility libraries seems rather good, the main problem is how to select the backend (e.g. creating a configuration menu).

Ideas for implementation

0. Making libraries which provide login and fetch methods for specific backends

I think for all implementations it would be useful to have libraries which do authentication and implement a subset of solid api-rest. They could use a common library for making solid-like responses (e.g. solid-rest). For example “solid-proxy-dropbox” and “solid-proxy-localstorage” libraries.

1. Including a special libraries

We could create a solid-auth-client-interoperable library and then use this in apps. It would need a way to set the preferred backend (e.g. auth.setBackend(SAFE)), a modified login method, and a modified fetch method. The fetch and login method would be specific to the chosen backend and could be dynamically loaded (import('./solid-proxy-dropbox')) to minimize loading time.

The solid-auth-client-compatible library could then be used inisde other libraries, or as parameter if the library allows it (like in the new solid-file-client). If necessary one could also override the window.fetch method to the compatible one.

2. Running a localhost

We could create a mini-server which serves a solid-pod endpoint locally which forwards to the specific backends. When it gets a request, it would make a request to the specific backend (e.g. dropbox), make the response solid-compatible, and then return it. Authentication would be over a local webpage.

3. Using Web Extension to replace libraries

We could create a web extension which replaces requests to import solid-auth-client with a solid-auth-client-compatible version. The user could choose the preferred storage and login in the web extension, and the web extension could use a content-script to update the settings in the app. When the app makes a request, it uses the fetch from solid-auth-client-compatible without knowing it.

Overview/Summary

Implementaion Usability Modify apps Install Starting Access Control
Including library good yes no no no
Web Extension ok no yes no no
Local Server bad no yes yes possible

I think that if we want to do this, my current suggestion is to do it in this order. This will probably change depending on discussions.

  1. Think about an interface for solid-proxy-dropbox and co
  2. Make such an interface for one or two backends
  3. Make solid-auth-client-compatible library which can switch between these proxies
  4. Make user guide how to make something compatible
  5. Make Web Extension which injects solid-auth-client-compatible

For this, I think the proxies and the authentication part will be the hardest things (regarding implementation and design).


#2

As a side note: This is similar to the work done by @happybeing to get Solid on SAFE working, but trying to make it work in a more general way and thus allowing a more diverse set of backends.


#3

Really good to see this @A_A and I think you’ve presented sensible options. To clarify, the approach I’ve taken with SAFE fits with your first option. At this point my implementation is not generalised in the way you suggest but would fit very well into the ‘generic library’ approach. I favour this because it creates only a small amount of work for the app developer, and no change for the user (no server to set up and run, no extension to download etc). The modified apps will ‘just work’.

One thing that I suspect can be done which you suggest might not be feasible is SPARQL support. I’ve not attempted to support this yet, but my intention was that this be done on the client rather than handed over to the server. I believe there are already client side JavaScript libraries that do this, so I think it is feasible to add that functionality into the ‘generic library’ approach, or indeed the others you described.


#4

I haven’t used SPARQL, but afaik it supports requests which conditionally load data from multiple resources. My concern is that when we support it, this will be harder to implement and also can lead to lots of requests if it is not done on the server itself. But I guess it is possible.


#5

Something that could slightly complicate the use of such diverse backends are API keys. If every app needs its own API keys for all backends it want to support it will be additional work to use solid-auth-client-compatible (or similar). I guess that it’s not possible to have an API key for a library, but I don’t have experience with this yet.


#6

There may be a need for API key’s in some cases, but IIRC remotestorage.io were able to support Google and Dropbox without this (or at least without users being aware of this), using a generalised UI to authorise with the backend chosen by the user. Worst case would be that each app would need to have a key for each backend that requires this.

It’s not an issue when using SAFE Network as the backend because it has an authorisation mechanism that can be handled using the login flow.


#7

Yes, API keys are not relevant for all use cases, I guess that many will work without them. For Dropbox and Google Drive it seems necessary though, remoteStorage also uses API keys for them:

remoteStorage.setApiKeys({
  dropbox: 'your-app-key',
  googledrive: 'your-client-id'
});

(from their tutorial)


#8

Yes, we are building cross-platform compatibility into all we’re working on through JSON and XML content. Then it will work will all these sorts of “World saving” solutions like Solid and Holo, and more to come.