Has anyone given any thought as to how to setup personal pod servers and make it scale easily?
I’m new to Solid but from my understanding to participate one needs a pod to store ones personal data. This pod needs to have or work with an OAuth server to control access to the data in the pod. Having a full-time server costs money which most people don’t want to fork out for. But AWS lambda functions don’t cost anything to maintain when they are not being used (but are sleep so might have a startup time cost).
Cloud storage is also ridiculously cheap, so I was thinking that to setup a pod one just need a pod server written as a lambda function which keeps all its data in an S3 bucket. S3 buckets can also be configured to server content directly, so for public data the pod could just link to the bucket directly. The pod server lambda could server private data from a private S3 bucket and also be an OAuth / OpenID server.
I’m using AWS as an example, any of the cloud providers could be used or Kubernetes cluster. I was thinking if i could get the Solid server written as a lambda then an AWS (or any other provider) deployment config could be setup so someone with a little knowhow could deploy their own pod server via lambda functions and it wouldn’t cost anything to run while the Solid Revolution built up momentum.
Can anyone see a flaw in my thinking?
This does sound potentially workable for the read path, but it’s the write path where you’ll encounter the complexities
how so? If the lambda is acting as the intermediary between the bucket and the client, shouldn’t it be OK as it also has access to the ACL? Unless there is a concurrency thing in the spec?
Where can i find the pod protocol spec?
So what is the complication with the writing?
On the write path you’ll need to understand ACLs, parse sparql patch updates, likely do some content-negotiation, etc. On the read side, you’ll need to do ACLs, content negotiation, etc. all whilst mapping to permissions in S3.
I’m not saying it’s impossible, but there is a lot of work needed to build a compliant solid server.
Yes, it does seem like a complicated spec but not so bad if you break it down.
- ACLs can be independent of the storage medium (assuming file paths can be absolute which they are on most cloud buckets). There should already be a a lib for this, right? (there is an example perhaps a more general lib could be cooked up in Haxe and compiled to all major languages)
- “parse sparql patch updates”, there is a library for handling this already and probably others in other languages.
- Reads are simple and “content negotiation” can be handled by an abstraction stub, which is the only real work I see here. The S3 permissions can be completely open for the solid server which is the point of the server, isn’t it, to act as an intermediary?
The hardest part will be trying to figure out what the specs say
What about an embeddable pod?
I think that no matter what, self hosting in the traditional sense most likely will not appeal to most users, full stop. Even as a developer, I would not do it, unless it was near completely hands off.
Instead, what if the pod stays right on the device itself? SyncThing does an excellent job of syncing a phone and a laptop in a consumer friendly way. The same idea could sync a pod across all users devices. Even just a basic ZIP file import export based backup could be powerful.
It would always be on localhost at some reserved port, and would work just like a regular pod, apps could store data straight to your device that would never touch a cloud server, no monthly fee, no extra hardware, just a regular app with an embedded server and eventually some P2P replication.