Hey,
Has anyone given any thought as to how to setup personal pod servers and make it scale easily?
I’m new to Solid but from my understanding to participate one needs a pod to store ones personal data. This pod needs to have or work with an OAuth server to control access to the data in the pod. Having a full-time server costs money which most people don’t want to fork out for. But AWS lambda functions don’t cost anything to maintain when they are not being used (but are sleep so might have a startup time cost).
Cloud storage is also ridiculously cheap, so I was thinking that to setup a pod one just need a pod server written as a lambda function which keeps all its data in an S3 bucket. S3 buckets can also be configured to server content directly, so for public data the pod could just link to the bucket directly. The pod server lambda could server private data from a private S3 bucket and also be an OAuth / OpenID server.
I’m using AWS as an example, any of the cloud providers could be used or Kubernetes cluster. I was thinking if i could get the Solid server written as a lambda then an AWS (or any other provider) deployment config could be setup so someone with a little knowhow could deploy their own pod server via lambda functions and it wouldn’t cost anything to run while the Solid Revolution built up momentum.
Can anyone see a flaw in my thinking?
This does sound potentially workable for the read path, but it’s the write path where you’ll encounter the complexities
1 Like
how so? If the lambda is acting as the intermediary between the bucket and the client, shouldn’t it be OK as it also has access to the ACL? Unless there is a concurrency thing in the spec?
Where can i find the pod protocol spec?
So what is the complication with the writing?
On the write path you’ll need to understand ACLs, parse sparql patch updates, likely do some content-negotiation, etc. On the read side, you’ll need to do ACLs, content negotiation, etc. all whilst mapping to permissions in S3.
I’m not saying it’s impossible, but there is a lot of work needed to build a compliant solid server.
Yes, it does seem like a complicated spec but not so bad if you break it down.
- ACLs can be independent of the storage medium (assuming file paths can be absolute which they are on most cloud buckets). There should already be a a lib for this, right? (there is an example perhaps a more general lib could be cooked up in Haxe and compiled to all major languages)
- “parse sparql patch updates”, there is a library for handling this already and probably others in other languages.
- Reads are simple and “content negotiation” can be handled by an abstraction stub, which is the only real work I see here. The S3 permissions can be completely open for the solid server which is the point of the server, isn’t it, to act as an intermediary?
The hardest part will be trying to figure out what the specs say 
What about an embeddable pod?
I think that no matter what, self hosting in the traditional sense most likely will not appeal to most users, full stop. Even as a developer, I would not do it, unless it was near completely hands off.
Instead, what if the pod stays right on the device itself? SyncThing does an excellent job of syncing a phone and a laptop in a consumer friendly way. The same idea could sync a pod across all users devices. Even just a basic ZIP file import export based backup could be powerful.
It would always be on localhost at some reserved port, and would work just like a regular pod, apps could store data straight to your device that would never touch a cloud server, no monthly fee, no extra hardware, just a regular app with an embedded server and eventually some P2P replication.
2 Likes
Yeah, I was thinking that I’d make a Pod server that works off lambda/cloud functions so your not paying for it when its not being used and it can scale up as needed and it has access to cloud storage i.e. buckets which are also very cheap.
TO make it easy to deploy for devs I’d make a repo for CDK or Terraform etc with one of those buttons for click to deploy. If you have your own domain you can clone and update a config file of some sort.
Your experience may vary, but from what I’ve heard, AWS lambda and similar turn out to actually be more expensive (and complicated to maintain) than just a small VPS with Object Storage (e.g., a $5 DigitalOcean droplet.
Something that may be worth a look at is can Solid be decentralized? Could it primarily run on someone’s home internet as just a device you plug-in? Perhaps with mirroring to keep it always available?
I guess the thing about lambdas is you only pay for when its used. So in the beginning, when one might go months without anyone/thing querying your Pod you will pay nothing for hosting, rather than a fixed amount every month (which i think students would not want to do). So if your backing up to cloud storage anyway, adding a Pod interface to your data might make that data more accessible and cost you nothing if you don’t use it. An easy and cheap opt-in approach.
From the intro to the Solid Spec
An overarching design goal of the Solid ecosystem is to be evolvable and to provide fundamental affordances for decentralised Web applications for information exchange
So it appears to be a goal but weather it can be decentralised is another question. A question that nobody seems to be asking is: What about data persistence? Self hosting anything is notoriously unreliable and usually doesn’t stand the test of time (blogs older than 10 years are usually only accessible on archive.org)
I would guess so if their NAS had HTTPS facilities. Why not? I don’t know if it’s easy atm but without someone providing a simple implementation of the standards involved almost nobody will go out of their way to get it there.