Absolutely agree, that’s the essence of a data coop: One shared platform for data, underneath the apps.
Agree that the issue is how to get people to create and use a shared platform of any kind, but I think a coop would be much easier to build on, and it’s the apps that will bring people in, not the data platform.
Not sure what you would replace servers with, because ultimately it’s always servers in the end. The idea of a coop is that we own and control the servers, through the coop. What could be better?
How do you envisage implementing this, specifically with interoperability between apps (so any app can read, understand, write data of any other app)? This is the essence of Solid LDP, so would you do this in some other way, or is your design different, different goals?
What could be better?
No servers
I think one aim of Solid pods as self administered or as a service, or indeed owned by a coop is to protect the servers / services from exploitation, centralisation and take-over. I think a coop is an improvement in some aspects of that, but brings other problems and unknowns.
Both retain servers along with the hassle and risk of servers and services.
Even if a trusted entity can be relied upon as protection against take-over and centralisation (which I think you can argue for a coop), you are left with funding/maintenance, backup and security. All very hard and constantly failing. I think both Solid pods and a coop mitigate these risks to an extent but am not satisfied with the solution because servers are so obviously broken and vulnerable.
Another issue is the business model. I haven’t mentioned that yet, but I think it is also crucial to mitigate centralisation risks. For Solid or coop, apps need to be developed and paid for. So I’m not sure how that would work with a coop. I know there are ideas around this for Solid, but am not sure they would transfer to the coop model unless it was based on Solid (or an equivalent) so I wonder if you have thoughts on that.
I’m not saying there isn’t a role for coops. I think they are a valuable component in democratising technology and business which important goals to me. I’m trying to understand the ways in which they can solve certain issues and where their limits lie. You could just run Solid pods with a coop, but you seem to be advocating an alternative rather than a combination.
In a coop, interoperability of applications works the same way that many applications can share a database: they either share only data in the database, or one application acts as a layer for the other.
Applications talk to services that talk to data stores. And whatever hosts services is a server, in my books. There’s no getting away from servers unless everything is local. Not much is local on the internet. Maybe I’m misunderstanding what you mean by servers?
I don’t see how decentralization eliminates exploitation. If you can put the data together again to use it, somebody else can put it together to exploit it, except for access controls. You need access controls whether you centralize or decentralize. It’s the access controls that make the difference, not decentralization?
I don’t know how the business model works in Solid yet. TSIS presumes the coop is part of a funded app, which is TSIS. Getting there is a pipe dream at this point. I just know that if I was building a new Facebook, I’d chose a simpler consistent data model, so a coop seems likelier to work than free-form decentralization. I’m saying “I don’t know, but Solid seems (much) harder to make work.”
Thanks very much for answering my questions which has given me a better picture of what you have in mind.
I think there are challenges, as I’ve mentioned related to funding and creating effectively a bespoke solution. One of the benefits of Solid is scalability - someone developing an app can hope that it will interest anyone who is using Solid storage, rather than the coop having to fund all the apps because they are only usable for the coop of whatever size - so a much higher development cost to fund per user. You could share costs between coops, but I think that’s a challenge to scale because it is hard to start each one, and those doing that work will have their own vision and ideas about how it will work, what apps, how they will be built etc. I think all that is made much easier with a common standard that decouples development from a particular group, so I would be looking to combine Solid with the coop model rather than seeing it as an alternative.
This is perhaps where I don’t see coops as simpler than Solid. In theory yes, just build a single bespoke system and set of apps. But hard to fund and do that work, and hard to scale because I don’t see it will be easy to replicate, or rather that it is not likely to be attractive to each new coop. I might be wrong about that.
Moving to other ideas that might be relevant. Have you seen @Glensimister’s post about his Devolution app?
He’s built a demo of this running on SAFE Network also, including a video here.
Devolution is one way of solving some of the problems I’ve highlighted within a coop model so it may be interesting to you, including perhaps helping with the setting up and running an online coop itself. It could run on Solid or SAFE.
The points I’ve made about servers would be addressed by using SAFE Network which also provides democratising business models and gives users permanent data security. And if we put Solid on SAFE we have best of all worlds, so that is my particular thing.
Solid (along with a series of W3C specifications) provides a toolset standardizing the interface between data and server logic (Quotes HERE). That’s technically what it is, no matter you willing to use it for “decentralization”, “centralized” coop or coops, or any other things crazy. A standardized interface can reduce the cost of design and developer adaption, so it would be more friendly if TSIS coops provide such an interface (No conflicts with any existing design, just extra endpoints).
You have a great question about the “business model”, which is also my team and the Toronto community now seeking. One thing we found is about the growing global regulations of Data Portability (eg. new GDPR). Obviously, exporting/transferring data without a cross-application standard could offer very limited usage thus with less applicability. I foresee this would become one of the mandated work to do even you think it’s “much harder” sooner or later.
We actually have a few other thoughts about what we can utilize Solid/PODs for the business models. I’ll bring the discussion in our meetup on May 13. I see you are also in Toronto, welcome to join us and bring different opinions.
You have some very good points. I see a coop at the core, for shared data that needs to be standardized and uniquely recorded (once), and “something else” around it, for all the other data.
I don’t think either Solid or coop solves both the problem of paying for storage as well as sharing storage for a new Facebook (with hundreds of millions of users and so many unique data models in Solid). Does that sum up the problem?
Thanks for the leads to SAFE. I do see how it addresses the economic aspect, but I don’t see building a new facebook or other major application on top of it. How do you make so many bits of spare capacity on so many desktops (which are sometimes turned off) into an effective application? It’s a solution but is it feasible? Maybe I’m not getting the point.
Why does everyone seem to think that running this on a Raspberry Pi means success? So you’ve got that backed up, encrypted and in multiple locations? You’ve got it on a UPS with multiple circuits? Multiple service providers? You’ve got DNSSEC setup, right? Even if you said yes to everyone of those it’s completely insane to think that would even begin to be a solution for most people is crazy.
This is a complete misrepresentation of what a coop is.
I agree scaling is one of the things to be worked out with Solid and SAFE, and indeed Solid on SAFE. But in the case of SAFE, even though data is stored in a decentralised way it is designed to cope with individual computers coming and going. So applications will access storage through a standard API and not be affected by that.
The challenge for large scale applications is to implement features on a different architecture than we are used to: simple standard backend, with complexity in the client/browser, but I have no doubt we will find solutions. I’m quite excited about working on that, but the basics are the priority for me for now.
I’m more concerned about performance, with storage scattered across so many desktops (some of them off).
Something still needs to unify bits of data from different stores to display them together, in large unified apps. This is my concern with Solid (and SAFE), more to go wrong, and lower performance? I like it as research, but is it not bounded by the initial ingredients?
Here are some examples of what I’m wondering. How do you index data for performance if it’s decentralized? How do you ensure and use key relationships if different data subsets use different keys? Solid on SAFE only has the notion of user identity (URI), if I got that right. Generally, how do you build organization into the data? Have I got the wrong notions? I’m beginning to think…
You’ve made me wonder about viability more clearly, so maybe that’s the bigger problem.
You ask the right questions, some I’m also not sure about others I could respond to wrt SAFE but not here - all are things that will need to be addressed. What makes it so important to me to try Solid on SAFE is that I see the potential advantages alongside any unsolved questions. I think it is essential given where we are and how much worse that is getting.
I suspect you and many are here because you recognise the problems and want to find ways of solving them, and it will also be important to have other options such as your approach.
I was pointing out that Solid can be run at home on a very basic server (as I am doing with one of my identities), or run on any one of the millions of commercial servers (which I am doing with two other identities).
Wherever I choose to host my Pods, I have complete control of them. I have complete control over who has access (in the case of my own server I can even control which server admins have access) and I can move the contents to any of the millions of different servers whenever I want.
That is a completely different concept to giving control of my date to a data coop:
If you only want to access your own Pod, that will work just fine. If you want to access data from many Pods (such as a new Facebook built on Solid), how will that scale? For large applications with millions of users, you effectively end up with silos, defined by performance constraints. What am I missing?
What you are missing is that even if I was a celebrity with millions of followers (which I am not) the only data that would be in my Pod would be my own comments, not everyone else’s.
Similarly, if I want to follow thousands of people and make a comment, my own comments are still the only comments actually in my Pod.
Think of the way website pages are created with databases. Each part of the website page is created from different tables in the database. From a user perspective you see the whole page, even though the content of that page originates from different tables in the database, or even from different databases. If you have Google ads on the page, that comes from a different source again.
Then forget about website pages and think social media, and forget about database tables and think Pods. From a user perspective the site may look identical to Facebook, but the social media page is actually a link between the various items of data contained in the different Pods. You may be able to view everything on a single computer screen, but the individual pieces of data are in different Pods. Only my data is in my Pod.
Finally, think of this forum. Many different users all commenting from many computers or phones, but the comments all appear on one screen and are hosted by a single server. In a Solid use it would look exactly the same on screen, but your comments would be stored in your Pod (and hosted on whatever server suits your use, home server or commercial server) and my comments would be stored in my Pod (which I could also host anywhere),
Solid (Social Linked Data) does not need a central server. It can be used by millions of people using millions of different servers, rather than millions of people using a single company’s (Facebook) server.
I understand everything you say. What I don’t follow is any reasonable performance expectation if everyone’s comments are stored separately, in your examples. Some apps (eg. twitter) won’t be impacted so much, some (e.g. a new facebook) will grind to a halt without indexing across so many individual Pods. Surely I’m not missing anything?
Fetching a single facebook page will become a monster of a job dependent on every Pod involved. It can be done, but do you see that the Pod data organization will make it very slow and prone to breaking?
The other thing worth noting is that of course someone can copy and paste something I have made public, but there is no general access to data I only show to family and friends.
At present, whatever my settings, Facebook can sell any of the data on their servers. They can even sell their whole company. Using Solid, if I mark something as private to friends and family then friends and family are the only people who have access to my Pod.
So there is only a little more security of data I mark as public, but complete control of data I mark as private.
Hackers would need to hack the many Solid servers of every individual user to get the same info that they can get now from a single hack of a facebook server.
I’m not comparing Solid to today’s facebook, I’m comparing to a data coop owned by users.
True, even for a data coop. However much of this same complexity that puts off hackers would also put off honest coders having to accommodate so many different Pods instead of a single data service.
There are not millions of comments on a single Facebook page. There are actually very few.
I am currently watching a smart tv over the internet, streaming films like millions of other people do every day on their phones.
Times are changing, and Solid is a completely new concept.
The argument about bandwidth was being made about tv not so long ago.
Every time you do a search of Google, Google provides links to millions of pages. And millions of people do Google searches every minute. The difference is that you only click on one of those links at a time.
Some people will need a commercial server for Solid, but I don’t anticipate I will be one of them.
The computational complexity of the underlying data stores will remain the same, and gets worse the more layers you add. Pulling unindexed comments from a dozen different stores with different rules will be that much slower, and prone to that much more breaking. If this is not intuitive, can you figure out what I’m missing?
Consider YouTube comments, where there are often hundreds of contributors on a page. If everybody has their own Pod, the browser will have to hit hundreds of Pods. Any incompatibility, any server down, will create an exception to be coded around and managed. How likely is that to perform poorly, and/or break? Personally, I expect it is very likely. Do you not? The same holds for any page shared by nontrivial groups.
@happybeing points out that the answer might be a hybrid of data coop and individual Pods. He also points out the greater concern that none of these ideas has a viable business model yet.
