As the “S” of Solid stands for Social network I am wondering where is the Solid network.
Up to what I can see there is currently no way for a Pod to communicate to another and of course to a whole community.
In the world of data communication (SOA, ESB, Microsoft Biztalk, Web/REST services etc…) where I have been working there are 2 network typologies: Peer to peer and Hub and Spoke.
There is no need to describe Peer to Peer network. It is good when one peer communicates to one or a few peers but becomes a mess when having to manage a lot of peers and groups. Obviously not the good choice for social networks.
The other way is Hub and Spoke, a central hub and connected spokes. In this case the hub is not only a repository but must run services, most of them about the publish/subscribe architecture: Someone publishes, other, Pods or applications, subscribe and so are aware of publications.
I think Linked data is the good standard to communicate information and Solid, with the WebId, the real good place for storing data and communicate with a hub.
But there is the lack of a hub server with associated services.
Is there something planned about this?
There are already social features to Solid, in existing apps (such as the various chats, meetulator etc) and other approaches being worked on. They are all part of a bigger ‘distributed social network’ that exists and will no doubt grow on Solid, and includes the p2p social ecosystem which you rule out.
So I disagree that you can rule out p2p from the category. I also think federated is an alternative to hub and spoke. Darcy/Ibex is a recently prototyped federated example, from the Solid gitter:
so… we kinda have finished our Hackathon: Meet the Darcy Ibex, a prototype social media stream based entirely on Solid: https://ibex.darcy.is For an introduction on how it works, check what is ibex - Darcy
Ref: https://gitter.im/solid/chat?at=5e1361df1c3a592052dfd6f5
What you seem to be equating with ‘social network’ is the Facebook style centralised version, which is what I would very much like to avoid recreating on Solid, though I’m sure many will try. I don’t accept that it is the only way to realise social networks or social applications though, and I don’t equate it with the ‘S’ in Solid.
I think there are lots of good reasons why one wants to go at least half-way centralised. One of the issues we ran into with Ibex for example is that in order for comments to work, everyone needs to allow every possible commenter write access to the activity inbox. As a result, you cannot post to public but ban certain people from commenting. Or close commenting on certain posts.
The suggested solution is to have the client application discard unwanted notifications, but I don’t see this scale up when we have millions of active users and a good chunk of them will be spambots.
The other reason why one wants some sort of centralisation is for discovery and search purposes. And things like caching of course.
But these things don’t need to be at odds: I think in the end, we can have a seamless transition between true peer-2-peer and a federated centralised model, where individual users can do both at the same time and move their activities freely.
As long as the actual content and relationship graphs are stored on Solid pods, we can use that data freely with and without central instances, depending on our current threats and usage needs.
Full disclosure: Darcy will be a bunch of federated central instances, to enable instance-wide moderation and policy control features, as we think this is pretty important for any large-scale online space in the long run.
Horrifying comparison : With Facebook you go to the (virtual) Facebook building and the security guy at the entrance ask you to empty your pockets and everything you leave here belongs to Facebook.
With a hub and spoke architecture you go to a fair and show people in a standardized way what they can see in your Pod and what they offer you to see in their Pods. You can use the fair search and discovery services or use a social network application providing services to help you to find and offer things of your interest. But neither the fair services nor the social network app will pretend to be the owner of your data.
OK, some services, (chat, mail, others …) will be better implemented with a peer to peer architecture. But keep in mind BitTorrent which is the most famous peer to peer network uses trackers that are kind of hub and spoke implementation to initiate communications.
The problem is each application reinvent its own communication protocol which is tedious and, most important, not compatible with other apps. While writing this I see @JollyOrc post who apparently worked on Ibex: If I create my own social network will I be able to reuse posts posted on Ibex? Probably not but I could do it if Ibex and all other apps use the Hub architecture as the fair I described.
if you use the same taxonomy as we do, you can indeed reuse the posts generated through Ibex. And as the source code is open, it should be pretty easy for you to do so. Ibex currently uses no Hub, but we plan to eventually introduce one (for reasons outlined above).
That Hub would store the actual content and friends graphs on the Solid pods and thus retain pure Solid Hub-less compatibility.
I’d like to go deeper into this but unfortunately don’t have the time. I have watched some of the discussions on how to scale, but my sense is that we don’t need to scale the same way as the dominant (centralised) services and that there are new models that avoid the need to centralise in order to achieve this. I think one possible problem is that Solid hasn’t really addressed some of the centralising issues, though that remains to be seen.
For example: @JollyOrc raises spam, and how to prevent random individuals posting to public areas. How can Solid address that without becoming centralised? Those kind of issues are potential weak points that Solid should focus on solving rather than ‘how can we build a social network’. If not, then I think you come to the obvious conclusion, which I believe goes against the vision of Solid.
I think there is no actual conflict here. There are two different kinds of centralisation:
- Centralisation of applications, services and presentation
- Centralisation of data storage
Solid is all about the decentralisation fo the data storage - that is what it does pretty well, most of the time. But it does not provide decentralised processing. There are basically no active components, everything is “just a file”.
As soon as we start to process data, want to react to data, aggregate it or do similar things, centralisation might come up. The trick is to do it in a way that keeps permissionless innovation possible: The actual data and relationships need to be store on Solid pods so that other applications, centralised or not, can make use of it if the users want them to.
I don’t agree that it only decentralises storage. Apps are hosted on pods too, so are you ignoring that.
What it doesn’t solve which I think is what you are alluding to, is decentralising processing at scale, which I would say differently: Solid is not designed to support large centralised processing. There is nothing to stop such applications interacting with Solid pods, cf. my shocking example gain: facebook. But why would they? Perhaps there are some good use cases here, but I think if you go for centralised processing in order to scale you are missing the point.
Solid may not have solved this though, that I agree with and this is another reason I’m not convinced about the server based architecture, even pods as a service will I expect become centralised, and one of the ways will be creating solutions such as those alluded to in the OP and by @JollyOrc above. I’d rather we look for ways to decentralise more than just storage. I think Solid does go some way towards that, but probably not far enough. But this is just a first iteration.
Right! And so those processes can only be provided by applications, most of time when the “just a file” is created. And if we are talking about communication processes it can be sync processes, but the target(s) must be online which is rarely the case in social networks or async communication and here comes the role of the hub.
They aren’t executed on the pods though. But yes, Solid allows for decentralised data processing. Most of the time it means client-side, where the apps for said processing can be either loaded from a random webpage or the Solid pod. That is indeed important, awesome, and shouldn’t be neglected!
The core concept of Solid is decentralised storage, combined with the option to execute apps from anywhere within the browser. But if the user hasn’t got the webpage open, nothing is happening. In order to process and react to data while the user is offline, we’d need to run code on servers that aren’t part of the Solid specifications. And for those servers, there are good reasons to pool resources and sort-of-centralise.
That doesn’t mean that I want us to end up with another Facebook-like solution. Think more of the likes of Diaspora/Mastodon, but with Solid underpinnings.
I agree mostly, but Solid has more than decentralised storage as it’s core.
For example, the separation of application from data which is crucial to enabling both data portability and creating competition between applications based on function rather than attempting to capture our data and therefore us.
Another is of course RDF/Semantic Web, which is essential for separation of apps from data but brings a lot more along with it.
All those are core capabilities to me. If I thought it was only decentralised storage I would have far less interest because I don’t think it does that particularly well.
I’m going to sound like a nitpicker, and for that I should apologise, but:
All these features and capabilities are an expression and features of the decentralised storage. That isn’t supposed to diminish these features, but they don’t provide any active features or computing capabilities. All the things that act on the data are outside of Solid. Again: This isn’t necessarily a bad thing. For all the reasons outlined in this conversation and elsewhere, Solid is still a great thing!
But my instinct is still that for certain things, some sort of distributed centralisation is the better way to go. If just because the average computer user doesn’t want to worry about how to maintain the software they are using.
That said: I would really welcome if Solid would add a few things to address a few of the bigger headaches. ACLs that allow “allow all, but…” scenarios for example. That way we can achieve more decentralisation.
Intuitive … but maybe not conscious mathematics: When you have n elements communicating together relationships are established and those relationships have costs in terms of memory, bandwidth, delays and management.
So you need to evaluate the number of connections your system will generate. Mathematics formula is, n being the number of connected points, (n2-n)/2 relationships.
For example :
2 people sending mails to each other, (n2-n)/2 = 1 relationships
5 people chatting in a group = 10 relationships
50 people in a social network group = 1.225 relationships
Then if you use some (evil) distributed centralisation:
2 people sending mails to each other: 2 relationships. A little bit more
5 people chatting in a group = 5 relationships, better than without hub
50 people in a social network group = 50 relationships, your network will not crash!
So all theoretical assessments may be right but at last reality wins!
There are other ways of working than centralisation. There are reasons for wanting to centralise that are to do with efficiency for example, and others that are to do with forms of control that we see can be harmful.
But centralisation is not the only way to solve the technical problems. It’s a simpler thing for human minds to think about, design, build and manage, but if you look at how nature works to solve these things you see decentralised solutions dominate. It’s fascinating to learn about so I recommend reading up on the area. And don’t tell me nature doesn’t care about efficiency.
The upshot though is that for various reasons humans are attracted towards centralising solutions and away from the alternative, and to an extent regardless of what is possible or the characteristics and appropriateness in a particular use case.
There is a reason the internet was originally designed to be decentralised. Anyone care to list the goals? There are reasons that it became centralised. There are reasons that nature outside of human systems trends to decentralise.
One of the key goals of Solid is to restore the decentralised open internet, yet here we are again with centralisation creeping into our thinking.
In case anyone is inclined to explore this through natural systems, look at how very intelligent decentralised systems solve problems. Ones I’ve learned a bit about are ant colonies, but I recently read this about bees so have it to hand and recommend it:
http://www.overcomingbias.com/2020/01/how-bees-argue.html
You might say, but that is inefficient, to which I reply that’s the cost of making it effective and reliable, and avoiding the problems we face today. If we don’t learn those lessons Solid will fail in its primary purpose IMO. At least, it won’t fulfil the goals that are why I’m here supporting it.
BTW I’m not saying nothing should be centralised, I’m saying that I don’t agree with the reasons being given in this topic, because I think they ignore parts the bigger picture that I believe are important, and I think are key to solving the problems that Solid is designed to solve.
PS One of the things that humans are drawn to is short term whereas nature tends to test its solutions over the long term and in general doesn’t focus on over optimising or short term. Neither is right or wrong. I highlight it because I think it helps to understand what attracts us to a particular solution or path, and what we may miss by following it.
In-band hypermedia, i.e. hydra, could let Solid or a successor spec allow processing on the pods and become a digital assistant spec. It could be open source and could eventually be competitive with Siri, Alexa, etc. The people’s digital assistant (think MLK not Stalin), that could replace the others. Then each pod could do spam filtering etc with AI thats under human, and hopefully not corporate, control. Decentralized (to the extent of one or a few digital assistants per person) but each pod would no longer be that simple. They could be configured by their owners in intuitive ways through use and example. I don’t think its that far away. Efficiency you say? Well, according to Moore’s law…
Who remembers Ray Ozzie’s Groove Workspace product?
This was a desktop collaboration that supported direct P2P replication and synchronization capabilities.
Microsoft bought Groove Networks and Groove Workspace became the SharePoint Workspace client. The underlying replication and synchronization infrastructure evolved to become Microsoft Live Mesh …a precursor to the current Microsoft OneDrive.