New decentralized social network, specialized in sharing public and Creative Common content


#14

Thank you @timbl for your feedback ! Generative Objects does much more that transforming Excel files (GO-XL is a sub-product), it is a fully fledge low code development platform that I am open sourcing and is used to build from scratch full web services and platforms. I have recently created a connector to SOLID and SPARQL data sources to create in minutes full applications leveraging SOLID and data sources like dbpedia.org. The demo application https://solid-demo.generativeobjects.com is built from scratch with Generative Objects with almost zero lines of manual code.

My intention is to connect Generative Objects to Solid and open source it to accelerate the creation of Solid applications and reach the critical mass of apps that would ensure the widespread and adoption of Solid. This is my gift to the community :wink:

And the plan is the use the Generative Objects platform to build the social network I am proposing in this post. And I plan on doing this and delivering the first working version in no more than 2 to 3 months thanks to the low-code productivity promise.

This is exactly it. And also creating a social network on top of it.

Yes indeed, this part is straight forward.

And this part (and much more) can be built very fast with Generative Objects technology. Especially now that we have a Solid connector for Generative Objects.

In fact it can be pretty straightforward if done this way : you just access the video clip from your friends you are connected with in the social network and who was sharing the file to you ! And if you share the file yourself, you embark it on your own POD and your own friends and connections can stream the video from your POD. The use case is therefore :

In the social network, on my feed lists : I can see a friend who shared a video

I watch the video, I am streaming it from my friends’ POD

I like the video and want to share it on my profile

At the time of sharing the video, I embark it on my pod

When my own friends see my video sharing, they can watch the video by streaming it from my POD

However, if we want a public repository where people could search for videos and access them, then yes indeed we would need to have a proper discovery service on top of SOLID decentralized architecture. This can be done but is already another use case.

My first vision is to build a social network and do it the way I describe above. Also because every user will most probably pay for their own POD provider, and the more followers they have, the most they will have to pay to have sufficient bandwidth. Thus it would be important to limit who can stream a given video from a user POD to the friends/connection of this user to not involve more hosting cost for the user.

And yes, the value would be full decentralization of data, and of the social network itself, speed, resilience, and also a new way to ensure that the videos will not be tampered with, without having to rely on resource intensive technology like block chain. Instead of relying on the unbreakable quality of blockchain, you rely on the fact that a given video is potentially replicated so many time that someone who would push a tampered version would actually change the hash of the video that will not be the same as all the others copies … All the copies would have to be changed to tamper with the video, which would be impossible because stored in personal PODs. This is a bonus statement :blush: I just thought of it, and writing while thinking, so it might not be fully relevant …


#15

This is why I mentioned PeerTube, because it is not just an ActivityPub federated service, but has P2P capabilities specifically for the video part, and to manage the issues you mention. Copied from joinpeertube.org:

The PeerTube software can, whenever necessary, use a peer-to-peer protocol (P2P) to broadcast viral videos, lowering the load of their hosts.

In this way, when you watch a video, your computer contributes to its broadcast. If a lot of people are watching the same video at the same time, their browser automatically send small pieces of the video to the other viewers. The server resources are not over-exploited : the stream is split, the network optimized.

It might not look like it, but thanks to peer-to-peer broadcasting, popular video makers and their videos are no longer forced to be hosted by big companies, whose infrastructure can stand thousands of views at the same time… or to pay for a robust but extremely expensive independent video host.


#16

Does it ? for me what takes time is when you can to a place and want to see the list of users who accessly visited this place, because I am scanning all registered users PODs, with no optimization / no parallelising. And this would definitively not work with more users registered without extra indexing done…

For me it looks like accessing a single resource from a Solid POD is actually fast, it is a classical HTTP request. The problem is when you need to access data from many PODs, and consolidate all the data. My 5 cents is that it is a matter of designing the application and the pages accordingly. For instance, showing a post with comments from many different users : it would not be efficient to get all the data from all PODs of the commenters with classical http requests, consolidate the data and then display at the end. It would be required to highly parallelize the calls using web sockets, and display the data on the fly whenever a new piece is available.

For this use case, the factor is not so much for me the volume and number of users. Solid has a decentralized approach, so resources are split on so many different servers / providers, the load is distributed and every single request are individual HTTP request with linear performance. For me (but I might be underestimate other factors) the performance is about the display of a page on the user browser, and the number of calls to do to many PODs to get all the data required to display the page. However: a single page for a given user is by nature limited in terms of how much information can be displayed on a single page ! Therefore it must be possible to find technics to optimize this part, and this performance is not at all related to how many users are using the platform, performance will be linear no matter how many new users are joining (provided we have client side applications, which is the idea with Solid)

What is time and resource consuming though on networks like Facebook or Linkedin is the creation of the feed list (or for a bank : working on data from millions of users having their own PODs ) . It is especially true when complex algorithms are used to optimize the feed, not only for the user own direct interest but for optimizing advertising and what we want the users to see in priority. For that part yes : it would be more challenging, and we would need some server side application to do this. And if we have a server side application for this, all is possible, as it is for facebook : we would use a database to index all PODs content and links, without actually importing all the data : just the bits necessary. Every time data is added or modified by an application on a user PODs, some information (not the data itself, can even also be encrypted) can be sent to a server to do some consolidation / indexing work. I am simplifying here, but I am sure we can find solution.

To conclude : for the matter of an application used to navigate my profile and my friends posts, and navigate links, I don’t see a too big challenge (this is how the web works). We can start with this for a simple social network as described in my post, and with simple feed list with no extra complex algorithm. This use case is, I think, feasible.

What do you think of this ?


#17

Thank you @aschrijver, I did not have time yet to check your link, definitively something to explore. The peer to peer approach of joinpeertube.org is somehow different to what I am proposing. My approach is maybe too simplistic ? At the same time simplicity is good :slight_smile: maybe there is something in the middle to figure out, or a composition of both approaches. I like though the idea to leverage Solid and build a full new approach that relies solely on the core HTTP principles. I have a deep intuition that there is something very interesting to create here !


#18

Thank you @aschrijver for these links !
Individious looks great ! however for what I understand we are still relying on deployed instances of indivious. It is decentralized in the sense that it does not belong to a big corporate entity, and you can have multiple instances, but still centralized in the sense that each instance of individious is a classical centralized client server application (I might be wrong… I haven’t gone in too much exploration but feels like it , please tell me if I am wrong…)

This is as I was sharing about mastodonde and each mastodonde instance being a classic client server application.

I think that with Solid we can go for a truly decentralized approach. Actualy, JoinPeerTube is decentralized since following a peer to peer approach. However I am wondering if with Solid we can invente a new way to decentralized.


#19

Indeed. I added Invidious not as a full project example, but because you mentioned also sharing links to existing YT videos, so there’s probably parts of the codebase interesting for closer examination.


#20

You can do that in parallel with HTTP. With HTTP/2 even the performance is good.

Exactly. Therefore you you can just load, let’s say a batch of 10 comments in parallel using HTTP, and further batches continuously while the user navigates through the comment list.


#21

I agree with you on this. In my opinion this is a weaknesses in the current Solid approach, it is not completely decentralized, you still have servers and location dependency, it is a kind of compromise. My feeling is that we should keep the strengths of Solid, RDF and the standards around it (WAC for example) as universal data format for the Web, but combine them with pure decentralized approaches like IPFS or SAFE network to get rid of servers.

I would aim for a serverless Solid, but I’m not sure whether we still need an intermediate server-based approach and switch over later. This is my current dilema.


#22

Yes @rodant , with what I am proposing however, there would large replication of all public / shared content, through the idea that when you share some content from others, you get a copy of this content on your own POD. Therefore there is decentralization AND resilience of the data, provided that …. You have many POD servers ! because if all PODs are hosted by only an handful set of POD servers, then it is not so much decentralized….

Also see my comment here : New decentralized social network, specialized in sharing public and Creative Common content , comment on @aveltens’s docker :

In the extreme case scenario : if we could all self-host our own POD server on our own local machine, with the intend to only host our own POD (and maybe the ones from our family / close friends), then we would actually have a fully decentralized system, combining kind of a peer-to-peer network approach and the promise of Solid for keeping ownership of data. Our POD becomes indeed a replicated container for publicly shared data (replication of shared content into multiple PODs) and as well for your own private data, with control on who can see.

I still think we would need a central server for consolidation / indexing capability and ensuring that we can build high performing systems. But this server would not hold data, just pointers to data. And if this server fails, it does not affect the data itself, just the raw performance of the applications, which will get back to normal when the central server is restored.


#23

I don’t exclude this possibility, applications could need some optimizations or additional processing implemented in an external service, which could also be high available. But I also don’t exclude that we find new decentralized solutions for those problems, the time will show that. Though, this is from my point of view the next step, first we must solve the central use cases in the best possible way.


#24

Oh yes completely agree ! there are solutions for everything. It is a matter of being clear about where we want to go, state the intention, and then create with full trust that together we will get there :slight_smile:

Also fully agreeing on the step by step approach : let’s start the ball rolling without waiting for everything to be perfect and we will improve with time. What we need now is to have a real world, production application, make it scale to many users and then iterate. We know this : the important matter is to get traction and wide adoption first.

I have the intention to build this decentralized social network we are talking about here, and do it fast, and do it together, so everyone is welcome to join to co-create ! I am 100% dedicated to this.

For those interested, please check the event I am proposing to host : Creation of Solid applications with low-code development platform Generative Objects , I plan to demonstrate how we can build very fast this social network with Generative Objects low-code technology. Please comment on the post for the event to say what is your time-zone if you are interested to participate


#25

@walter.almeida I just want to share with you this repo from Redecentralize.org , where you can find different decentralized projects that might be of your interest (like D.Tube) or could give you more ideas around your app :wink:


#26

Great ! Thank you @alexcorvis84


#27

Yes, and awesome peer-to-peer is also a good one.


#28

This conversation continues in Deploying solid in kubernetes, anyone has experience and can help us?


#29

This seems even more interesting with considering the video data static, and having to be duplicated easily between machines. Pinning to IPFS pods and sharing the videos’ metadata via a simple Solid application sounds even more approachable, than turning Solid into a (fault-tolerant, duplicating) CDN. Eventually one could even reuse concepts or experiments with https://ipld.io/ ?


#30

Ok I will really study this, see what route we take. I am definitively open to suggestion and leveraging existing open source projects !


#31

It would be more than a =basic CDN, it would be a contextual CDN with access ruled by your personal connections => you access the resource from the friend who shares it, and you yourself become a node for your own friends.

But indeed, I am not familiar with IPFS, I really need to check it !


#32

Hi Walter, It seems my use case would be a nice test of your network. It is a meaningfull use case to share one’s medical dossier. If you want more information drop me a note. Thanks!

Hans


Re-invent the social ẃeb
Re-invent the social ẃeb
#33

Hi Walter,

these are good starting points to get into the depths of IPFS:

This is where it basically started ~ 2014/15:

It’s like torrent, https://cobox.cloud/ or http://dat.foundation/, but slightly different.

You’ll see how it’s useful for immutable data. IPLD then is a metadata language used in IPFS for itself and to bridge to other networks. https://ipld.io/

With it we get nice things, such as: