New decentralized social network, specialized in sharing public and Creative Common content

Regarding how videos will be distributed, your description reminds me how Mastodon works…

Really interesting idea this new SN, we need this kind of decentralized apps if we want to change the current state of the www :wink:

1 Like

Thank you @alexcorvis84 for the link

Question is : does it make sense to create a new platform like this with Solid ? Or it would be redundant to others platforms (like mastodon for instance) that already exist ? is there a unique value brought by what I am envisionning here ? What is really missing today and where can we bring real value ?

1 Like

IMHO the most important point is not only to create decentralized apps besides work in the data interoperability.

i.e: Mastodon is a great decentralized network…but with the same ‘issue’ that many SN nowadays…What if I decide to change to a new SN? My data get lost and would like to still have my own data. I think you get the point of what I mean…

That’s why I see Solid interesting, as it offers these 2 crucial points: decentralization & data interoperability.

2 Likes

Ok @alexcorvis84, for what I see of mastodon, it is not as decentralized as what we can do with SOLID and my proposal here. They say it is “decentralized” because anyone can setup an instance of mastodon and run their communities, and mastodon instances can communicate with each other.

However … every single mastodon instance is itself centralized !

The proposal here is to fully decentralized the whole SN, each user having its own data in their POD, and the network running client side, thus only requiring a browser ! (well as said : there still could be some centralization for optimization / consolidation purposes, but limited to a minimum, or potentially done peer to peer)

1 Like

Yes got it. So with the proposal here, every user definitively keeps his/her data and yes : should be structured in a interoperable way, using the proper namespaces / ontologies.

Moreover, I have this idea of getting copies of content when sharing it, so that not only the user keeps his data, but embarks the data he shared. It is an extra value, based on encouraging to maximize shared / open data. And of course, the user initially sharing a piece of data can decide to allow such a public sharing or forbid it.

It will allow for a single piece of data to be shared with millions of people without requiring a big centralized platform like youtube to handle the load, because any user interesting in accessing the data would access it from the POD of the user sharing it with him.

Yes, however I see the value of also decentralizing the architecture of the SN, to make it fully resilient to failure and not depending on a single operator

1 Like

@walter.almeida It took me a while figure our what your Generative Objects system does, but I understand you take an existing Excel spreadsheet, and with th e help of some annotations to each column to tell the system about it, you then generate a (google forms like) web app automatically, which includes both the tabular list of items as well as an item editor screen.

Thank your for your presentation to solid world about making it work with Solid.

But the actual subject of this forum topic is specifically about a way of using solid to distribute videos (like the things people put on YouTube) across people’s pods so that they could collectively replace the central stores like YouTube and Vimeo. That sounds basically straight forward as you can store videos on solid pods. (The current data browser while very clunky does allow you to upload a video to a folder, and then to play you can with sounds files) So what we need is the social aspect around that. But the most interesting take-away for me from your post was that for video clips, we may need to build on top of solid a high level protocol for offering to store and serve other people’s videos under certain circumstances, and finding a copy near you of a popular video. And this to distribute server load and server capacity, and perhaps also give redundancy. This is much like what a Content Distribution Network does, but in our case you would share video storage through the social net. Interesting. Also interesting because you local computer or your local disk is the part of your solid pod configuration which needs to have a copy, so you can watch it without glitches. So it connects to Local first schemes, and also systems where your solid contacts are actually kept in many people’s pods for resilience and speed. Interesting.

4 Likes

One of the big problems I see in any kind of ‘Solid’ solutions is performance

As you can check now…ie: a Solid app that lists your friends data -> It takes time to load their profile pic

Can we imagine a Social Network, with millions of Photos, videos…data with a great performance? Or can we imagine a bank using data from million accounts stored in their client Pods? Seems like a dream so far…

As I said recently over the forum, I think if we want to see Solid succeeding, two crucial points must be solved: An UX/UI as close as any current app & and great (at least acceptable) performance.

2 Likes

Thank you @timbl for your feedback ! Generative Objects does much more that transforming Excel files (GO-XL is a sub-product), it is a fully fledge low code development platform that I am open sourcing and is used to build from scratch full web services and platforms. I have recently created a connector to SOLID and SPARQL data sources to create in minutes full applications leveraging SOLID and data sources like dbpedia.org. The demo application https://solid-demo.generativeobjects.com is built from scratch with Generative Objects with almost zero lines of manual code.

My intention is to connect Generative Objects to Solid and open source it to accelerate the creation of Solid applications and reach the critical mass of apps that would ensure the widespread and adoption of Solid. This is my gift to the community :wink:

And the plan is the use the Generative Objects platform to build the social network I am proposing in this post. And I plan on doing this and delivering the first working version in no more than 2 to 3 months thanks to the low-code productivity promise.

This is exactly it. And also creating a social network on top of it.

Yes indeed, this part is straight forward.

And this part (and much more) can be built very fast with Generative Objects technology. Especially now that we have a Solid connector for Generative Objects.

In fact it can be pretty straightforward if done this way : you just access the video clip from your friends you are connected with in the social network and who was sharing the file to you ! And if you share the file yourself, you embark it on your own POD and your own friends and connections can stream the video from your POD. The use case is therefore :

In the social network, on my feed lists : I can see a friend who shared a video

I watch the video, I am streaming it from my friends’ POD

I like the video and want to share it on my profile

At the time of sharing the video, I embark it on my pod

When my own friends see my video sharing, they can watch the video by streaming it from my POD

However, if we want a public repository where people could search for videos and access them, then yes indeed we would need to have a proper discovery service on top of SOLID decentralized architecture. This can be done but is already another use case.

My first vision is to build a social network and do it the way I describe above. Also because every user will most probably pay for their own POD provider, and the more followers they have, the most they will have to pay to have sufficient bandwidth. Thus it would be important to limit who can stream a given video from a user POD to the friends/connection of this user to not involve more hosting cost for the user.

And yes, the value would be full decentralization of data, and of the social network itself, speed, resilience, and also a new way to ensure that the videos will not be tampered with, without having to rely on resource intensive technology like block chain. Instead of relying on the unbreakable quality of blockchain, you rely on the fact that a given video is potentially replicated so many time that someone who would push a tampered version would actually change the hash of the video that will not be the same as all the others copies … All the copies would have to be changed to tamper with the video, which would be impossible because stored in personal PODs. This is a bonus statement :blush: I just thought of it, and writing while thinking, so it might not be fully relevant …

This is why I mentioned PeerTube, because it is not just an ActivityPub federated service, but has P2P capabilities specifically for the video part, and to manage the issues you mention. Copied from joinpeertube.org:

The PeerTube software can, whenever necessary, use a peer-to-peer protocol (P2P) to broadcast viral videos, lowering the load of their hosts.

In this way, when you watch a video, your computer contributes to its broadcast. If a lot of people are watching the same video at the same time, their browser automatically send small pieces of the video to the other viewers. The server resources are not over-exploited : the stream is split, the network optimized.

It might not look like it, but thanks to peer-to-peer broadcasting, popular video makers and their videos are no longer forced to be hosted by big companies, whose infrastructure can stand thousands of views at the same time… or to pay for a robust but extremely expensive independent video host.

3 Likes

Does it ? for me what takes time is when you can to a place and want to see the list of users who accessly visited this place, because I am scanning all registered users PODs, with no optimization / no parallelising. And this would definitively not work with more users registered without extra indexing done…

For me it looks like accessing a single resource from a Solid POD is actually fast, it is a classical HTTP request. The problem is when you need to access data from many PODs, and consolidate all the data. My 5 cents is that it is a matter of designing the application and the pages accordingly. For instance, showing a post with comments from many different users : it would not be efficient to get all the data from all PODs of the commenters with classical http requests, consolidate the data and then display at the end. It would be required to highly parallelize the calls using web sockets, and display the data on the fly whenever a new piece is available.

For this use case, the factor is not so much for me the volume and number of users. Solid has a decentralized approach, so resources are split on so many different servers / providers, the load is distributed and every single request are individual HTTP request with linear performance. For me (but I might be underestimate other factors) the performance is about the display of a page on the user browser, and the number of calls to do to many PODs to get all the data required to display the page. However: a single page for a given user is by nature limited in terms of how much information can be displayed on a single page ! Therefore it must be possible to find technics to optimize this part, and this performance is not at all related to how many users are using the platform, performance will be linear no matter how many new users are joining (provided we have client side applications, which is the idea with Solid)

What is time and resource consuming though on networks like Facebook or Linkedin is the creation of the feed list (or for a bank : working on data from millions of users having their own PODs ) . It is especially true when complex algorithms are used to optimize the feed, not only for the user own direct interest but for optimizing advertising and what we want the users to see in priority. For that part yes : it would be more challenging, and we would need some server side application to do this. And if we have a server side application for this, all is possible, as it is for facebook : we would use a database to index all PODs content and links, without actually importing all the data : just the bits necessary. Every time data is added or modified by an application on a user PODs, some information (not the data itself, can even also be encrypted) can be sent to a server to do some consolidation / indexing work. I am simplifying here, but I am sure we can find solution.

To conclude : for the matter of an application used to navigate my profile and my friends posts, and navigate links, I don’t see a too big challenge (this is how the web works). We can start with this for a simple social network as described in my post, and with simple feed list with no extra complex algorithm. This use case is, I think, feasible.

What do you think of this ?

1 Like

Thank you @aschrijver, I did not have time yet to check your link, definitively something to explore. The peer to peer approach of joinpeertube.org is somehow different to what I am proposing. My approach is maybe too simplistic ? At the same time simplicity is good :slight_smile: maybe there is something in the middle to figure out, or a composition of both approaches. I like though the idea to leverage Solid and build a full new approach that relies solely on the core HTTP principles. I have a deep intuition that there is something very interesting to create here !

1 Like

Thank you @aschrijver for these links !
Individious looks great ! however for what I understand we are still relying on deployed instances of indivious. It is decentralized in the sense that it does not belong to a big corporate entity, and you can have multiple instances, but still centralized in the sense that each instance of individious is a classical centralized client server application (I might be wrong… I haven’t gone in too much exploration but feels like it , please tell me if I am wrong…)

This is as I was sharing about mastodonde and each mastodonde instance being a classic client server application.

I think that with Solid we can go for a truly decentralized approach. Actualy, JoinPeerTube is decentralized since following a peer to peer approach. However I am wondering if with Solid we can invente a new way to decentralized.

2 Likes

Indeed. I added Invidious not as a full project example, but because you mentioned also sharing links to existing YT videos, so there’s probably parts of the codebase interesting for closer examination.

1 Like

You can do that in parallel with HTTP. With HTTP/2 even the performance is good.

Exactly. Therefore you you can just load, let’s say a batch of 10 comments in parallel using HTTP, and further batches continuously while the user navigates through the comment list.

1 Like

I agree with you on this. In my opinion this is a weaknesses in the current Solid approach, it is not completely decentralized, you still have servers and location dependency, it is a kind of compromise. My feeling is that we should keep the strengths of Solid, RDF and the standards around it (WAC for example) as universal data format for the Web, but combine them with pure decentralized approaches like IPFS or SAFE network to get rid of servers.

I would aim for a serverless Solid, but I’m not sure whether we still need an intermediate server-based approach and switch over later. This is my current dilema.

3 Likes

Yes @rodant , with what I am proposing however, there would large replication of all public / shared content, through the idea that when you share some content from others, you get a copy of this content on your own POD. Therefore there is decentralization AND resilience of the data, provided that …. You have many POD servers ! because if all PODs are hosted by only an handful set of POD servers, then it is not so much decentralized….

Also see my comment here : New decentralized social network, specialized in sharing public and Creative Common content , comment on @aveltens’s docker :

In the extreme case scenario : if we could all self-host our own POD server on our own local machine, with the intend to only host our own POD (and maybe the ones from our family / close friends), then we would actually have a fully decentralized system, combining kind of a peer-to-peer network approach and the promise of Solid for keeping ownership of data. Our POD becomes indeed a replicated container for publicly shared data (replication of shared content into multiple PODs) and as well for your own private data, with control on who can see.

I still think we would need a central server for consolidation / indexing capability and ensuring that we can build high performing systems. But this server would not hold data, just pointers to data. And if this server fails, it does not affect the data itself, just the raw performance of the applications, which will get back to normal when the central server is restored.

2 Likes

I don’t exclude this possibility, applications could need some optimizations or additional processing implemented in an external service, which could also be high available. But I also don’t exclude that we find new decentralized solutions for those problems, the time will show that. Though, this is from my point of view the next step, first we must solve the central use cases in the best possible way.

1 Like

Oh yes completely agree ! there are solutions for everything. It is a matter of being clear about where we want to go, state the intention, and then create with full trust that together we will get there :slight_smile:

Also fully agreeing on the step by step approach : let’s start the ball rolling without waiting for everything to be perfect and we will improve with time. What we need now is to have a real world, production application, make it scale to many users and then iterate. We know this : the important matter is to get traction and wide adoption first.

I have the intention to build this decentralized social network we are talking about here, and do it fast, and do it together, so everyone is welcome to join to co-create ! I am 100% dedicated to this.

For those interested, please check the event I am proposing to host : Creation of Solid applications with low-code development platform Generative Objects , I plan to demonstrate how we can build very fast this social network with Generative Objects low-code technology. Please comment on the post for the event to say what is your time-zone if you are interested to participate

2 Likes

@walter.almeida I just want to share with you this repo from Redecentralize.org , where you can find different decentralized projects that might be of your interest (like D.Tube) or could give you more ideas around your app :wink:

1 Like

Great ! Thank you @alexcorvis84

1 Like