New decentralized social network, specialized in sharing public and Creative Common content

Build a Solid App Ideas for Solid Apps
1

Hello guys !

Following my successful experimentation of integrating SOLID and Generative Objects (check here : Creation of Solid applications with low-code development platform Generative Objects),

I am planning to start building a new social network, specialized in sharing information, documents and videos, in a decentralized and open way.

What I foresee it the following : while I would keep linking to youtube and other existing video platform for resources that are already there, the idea is to promote uploading your videos to your POD to decentralize the video sharing system.

And then: when you share videos: your friends can access them and stream them from your POD. But whenever a friend wants to share your video: a copy is uploaded to their POD and it is streamed to your friendsā€™ friends from your friendā€™s POD. And so on. We can anticipate that you have a limited number of friends, so your pod server will not be overloaded by a flood of requests, the load is distributed to all the POD / POD servers. And if you are an influencer with large number of friends, it is up to you to pay for a provider who can sell you the right bandwidth for your audience.

This decentralized hosting of videos also implies that the videos can actually be shared this way. So a user can decide if he/she authorizes the copy of his/her videos and further sharing, which would be encouraged in our platform. Therefore, we encourage Creative Commons licensed videos.

On a technical / ecological view point : we could say that it is not an efficient use of resources because data is duplicatedā€¦ However, it is balanced by distributed server resources for the streaming. I have no idea if the balance is even, if anyone has a clue, I welcome the information to estimate the relevance of this architecture !

So data will be decentralized through a double approach :

  • Using personal Solid PODs
  • Sharing content by duplication in multiple PODs

The application will also be decentralized (ultimately, as for now GO is designed to generate client server applications, but this will change!), as it would mainly run client side, in the browser. I anticipate though that we will still need some server side services, especially for optimizations and consolidation purpose, building the feeds etc. but this will/should be drastically limited compared to classical centralized platforms.

I am not planning this to be a business, but a community platform, that will belong to all of us, the users of the platform.

And for this to work, we need to have POD with a good quota in size, and I did not find any existing provider doing this ā€¦

So I am still wondering if Solid is the proper way to go, but I am really interested to do it with Solid ! Your feedback is welcomed if you have any input / feedback.

So, for the Pod provider, I am planning to set up a new Solid Pod Server for the purpose, with high quota. I am financing the server so far, this idea is that when it is working and growing, we will distribute the cost of the servers to the users of the application, according to the quota size they are wanting. And then of course : anyone can use his own server, or choose any POD provider that can provide with sufficient quota.

My plan is to have a first running version pretty fast, in the coming one to two months, and it is definitively possible with the GO low-code platform !

I am welcoming all the help from anyone who is called by this project. Please comment this post or reach me if you are interested.

@Smag0 already proposed to help in setting up a node server on a machine I will provide. Please @Smag0 you can confirm if you are still interested, with all this extra information on the project !

Looking forwad to connecting with anyone who would be interested in this project !

8 Likes
2

Wow, very ambitious plans, cool. I think there is nothing wrong to link to existing videos anywhere on the web, they do not necessarily need to be hosted on a Solid Pod.

.I am already doing this in my data, check out the recording linked in this talk for example.

I am embedding the data and the video recording on my pod homepage here.

3

I agree there is nothing wrong to link to existing videos. My proposal is for a new approach / new values :

There is a reason for encouraging not linking to existing data but rather upload content/videos to the PODs :

  • to not rely anymore on a centralized platform like youtube
  • to promote creative commons / redistributable content, as opposed to copyrighted content

And I will check your links ! thank you @aveltens

4

Great move Walter. This is exactly what Solid needs - leadership and testing out what it can do, where it needs work, what it canā€™t do etc.

Good luck.

1 Like
5

Maybe IPFS is more suited for the video use case?

2 Likes
6

On the video side thereā€™s also PeerTube to be inspired by;

And for the linking:

3 Likes
7

Regarding how videos will be distributed, your description reminds me how Mastodon worksā€¦

Really interesting idea this new SN, we need this kind of decentralized apps if we want to change the current state of the www :wink:

2 Likes
8

Thank you @alexcorvis84 for the link

Question is : does it make sense to create a new platform like this with Solid ? Or it would be redundant to others platforms (like mastodon for instance) that already exist ? is there a unique value brought by what I am envisionning here ? What is really missing today and where can we bring real value ?

1 Like
9

IMHO the most important point is not only to create decentralized apps besides work in the data interoperability.

i.e: Mastodon is a great decentralized networkā€¦but with the same ā€˜issueā€™ that many SN nowadaysā€¦What if I decide to change to a new SN? My data get lost and would like to still have my own data. I think you get the point of what I meanā€¦

Thatā€™s why I see Solid interesting, as it offers these 2 crucial points: decentralization & data interoperability.

2 Likes
10

Ok @alexcorvis84, for what I see of mastodon, it is not as decentralized as what we can do with SOLID and my proposal here. They say it is ā€œdecentralizedā€ because anyone can setup an instance of mastodon and run their communities, and mastodon instances can communicate with each other.

However ā€¦ every single mastodon instance is itself centralized !

The proposal here is to fully decentralized the whole SN, each user having its own data in their POD, and the network running client side, thus only requiring a browser ! (well as said : there still could be some centralization for optimization / consolidation purposes, but limited to a minimum, or potentially done peer to peer)

1 Like
11

Yes got it. So with the proposal here, every user definitively keeps his/her data and yes : should be structured in a interoperable way, using the proper namespaces / ontologies.

Moreover, I have this idea of getting copies of content when sharing it, so that not only the user keeps his data, but embarks the data he shared. It is an extra value, based on encouraging to maximize shared / open data. And of course, the user initially sharing a piece of data can decide to allow such a public sharing or forbid it.

It will allow for a single piece of data to be shared with millions of people without requiring a big centralized platform like youtube to handle the load, because any user interesting in accessing the data would access it from the POD of the user sharing it with him.

Yes, however I see the value of also decentralizing the architecture of the SN, to make it fully resilient to failure and not depending on a single operator

1 Like
12

@walter.almeida It took me a while figure our what your Generative Objects system does, but I understand you take an existing Excel spreadsheet, and with th e help of some annotations to each column to tell the system about it, you then generate a (google forms like) web app automatically, which includes both the tabular list of items as well as an item editor screen.

Thank your for your presentation to solid world about making it work with Solid.

But the actual subject of this forum topic is specifically about a way of using solid to distribute videos (like the things people put on YouTube) across peopleā€™s pods so that they could collectively replace the central stores like YouTube and Vimeo. That sounds basically straight forward as you can store videos on solid pods. (The current data browser while very clunky does allow you to upload a video to a folder, and then to play you can with sounds files) So what we need is the social aspect around that. But the most interesting take-away for me from your post was that for video clips, we may need to build on top of solid a high level protocol for offering to store and serve other peopleā€™s videos under certain circumstances, and finding a copy near you of a popular video. And this to distribute server load and server capacity, and perhaps also give redundancy. This is much like what a Content Distribution Network does, but in our case you would share video storage through the social net. Interesting. Also interesting because you local computer or your local disk is the part of your solid pod configuration which needs to have a copy, so you can watch it without glitches. So it connects to Local first schemes, and also systems where your solid contacts are actually kept in many peopleā€™s pods for resilience and speed. Interesting.

4 Likes
13

One of the big problems I see in any kind of ā€˜Solidā€™ solutions is performance

As you can check nowā€¦ie: a Solid app that lists your friends data -> It takes time to load their profile pic

Can we imagine a Social Network, with millions of Photos, videosā€¦data with a great performance? Or can we imagine a bank using data from million accounts stored in their client Pods? Seems like a dream so farā€¦

As I said recently over the forum, I think if we want to see Solid succeeding, two crucial points must be solved: An UX/UI as close as any current app & and great (at least acceptable) performance.

2 Likes
14

Thank you @timbl for your feedback ! Generative Objects does much more that transforming Excel files (GO-XL is a sub-product), it is a fully fledge low code development platform that I am open sourcing and is used to build from scratch full web services and platforms. I have recently created a connector to SOLID and SPARQL data sources to create in minutes full applications leveraging SOLID and data sources like dbpedia.org. The demo application https://solid-demo.generativeobjects.com is built from scratch with Generative Objects with almost zero lines of manual code.

My intention is to connect Generative Objects to Solid and open source it to accelerate the creation of Solid applications and reach the critical mass of apps that would ensure the widespread and adoption of Solid. This is my gift to the community :wink:

And the plan is the use the Generative Objects platform to build the social network I am proposing in this post. And I plan on doing this and delivering the first working version in no more than 2 to 3 months thanks to the low-code productivity promise.

This is exactly it. And also creating a social network on top of it.

Yes indeed, this part is straight forward.

And this part (and much more) can be built very fast with Generative Objects technology. Especially now that we have a Solid connector for Generative Objects.

In fact it can be pretty straightforward if done this way : you just access the video clip from your friends you are connected with in the social network and who was sharing the file to you ! And if you share the file yourself, you embark it on your own POD and your own friends and connections can stream the video from your POD. The use case is therefore :

In the social network, on my feed lists : I can see a friend who shared a video

I watch the video, I am streaming it from my friendsā€™ POD

I like the video and want to share it on my profile

At the time of sharing the video, I embark it on my pod

When my own friends see my video sharing, they can watch the video by streaming it from my POD

However, if we want a public repository where people could search for videos and access them, then yes indeed we would need to have a proper discovery service on top of SOLID decentralized architecture. This can be done but is already another use case.

My first vision is to build a social network and do it the way I describe above. Also because every user will most probably pay for their own POD provider, and the more followers they have, the most they will have to pay to have sufficient bandwidth. Thus it would be important to limit who can stream a given video from a user POD to the friends/connection of this user to not involve more hosting cost for the user.

And yes, the value would be full decentralization of data, and of the social network itself, speed, resilience, and also a new way to ensure that the videos will not be tampered with, without having to rely on resource intensive technology like block chain. Instead of relying on the unbreakable quality of blockchain, you rely on the fact that a given video is potentially replicated so many time that someone who would push a tampered version would actually change the hash of the video that will not be the same as all the others copies ā€¦ All the copies would have to be changed to tamper with the video, which would be impossible because stored in personal PODs. This is a bonus statement :blush: I just thought of it, and writing while thinking, so it might not be fully relevant ā€¦

15

This is why I mentioned PeerTube, because it is not just an ActivityPub federated service, but has P2P capabilities specifically for the video part, and to manage the issues you mention. Copied from joinpeertube.org:

The PeerTube software can, whenever necessary, use a peer-to-peer protocol (P2P) to broadcast viral videos, lowering the load of their hosts.

In this way, when you watch a video, your computer contributes to its broadcast. If a lot of people are watching the same video at the same time, their browser automatically send small pieces of the video to the other viewers. The server resources are not over-exploited : the stream is split, the network optimized.

It might not look like it, but thanks to peer-to-peer broadcasting, popular video makers and their videos are no longer forced to be hosted by big companies, whose infrastructure can stand thousands of views at the same timeā€¦ or to pay for a robust but extremely expensive independent video host.

3 Likes
16

Does it ? for me what takes time is when you can to a place and want to see the list of users who accessly visited this place, because I am scanning all registered users PODs, with no optimization / no parallelising. And this would definitively not work with more users registered without extra indexing doneā€¦

For me it looks like accessing a single resource from a Solid POD is actually fast, it is a classical HTTP request. The problem is when you need to access data from many PODs, and consolidate all the data. My 5 cents is that it is a matter of designing the application and the pages accordingly. For instance, showing a post with comments from many different users : it would not be efficient to get all the data from all PODs of the commenters with classical http requests, consolidate the data and then display at the end. It would be required to highly parallelize the calls using web sockets, and display the data on the fly whenever a new piece is available.

For this use case, the factor is not so much for me the volume and number of users. Solid has a decentralized approach, so resources are split on so many different servers / providers, the load is distributed and every single request are individual HTTP request with linear performance. For me (but I might be underestimate other factors) the performance is about the display of a page on the user browser, and the number of calls to do to many PODs to get all the data required to display the page. However: a single page for a given user is by nature limited in terms of how much information can be displayed on a single page ! Therefore it must be possible to find technics to optimize this part, and this performance is not at all related to how many users are using the platform, performance will be linear no matter how many new users are joining (provided we have client side applications, which is the idea with Solid)

What is time and resource consuming though on networks like Facebook or Linkedin is the creation of the feed list (or for a bank : working on data from millions of users having their own PODs ) . It is especially true when complex algorithms are used to optimize the feed, not only for the user own direct interest but for optimizing advertising and what we want the users to see in priority. For that part yes : it would be more challenging, and we would need some server side application to do this. And if we have a server side application for this, all is possible, as it is for facebook : we would use a database to index all PODs content and links, without actually importing all the data : just the bits necessary. Every time data is added or modified by an application on a user PODs, some information (not the data itself, can even also be encrypted) can be sent to a server to do some consolidation / indexing work. I am simplifying here, but I am sure we can find solution.

To conclude : for the matter of an application used to navigate my profile and my friends posts, and navigate links, I donā€™t see a too big challenge (this is how the web works). We can start with this for a simple social network as described in my post, and with simple feed list with no extra complex algorithm. This use case is, I think, feasible.

What do you think of this ?

1 Like
17

Thank you @aschrijver, I did not have time yet to check your link, definitively something to explore. The peer to peer approach of joinpeertube.org is somehow different to what I am proposing. My approach is maybe too simplistic ? At the same time simplicity is good :slight_smile: maybe there is something in the middle to figure out, or a composition of both approaches. I like though the idea to leverage Solid and build a full new approach that relies solely on the core HTTP principles. I have a deep intuition that there is something very interesting to create here !

1 Like
18

Thank you @aschrijver for these links !
Individious looks great ! however for what I understand we are still relying on deployed instances of indivious. It is decentralized in the sense that it does not belong to a big corporate entity, and you can have multiple instances, but still centralized in the sense that each instance of individious is a classical centralized client server application (I might be wrongā€¦ I havenā€™t gone in too much exploration but feels like it , please tell me if I am wrongā€¦)

This is as I was sharing about mastodonde and each mastodonde instance being a classic client server application.

I think that with Solid we can go for a truly decentralized approach. Actualy, JoinPeerTube is decentralized since following a peer to peer approach. However I am wondering if with Solid we can invente a new way to decentralized.

2 Likes
19

Indeed. I added Invidious not as a full project example, but because you mentioned also sharing links to existing YT videos, so thereā€™s probably parts of the codebase interesting for closer examination.

1 Like
20

You can do that in parallel with HTTP. With HTTP/2 even the performance is good.

Exactly. Therefore you you can just load, letā€™s say a batch of 10 comments in parallel using HTTP, and further batches continuously while the user navigates through the comment list.

1 Like