Solid scope and ecosystem

I don’t think it’s huge compared to the total amount of data online. Yes there’s a lot, but it’s like saying Firefox is huge, when it’s actually a small fraction of the market.

I want the Semantic Web, but I don’t think it is near dominant yet and I’m not confident it ever will be.

There is enough public semantic data out there to make it very useful, which is why I’m building stuff to help make use of it.

1 Like

But it seems like the biologists and geneticists are leading the way, so a lot of things will be built on that.

I would like to stay on topic with:

  • Is the scope of Solid clear?
  • Is the scope of Solid clearly communicated?
  • Is the scope of Solid as small as described in my first post?

This is what I would like to know when doing a software selection and evaluating Solid. I pose that:

The chance to be confused about Solid is 99% and Solid is not selected on the shortlist of the passer-by.

What I see now is many different standards and functionality being discussed as if they were a natural part of Solid that refer imho to a much, much broader scope than I expected. At the same time many project that are part of the initiative use a very specific techstack, which may put people off who are not using said stack. They then leave with impression that Solid is yet another app framework.

1 Like

As you know I love this topic and feel that it is fundamental to the community to get this nailed down sooner rather than later.
@aschrijver I think you are doing a great job of trying to articulate the drivers and gaps. But I think what would really help here is for Inrupt to wade in with a view. They are commercially driven to make this successful, and are funding the majority of the project as I understand it, so it would be great to hear their vision. Anyone out there who could comment?

3 Likes

Having been an active participant in the early years of the web, my memory is that it took off because a bunch of us said “Yes! This is how knowledge wants to be.” because we saw the long range implications and that is what drew us to it.

In my view, this is how Solid wants knowledge to be (with tech implementation in parentheses):

  • Each person’s data is under their own control. (storage/access: wac,pods)
  • Meaning is assigned by humans (processing: rdf)
  • A person’s identity is not controlled by centralized providers of other services (identity: webId)
  • Interface choice is not controlled by centralized providers of other services (interface: apps, not servers)
  • Linking and co-creating are easy (collaboration : rdf,wac,read-write web)
  • Filtering of bad actors is easy (wac)
5 Likes

Thank you @james. And thanks too @jeffz such kind of list bring clarity and should be on the For Developers page as feature / vision cards, with a dril-down to specifics and some diagrams clearly showing how it all sticks together.

So my 2 bullet points correspond to your 1) and 3) and then you get 6) as a natural result of that - as I read it - or are there additional chunks of functionality in that filter that are scope extending? I see bullet 2) assigning meaning, as a precondition to Solid, but it is not one that needs an entire semantic web… I could apply Solid in any bounded context where you have that.

Bullet point 4) and 5) are real scope expanders! Both are, to me, nice-to-haves compared with 1), 3). Question is if 4) and 5) are being worked on to the detriment of progress in the ‘core features’? And to what extend can I use these features in isolation?

I recommend reading the paper of the ‘competing’ standards-effort on Encrypted Data Vaults presented at Rebooting the Web of Trust, Prague. It makes a comparison of existing approaches, including Solid, and builds that out to requirements + architecture etc.

Regarding 4) they give this clear description:

“In the case of Solid, NextCloud, and Identity Hubs, end users have the option of installing and running the server portion of the data store on a device they control, or signing up to an already configured instance hosted by a trusted third-party (eg. a commercial provider, affiliated institution, or friend).”

Aha! This insight is not easy to be had in the Solid world, and before that you will be so confused with ‘Apps’ and all the complexity of 4) and 5) being discussed, that you probably miss it.

I would urge Solid to use different terminology than ‘Apps’!

What I would like to see for solidproject.org content strategy to communicate its message is:

  1. This is in layman’s terms the crystal-clear elevator pitch and bullet points of what Solid adds to the existing linked data web

    1. This is the intuitive roadmap that we follow to bring this to you
    2. Drilling down on each of the bullet points will lead you progressively to the nitty-gritty details
    3. This is the process that we follow, and the parties that are involved and their roles and responsibilities.
    4. We have a commercial, business-driven side, and we are starting to support a more open community-driven side which we want to be closely evolved in the evolution of Solid. (We recognize that until now we spent less attention to the OSS and free software movement, but that will change).
  2. Standardization of all of this is our primary concern. This is why Solid exists.

    1. Our goal is for all this to become a natural part of the future web
    2. Until web-scale adoption is reached here are the use cases where you can already benefit from this
    3. Here are the exact interdependencies between our specs and components, and here are the places where our specs are not places and which you should avoid or expect frequent changes.
    4. This is the complete overview of how Solid relates to other initiatives: how they overlap, and how they are complementary, and what our cooperation efforts are.
  3. To help guide us and guide our community of implementers we are building an ecosystem around the standards

    1. In this ecosystem we make specific technology choices, like JS-based techstack, but the Solid standards are for any techstack
    2. Here are our reference implementations, here are our example applications and here are our community-driven projects
  4. Here’s our community of friends that help us in our quest

    1. And this is the way to come aboard, where we gladly welcome you, to help the community to be more than just a forum.
    2. Here are the exciting community plans and we’ll leave it to them to guide you through it (community independence).

A bit tangential to this topic…

In general I find the approach of RWOT more appealing, because of a) pure focus on standardization without confusing mix of impl/ecosystem building, b) they seem to take a broader perspective than the more of a ‘this is how the web should be’ approach of Solid, and this reflects well on c) the standards they are creating which I not only find quite exciting (and well documented), they are also closer on the W3C standardization track.

Luckily I got a satisfying answer that Solid is aware and cooperating with RWOT and considering adding support for said standards.

For instance in Proposal: Support Decentralized Identifiers (DIDs) in addition to Web IDs, where @codenamedmitri also mentions “Inrupt, as well as the Solid community, should join the DID WG!” and Project: Support W3C Verifiable Credentials on Solid.

@aschrijver below is something that you may find useful as a template and also to compare and contrast with SAFE, published by Maidsafe to clarify the goals of that project.

It would be really useful to have something like this from those at the heart of Solid. We can infer here, but it would I think be helpful to have input of this form from @timbl and Inrupt.

2 Likes

Ah yes, a list of top-level use cases and requirements. I imagine these should be available somewhere for Solid?

1 Like

As far as linked data, or rdf, I don’t see how its off topic here.

If you look at

https://www.lod-cloud.net/

the number of rdf datasets is not growing so dramatically lately, but that doesn’t measure the number of links. But if you look at

https://www.lod-cloud.net/clouds/lod-cloud.svg

the links in the bio area are very dense. Its getting a lot smarter. How can all that be replaced so easily?

They don’t even count datasets under 1000 triples, which would probably leave out most foaf files and solid pods.

They are not in project scope, unless there is a need to extend the specs, or - in an effort to reboot the semantic web - solid has widespread adoption as their objective.

Imho, it will indeed not be replaced, probably even grow some more. But the big breakthrough that many hope for is far from certain.

Adoption-wise. There is risk of confirmation bias in this space, because solid project is the main focal point for all things linked data (see for yourself by searching related terms on github). Most devs hardly know what linked data entails, even though they know about SEO, Google Knowledge Graph and schema.org. (BTW Most graph databases - also the new ones - are not using linked data).

For business types most of this stuff is so overly technical, with unappealing UI’s and such, that there is not so much yet to pique their interest. From the semantic web era I can’t remember a single application with a good user experience, and most uptake was only in academic realms (that is why I’m happy to see people like @glensimister here). SemWeb was a hype then, and the fact that it didn’t succeed meant that devs ‘moved on’. It will be harder to convince them a second time.

Then there is the risk of competing standards prevailing, like the aforementioned RWOT specs. Also in general, with its current positioning Solid may be perceived as just one in a forest of privacy-related solutions.

Finally there is EEE, or embrace, extend, extinguish (just recently mentioned in a discussion about Offer support by personal offerbots on SocialHub forum, as a big risk for the fediverse), where eventually some FAANG takes control. A semi-proprietary linked data web could arise (I saw renewed activity at viv.ai, acquired by Samsung, where in the intro video at 8.50min. you see a potential such candidate).

Also internet activist Aral Balkan mentioned to me that he thinks that - since Inrupt is VC-funded, which he is firmly against - this means they will aim for a Big Tech buy-out. I don’t know, but it is possible.

1 Like

I can’t cite numbers, and there is so much that goes on out of view that unfortunately I’m only left with vague beliefs. I think TimBL is on the level. I think he’s doing this because he wants a better world, not because he wants to sell out to FAANG. I think the purpose of linked data was and is so that we can reason with machines and especially with each other. I think Solid is intended to protect our privacy, not to hype linked data.

1 Like

I agree and think so too, plus hypes are never good. But unfortunately the tech world as a whole decides what will be a hype or not (and it is very hype-sensitive… we’ve just left the AI hype and now its waiting what will be next). Then we should be prepared to deal with that.

I think the read/write web motto is very important but somewhat underestimated. I see quite a few efforts to introduce immutability and other append-only content-addressed storage technologies, which most certainly have uses (such as storing the vocabularies, or videos, or anything that demanded a lot of work before being published) but it will not suit all social interactions because everyone makes mistakes, and correcting or removing the data will always lead to better understanding of each other than piling up more data on top of the incorrect data.

1 Like

@aschrijver wrote

assigning meaning, as a precondition to Solid, but it is not one that needs an entire semantic web… I could apply Solid in any bounded context where you have that.

Please give an example of accomplishing this goal without the semantic web.

I can imagine a situation in which we all had control of our data and identities (1&3) but in which a limited number of centralized servers provided the only interface and processing mechanisms (4&5). That would not be much improved over the current situation. So I believe they are not simply nice-to-haves.

Your point about apps strikes me as very off base. First you quote something which says that a pod can be on a remote host or a local device. Then you claim that talk of apps obscures that point. What has one got to do with the other? The first has to do with where servers are located and relates entirely to data storage and and has nothing to do with apps which relate to interface and processing.

You suggest a different terminology than “Apps”. Please explain how Solid can accomplish the five points in my scope statement without using the term “apps”. And especially, please explain it in a way that a developer can get the idea that designing for Solid needs a reorientation away from the currently standard server-based approach to processing - the most important thing for new Solid developers to recognize.

Yes, you rightly point this out. It can be confusing. Where I say ‘semantic web’ I mean the effort to migrate the entire web to linked data standards.

I am not saying that in general they are nice-to-haves, but to me in the current use that is interesting to me they are. Given that the scope of Solid may be much larger than what is in plain sight, this may mean that for my current use case I better look elsewhere, until Solid gets a more complete and stable state.

Yes, that could well be… I am still on the quest to untangle the scope of solid. Hence this thread.

Ah, this is interesting. Here I think you refer to Apps in the same understanding as I have of this concept: Any apps / applications and probably even services that any developer creates.

But if you say Solid Apps, then you create a confusion. Is that different than Android Apps, or Apple Apps, or Atlassian Apps, or whatever else kind of apps there are?

If however, I do not have the same understanding of apps you just referred to, and there indeed is something solid-specific to these apps (other than integrating, say, a library to achieve solid-standards-compliance), then talking about Solid Apps is still confusing. That android developer when hearing about Solid will say “Solid Apps? Nah, I create Android Apps”.

What imho in both cases would be much better is to make it crystal-clear that this is about Solid-compliant apps, i.e. apps that implement solid standard in some way. So that android developer will say “Solid? Sure I make my app solid-compliant after hearing these benefits”.

PS. All I’ve just said would be invalid if Solid does not have the ambition to be a universal standard, and is content to be just for a subset of devs that see merit in that and happen to also like the accompanying techstack. This is imho likeliest interpretation an outsider will get when they casually bump into solidproject.org.

1 Like

I do see your point about “Solid compliant app” instead of “Solid app”. I would prefer to think of it as “an app operating in the Solid ecosystem”. Maybe this picture will explain why we keep referring to apps as a key part of the architecture

.

1 Like

Yes, I see what you mean, and that is a clear diagram. Someone reading it will infer that “app” could also include their app if they implement / integrate solid standards support. It could also say “compliant app”, but not “solid app” or the confusion returns.

Note that I am nit-picking on this, because I think it is crucial to Solid’s widespread adoption. Most times I’ve seen Solid discussed outside of solid community e.g. on Hacker News (not unimportant), it was with a lot of confusion and misunderstandings.

In that light I think “an app operating in the Solid ecosystem” should also be avoided, because now I have the impression that I have to integrate with an entire ecosystem, and build my app around that.


I find the terminology in Encrypted Data Vaults to be much more intuitive.

  • Encrypted Data Vault vs. Personal Online Data Storage (“Ah, you mean like Google Drive, ‘Free Cloud Storage for Personal Use’…”). The first is much more descriptive. If you use POD’s then it must be followed with an explanation of the acronym.
  • Uses familiar Client + Server terminology and avoids Services and the overloaded term Apps altogether
  • Having a Vault it is clear that I could host it on either a Client or a Server
  • Storage server vs. POD Provider. A storage server is a universal concept. Is a POD provider a specialized storage server?

I also really like the Core Use Cases section. It is written so that most non-techies can understand

  • It also avoids words that can be confusing, like ‘privacy’ and 'encryption. It just says e.g. “only I can see and use the data”. Instead “Prioritize Privacy” is a Design Goal.
  • It avoids even ‘personal data’ saying just ‘data’. What is personal data? When does it go from being data to personal data or back? Why would I not store more than personal data in my vault? The word data is sufficient. ‘Personal data’ is open to interpretation and there is much discussion on what is an is not personal (e.g. see GDPR-related discussions).
1 Like

In my opinion, that impression would be correct and is a key point that needs to be gotten across to developers - Solid apps are meant to work in the Solid ecosystem and can not operate entirely outside of that ecosystem.

As far as using Solid as an adjective on the word “app”, let’s look at other adjectival uses:

  • Javascript App - an app specific to a given language
  • Social Media App - an app specific to a given function
  • Android App - an app specific to a given operating system
  • Web App, Desktop App - apps specific to given environments

When I use the phrase “Solid app”, I mean it in that fourth sense, an app specific to a given environment. Nothing you’ve said has convinced me that is an improper usage.

That is great information. As an early adopter one should clearly know - be able to glean quick insight to - exactly what other parts of the ecosystem or requirements are needed.

Using the term ‘Solid App’ makes it a subset of the world of apps. It is a new term that newcomers have never heard of, and now they have to determine how that slices with all the examples of app categorizations (and there are more) that you just mentioned.

I was preparing another response when you posted, so I just copy that below:


@jeffz regarding confusion… We had the same understanding on what “apps” mean, previously. But the Solid Project website is telling an entirely different tale.

Developer drill-down:

Landingpage ➜ For Developers ➜ Build Solid Web Apps ➜ Build your first Solid Web App:

“This tutorial assumes you are familiar with modern JavaScript and its ecosystem […] and are interested in building one that reads and writes data to a Solid Pod.” The entire code walkthrough gives no clarity how a Solid (Web) App is related to the common understanding of “apps”.

  • Impression: Solid Web Apps are NodeJS apps that use the Solid web framework…

Landingpage ➜ For Developers ➜ Build Solid Web Apps ➜ Check out the existing Solid apps:

This time apps is not capitalized. On the ‘Use Solid Apps’ page there is a hopeful start: " You wrote a Solid-compatible app that’s not listed? Feel free to add it to the list."

But then the list, as far as I can see, only contains apps that were built from the ground up with Solid in mind. There are no integration libraries, plugins, addons, extensions to existing projects / products out in the wild.

  • Impression: Yes indeed, this is an alternative framework than what I am using now.

Landingpage ➜ For Developers ➜ Tools and libraries overview:

Here you find Solid specific tools and Solid specific libraries, but once again none of the tools and libraries that people are familiar with. No integrations with existing systems.

  • Impression: Wow, I have to learn an entire new ecosystem to stick a full-blown app together.

Landingpage ➜ Implement Solid

Implement Solid sounds interesting, BUT: “Contact us on our email to state your need”. Full stop.

  • Impression: Ah, this is probably a .org site driven by a business that develops the flagship products for this framework.

User drilldown

Costs me too much time to expand now, but the perception is the same developers will get.

FAQ

Here there is an entirely new set of confusions to address. It makes everything Solid.

  • Solid users ➜ Is that different than just “users”?
  • When using Solid… ➜ I am not really using Solid, I just use a solid-compliant app

And more things that do not really make it easier.

If you have specific suggestions for rewriting that part of the documentation, please submit some PRs on the github repo.

There are no integration libraries, plugins,

Those are found in a different part of the documentation, one that I happen to be writing. So I’d value if you gave me a similar critique of the Solid tools and libraries page and I can directly take your input into account as I rewrite.

2 Likes