Is RDF "hard"?

My apologies for a long post. But I have added a TL;DR of sorts:

Summary

  • Solid project seems most focused on a Grand Vision where its success hinges on tackling enormous complexity and widespread adoption of standards and ecosystem projects.

  • There’s no gradual path to adoption with stable well-productized milestones all along the roadmap and options to choose from. No way for people to get acquainted with Solid without going all-in and face the brunt of the complexity.

  • There’s little focus on the process of software development, how Solid fits, and what benefits it brings in the short term (i.e. before the Grand Vision is realized).

  • While there is a deep technical focus, there seems to be almost a business myopia, as to all the process and design best-practices that are also needed to create interoperable apps that satisfy people’s needs. Social aspects of development are neglected.

Recommendations:

  • Focus on all the practical things that help average developers leverage Solid technology right now in their apps. Tools, documentation, different langauges supported, etc. And with these people now invested in Solid, entice them towards deeper community involvement.

  • Ensure that not just tech is covered, but that Process is as well. How do I design my app with reasonable expectations for interoperability? How can and should I collaborate with others, and what organization structure and tools can we offer to help with that?

I notice the tendency to deep-dive into code, and I understand that as you are all coding apps and libraries. I am just chiming in as an outside observer going deliberately beyond the technical. And I may see this all wrong, of course, but I’m writing this down in hopes it may be useful, not to be critical :hugs:

Developer experience

Some have said that RDF is actually pretty simple, but that’s just when looking at the format and how it presents information. However, presenting meaningful, machine-readable information that is interoperable with other apps is where it gets tricky and where - according to the HN comment - “layers upon layers of complexity” are added and “no good library and tool support in general” exists. As @NoelDeMartin mentions as soon as a standard is not implemented in your language you have to create it from scratch.

:thinking: My view: Unless you don’t care about broad interoperability things become pretty hard pretty quick on a technical level.

Semantic Interoperability

Why go through all this trouble? That is the potential ease of interoperability, i.e. of others reusing and building upon your data model without having to closely cooperate with you to formalize how that looks like, right? The work has been done, and a common vocabulary is for grabs.

@jeffz outlines outlines a process for doing so, i.e. first go to the most widely used ontology at schema.org and then move on to LOV ontology search engine and find more ontologies to fill the gaps. This focus on the Process is a big step in right direction, and I feel that it is exactly the process part that gets overlooked by the more tech-oriented people in the community.

@anon85132706 wrote in another topic:

I don’t know about that, but I argue that the process above only partly solves interoperability issues. By choosing from common ontologies we’ve only increased the potential for our data to be reused in interoperable ways by others, and without being super duper careful while doing so we may have increased the chances that that happens in the wrong way and will lead to problems down the road.

:thinking: My view: Unless having a very well defined and deep understanding of the process and following it properly, the technical deep-dive may not be a worthwhile exercise at all.

What do I mean here? Given the example above I have a certain semantic interpretation of a “Customer” as it exists in my app. Now I start to look for a representation in a common ontology that I think matches my idea of the concept. Now, if someone else using the same ontology has a slightly different interpretation, then now we have a semantic mismatch. And these mismatches are I think very easy to make, and everyone makes them all the time and at all different places in the interconnected graph. I can query with SPARQL all I want, but the further I get from my application boundary the more meaningless the data becomes. I cannot trust the results to be useful, unless the common understanding of their meaning is still guaranteed.

Of course, having really well-designed ontologies helps here. And some of these exist, where their concepts are so ubiquitous that they are hard to misinterpret. But it is as @dynamoRando says:

In reaction to @anon85132706 a reference was made by @justin to Solid Application Interoperability, a new draft standard that builds upon other standards, and aims to address Problems and Goals for Interoperability, Collaboration, and Security. That is great, but in short term that makes the circle round to adding more “layers upon layers of complexity” and “lack of tool support”.

:performing_arts: The Faces of Solid

Some personal impressions, that may be wrong… To me it seems like there are 2 tracks to the Solid project that are intertwined:

  1. The practical path. Defining ways to have secure personal data vaults that are decoupled from their application, and where the linked data in them increases the potential for semantic interoperability (lowers the barrier to reuse). Comprehensive, mostly existing standards suffice.

  2. The Big Vision. Broadscale, seamless interoperability with ease. Building on 1) with many additional specs and layers of complexity, but abstracted away by a rich and mature ecosystem. A rebooting of the Semantic Web.

You can very well build apps that align to 1) currently and then RDF will be “manageable”, but if you want to also comply or evolve to 2) then RDF is “hard” and you should accept years of ‘growing pains’ of the ecosystem yet to come. While 1) is within reach now, the visionary path 2) may never be reached. It requires broad acceptance of the standards and widespread adoption for it to materialize.

To me it seems that 2) The Broad Vision is what the core team is most interested in, and I sometimes wonder if there shouldn’t be much more attention to a gradual transition path from track 1) to track 2) and a clearer distinction between both. That track 1 is neglected as a valid approach for people who just want to make a start with Solid technology.

After all that is what this whole thread is about. We get smacked around the ears with all the inter-related and still very much emerging standards and their low-level technical implementations.

Process

Back to the process. What do we try to do? Make delightful software that addresses people’s needs!

The whole “what RDF should I have?” is much further down the line of the software development lifecycle. It is a technical concern. And so too is the answer to the question “How interoperable does my application need to be?”, which ultimately translates to technical requirements.

In all this technical onslaught we forget that software development is in large part a social process.

I find this whole notion mostly missing from this community, and frankly so too do I miss that in the SocialHub community (dedicated to evolving the Fediverse) where I am more active. (It is I think the reason why many apps when adding federation support seem to me like they just added Microblogging capabilities. Because that’s the technical interop infrastructure that the first popular app Mastodon put in place. And with a pure tech focus it seems that that should be “plugged into”.)

Where is the Process? We can code as much as we want, but without proper process how do we know we build the right thing? That we translate real needs into code? Even more so, how - without attention to process every step of the way - do we know that track 2) of The Big Vision will eventually even support the processes we need? That things will live up to their promise?

I feel that Solid project would be much stronger if it provided good stepping stones towards the big vision, with production-ready deliverables all along the way and proper guidance in place to go from one stone to the next. That way it can take an ever growing developer base along its trajectory, instead of just the most adventurous developers that are in for the long haul.

Domain modeling

Stepping away from Solid for a bit, and looking at software development. In general we analyse stakeholder needs to learn about the business domain, then we create a business model - which has both a particular data structure as well as business rules / logic - and only then we start our elaboration into ever more technical details. It can be iterative and agile, but we always do this, even when not being explicit about it.

Strategic Domain Driven Design is a more explicit method. It tries to capture the common or Ubiquitous Language of the business and then split them into Bounded Contexts which are like the scope where the semantic meaning of concepts is well-defined. A “Customer” in Ordering isn’t the same as the “Customer” in Shipping context. And that makes them manageable.

If we want to fall back on using well-known ontologies to express ‘universal semantics’ we are not only reusing a data format, but we are reusing all the social interaction that led to their design. We have to understand and adhere to the business domain they were defined for. Or, alternatively, we have to have these layers of complexity in place that can parse the universal semantics in machine-readable fashion.

This is not the first time I wrote about this. In Aligning efforts in LD schema / ontology design + adoption I quoted from Kevin Feeney:

“[…] the big problem is that the well-known ontologies and vocabularies such as foaf and dublin-core that have been reused, cannot really be used as libraries in such a manner. They lack precise and correct definitions and they are full of errors and mutual inconsistencies [1] and they themselves use terms from other ontologies — creating huge and unwieldy dependency trees.”

I feel that it is worthwhile to ask what we want to achieve with Solid in terms of benefits, both in the short and in the long term, and pay more attention to where it sits in the process of going from people’s needs to actual code.

3 Likes