When someone asked a slew of good, related questions on gitter recently I felt that I wanted to respond not on gitter, where there is no structure, but as a set of answers which could be jointly curated as part of the Solid Project’s institutional memory.
How about a Solid App for this? Things it wold be able to do should include:
Ability for a given group or groups to able add a question
Ability for the same or different set of people to be able to add answers
An associated solid chat for discussion of the questions and answers
An ability to create and nest and manipulate FAQ sections (like bookmark folders?)
An ability to “soft link” FAQ sections to existing or new other FAQs
Questions and answers could be thought of as special cases of messages, so message code from chat could maybe be adapted for them. This would include the ability to like, bookmark, share in various ways. A difference, though, is that chat messages are generally written once and rarely if ever edited, but FAQ answers would be constantly edited to keep them up to date. The authors of the Question may be less important or anonymous. So might the authors of the answers.
FAQs would be useful wherever a group currently uses a wiki.
Those old enough may remember that internet newsgroups had FAQs which were build up by the community and circulated periodically or stored on FTP sites, so the community had a place for new people to get up to speed, and a reference for what the group’s common understanding. Let’s use Solid for that.
A chat and and FAQ go well together, so a FAQ could be an optional feature of any solid chat. Any FAQ should have a pointer to the chat where it is discussed.
I would like to further elaborate on this idea. We all use many online channels in day-to-day communications. Inevitably while typing somewhere - e.g. in a Discourse discussion like this - we will generate “knowledge” that is more more valuable in a broader context. We would like to collect it somewhere, and maybe even duplicate / archive to different locations.
If I typed something useful here, you might respond “Please add that to the FAQ Solid App”, and I might either take the time for this manual action, or just forget.
So I am thinking of turning things around. Make knowledge aggregation from any channel easier.
In this case the FAQ has a simple “Q&A” semantic vocabulary, but it could also be a “Best-practice”, or a “Rule”, or “Guidance”, etc. In other words some snippet of Linked Data.
Question: How can we make knowledge aggregation easier?
Answer: By aggegrating it automatically from the channel you are currently explaining the knowledge. At this time, while your mind focused on it, you are most likely to describe it properly, especially when you are aware it will be reused elsewhere.
Since the channel here is Discourse forum, there’d be a plugin that parses this quote text snippet from my comment and sends it to a pre-configured aggregation server. This can be any type of application, in this case maybe the “Solid FAQ Center”, a website where it enters a review queue.
There’s a publish/subscribe model, and there can be many aggregators (a federation model) as well as many supported channels (Gitter, Github, MatterMost, etc.). For instance, the W3C may have a Linked Data Glossary wiki that is fed from incoming data that listens to a “Solid” channel. I hereby send it a new term via the imaginary Discourse plugin:
Glossary: POD
Personal Online Datastore. Pods are like secure personal web servers for data. When data is stored in someone’s Pod, they control which people and applications can access it. A pod lets people store their data securely in a decentralized location.
Federation
I’ve had this idea for some time. It would fit Solid perfectly. It is also similar in concept to the Murmurations protocol. Now Murmurations has the issue that the index server is centralized. So I imagine that to be federated instead, using ActivityPub and the whole semantic knowledge network becomes both a Solid as well as a Fediverse application. Federation is useful as it removes single points of failure, and at any time there are servers available that allow you to subscribe to channels of interest.
WDYT?
It seems most feasible to me. A MVP would be:
A single channel plugin e.g. for Gitter with schema + template + config features
A PubSub server that handles Linked Data snippets, stores in pods
A consumer that subscribes to the server, presents data, e.g. a FAQ website
[Optional] Federation, sync between PubSub servers
You can find related discussion and more elaboration by me on this Fediverse sub-thread:
In agreement with the direction described by @aschrijver, I think the key advantage of solid here over e.g. stackoverflow is the expression of the underlying data/snippets as linked data, and therefore the ability to reference them, expose alternative interfaces, and for anyone to be able to host their own FAQ in an app, mixing data from their pod and others’.
I want to be able to make the FAQ (or other knowledge representation) my own - to be able copy parts of it to my pod, add my own personal annotations, subscribe to and vet changes from this and other versions, and potentially have others subscribe to changes to my version. In addition to the ability to collaboratively edit a single document, I want an/another interface that provides this functionality over the same data.
A gitter etc. plugin is yet another interface with a similar intent.
There are two parts to this: one is the UI design and associated vocab development, which is specific to a particular knowledge representation (e.g. FAQ), though there would be common UI elements and vocab regarding saving and subscribing.
The second part relates to the knowledge aggregation functionality. In principle Solid provides the ability to discover the user’s desired FAQ data registry and to save to it. Even without PubSub, it seems plausible that an app could use LDN to receive subscription requests, and deliver notifications at the time a change is made. Integration with PubSub would presumably strengthen scaling.
In my mind, the greatest difficulty in getting started is dealing with versioning (for both federation and collaborative editing).
There’s etags, m-ld, braid, memento, and prov vocabulary, but it’s not yet clear to me how to put the different parts together in a Solid world where the user determines what additional functionality their pod might make available.
can easily become untrusted if content is inaccurate.
I think the narrative flow of an Answer is really important, in addition to its content. This suggests to me that the format of a single Answer is best represented as marked-up text, and it lives in a structured metadata context shared with other Qs and As.
I agree with @josephguillaume that versioning is essential. Because of the above, I think it’s key that updates are human-mediated, to update the narrative, so a wiki or VCS-style versioning approach is best.
While it would be nice to have collaborative editing (of course ) I think the most valuable improvements we can make to existing wiki-like systems is to be able to:
prompt authors to react to changes in the references (e.g. a new version of an API);
aggregate from chats & discussions, etc., via UI plug-ins or automatically.
(Incidentally, I’m wary of the latter idea to aggregate automatically. This could have the unintended consequence of helpful people becoming hesitant to answer anything they’re not totally sure about, especially in informal chats. Maybe this could be worked around with some UX, e.g. a fine-grained opt-out.)
Indeed. I think the general need that @timbl expressed could be described as:
Need: Make knowledge generated in one place (channel) easily available (‘pubsub’) to any other place (app) where its valuable.
(Note that ‘PubSub’ is just conceptional, not necessarily a direct component of the technical implementation)
This refers to where you subscribe to the knowledge, which is either directly to the channel where it is generated, or to some aggregation point. The latter might be fed with knowledge coming from many different channels. The aggregator decouples from the channels.
This refers to the use case that is interesting to you, i.e. the ‘app’ part of the equation that consumes knowledge from connected aggregators or channels.
Your app has generated new knowledge as it were, and to make that available to other you now take the role of a ‘channel’ that published it.
I perceive about 4 parts possibly, namely:
the vocab design and that may include rules for how it should be represented in UI’s
the actual channel plugin that is capable of rendering the UI for that particular channel
the whole infrastructure for distributing the appropriate locations
the storage and aggregation of knowledge
For my case I see the best role in 3) is ActivityPub + Fediverse. In 1) a big role for Linked Data. Part 2) is wholly channel-specific, and 4) seems best suited to what Solid brings to the table on top of Linked Data.
For a generic applicability of the solution the separation of concerns this offers is important. Reformulating the above the general process is:
Extraction → Distribution → Aggregation → Consumption
Each of these steps may have some tailored processes that depend on the use case. For a MVP we should keep as simple as possible but recognize the extensibility points that exist in this top-level process.
I think the main question is: Could we define a good wire format that captures a FAQ knowledge snippet? I guess that’s a Yes (e.g. JSON-LD). Other than that:
Extraction: depends on a) the channel you wanna use b) your requirements for gathering data from said channel.
Distribution: might be provided mostly out-of-the-box, without custom requirements to consider.
Aggregation: might also be mostly readily available as general components to deploy.
Consumption: is then the part where most of custom needs must be addressed (e.g. collaborative editing)
You might object that Aggregation cannot be box-standard, because review + collaborative editing must first occur to assure knowledge quality. But I see that as just another Consumer app / component in the overall flow, where the processed result (‘enriched knowledge’) is fed back in the system.
I see this as a concern in the Extraction step of the process flow. How are snippets of knowledge created, and how does that affect how people interact in that particular channel?
There may some rather generic channel implementation available for the most simple use cases. Other than that custom requirements may arise. One such requirement may be that only particular stakeholders have the ability to define knowledge snippets for aggregation.
I will give an example scenario, completely different than @timbl’s Solid FAQ app that has this requirement. Here goes…
Use case: Improve Github issue management
Problem statement
In our project we have great interaction from our development community and contributors. Many issues are created and thoroughly discussed until either a PR is created or the issue is closed as “Not doing”. But as our project grows in size and complexity we have to tackle some important bottlenecks:
Our maintainers spend more time than needed in issue management to follow-up in all ongoing discussions.
Even when an issue is still open the amount of text gets Too Long; Didn’t Read for many, leading to inefficient communication.
After the issue is closed the discussion contains many implicit design + architecture decisions that become hard to find.
Solution
Give maintainers the ability to markup knowledge snippets in their responses to issue comments, that are automatically aggregated to the various documentation artifacts of the project. An example of such response might look like:
@TheMaintainer wrote:
Thank you @SomeUser for that explanation. I wasn’t aware that our project was in use for such large-scale deployment. We see that we must support this for you and future customers, so I hereby define:
Requirement: The server MUST support at least 500 concurrent connections.
As for your other points … [bla bla yada yada] …
@SolidKnowledgeBot: I added 1 Requirement to Functional design on behalf of @TheMaintainer and related to #3456.
SolidKnowledgeBot might update docs directly, but more likely would create PR’s if the functional design was a Markdown file in /docs. But there might be multiple consumers, such as a Trello board where a card is created, or a Discourse forum where a discussion topic is created for project community.
I’m keen to further explore the generic/solidos aspects of the aggregation+consumption workflows.
Looking at the GitHub use case (thanks!), I think the “update directly” flow is already addressed using Solid (or covered by generic data interoperability/discovery mechanisms).
What we’re after is therefore some form of PR mechanism/Trello card staging/discussion topic moderation on a solid server.
Trello card staging seems easy enough - with the schema used by the issue-pane, a new issue would be created with the type used for wf:initialState. The (card) UI already has the ability to differentiate between issue states.
The distribution tool would also need to be granted write/append privileges.
Discussion topic moderation is a little trickier. The chat-pane looks like it builds a list of “other chats” from other registered LongChat objects. In principle, aggregation could just involve registering a new LongChat, but probably a class is needed that provides an index of LongChat objects. That doesn’t seem to exist yet?
Marking a chat as being new/unchecked obviously isn’t supported yet either.
Both of these involve a fairly simple aggregation: an index in which new objects can be added, with the new object being marked as such.
For an FAQ, this idea already provides two levels of possible aggregation: adding a new FAQ topic to an FAQ index, and adding a new FAQ answer to a question. In both cases, the status of the new addition needs to be stored.
The PR is more difficult again as it implies merging of data. If the new data is just an addition to a list, then it reduces to the cases above. If it involves deletions or modifications, then we need to be able to manipulate diffs or multiple versions, and deal with conflicts if the FAQ has been edited since the version referred to in the extraction channel.
This is probably not in the MVP, and I agree is probably for another topic.
An inspiring follow-up, Joseph, thanks! Since the idea has such broad application, and with your further elaboration on the issue management use case I was inspired to cross-post the idea to the FedeProxy community in this post:
I consider FedeProxy to be among the most important FOSS projects, as the community intends to free software development from the shackles of specific code forges, like Github and Gitlab, that are increasingly turning into walled gardens and having network effect that lead to lotsa FOMO to use other forge software, or self-host ones repositories. FedeProxy was the main driver behind the move to Federate Gitea, a project that is now ongoing.
(The idea is not in scope for FedeProxy MVP, but may fit in a future vision to move towards United Software Development)
I will look more in-depth to your writings later this week, but also hope other people will join this brainstorm, and maybe @timbl can reflect on it too wrt to the original proposal.
) I think the most valuable improvements we can make to existing wiki-like systems is to be able to:
: