Integrating ActivityPub within Solid specs

Solid Specification
1

Hello everyone,

Following last week’s Solid Practitioners meeting where I’ve presented ActivityPods and seen (perhaps for the first time) a lot of enthusiasm regarding the idea of integrating ActivityPub as a communication protocol within Solid, and at the request of several attendees, I’ve started to compile a list of what would be needed to add an ActivityPub layer to Solid Pods.

But first a little intro to give some context…

What is ActivityPub ?

ActivityPub is a simple communication protocol that allows any actor on the web to communicate with any other actor, no matter what software or server they are using. Actors can be persons, groups, organizations or softwares (bots).

Every actor can publish activities through its own outbox. The ActivityPub server then forwards these activities to the recipients’ inboxes (using HTTP signature for authentication). Inboxes and outboxes can also be fetched to see the activities that have been posted (if permissions allow).

In a way, ActivityPub is like email, but thanks to the use of RDF, it can describe any kind of social activity. Many commonly-used social activities are described in the ActivityStreams (AS) ontology, but it can be extended as much as needed with other ontologies.

Why ActivityPub is needed by Solid ?

The other day, @nikoNG from NextGraph shared with us the three types of communication protocols that can be abstracted (and that he also implemented in NextGraph): PubSub (Streaming), RPC (Remote Procedure Call) and Inbox. Here is the slide I showed during last week’s meeting:

8dfaf2f8-fc58-431e-8a00-8ce2231d21ed
8dfaf2f8-fc58-431e-8a00-8ce2231d21ed1054×589 68.6 KB

As we can see in the violet area, Solid already use all three types of communication protocols, but in our opinion the Linked Data Notification (LDN) is too lightweight to fully address the related needs (ActivityPub was built upon LDN, which is why it uses the same predicate for the inbox)

The real need is that every WebID user has a clear way to be reached, and that it is also able to respond if needed. Once WebIDs can communicate with each others, many other things become possible and, maybe, we can rely less on the RPC-type of communication that is common in Solid specs.

Regarding Solid Notifications, it is a good protocol, but it requires the client to subscribe first, otherwise they don’t get any notifications. So it is good to watch resources (and we use it in ActivityPods to watch for inbox/outbox activities) but it is too limited for open communication between actors, since an actor cannot listen to the all other actors on the web.

What MUST be implemented ?

Here’s what must be implemented to have a very basic ActivityPub communication between actors:

  • Generate a private/public keypair on the Pod and attach the public key to the WebID
  • Attach a as:outbox endpoint to the WebID (ref.)
    • POST allow the actor (WebID owner) to post activities, which are then forwarded to recipients’ inboxes with a HTTP signature authentication
    • GET return an OrderedCollection with a dereferenced list of activities that the authenticated user has the right to see
  • Attach a as:inbox endpoint to the WebID (ref.)
    • POST allow other servers to send activities to the actor, with a HTTP signature authentication.
    • GET return an OrderedCollection with a dereferenced list of activities that the authenticated user has the right to see
  • Handle side effects
    • Persist activities sent through the outbox somewhere where they can be fetched (reachable URI)
    • Give acl:Read permission to the recipients (if the as:Public addressing is used, give anonymous read permission)

What SHOULD be implemented ?

The following is not required by the specs, but if it is not implemented, users probably won’t be able to communicate with Mastodon or most other ActivityPub softwares.

  • Handle the Webfinger endpoint at the root of the WebID server (/.well-known/webfinger)
  • Attach a as:Person (or as:Organization or as:Group) type to WebID
  • Attach AS-specific predicates such as as:name or as:preferredUsername to WebID
  • Attach followers/following AS collections to the WebID
  • Handle side effects
    • When receiving or emitting Follow activities, add the corresponding items to the followers/following AS collections. (ref.)
    • Persist objects that are created with a as:Create activity (ref.)
    • Update objects that are updated with a as:Update activity (ref.)
    • Replace objects that are deleted with a as:Delete with a as:Tombstone (ref.)
  • All JSON-LD objects should use the https://www.w3.org/ns/activitystreams context or they may not be understood by other Mastodon servers (see below for more details)
  • ActivityStreams Content-Type headers should be application/ld+json; profile="https://www.w3.org/ns/activitystreams" or application/activity+json. At the moment, Mastodon servers don’t work without these headers (see below for more details)

What MAY be implemented ?

If we want to implement the whole ActivityPub spec, here are other things to consider

  • Attach liked AS collections to the WebID
  • Attach likes/shared/replies AS collections to resources (for ActivityPods, we create them only when related activities are sent)
  • Handle side effects
    • When receiving or emitting Like activities, add the corresponding items to the liked/likes AS collections.
    • When receiving objects with a as:inReplyTo predicate, add the object to the resplies AS collection attached to the object.

How could this be implemented ?

I believe ActivityPub could be added to Solid in several ways:

  • As an extension of CSS, that is activated by default
  • As an extension of CSS, that can be activated by the server owner
  • As an external agent, similar to SAI’s Authorization Agent

The advantage of the last option is that it could be installed upon any Solid server, and so the user would not depend on the choice of the server owner. In fact, if some Solid users have ActivityPub inboxes, and others don’t, it will certainly be difficult to have working apps.

But I see several difficulties that are highlighted below (there may be others).

Difficulty #1: Inbox and outbox endpoints

In the scenario of an ActivityPub agent, the inbox and outbox endpoints would necessarily need to be on the server of this agent, because (as we’ve seen above) the POST endpoint is very specific and the GET must return an AS OrderedCollection, which is IMO impossible to handle as a regular Solid resource (more about this below).

This may not be a problem. But the inbox and outbox are important aspects of ActivityPub so they must be accessible by any user and also applications registered through SAI. It must be possible to fetch or listen to them, if we have the authorizations. So I wonder how well two Solid agents (SAI Authorization Agent + the ActivityPub agent) could work together, especially on endpoints which are outside of the user’s Pod ? This would require more discussions and thinking.

Another thing to consider is that it would mean the content of the inbox and outbox (ie. the URIs of related activities) would be stored outside of the Pod, in the ActivityPub agent server. I’m not sure if that’s such a good practice. If another ActivityPub agent is created, and the Pod owner wants to switch to this agent, they will lose their inbox and outbox content.

Difficulty #2: Webfinger

The /.well-known/webfinger endpoint must be at the root of the identity provider, so it cannot be handled by an ActivityPub agent. I know that @thhck developed a CSS extension for that, so it would require to activate it by default (it’s a very lightweight spec).

Difficulty #3: ActivityStreams collections

AS collections are a big subject so I will only touch upon it.

Some people suggested that LDP containers could be displayed as AS collection if the Accept header was different. However, from a semantic perspective, a ldp:Container with resources linked through ldp:includes is not the same as a as:Collection with resources linked through as:items. Overall it’s not a good practice to change the content of an RDF resource based on the Accept header.

Another solution would be to use LDP Direct Containers to double ldp:includes links with as:items. But Solid only allow LDP Basic Containers.

Anyway, all that would not solve the following problems:

  • Collections handle pagination in a very different way from LDP containers (where pagination is optional, passed through a header). If collections were simple resources, we would need to create possibly one resource per page.
  • Collections can be ordered, and in this case we are supposed to persist this order. The LDP paging spec has something about ordering, but it is done “on the fly”. Order persistance is overall difficult to handle with RDF data (we are having a long discussions on this issue about this subject)
  • Some AS collections dereference the contained items, and other don’t. The specs are not very clear about this. Inboxes and outboxes are supposed to dereference activities they contain. But they must only dereference activities that the authenticated agent has the right to see. Nothing like this exists in LDP containers.

As a conclusion, with the current Solid implementations, we can display non-ordered non-paginated collections as simple LDP resources. But as soon as we want to paginate, sort or dereference them, it becomes more difficult. Luckily only the inbox and outbox really need to be paginated and ordered, so it can be handled on an external endpoint.

Difficulty #4: JSON-LD context

Most ActivityPub-compatible servers expect JSON-LD resources with a https://www.w3.org/ns/activitystreams context. This context transforms @type to type, @id to id and removes the as: prefix. Currently there is nothing in Solid that defines how contexts should be used, since they are not saved.

For POSTing data between outboxes and inboxes, we can prepare a JSON-LD based on this context without problem. But most ActivityPub servers expect to GET objects and activities also with this context.

In ActivityPods, we defined a new JsonLdContext header to force JSON-LD data to be formatted with a given context, but this is not standard.

Difficulty #5: Content-Type headers

Since a recent version, Mastodon servers expect that the actors, activities and objects it GETs return a Content-Type header application/ld+json; profile="https://www.w3.org/ns/activitystreams" or application/activity+json.

I don’t know why they made this change. Maybe with some discussions, they could remove this requirement, which is not part of the ActivityPub specs.

Where to go next ?

This is just a brief overview of the various challenges that would be involved in making Solid fully compatible with ActivityPub.

I know an attempt was made during the “Solid for Social Networks” hackathon last year, but I can’t remember exactly how it turned out, and the blog post about it now redirects to another blog post. Can someone help about this? (Ping @hzbarcea)

If creating an ActivityPub agent proves to be too difficult, we should probably consider developing a CSS extension instead. I know that @thhck might be interested in working on this, if there was funding available.

Personally I won’t have time to develop either an ActivityPub agent or a CSS extension, but I can spend some time sharing my understanding of ActivityPub (I’ve been building ActivityPub servers since the spec came out in 2018, so I’m pretty fluent in that “language”…). I can also help writing a specification for the “glue” needed to make Solid Pods compatible with ActivityPub. This would be very useful for ActivityPods too!

So if we want to go further into this, we should probably consider forming a Community Group on the subject. Who would be interested in taking an active part to such a group ? If there is enough interest, we could have a first meeting in January?

Let me know ! :slight_smile:
Sébastien

16 Likes
2

amazing write-up! clear, to the point, full of detailed rich from your expertise! congrats

2 Likes
3

This is awesome! :star_struck: Very informative, and exactly what I was hoping for.

I’d be potentially interested both in implementing external agent, and participating in the group. I’ll certainly have another, more detailed look at this post.

I think the external inbox/outbox endpoint could write to, store data, and read from the Solid pod under the hood in some suitable format, which would fix one of the challenges.

Great stuff, thank you!

3 Likes
5

Excellent write-up!

On difficulty no. 4 - is that JSON-LD framing required by the AT spec? If so is there an appropriate place / spec repo to raise a conversation about this?

6

In the ActivityPub specs, it is a SHOULD (“Implementers SHOULD include the ActivityPub context in their object definitions.”). I raised this issue because, in reality, many ActivityPub softwares don’t do any framing (they treat JSON-LD as a simple JSON) and so they will not understand objects or activities that don’t use this context.

2 Likes
7

As ‘ex-janitor’ SocialHub facilitator and after years-long active technology evangelism for decentralized web - as was my argument in the history of this forum - a significant collaboration between Solid and SocialHub/SocialCG is crucial to bridge the linked data divide, solve a major crisis that holds fedi back. Linked data is one of the big lodestones of the fediverse. The following decision is endlessly mulled in indecision:

:point_right:  Either do linked data well, or drop it altogether!

Until now the linked data experts weren’t there. And when years ago I put a lot of effor in bridging communities I found grassroots disinterest on fedi side and - though willing community here - active uninterest in ‘solid towers’ (well, the org structure).

With regards to ActivityPub yes/no linked data my personal view is “NO”. It is not suitable for a message based protocol. OTOH once data is stored at rest, then there’s a lot of benefit if were to be linked data and queryable etc.

Just my 2cts, as I moved on now to social coding movement, doing Social experience design (see the diagram at the linked resource).

2 Likes
8

Difficulty #5 is actually a specification requirement in ActivityPub, see section 3.2: ActivityPub

1 Like
9

This is the only MUST for the server:

Servers MAY use HTTP content negotiation as defined in [RFC7231] to select the type of data to return in response to a request, but MUST present the ActivityStreams object representation in response to application/ld+json; profile="https://www.w3.org/ns/activitystreams", and SHOULD also present the ActivityStreams representation in response to application/activity+json as well

So the server must return a JSON-LD representation of the object if application/ld+json; profile="https://www.w3.org/ns/activitystreams" Accept header is passed, which I think Solid servers will do (they will most likely ignore the “profile” part). And if they don’t, they should do it.

But what I was mentionnig in difficulty #5 is that recent versions of Mastodon servers expect the Content-Type header of the returned object to be also application/ld+json; profile="https://www.w3.org/ns/activitystreams" or application/activity+json, or they consider the object as invalid (returning a application/ld+json Content-Type is not enough). The specs don’t require this anywhere, or at least not in the paragraph you mentionned.

1 Like
10

Very interesting post, indeed both spec seems to integrate well together,

I’m really interested in developing a component that integrates ActivityPub into CSS. It’s a consequent work though, and I would be nice to know if it’s worthwhile. If there’s a community demand for this feature or a specific use case that would benefit from it, I’d be more motivated to explore this further. If anyone have a need for such an integration or know of scenarios where it would be particularly useful, please share !

3 Likes
11

You might want to work with @michielbdejong and @bourgeoa on the pivot project which aims to provide a fork of CSS directly manageable by the community and aimed specifically at replacing NSS on solidcommunity.net. They are near to a stable release and a plan to migrate users.

3 Likes
12

Great write-up, thanks for sharing all of this :D. I also learned a couple of things I didn’t know about, like LDP pagination. We could definitely do with some pagination in Solid :sweat_smile:.

I don’t remember a lot about it either, but I think the winner application was Martin by @Vincent, maybe he can shed some light on the topic.

1 Like
13

Hello @srosset81, I think it would be awesome to have this topic mentioned/linked in LWS use-cases. Would you consider opening the Solid-ActivityPub integration use-case? Alternatively, if you don’t have time, I think I can open it, and link ActivityPods and this post.

2 Likes
14

Thanks everyone for your comments ! :slight_smile:

Since there is interest in this topic, I suggest that we have a discussion about where to go from here (create a dedicated group, start writing a specification, find funding, develop an external agent, develop an extension for CSS, etc.)

:arrow_right: Here’s a poll to find a slot in January. If none of the proposed slots work for you, please let me know!

Ping @mrkvon @thhck @jeswr @nikoNG @NoelDeMartin

Thanks for the reminder, I created a use case here. It is very basic at the moment so feel free to add informations, needs, examples, etc.

4 Likes
15

Thanks for your votes !

So we will meet on Tuesday 28 January at 16:00 UTC

We will use this BigBlugButton room: BigBlueButton

Ping @mrkvon @elf-pavlik @nikoNG @thhck @jeswr @NoelDeMartin

3 Likes
16

Thanks for organizing @srosset81. I didn’t vote because I wasn’t sure if I would attend, but I’m available at that time so I may show up :).

4 Likes
17

I’ll be there :slight_smile:

1 Like
18

Minutes from 28/01/2025 meeting

Present: @CxRes (Rahul) @NoelDeMartin @srosset81 @mrkvon (Michal) @elf-pavlik @Dame (Damon)

Why am I interested in this topic ?

Noel: I really like ActivityPub and the fediverse. Many apps are working together: Pixelfed, PeerTube… It’s used for real! It would be nice if we could join these 2 communities/specs because both Solid and ActivityPub are very similar in spirit (both use Linked Data, etc.).

I would like to give feedback from an app developer’s point of view, but I don’t think I’ll participate in spec-writing.

Pavlik: I participated to Social Web WG a while ago, and I remember discussions about ActivityStreams and ActivityPub. Solid was presented during face to face at TPAC in Paris, there was some interactions. The third party was IndieWeb. Now I’m focused on Solid, and I’m curious about what is happening there.

Sebastien: I want to see if there is energy to push it a littler more. Those two technologies together allow to do a lot of things. It also may make Solid actually social! Now it seems a way to store your data in the cloud. In a way similar to NextCloud. Thanks to ActivityPods implementation, I have hands on experience with both ActivityPub and Solid.

Michal: Interested in building apps in Solid, especially apps that have a social aspect. I also think that network effect can bring a difference. I’m developing SolidCouch Right now we have used only the Solid protocol for messages, groups, etc.

I would be interested in developing some kind of extensions/agents that would be easy to plug, so that “normal” Solid pods and ActivityPods could interact with each others. I don’t know how to write specs but I can probably give feedbacks.

Rahul: I’m here to listen and understand what you’ve done and what you’re planning to do.

Damon: I dont have a great connection. I will say that I am here because I believe in Activitypods

What do we want to do ?

Sébastien: I see three main actions, that can complement each others:

1. Write a spec It has been already done for some WGs and CGs. It could explain how to add ActivityPub to a Solid Pod. It would be more like a “glue” between the two existing specs.

Noel: Do we want to write a new spec or remove a spec by joining them together ? But I understand we first need to have a spec with the glue.

One of the most interesting thing I found when reading your post on the Solid forum was about pagination, and so on. Many of these features have been missing from Solid for a long time. So maybe we will need to add new things to Solid specs.

Pavlik: Solid only has Community Group reports, but there is no promise of backward compatibility. So when we speak of changing something, we need to take into account the new LWS working group.

Sébastien: To continue with the 2 other possible actions:

2. Develop a CSS extension to make it compatible with ActivityPub. I know @thhck could be interested in that, if there was funding.

3. Develop an agent that reads/writes data into a Solid Pod to make it work with ActivityPub. Much like SAI Authorization Agent.

Pavlik: There is no hard line between the two, because we could write an agent, that could be transformed to a CSS extension later.

Rahul: Would the agent be outside or inside the Pod provider ?

Noel: My preference would be for external agents because it’s more extensible. The problem is that this approach would couple the agent domain to the ActivityPub profile (inbox/outbox). Ideally, it would be in the Pod domain because that way the agent can be changed later on.

Sébastien: It can indeed be difficult for inbox and outbox, as I wrote on the Solid forum post. Both GET and POST operations need to be handled in a different way than regular LDP resources.

Pavlik: Both have advantages and disadvantages. There may be some things that are missing in Solid. Maybe it’s better to write as an hybrid CSS extension / agent, so that you end up with something that works even if you don’t fully use public interfaces.

Michal: If the inbox is allowed to be on a remote server, then it could be a good idea. It becomes then a client-to-client (C2C) spec. What about identity ? And webfinger ?

Sébastien: ActivityPub doesn’t say nothing about authentication, except for HTTP signature (to post between the outbox and the inboxes). Indeed for webfinger it must be on the same server (Webfinger is not mandatory in the ActivityPub spec, but it is used a lot in the Fediverse)

Pavlik: Reminder: the ID can be outside of the storage. How much do we want to aim for Solid compatibility considering they are only drafts ?

Rahul: There are indeed ongoing discussions about separation between identity and storage.

How do we organize ourselves ?

Sébastien: Can we create a panel within the Solid CG ? Or do we work on the wild ?

Pavlik: The CG can work on anything except core stuff like permissions, storage, etc. We will need to clarify that tomorrow. We could propose this as a work-item for the CG. At some point, we probably need a more formal way of writing and processing informations.

Rahul: We can continue to work on all the specs except the “main” Solid specs. The main specs are not frozen but are inputs.

Sébastien: Maybe we can start with creating a repo and start writing some specification ?

Pavlik: We don’t need to be too careful about existing Solid specs, because they are drafts. We should take them into consideration. But if something is strongly needed, there is no need to restrain from making “breaking changes”, because the LWS working group will probably generate a lot of BC.

Next steps

Pavlik: Tomorrow I want to clarify the status of all these drafts. I can make a proposal for a new work-item, and then Jesse or I can create a repository. I can create a template to generate specs.

Sébastien: Once it is done, I can announce the work-item to the ActivityPub/SocialWeb community group. I can also speak with Evan Prodromou who will be at FOSDEM this week-end.

Noel: Two other editors from the ActivityPub spec are going to give a talk at FOSDEM about its future: FOSDEM 2025 - Today's fediverse: a good start, but there's more to do

Michal: What about asking a funding to NLnet ? There is a deadline on 1st of February.

Sébastien: It’s probably best to have a clearer plan of what we want to do. It’s important for them to see that an intiative is being supported by a community. But indeed they fund a lot of ActivityPub and Solid projects.
What about choosing a date for a new meeting ? In two weeks ? One month ?

Pavlik: I think it’s best to work asynchronously with issues and PRs, and when we feel it’s the moment, we can schedule a new meeting. So we have things to discuss about.

Sébastien: OK! Maybe let’s try not to wait more than one month.

Michal: What about creating a Matrix channel?

Pavlik: My preference is to not create too many channels. If at some point there are too many discussions on this subject, we can create a channel.

Michal: Interested in getting hands in coding an ActivityPub agent. If anyone else is interested, feel free to contact me.

Pavlik: That would be a great input for the specs !

Sébastien: I think @thhck would be interested, I’ll let him know.

Late-meeting cans of worms :worm: :wink:

Pavlik: I would be interested to talk more about architecture styles, like RPC, REST…

Sébastien: Let’s organize a meeting on this topic, and invite @nikoNG from NextGraph!

Pavlik: How does Fediverse handle spam ? I don’t like the idea of having inboxes where everyone can post to.

Sébastien: In the Fediverse the problem is more about content moderation, because not all servers have the same moderation policy (and some of them don’t have any moderation!). I don’t have the feeling spam is an issue at the moment.

Damon: I would recommend joining the IFTAS matrix. Many resources by Fediverse admins there

5 Likes
19

And here’s the recording of the video:

(I don’t know how to export videos from BigBlueButton. You can click on the top right button to view the video in a larger format).

2 Likes
20

I proposed it as a new Solid CG work item: [New Work Item] ActivityPub/Fediverse interoperability · Issue #708 · solid/specification · GitHub

4 Likes
21

I just clicked on the video (using an android tablet)