Migrating centrally-stored user data to Solid Pods

I’m surprised there isn’t more public information and interest on this topic. I would hope there are many companies with the same desire: migrating user data off our servers and onto personal Pods.

Absent of discussion or precedence of best practices, we have created our own solution. I’ll share it here in case any others want to build an app today with an eye towards Solid Pods in the future.

We already use PostgresSQL. So the simplest and most portable solution was storing a publicly-developed (such as foaf, etc…) schema as JSONB.

According to Designing JSON Documents, updating could get tricky:

Although storing large documents is practicable, keep in mind that any update acquires a row-level lock on the whole row. Consider limiting JSON documents to a manageable size in order to decrease lock contention among updating transactions.

This is a bit of a concern because we currently want to store history as a series of events (see my previous post). I’m also a bit blind to any practical concerns related to event-driven data stored on Pods, but it’s a risk we can take because our business logic can also tolerate a simplified view of this particular data.

Section 8.14.2 of Designing JSON Documents also had some other advice I found helpful:

The structure is typically unenforced (though enforcing some business rules declaratively is possible), but having a predictable structure makes it easier to write queries that usefully summarize a set of “documents” (datums) in a table.

As @RubenVerborgh notes in Let’s talk about pods, Pods are currently document-centric. So I also like that this approach with JSONB may grant us more flexibility when migrating from a data-centric database model to a model of documents that contain structured semantic data.

At least that’s my thinking. Even though we have already started implementation, I would gladly welcome more conversation with folks that have grappled with (or even thought deeply about) this problem.

2 Likes