I was wondering if there is a test suite which you can point at a Solid identity provider and another at a Solid pod provider and run a series of tests against their APIs to see what’s current functional / absent / different. i.e. you give it your account credentials and it uses a headless browser to run a localhost app and perform a start_login (session.login()), finish_login (handleIncomingRedirect()), followed by a series of API calls. I don’t know how you’d automate the signin / authenticate app steps so perhaps you run the first version of this proposed test suite in your browser and manually perform these steps for it.
I ask as I was trying to use some simple @inrupt/solid-client code against a different pod provider and it was erroring where as pod.inrupt.com was passing. Would be great to see a table of different pod providers and how their services pass / fail different features.
There’s two initiatives that aim to verify interoperability and adherence to the specifications. There’s a test-suite run by independent volunteers, and Inrupt is working on their Solid Conformance Test Suite, which should be open source but which I can’t find right now (possibly because it’s not ready for use by others yet, but I’m not sure - @Kevin might know). People are also chatting about them on Gitter’s solid/test-suite.
Following on from @Vincent 's comment, I can confirm the Conformance Test Harness (CTH) developed by Inrupt is open-source and available for use. I fact, we have just wrapped up the v1.0 release which now runs from a Docker image so there are no other dependencies to worry about. This makes it easy to use the CTH in a variety of environments, whether testing a Solid server implementation with existing tests, developing new tests, or incorporating it into CI workflows.