Solid file manager

bourgeoa · March 11, 2019, 6:35pm

The error still there but it can be published with it.

To reproduce the error :

login and enter your pod
logout and validate logout
click enter your pod

Nice work

A_A · March 11, 2019, 6:51pm

I’ve updated the UI now, so the “Enter your pod” button is disabled when logged out. So the user always has to be logged in with some account before trying to access the pod (it doesn’t check whether it is the correct one though).
(You will probably need to refresh without cache for the path to apply)

Do you think that’s a good fix for the error, @bourgeoa ?

ztein · April 22, 2019, 4:05pm

(moving from other forum thread)

Solid Plume - simple blog app

@ztein Could you ch
ange the copyFile method in src/Api/Api.js to following and try it again?
export async function copyFile(originPath, originName, destinationPath, destinationName) {
    const fileResponse = await fetchFile(originPath, originName);
    const contentType = fileResponse.headers.get('Content-Type');
    const content = (contentType === 'application/json') ?
        await fileResponse.text()
        : await fileResponse.blob();

    // Note: Not passing contentType as parameter as this would lead to errors when creating .json files
    return createItem(destinationPath, destinationName, content);
}
I can’t reproduce it with the solid.community version (I think it’s NSS 4.x.x, so it’s probably an incompatibility with NSS v5.x) and don’t have a v5 pod for testing it so it maybe fixes it and maybe does nothing.

I 've changed it now - first time I actually build a npm module, but think I got it working and deployed ok

Anyway, there is a difference but not much - those folders and files are still converted to turtle format, but now without the .ttl extension. The content is pretty much the same, example:

cat css 
@prefix : <#>.
@prefix css: <>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix terms: <http://purl.org/dc/terms/>.
@prefix XML: <http://www.w3.org/2001/XMLSchema#>.
@prefix fonts: <fonts/>.
@prefix st: <http://www.w3.org/ns/posix/stat#>.
@prefix cs: <http://www.w3.org/ns/iana/media-types/text/css#>.

    css:
        a ldp:BasicContainer, ldp:Container;
        terms:modified "2019-04-22T15:47:33Z"^^XML:dateTime;
        ldp:contains
            <blog.css>, <font-awesome.min.css>, fonts:, <grids-responsive-min.css>,
            <pure-min.css>, <simplemde.min.css>;
        st:mtime 1555948053.643;
        st:size 4096.
    <blog.css>
        a cs:Resource, ldp:Resource;
        terms:modified "2019-04-22T15:47:33Z"^^XML:dateTime;
        st:mtime 1555948053.451;
        st:size 14221.
    <font-awesome.min.css>
        a cs:Resource, ldp:Resource;
        terms:modified "2019-04-22T15:47:33Z"^^XML:dateTime;
        st:mtime 1555948053.459;
        st:size 27340.
    fonts:
        a ldp:BasicContainer, ldp:Container, ldp:Resource;
        terms:modified "2019-04-22T15:47:34Z"^^XML:dateTime;
        st:mtime 1555948054.415;
        st:size 4096.
    <grids-responsive-min.css>
        a cs:Resource, ldp:Resource;
        terms:modified "2019-04-22T15:47:33Z"^^XML:dateTime;
        st:mtime 1555948053.459;
        st:size 9798.
    <pure-min.css>
        a cs:Resource, ldp:Resource;
        terms:modified "2019-04-22T15:47:33Z"^^XML:dateTime;
        st:mtime 1555948053.459;
        st:size 17286.
    <simplemde.min.css>
        a cs:Resource, ldp:Resource;
        terms:modified "2019-04-22T15:47:33Z"^^XML:dateTime;
        st:mtime 1555948053.643;
        st:size 10710.

Here’s what I did, for reference:

Changed the code in Api.js
npm install
npm start (to check if it worked)
npm run build
rsync build/ to servers app folder
Opened it in browser, refreshed, and tested to upload, extract and rename folder

A_A · April 22, 2019, 4:30pm

Thanks for trying it out!
In the pod, folders are stored as turtle files like the one you’ve posted (the css folder). The contents of it show, that it contains blog.css, font-awesome.min.css, a fonts folder and so on and all of those seem good to me (it is identical to my css folder, except for different modification times). So the bug isn’t happening while storing the folder, but maybe while displaying it. Does it work to open the css folder directly (not via the solid-filemanager)? And could you describe the error you get a bit more in detail?

A_A · April 22, 2019, 4:36pm

And after installing it should be enough to run npm start. It automatically refreshes on code changes and (if you’ve given permission for localhost:xxx) should be able to login and make changes to your pod.

ztein · April 22, 2019, 7:30pm

Yeah it’s the same in the servers filesystem - those folders have been converted to turtle format. I’m checking the (real) filesystem a lot, to learn how the solid server use it (and perhaps figure out other ways to implement it in the future). I also checked for every step in the upload and rename process, and the conversion happens when I rename the main folder. After it’s unzipped, it’s still a normal folder. But after renaming, it’s suddenly in turtle format.
It’s also odd that this only happens with some folders and some files - so far it seems to happen constantly on the following objects:

Folders named css,img,js
Files ending on .json

,js files however, are not converted.

I haven’t seen it happen on any other folders or files yet, so it’s a mystery why it’s only happening wiht those mentioned.

I also think the server supports both types of folders - turtle folders and normal folders. But maybe they have different content type or something? I don’t know how the filemanager works, but maybe there is a way to specify which type of folder you want to upload?

Well I don’t know, but something is odd. I don’t get any specific error message btw, but the initial problem was when I uploaded the solid-plume app into my public pod to check it out, and it didn’t work - because it couldn’t read those turtle folders (with js and css etc).

I figured out a better way to host shared apps for the solid domain pods btw - rather than putting the app files in the solid pod space, I’m now bypassing a path (/apps for now) in apache, so they’re not going via solid server anymore, but are served directly by apache - on the same domain, but from a “normal” filesystem

And before I forget: the solid-filemanager works really well in general, so thanks for the effort and keep up the good work!

ztein · April 22, 2019, 7:35pm

Thanks for the tip, I think I will look more into npm development to understand it a bit better
But I can’t login to my online pod server from local host it seems - it just hangs. Well perhaps it’s time to install a local solid server as well on the workstation, now that you got me into npm building

A_A · April 22, 2019, 8:11pm

(Assuming you’ve unzipped via the solid-filemanager)
This really confuses me, because the extract method and rename method both use the same function for creating a folder (createFolder from Api.js). So the requests should be pretty much the same (except for name, path, …) and hence the server should respond in a similar manner. If you have time, could you check if the network requests for creating a folder via the extracting method differs to the requests from the rename method? (i.e. look into the browser devtools, go to Network. The createFolder method first checks if the folder already exists, so it’s likely the request 2 below a 404 request (GET-404, OPTIONS-204, POST-201). Content-Type should be ‘dir’ and slug is the name of the folder).

Edit: I’ve just realized that content-type should be set to “text/turtle” for creating folders (as mentioned in the api spec). I will change that soon, you could test if this resolves your issue (just replace ‘dir’ with ‘text/turtle’ in Api.js → createFolder)

What exactly do you mean, with “it’s in turtle format”? Does the solid-filemanager still recognizes it as a folder (i.e. it has a folder icon) and can you open it to view the files it contains (similar to the public folder)?

And I’m glad that you like the solid-filemanager, feel free to give more feedback or leave suggestions if you want

ztein · April 22, 2019, 9:47pm

I mean in the real filesystem - they are converted from normal folders to “turtle” files. I don’t know what to call them but like in the css file I quoted above - this was originally the folder named css.

But good you asked - I just tested with filemanager, and it can’t open those folders anymore. And that happens just after extraction - while the folders are still real folders in the filesystem. So looks like filemanager can not read them for some reason?

See if you can open these links, first in solid and then in filemanager - it’s folders that was zip extracted:

https://stein.solid.nett.org/public/solid-plume-0.10/css/
https://stein.solid.nett.org/public/solid-plume-0.10/app/

Both of them works ok for me in solid, but the css folder doesn’t work in filemanager.

I have to check those network requests and the suggested patch later - it’s getting late and back to work tomorrow. But got both the filemanager and solid installed on my workstation now, so ready for more testing later on

A_A · April 22, 2019, 10:11pm

Thanks for providing these examples, I’ve found out why it’s not displaying it as a folder now:
In the code it accidentally only checks if the first found type equals to ldp#BasicContainer (aka it’s a folder), instead of iterating through all files. To fix it, you can go to the isFolder function from folderUtils.js and remove the else return false statement inside the loop (and the console.error after it). I’ve added it to the ToDo list for the next deployment.
Somehow it added a media-type to the css folder to the first position of the types, and therefore the loop didn’t find the folder type.

# for app
n0:
    a ldp:BasicContainer, ldp:Container, ldp:Resource;
# ...
# for css
css:
    a cs:Resource, ldp:BasicContainer, ldp:Container, ldp:Resource;
# ...

And I feel like it’s ok if they are converted to turtle files in your folder, because that’s how they are retrieved via the api. But I’m not really familiar with pod servers -_-

UPDATE: I’ve made the changes I’ve suggested to you and a few others and pushed it to the master branch. Could you git pull and test it with this version?

ztein · April 23, 2019, 6:35am

Hey nice you found it, tested the fix now and it works great - those folders works as expected again!

I don’t mind if they are converted to turtle files as long as they still work as folders, but with this bug the actual files inside the folders got lost as well, which is not so good

Well thanks for the quick fix, one mystery solved!

Edit: I didn’t see the update so only tested a local fix. But will check out the new git version when I’m back from work!

happybeing · April 23, 2019, 9:37am

I’m not sure this is the case. They are retrieved as RDF because that is the LDP interface response (an LDP client expects RDF, and the default format for an LDP response is text/Turtle).

The server can implement the backend any way it likes, and NSS represents LDP containers as native filesystem directories, not RDF files. So I think you are not doing what you think you are when you write a container as a file!

I’m not sure though, I’m a bit surprised it works at all because I’d have thought the server should either not accept your request, or realise you are creating a container and make the required directory. So it might also be an NSS 5 server bug. @megoth?

There might also be a link to the problem I found trying to write a json file in NSS 4, because some of the symptoms @ztein reported look similar.

A_A · April 23, 2019, 10:26am

From the client side, I am pretty sure that when creating a folder we should send a POST request for creating a turtle file, as stated in the API spec.

But I can’t speak for the server side, it could be represented completely differently there.

I’ve found an issue which is probably related to this. However, it seems to work fine when using POST with explicitly setting Content-Type: undefined and body: jsonBlob.

happybeing · April 23, 2019, 10:57am

Yes, I think you are correct in essence about how to create a container but I think there is a problem if the result is a file. I create the container for posts in Plume like this:

github.com

happybeing/solid-plume/blob/gh-pages/app/app.js#L268


      
          }
          Solid.web.head(url).then(
              function(container) {
                  if (container.exists) {
                      config.postsURL = url;
                      fetchPosts(url);
                  } else if (container.xhr.status < 500) {
                      // try to create container for posts in user's POD
                      const slug = url.slice(url.lastIndexOf('/') + 1)
                      const parent = url.substring(0,url.length-slug.length)
                      const isContainer = true
                      Solid.web.post(parent, slug, null, true).then(
                          function(res) {
                              if (res.url && res.url.length > 0) {
                                  const arr = url.split('/');
                                  config.postsURL = arr[0] + '//' + arr[2] + res.url;
                                  // return saveConfig() disabled to avoid need for configuring an access control file
                              }
                            }).then( function() {
                              // add dummy post
                              var acme = {

You could run Plume (delete or rename /public/posts first) and compare the HTTP requests with that of solid-filemanager to see why the difference. One might be Plume’s use of the slug.

Thanks, I’ll make a note of that. Pretty sure I didn’t try not specifying the content type!

A_A · April 23, 2019, 11:25am

I forgot to mention it, but I also use the slug when creating folders (and files). As far as I’ve seen our requests have the same format. Something that I am not sure of yet is, if it is needed to also send data with the POST request as they do in the api spec example (<> <http://purl.org/dc/terms/title> "Basic container" .). Currently I send an empty body when creating a folder.
If you want to take a look at the full Api implementation of the solid-filemanager, here is the current version. If you think it’s of interest, I could also create a npm package for it, but it’s pretty similar to the solid-file-client so I don’t really think it would be useful.

And setting Content-Type to undefined seems like a hack, so if you know a more “official” way to do this, please let me know
You can take a look at it in action here (e.g. create and copy a test.json file): https://solid-filemanager.solid.community/public/nightly/

happybeing · April 23, 2019, 12:01pm

I suggest you try ‘text/turtle’ as this may be causing the server to create a file rather than a directory. This is specified in the spec link you gave me, and is also how Plume does it:

github.com

happybeing/solid-plume/blob/gh-pages/app/solid.js#L310


      
                  f.nowOrWhenFetched(docURI,undefined,function(ok, body, xhr) {
                      if (!ok) {
                          reject({status: xhr.status, xhr: xhr});
                      } else {
                          resolve(g);
                      }
                  });
              });
          
              return promise;
          };
          
          // create new resource
          // resolve(metaObj) | reject
          var post = function(url, slug, data, isContainer) {
              var resType = (isContainer)?LDP('BasicContainer').uri:LDP('Resource').uri;
              var init = {
                          method: 'POST',
                          headers : {
                             'Content-Type': 'text/turtle',
                             'Link': '<'+resType+'>; rel="type"'

Thanks for the reminder about the Solid docs, I tend to go to the LDP Primer which is much harder to understand!

A_A · April 23, 2019, 12:03pm

Would this also work for non-turtle files like html, json, etc.?

happybeing · April 23, 2019, 12:13pm

I don’t think so. I used ‘text/plain’ to write to ‘config.txt’, so I think you should use an appropriate content type. Unfortunately though, it didn’t work using ‘application/json’ trying to write to ‘config.json’. Other types worked I think (eg ‘text/html’), but I didn’t get it to work with ‘.json’ files.

I could easily be wrong about this though!

A_A · April 23, 2019, 12:37pm

Thanks for your input, I will take a closer look at it

A_A · April 23, 2019, 4:06pm

Ok, so the behavior I noticed is quite strange:
Using slug: filename.ext without setting the Content-Type works for the solid.community version (4.x afaik) and (except for filename.json!) also for 5.0.0.
Using slug: filename; Content-Type: application/json should have worked and was fixed in this PR about one month ago and hence only works in 5.x.

So I currently don’t see a possibility to support both versions. I’ve documented this behavior in this issue.

Topic		Replies	Views
Solid FileManager app not working	2	216	March 16, 2023
Announce : much updated Solid-File-Client	40	2494	January 9, 2020
Local Solid Pod App Testing	4	1746	November 3, 2019
Announce : updated Solid IDE	0	495	January 28, 2020
REQUEST FOR TESTERS (Just a few minutes of your time, all you need is a POD) App Testing	8	1259	March 11, 2019

Solid file manager

Related Topics