I've written before about how JSFS could evolve into an operating system for the web. I've been thinking about this a lot since then and I've had a few revelations.
The first is that JSFS code is incredibly hard to work with, and especially hard to integrate with contemporary JavaScript. JSFS was written a long time (over a decade) ago, before Node.js modules were standarized, before Promises & Async/Await, and before many of the browser API's for host feature integration existed. Because of this, simple things like using a module to store blocks in S3 are difficult if not impossible.
Over the years I've taken a stab at "modernizing" it but this has almost exclusively by re-implmenting it in differenet languages. The goal was always to maintain compatibility with both the JSFS API and the on-disk structures (blocks and inodes) so that regardless of the implementation the server remains compatible with both existing clients and storage pools. I never finished any of these however for various reasons that all boil down to simply taking too long.
I've come to realize that if I'm going to move the project forward and make progress on the operating system for the web, the solution is to return to JSFS's roots and simplify it. In hindsight this seems obvious, especially thanks to the advancemetns in both JavaScript/Node.js and web browsers.
Here's the basic plan:
On the server-side, refactoring the callback-oriented structure to use Async/Await will go a long way to making the code easier to work with. Use of newer Node.js API's for both disk and network I/O will also improve maintainability as well as performance (and maybe even reliability).
Another significant area of simplification will be removing unused/obsolete features or moving them out of the "core" by delivering support for executable files (traditionally refered to as "JSFS-X"). Features like optional block encryption were never really used and probably shouldn't be add complexity to the core and provide little to no value. Other features like metadata extraction are better suited implemented as executables that can be used as optional components. These reductions and extractions should make it possible to substantially reduce the size of the core (or "kernel" if you prefer) and both improve runtime qualities as well as maintainability. This will also allow for the removal of third-party modules from the core, allowing Node.js to be the only prerequisite to running the server once again.
On the API side, parameters needed to support features removed from the core can themselves be removed to reduce the complexity of the API. In addition to this, adoption of contemporary web standards will make the API specification smaller while at the same time easier and more familiar for consumers (for example, perhaps using the standard Authorization header in place of x-access-token). It may be possible to remove other parameters as well through the use of conventions (for example, I would like to make the public/private distinction a matter of convention to make this distinction more intuitive and obvious to consumers).
In addition to removing things, this plan includes adding a few things as well. Support for executing uploaded files will be included as part of the core. This is necissary in order to preserve feature parity (as some existing features will be pushed out of the core as described above) but it is also essential to fulfilling the role as an operating system, and opens a vast new range of applications for the system. The other addition will be federation, another feature that has been part of the roadmap since almost the beginning and has been experimented with in several forms over the years. Execution and federation allow the core to be used for any web application I can imagine, and I feel are essential to fulfilling the objective of becoming an operating system for the web.
Since these changes both expand the applications for JSFS and introduce breaking changes to the API, I think it's a good time for a new name as well: OSW, the Operating System for the Web. This name codifies the intentions of the future of JSFS and is an unambigious statement of the departure from it being simply a file system.
I will be slowly updating the various projects I've started along the way to point at this canonical repository where the OSW work will take place. I'll also be following-up with new posts here as the details about each change take shape.