Friday, November 22, 2013

Some more notes on Parallel FS

OrangeFS looks kinda like what I want, but it has the usual Linux-only aspect.

http://www.orangefs.org/

The lack of real portability bothers me. I do most of my Java Dev work on OSX, 2nd-most on Windows, with Linux a distant third, mostly for concerns about portability. (I do note that that is really the reverse-order list of "able to patch the OS" behavior).

So of course in terms of what I can do for myself, it has to completely be portable across OSes, and not require any sort of "kernel patch" because there's no way I'm going to do that--just not interested.

Orange FS does have similar aspects, but at least in the doc reading I've done, they don't quite have a conceptual theme/analogy.

------

Related, but a little weird: I'm actually thinking about writing some/all of this in Lisp (ok, that'd have some portability issues, but there's no reason a Librarian couldn't be written in Lisp). I haven't done any work in Lisp in years, so that'd be kinda cool...and there are some free Lisp versions that are pretty good these days (I recently re-discovered CLisp for Windows, it's a Cygwin package). Already I can see an issue: I need a build that includes multi-threading, and the basic CLisp does not.

------

Desiderata for my Grid FS:

OS-agnostic
Can make use of any/all machines on the local net (scales)
Doesn't require mount-points for every machine (doesn't scale)
All shared space is available to any machine (scales)
Fault-tolerant about machines coming/going; system contains much self-discovery (scales)
Local apps don't really have to know much about the actual system, they're just going to interact with local files (scales)
Has some amount of redundancy
Doesn't require a bunch of special services

No comments: