I want a way to have my home directory on my server automatically sync'd with my home directory on my laptop (and probably my work desktop). The filesystem needs to appear to be local (from a convenience and performance point of view), needs to handle disconnected operation seemlessly and deal with editing collisions in a sane way. It needs to run under DebianLinux and AppleOsx (Windows would be nice but not required). -- Adam.
- Starfish Distributed Filesystem
- Starfish is a highly-available, fully decentralized, clustered storage file system. It provides a distributed POSIX-compliant storage device that can be mounted like any other drive under Linux or Mac OS X. The resulting fault-tolerant storage network can store files and directories, like a normal file system - but unlike a normal file system, it can handle multiple catastrophic disk and machine failures.
http://wiki.digitalbazaar.com/en/Starfish_Distributed_Filesystem
- LUFS
- LUFS is enabling you to mount into your file hierarchy a remote computer's file system, which is accessible by various means (ftp, ssh, etc.). Then, the access to the remote files will be completely network transparent. In other words, you'll be able to read/modify remote files as if they were local, watch movies/listen to MP3s from FTP/SSH/Gnutella servers without copying them locally. Sheer magic. Now skip to the next section.
- Monotone
- Monotone is an interesting looking distributed versioning system that supports disconnected operation and uses NNTP or HTTP for transport.
- SFS
- SFS is a secure, global network file system with completely decentralized control. SFS lets you access your files from anywhere and share them with anyone, anywhere. Anyone can set up an SFS server, and any user can access any server from any client. SFS lets you share files across administrative realms without involving administrators or certification authorities.
http://www.fs.net/sfs/@amsterdam.lcs.mit.edu,bkfce6jdbmdbzfbct36qgvmpfwzs8exu/pub/
- SHFS/SSHFS
- SHFS is a simple and easy to use Linux kernel (2.4) module which allows you to mount remote filesystems using plain shell (ssh/rsh) connection. It supports some nice features like number of different caches for access speedup, target system optimisations, etc.
- Unison
Basically a bidirectional rsync over SSH. It's ghetto but has the redeeming quality of simplicity. Full copies on all machines for caching/performance. Command line to update/sync (example configuration).
Right now I'm thinking that Unison is the best option.
Adam.
"OpenAfs was the solution I chose because I have the experience with it from college. For performance, AFS was built with an intelligent client-side cache, but did not support network disconnects nicely. But there are other alternatives out there. Coda appears to be a research fork from an earlier version of AFS. Coda supports disconnected operations. But, the consensus on the Usenet (when I looked into filesystems a while ago) was that Coda was still too 'experimental.'
InterMezzo looks like it was started with the lessons learned from Coda, but (again from Usenet) people have said that it is still too unstable and it crashes their servers. The last 'news' on their site is dated almost a year ago, so I don't even know if it's being developed or not"
So if you were to recommend a distributed filesystem for Linux machines, would you choose one of the three filesystems listed here, or something else entirely?
Source: http://ask.slashdot.org/article.pl?sid=03/05/13/2049210
More comments:
I believe that Peter Braam (one of the authors of InterMezzo) has been working on a new file system named Lustre that builds on top of it's ideas. It's out in beta There is a detailed document here. Braam gave a nice talk on it at LinuxWorld this last January. I haven't tried it yet.
I've tested coda:
Coda for me indeed seems too experimental. It was not easy to setup and it looks like it requires alot of resources.. it took very long (several hours) for me to copy about 30 MB's.. maybe I didn't set it up in the right way. But then it is still alot of effort to make it run. And I think maintainability is an issue with Coda.. the system is quite sophisticated and I think if you want to seriously run Coda you require alot of knowledge about this sophisticated system. I've tried SFS just now.. and the installation and configuration was almost zero, it works almost out of the box (on Debian).
Information on Google's distributed filesystem, GFS (GoogleFS).
http://developers.slashdot.org/developers/03/09/29/2227224.shtml
http://www.cs.rochester.edu/sosp2003/papers/p125-ghemawat.pdf
Another SlashDot article, "Building a Better $HOME"