Thursday, December 24, 2009

jFUSE 0.2

jFUSE is a framework for developing cross-platform FUSE-based filesystems in Java. I quietly released the first version, 0.1, as a response to some messages in the MacFUSE Google Group (see here), but now as it has reached version 0.2 and matured somewhat, it's time to make a public announcement.

jFUSE allows you to develop FUSE 2.6 filesystems in Java (including the MacFUSE 2.0 extensions for Mac OS X). It also provides the programmer with access to relevant system constants and tools to make development a little easier.
The library consists of a native part (written in C++) and a Java part. Beginning with jFUSE 0.2, the native library has been tested on Mac OS X 10.4-10.6 and the latest versions of GNU/Linux (Debian,Ubuntu), OpenSolaris, FreeBSD and NetBSD (using librefuse).

Download:
  • jFUSE 0.2 (jar file with library and example file systems, Java sources and compiled native library for Mac OS X)
  • libjfuse 0.2 (JNI native library sources)
    Sources now also available on GitHub.
Two example filesystems are included. The first one is the mandatory HelloFS, the simplest filesystem you can find, and the other is a more complete in-memory filesystem called TestFS. They can be found in the org.catacombae.jfuse.examples package.

I have created two additional filesystems for jFUSE as separate projects (download links coming soon):
  • hfs-jfuse - A read only HFS/HFS+/HFSX filesystem driver.
  • rarfs-jfuse - A filesystem for mounting RAR files, displaying their contents and allowing direct access to uncompressed entries.
These two filesystems were trivially created by using libraries that I had written earier for HFSExplorer and RARMounter.
In fact, the inspiration for jFUSE as a more general Java-FUSE layer came when working on the older 'rarfs' code, that also consisted of Java code bound to FUSE callbacks, but in a less reusable way.

For developers that quickly want to start creating jFUSE filesystems, all that's needed is to extend the class org.catacombae.jfuse.FUSEFileSystemAdapter and override those methods that you want to implement (see org.catacombae.jfuse.FUSE26Operations for a description of the operations that can be overridden).

Developers that want to take advantage of the MacFUSE extended callbacks introduced in MacFUSE 2.0 can override org.catacombae.jfuse.MacFUSEFileSystemAdapter, which includes the FUSE 2.6 operations + the extended MacFUSE callbacks.
(Using the MacFUSE extended callbacks doesn't affect compatibility with non-MacFUSE platforms. They will simply not be invoked for those platforms.)

The Java API is not guaranteed to stay stable between releases, but it's also not likely to undergo any major changes.

9 comments:

Anonymous said...

Great thing, thanks for your efforts. On OS X 10.5 everything works fine, on 10.6 testfs boils out with an Input/Output error. Is it supposed to work on Snow Leopard?

Erik said...

Anonymous, February 3, 2010 2:49 AM:

On Snow Leopard, you must make sure to link against libfuse_ino64 when compiling instead of libfuse since they changed the default stat structure. This commonly leads to I/O errors.

Alternatively, you can compile against the Mac OS X 10.5 SDK. I may be able to provide an easy fix for this in the jFUSE source code... I'll have a look at it.

Anonymous said...

OK, letting point libfuse.2.dylib -> libfuse_ino64.dylib in /usr/local/lib and recompiling libjfuse makes it work. Thanks for the hint.

Ian said...

I found a minor bug in Timespec.toMillis().

I believe it needs to be 1000L to ensure the initial multiplication is done as a long, otherwise precision is lost.

return this.sec*1000l + this.nsec/1000000;

Great library, thank you for making it. It has been working nicely for me on Snow Leopard.

Erik said...

Ian, June 20, 2010 11:32 PM:

That seems reasonable, thanks. I commited this change to my private git repository.

I might publish this repository on github later if there is interest.

Erik said...

Ian:

Here's my newly created github repository for jFUSE:
http://github.com/unsound/jfuse

wafflebatter said...

Great library, but I have a question about multi-threading. Is the JNI wrapper multi-threaded? In my testing, on Mac, it seems like it isn't.

I have very similar loopback examples running using the canonical MacFUSE loopback.c filesystem, LoopbackFS Objective-C filesystem, and a loopback I created in Java using your library.

In my code, I block on read long enough to download a remote file to my loopback dir, at which point I read the bytes from it and return.

This works fine in all the examples, but I can only access one file at a time using your library, whereas with the other libraries I can access other files simultaneously while blocking on the read of the first file.

I've tried messing with the -s and -oasync_read flags, to no avail. I've also rebuilt the library.

Any advise would be greatly appreciated. Thanks. -- Mike

Erik said...

wafflebatter:

The thing is that when mounting, fuse_loop is invoked and not fuse_loop_mt (see org_catacombae_jfuse_FUSE_26.cpp line 512) which is the multi-threaded event loop.

So the file system will be inherently single-threaded. The reason for this is to simplify and the fact that I have not been able to do any testing with multiple threads.

If you change this line to test multi-threaded operation, then you probably also need to to pass -pthread/-lpthread in CFLAGS when building the C++ support library.

However I think that more changes are needed... for instance I think one needs to 'attach' the foreign thread to a Java thread upon each call into the Java portion of the library, which is not done in the FUSE callbacks at this point. It's not that difficult to fix, but I have no idea about whether other things also need to be changed.

wafflebatter said...

Thanks for the quick reply. I've already added some code to attach to the current Java thread, and I built with -pthread/-lpthread, but I missed the call to _mt. I'll add that now and let you know how it goes.

If it works, I will of course be happy to submit the modifications back (though at present they amount to about 5 lines). Thanks again. -- Mike