Compression of scientific data with MAFISC

MAFISC is a newly developed compression algorithm that aims to compress multidimensional scientific data well. Currently, it uses lzma as its compressing backend. It is currently provided as an HDF5 filter, which allows it to work transparently.

As always, usage of this software is completely at your own risk, including possible loss of data. Check the source code for the full disclaimer.

Since the introduction of the filter plugin interface into the HDF5 library, using MAFISC has become quite simple. The steps still required are:

1. Make sure that you have HDF5 version 1.8.15 or later installed.
2. Download the MAFISC source code, and unpack it.
3. Change directory into the unpacked mafisc directory and run `make` to build it. There are two environment variables that you may want to use at this point:
   • PREFIX: `make install PREFIX=…` will install the plugin within `$PREFIX/lib/plugin`, defaults to `/usr/local/hdf5`
   • HDF5_PATH: tells `make` where to find the HDF5 includes and libraries, defaults to `/usr`
4. Run `sudo make install` to install the MAFISC plugin below '/usr/local/hdf5' where the HDF5 library automatically looks for it.

That's it, after these simple steps the HDF5 library will automatically load the MAFISC plugin whenever it encounters a MAFISC compressed file.

To compress an existing HDF5 file with MAFISC, you can use the `h5repack` tool distributed with HDF5:

#32002 is the ID assigned to the MAFISC filter.
h5repack --filter=UD=32002,1,0 uncompressedFile.h5 compressedFile.h5

Instead of installing the MAFISC plugin to the default location, you may specify another location using `make install PREFIX=some/other/path`. This, however, will require you to tell HDF5 where to look for filter plugins by running `export HDF5_PLUGIN_PATH=some/other/path/lib/plugin`.

A paper on Mafisc has been published at the International Supercomputing Conference 2012, it can be downloaded at http://dx.doi.org/10.1007/s00450-012-0222-4