10 Seconds of Fame

This blog is actually linked to by the Wikipedia article on the HAMMER File System, and it wasn't me. That's actually very cool, thank you :)

Error-Driven Development

1. Compile
2. Fix errors
3. Goto 1

I like it.

daniel@daniel:~/linux-2.6.29.1$ make ARCH=um 2>&1 | grep 'error: ' | sed -e 's/.*error: //g' | sort | uniq
'EFTYPE' undeclared (first use in this function)
'FREAD' undeclared (first use in this function)
'FSCRED' undeclared (first use in this function)
'FWRITE' undeclared (first use in this function)
'LK_EXCLUSIVE' undeclared (first use in this function)
'LK_RETRY' undeclared (first use in this function)
...

hammerread.c + FUSE = Read HAMMER Filesystem on Linux

I hacked together a FUSE module using hammerread.c, so you can now mount a HAMMER device in read-only mode on Linux. I haven't done any extensive testing, yet, so things are probably still very flaky. You can get the FUSE module from here: fusehammer-0.1.tar.gz and test with hammerdisk.raw.bz2.

Here's a walkthrough:

daniel@daniel-desktop:~$ wget http://hammerfs.googlepages.com/fusehammer-0.1.tar.gz
daniel@daniel-desktop:~$ tar zxf fusehammer-0.1.tar.gz
daniel@daniel-desktop:~$ cd fusehammer
daniel@daniel-desktop:~/fusehammer$ make
daniel@daniel-desktop:~/fusehammer$ ./fusehammer
HAMMER filesystem for FUSE (readonly)
Usage: ./fusehammer <dev> <mount_point> [<FUSE library options>]
daniel@daniel-desktop:~/fusehammer$ mkdir mnt
daniel@daniel-desktop:~/fusehammer$ ./fusehammer ../hammerdisk.raw mnt
daniel@daniel-desktop:~/fusehammer$ ls -l mnt
total 0
drwxr-xr-x 0 root root 0 1970-01-01 01:00 a
-rw-r--r-- 0 root root 12 1970-01-01 01:00 test
daniel@daniel-desktop:~/fusehammer$ cat mnt/test
Hello
World
daniel@daniel-desktop:~/fusehammer$ fusermount -u mnt

Note that the file attributes are wrong (mtime, ctime ..). I need to investigate that.

POSIX File System Test Suite

Just found this: POSIX File System Test Suite

"The list of system calls tested is: chmod, chown, link, mkdir, mkfifo, open, rename, rmdir, symlink, truncate, unlink. There are currently 3601 regression tests"

Sounds cool, maybe something to look into at a later date.

In an unrelated matter, I read Amit Singh's (MacFUSE fame) blog on AncientFS, where he mentions:

"In future, it [the UnixFS layer] may also evolve into a "LinuxFS" layer that could make it even easier and faster to systematically make Linux kernel-based file systems work on other platforms with very few code changes."

Now that sounds interesting.

#define student 04302402740280438773

#define student 04302402740280438773

in blag included from the intertubes.

hammerread.c ported to Linux

I ported hammerread.c to Linux. I was surprised how little I had to change in the source code. The Makefile contains the following switches:

# -D_BSD_SOURCE for DT_FIFO, DT_CHR .. in <dirent.h>
# -D_FILE_OFFSET_BITS=64 for large ino_t in <dirent.h>
# -D_XOPEN_SOURCE=500 for pread64 in <unistd.h>

You can test the application yourself. Grab a disk image from here: hammerdisk.raw.bz2. The contents of the image are:

$ find . -type f -ls
6400704     0 -rw-r--r--    1 root   wheel  12 Mar 23 20:04 ./test
12801152    0 -rw-r--r--    1 root   wheel   4 Apr 11 00:56 ./a/b/c/bar

Now download hammerread-0.1.tar.gz and use it as follows:

daniel@daniel-desktop:~$ wget http://hammerfs.googlepages.com/hammerread-0.1.tar.gz
daniel@daniel-desktop:~$ tar zxf hammerread-0.1.tar.gz
daniel@daniel-desktop:~$ cd hammerread
daniel@daniel-desktop:~/hammerread$ make
daniel@daniel-desktop:~/hammerread$ ./hammerread ../hammerdisk.raw
signature: valid
name: HAMMER
daniel@daniel-desktop:~/hammerread$ ./hammerread ../hammerdisk.raw /
signature: valid
name: HAMMER
/ 0/0 40755 0
test 8 10061aac0
a 4 1007a1650
daniel@daniel-desktop:~/hammerread$ ./hammerread ../hammerdisk.raw /test
signature: valid
name: HAMMER
/test 0/0 100644 12
Hello
World

STOP!

Hammer time!

My work on the hammer file system is completed. You may download the module here: hammerfs-0.1.tar.gz. Here's a short walkthrough:

root@daniel-desktop:~# wget http://hammerfs.googlepages.com/hammerfs-0.1.tar.gz
root@daniel-desktop:~# tar zxf hammerfs-0.1.tar.gz
root@daniel-desktop:~# cd hammerfs
root@daniel-desktop:~/hammerfs# make
root@daniel-desktop:~/hammerfs# insmod hammer.ko
root@daniel-desktop:~/hammerfs# mount -t hammerfs nodev /mnt
root@daniel-desktop:~/hammerfs# cd /mnt
root@daniel-desktop:/mnt# ls -l
total 0
-rw-r--r-- 1 root root 434 2009-04-10 00:26 hammertime
root@daniel-desktop:/mnt# touch hammertime
touch: setting times of „hammertime“: Operation not permitted # can't touch this
root@daniel-desktop:/mnt# cat hammertime

                             
            .d:.            
       ...,;l0kc.           
       .l;oddO0xOd:.        
        .'dxOKOd;..         
         ;0kdo:.            
       .xNNXKKl             
       dNNNNNNX;            
       dNNNNNNNx            
       'XNKo0NNk            
       :XK, .ONK.           
       ;Xl   .kK.           
       .Oc    '0.           
        ..    .d;           

The special file hammertime will display a new frame each time it is cated. To watch the entire animation, type watch -n0.1 cat hammertime

fat free

fat_free, sb_breadahead ..

kernel identifiers are funny. ha-ha.

Sample filesystems

Seems, there is quite a few of them:

• rkfs
  
• lwnfs
• samplefs (src)
• ramfs

Debugging Linux kernels with VMWare

Still trying to find something, since UML didn't work for me. This here works with VMWare Fusion, too:
Debugging the virtual world.: Debugging Linux kernels with Workstation 6.0

I had to edit the .vmx file and add debugStub.listen.guest32=1 to it, then the log said:

Apr 03 14:33:57.701: vmx| VMware Fusion is listening for debug connection on port 8832.
Apr 03 14:33:57.701: vmx| target remote localhost:8832

yay. I can't load the debug symbols of the kernel, though.

(gdb) file vmlinuz-2.6.27-11-generic
"/Users/daniel/vmlinuz-2.6.27-11-generic": not in executable format: File format not recognized

Update: Seems like I need to get the uncompressed kernel (vmlinux) for the debug symbols, which involves getting the appropriate .debs from ddebs.ubuntu.com. I didn't follow that further, as I got UML working which I find more comfortable anyway.

The Linux Kernel Module Programming Guide

So, you want to write a kernel module. You know C, you've written a few normal programs to run as processes, and now you want to get to where the real action is, to where a single wild pointer can wipe out your file system and a core dump means a reboot.

Yes, I do.

Volume header and endianness

The volume header is defined as follows (vfs/hammer/hammer_disk.h):

/*
* Volume header types
*/
#define HAMMER_FSBUF_VOLUME 0xC8414D4DC5523031ULL /* HAMMER01 */
#define HAMMER_FSBUF_VOLUME_REV 0x313052C54D4D41C8ULL /* (reverse endian) */

and

struct hammer_volume_ondisk {
u_int64_t vol_signature;/* Signature */
..
char vol_name[64]; /* Name of volume */
..
}

now

-bash-4.0# dd if=/dev/ad1s0e of=dump bs=1B count=1
1+0 records in
1+0 records out
512 bytes transferred in 0.000320 secs (1600211 bytes/sec)
-bash-4.0# hexdump -C dump | head
00000000 31 30 52 c5 4d 4d 41 c8 00 00 04 00 00 00 00 00 |10R.MMA.........|
00000010 00 00 14 00 00 00 00 00 00 00 24 00 00 00 00 00 |..........$.....|
00000020 00 00 40 1f 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............|
00000030 ab 3a fa 2a dc 17 de 11 92 2e 01 0c 29 6d 7b af |.:.*........)m{.|
00000040 ac 63 dc 61 38 6e dc 11 85 13 01 30 1b b8 a9 f5 |.c.a8n.....0....|
00000050 48 41 4d 4d 45 52 00 00 00 00 00 00 00 00 00 00 |HAMMER..........|
00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000090 00 00 00 00 01 00 00 00 01 00 00 00 bb d0 4b bd |..............K.|
000000a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

The volume signature (10R.MMA.) is backwards and not even correct (partly due to endianness), while the volume name appears correctly.

Maybe something to investigate.

Kernel Hacking with UML

Kernel Hacking with UML

Fails inside Linux running on VMWare Fusion on my Mac, but works on a physical machine. Seems to be the way to go. I chose Debian 4.0 from the pre-made filesystems.

B+Tree

Ubiquitous B-Tree, hammer_btree.h, hammer_btree.c

Project Proposal

For the project proposal, I found a very good template/guideline.

Here's the proposal I handed in: proposal-lorch.pdf.

hammerread.c

hammerread.c is a self-contained library for the bootloader to read the hammer filesystem. It can be compiled as a command line tool using -DTESTING.

First get the dragonfly sources with cvsup, then:

-bash-4.0# cd /usr/src/lib/libstand
-bash-4.0# cc -DTESTING -std=c99 hammerread.c -o hammerread
-bash-4.0# ./hammerread
usage: hammerread
-bash-4.0# ./hammerread /dev/ad1s0e
signature: valid
name: HAMMER
-bash-4.0# ./hammerread /dev/ad1s0e /test
signature: valid
name: HAMMER
/test 0/0 100644 12
Hello
World

Discovering HAMMER

Time to play. I installed a hammer filesystem.

-bash-4.0# echo Hello > test
-bash-4.0# echo World >> test
-bash-4.0# hammer history test
test 000000010061aac0 clean {
00000001007a1520 23-Mar-2009 20:04:11
00000001007a1580 23-Mar-2009 20:04:43
}
-bash-4.0# cat test@@0x00000001007a1520
Hello
-bash-4.0# cat test@@0x00000001007a1580
Hello
World
-bash-4.0# cat test
Hello
World

Snapshot creation:

-bash-4.0# hammer snapshot /mnt /mnt/snap
/mnt/snap
-bash-4.0# ls -l snap
lrwxr-xr-x 1 root wheel 25 Mar 23 20:07 snap -> /mnt/@@0x00000001007a15c0
-bash-4.0# ls snap/
test

Getting the Code

development(7): git clone -o crater git://crater.dragonflybsd.org/dragonfly.git dragonfly.git

Got the code. There is gitweb, too.

Hello

I am going to port the HAMMER File System from DragonflyBSD to Linux as part of a semester project and thought it might be useful to keep a record of what is being done.