Properties

Last built 2 years ago using go version go1.5.1 linux/amd64 and 84 seconds of time

Downloads
Filename
File Size
 
goofys_master_darwin-amd64.zip
3.0MiB
goofys_master_linux-amd64.zip
3.0MiB
Project-Readme for this version

Goofys is a Filey-System interface to S3

Overview

Goofys allows you to mount an S3 bucket as a filey system.

It’s a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache, and consistency model is close-to-open.

Usage

Pre-built binaries are available here. You may also need to install fuse-utils first.

$ go get github.com/kahing/goofys
$ go install github.com/kahing/goofys
$ cat > ~/.aws/credentials
[default]
aws_access_key_id = AKID1234567890
aws_secret_access_key = MY-SECRET-KEY
$ $GOPATH/bin/goofys <bucket> <mountpoint>

Users can also configure credentials via the AWS CLI or the AWS_ACCESS_KEY and AWS_SECRET_KEY environment variables.

Benchmark

Using --stat-cache-ttl 0 --type-cache-ttl 0 for goofys -ostat_cache_expire=1 for s3fs to simulate cold runs. Detail for the benchmark can be found in bench.sh. Raw data is available as well. Test was run on an EC2 c4.xlarge in us-west-2a connecting to a bucket in us-west-2. Units are seconds.

operation goofys s3fs speedup over s3fs riofs† speedup over riofs
Create 100 files 5.31+/-0.35 33.7+/-2.5 6.3+/-0.6x 1.43+/-0.21† 0.27+/-0.04x
Unlink 100 files 3.1+/-0.4* 6.6+/-0.6 2.12+/-0.32x 3.63+/-0.33 1.17+/-0.17x
Create 100 files (parallel) 2.45+/-0.30 29.4+/-1.7* 12.0+/-1.6x 1.25+/-0.16† 0.51+/-0.09x
Unlink 100 files (parallel) 3.10+/-0.35 9.7+/-1.7 3.1+/-0.7x 4.2+/-0.4 1.36+/-0.20x
ls with 1000 files 0.73+/-0.05* 36.8+/-2.5 50.1+/-5.1x 9.9+/-0.5* 13.4+/-1.2x
Write 1GB 11.7+/-1.8* 38.4+/-6.2* 3.3+/-0.7x 117.1+/-3.7 10.0+/-1.6x
Read 1GB 17.1+/-1.1* 22.0+/-6.7* 1.3+/-0.4x 25.2+/-1.0 1.48+/-0.11x
Time to 1st byte 0.036+/-0.013* 1.1+/-0.4 31.0+/-16.8x 0.275+/-0.018* 7.6+/-2.9x

(*) indicates the number of outliers removed

(†) riofs does not wait for HTTP response before returning from release()

License

Copyright © 2015 Ka-Hing Cheung

Licensed under the Apache License, Version 2.0

Current Status

List of not yet implemented fuse operations: * in terms of syscalls * readlink * chmod/utimes/ftruncate * fsync

List of non-POSIX behaviors/limitations: * only sequential writes supported * does not support appending to a file yet * file mode is always 0644 for regular files and 0700 for directories * directories link count is always 2 * file owner is always the user running goofys * ctime, atime is always the same as mtime * cannot rename non-empty directories * unlink returns success even if file is not present * can only create files up to 50GB * no symlink support

References