Zfs
Basic Commands
ZPool All Properties:
zpool get all Z5000LUKS_zfs
Dataset All Properties:
zfs get all Z5000LUKS_zfs zfs get all Z5000LUKS_zfs/fs
List all commands executed:
zpool history Z5000LUKS_zfs
List pools and status:
zpool status zpool status -v
List datasets:
zfs list
List all snapshots:
zfs list -t snapshot
Show space used by snapshots (https://blogs.oracle.com/observatory/understanding-the-space-used-by-zfs):
zfs list -t all -r zfs list -t all -r -o space
- USEDSNAP = used by snapshot
- USEDDS = used by dataset
- REFER = size of all files when snapshot was taken
- USED = ?
Check the compression achieved
zfs get compressratio Z5000LUKS_zfs/fs
Cloning a ZFS filesystem from a snapshot
bash-3.00# zfs clone demovol/testing@snap21 demovol/clone22 bash-3.00# zfs list NAME USED AVAIL REFER MOUNTPOINT demovol 1.00G 900G 39.9K /demovol demovol/clone22 0 900G 32.6K /demovol/clone22 demovol/testing 32.6K 10.0G 32.6K /demovol/testing demovol/testing@snap21 0 – 32.6K -
Performance IO Monitoring the ZFS storage pool
bash-3.00# zpool iostat 1
capacity operations bandwidth
pool used avail read write read write
———- —– —– —– —– —– —–
demovol 4.95M 900G 0 0 0 35
demovol 4.95M 900G 0 0 0 0
demovol 4.95M 900G 0 0 0 0
demovol 4.95M 900G 0 0 0 0
Using a USB ZFS drive
##Connect the disk. zpool import mybackup ##Do the backup. zpool export mybackup ##Unplug the disk.
Step by step to create a new zfs disk
Find/view drive:
fdisk -l or $ mount
Create pool:
# zpool create -f YOURPOOLNAME /dev/sdX zpool create -f Z5000LUKS_zfs2 /dev/mapper/Z5000LUKS
Create filesystem
# zfs create YOURPOOLNAME/YOURNAME zfs create Z5000LUKS_zfs2/fs
Set features
zfs set compression=on Z5000LUKS_zfs2 zfs set mountpoint=/media/Z5000LUKS_zfs2 Z5000LUKS_zfs2 # zfs set compression=off YOURPOOLNAME/YOURNAME # zfs set copies=2 YOURPOOLNAME/YOURNAME
Mountpoints
Add new
zfs set mountpoint=/media/Z5000LUKS_zfs2 Z5000LUKS_zfs2/fs
Remove mountpoint
zfs set mountpoint=none Z5000LUKS_zfs2/fs
Auto snapshots
sanoid
sanoid also provides Syncoid
https://www.svennd.be/zfs-snapshots-of-proxmox-using-sanoid/
apt-get install libconfig-inifiles-perl git cd /opt git clone https://github.com/jimsalterjrs/sanoid ln /opt/sanoid/sanoid /usr/sbin/ ln /opt/sanoid/syncoid /usr/sbin/ mkdir -p /etc/sanoid cp /opt/sanoid/sanoid.conf /etc/sanoid/sanoid.conf cp /opt/sanoid/sanoid.defaults.conf /etc/sanoid/sanoid.defaults.conf
/etc/sanoid/sanoid.conf
####################
# sanoid.conf file #
####################
[tank/subvol-104-disk-1]
use_template = production
#############################
# templates below this line #
#############################
[template_production]
# store hourly snapshots 36h
hourly = 36
# store 30 days of daily snaps
daily = 30
# store back 6 months of monthly
monthly = 6
# store back 3 yearly (remove manually if to large)
yearly = 3
# create new snapshots
autosnap = yes
# clean old snapshot
autoprune = yes
sudo crontab -e
*/5 * * * * /usr/sbin/sanoid --cron
or
0 * * * * /usr/sbin/sanoid --cron
Alternative: zfs-auto-snapshot
For Ubuntu 16.04:
https://romaco.ca/blog/2016/09/04/install-zfs-auto-snapshot-on-ubuntu-16-04-lts/ You can download the Ubuntu 16.04 package here, and install it by running:
wget https://furneaux.ca/w4ca8cqmwx234io/zfs-auto-snapshot-trustyport.deb dpkg -i zfs-auto-snapshot-trustyport.deb
To avoid too frequently
rm /etc/cron.d/zfs-auto-snapshot rm /etc/cron.hourly/zfs-auto-snapshot
Or configure frequency for the individual pools https://askubuntu.com/questions/322541/activate-zfs-snapshots
sudo zfs set com.sun:auto-snapshot=true tank/backup sudo zfs set com.sun:auto-snapshot:monthly=false tank/backup sudo zfs set com.sun:auto-snapshot:weekly=false tank/backup sudo zfs set com.sun:auto-snapshot:daily=true tank/backup sudo zfs set com.sun:auto-snapshot:hourly=false tank/backup sudo zfs set com.sun:auto-snapshot:frequent=false tank/backup
or disable
sudo zfs set com.sun:auto-snapshot=false tank/tmp
Other Alternatives:
http://wiki.complete.org/ZFSAutoSnapshots
- simplesnap - a simple and powerful cron-based network snapshot transfer tool by John Goerzen.
- Has a simple and secure wrapper so unfettered root access isn't needed.
- Designed to work with zfSnap or other snapshot-making tools on this page.
- Supports multiple backup sources, targets, and target sets (such as offsite media rotation)
- zfs-auto-snapshot - a simple cron-based service. Different names for different snapshots, with a simple count of how many to retain. Can exclude based on property.
- snapxfer, a wrapper around:
- zfSnap - a flexible snapshot-creation system. Can embed an expiry age in snapshot name, or override from command line.
- zxfer - a flexible snapshot-transfer system. Can use rsync or zfs snapshots. Used together with another snapshotting system. For remote transfers, requires root ssh authorized_keys or sudo access.
- zfs-snapshot-rotation-and-backup, a Python library and example scripts that use it. Expects ssh root access to remote.
- zsnap -- looks flexible and powerful, last commit 5 years ago. Requires Ruby.
- Suggested on ZFS on Linux mailing list:
- zetaback - feature-packed. Has a server-side agent to run on systems being backed up. Buggy config parser without sufficient validation on client.
- zrep - for zfs based replication (possibly remote) of filesystems. Can also be used just for backups. License prohibits public redistribution. Does failover too. Doesn't work on nested datasets. Designed to run once per minute. Same snapshot retention on both sides.
- zfs-auto-snapshot - as above, for snapshotting
Sync to another machine using syncoid (from sanoid)
syncoid website: https://github.com/jimsalterjrs/sanoid/
info: http://unicolet.blogspot.de/2016/05/from-0-to-zfs-replication-in-5m-with.html
comparative with rsync: https://arstechnica.com/information-technology/2015/12/rsync-net-zfs-replication-to-the-cloud-is-finally-here-and-its-fast/
sudo apt-get install mbuffer lzop pv git
installation explain in section above
Syncoid connecting without root
if it's run without root, syncoid uses sudo automatically
visudo
# Syncoid commands backup ALL=NOPASSWD: /sbin/zfs get * backup ALL=NOPASSWD: /sbin/zfs snapshot * backup ALL=NOPASSWD: /sbin/zfs send * # backup ALL=NOPASSWD: /sbin/zfs list * # We only want to destroy snapshots backup ALL=NOPASSWD: /sbin/zfs destroy *@syncoid_backup*
https://github.com/jimsalterjrs/sanoid/issues/82
Alternative: znapzend
https://github.com/jimsalterjrs/sanoid/issues/102
Native Encryption
Not yet supported with standard repository
Info: https://blog.heckel.xyz/2017/01/08/zfs-encryption-openzfs-zfs-on-linux/#Compile-and-install
Installation
The encryption is not in the ubuntu repository version of zfs, it needs to be compiled from source:
Dependencies:
sudo apt-get install zlib1g-dev uuid-dev libattr1-dev libblkid-dev libselinux-dev libudev-dev libssl-dev sudo apt-get install build-essential autoconf libtool gawk alien fakeroot linux-headers-$(uname -r) sudo apt-get install parted lsscsi ksh
mkdir zfssource cd zfssource git clone https://github.com/zfsonlinux/spl git clone https://github.com/zfsonlinux/zfs
Method 1: Custom packages
Info: https://github.com/zfsonlinux/zfs/wiki/Custom-Packages
sudo apt-get install build-essential autoconf libtool gawk alien fakeroot gdebi linux-headers-$(uname -r) sudo apt-get install zlib1g-dev uuid-dev libattr1-dev libblkid-dev libselinux-dev libudev-dev libssl-dev parted lsscsi wget ksh gdebi
cd spl git checkout master ./autogen.sh ./configure make deb rm *devel*.deb for file in *.deb; do sudo gdebi -q --non-interactive $file; done
cd ../zfs ./autogen.sh ./configure make deb rm *devel*.deb for file in *.deb; do sudo gdebi -q --non-interactive $file; done
Method 2 (not used)
Info oficial web: https://github.com/zfsonlinux/zfs/wiki/Building-ZFS
cd spl git checkout master ./autogen.sh ./configure --prefix=/usr # << Don't forget the --prefix make -s -j$(nproc) make install cd ..
cd zfs ./autogen.sh git checkout master ./configure --prefix=/usr # << Don't forget the --prefix make -s -j$(nproc) make install cd ..
To load the new modules:
# Remove existing modules (repeat until successful!) modprobe -r splat modprobe -r zavl modprobe -r zcommon modprobe -r zunicode modprobe -r znvpair modprobe -r icp modprobe -r spl modprobe -r zfs # Insert newly compiled modules (all of these must succeed!) cd /lib/modules/$(uname -r)/extra insmod avl/zavl.ko insmod unicode/zunicode.ko insmod spl/spl.ko insmod nvpair/znvpair.ko insmod zcommon/zcommon.ko insmod icp/icp.ko insmod zfs/zfs.ko
Alternatively:
sudo ./scripts/zfs.sh
zfs.sh: The freshly built kernel modules can be loaded using zfs.sh. This script can latter be used to unload the kernel modules with the -u option.
Enabling feature in pool
zpool set feature@encryption=enabled testpool
Creating a encrypted zfs
zfs create -o encryption=on -o keyformat=passphrase -o keylocation=prompt testpool/enc1 zfs create -o encryption=on -o keyformat=passphrase -o keylocation=prompt -o pbkdf2iters=500000 testpool/enc1
Reading encryption properties
zfs get -p encryption,keystatus,keysource,pbkdf2iters Z4000LUKSB_zfs
new:
zfs get -p encryption,keystatus,keyformat,keylocation,pbkdf2iters Z4000LUKSB_zfs
Disk failure
Info: https://docs.joyent.com/private-cloud/troubleshooting/disk-replacement
Mount snapshots as readonly folders automatically
As of today, ZFS on Linux (ZoL) now also supports read-only access to snapshots via a invisible (not hidden, invisible) directory in the root of the dataset.
# ls -a /tank/test ./ ../ boot.tar text.tar text.tar.2 # cd /tank/test/.zfs/ # ls -a ./ ../ shares/ snapshot/ In the snapshot directory you'll find read-only copies of all your snapshots.
You can make .zfs visible via zfs set snapdir=visible tank/test.
Source: https://pthree.org/2012/12/19/zfs-administration-part-xii-snapshots-and-clones/