Zfs
Contingut
Basic Commands
ZPool All Properties:
zpool get all Z5000LUKS_zfs
Dataset All Properties:
zfs get all Z5000LUKS_zfs zfs get all Z5000LUKS_zfs/fs
List all commands executed:
zpool history Z5000LUKS_zfs
List pools and status:
zpool status zpool status -v
List datasets:
zfs list
List all snapshots:
zfs list -t snapshot zfs list -t snapshot -o space
List all snapshots from a disc
zfs list -r -t snapshot Z3000LU_zfs/oce1
Show space used by snapshots (https://blogs.oracle.com/observatory/understanding-the-space-used-by-zfs):
zfs list -t all -r zfs list -t all -r -o space
- USEDSNAP = used by snapshot
- USEDDS = used by dataset
- REFER = size of all files when snapshot was taken
- USED = ?
Check the compression achieved
zfs get compressratio Z5000LUKS_zfs/fs
Cloning a ZFS filesystem from a snapshot
bash-3.00# zfs clone demovol/testing@snap21 demovol/clone22 bash-3.00# zfs list NAME USED AVAIL REFER MOUNTPOINT demovol 1.00G 900G 39.9K /demovol demovol/clone22 0 900G 32.6K /demovol/clone22 demovol/testing 32.6K 10.0G 32.6K /demovol/testing demovol/testing@snap21 0 – 32.6K -
Performance IO Monitoring the ZFS storage pool
bash-3.00# zpool iostat 1
capacity operations bandwidth
pool used avail read write read write
———- —– —– —– —– —– —–
demovol 4.95M 900G 0 0 0 35
demovol 4.95M 900G 0 0 0 0
demovol 4.95M 900G 0 0 0 0
demovol 4.95M 900G 0 0 0 0
Using a USB ZFS drive
##Connect the disk. zpool import mybackup ##Do the backup. zpool export mybackup ##Unplug the disk.
Rename
to change the name of the file system only, you would type:
zfs rename tank/home/eric tank/home/eric_old
To relocate a file system, you would type:
zfs rename tank/home/mark tank/ws/mark
To rename a pool:
zpool export oldname
And then imported it with the correct name:
zpool import oldname newname
After the import completed, you can see the new name:
zpool status -v
Scrub
Start scrub in the background
zpool scrub poolname
Stop a scrub running
zpool scrub -s poolname
Step by step to create a new zfs disk
NEW: See specific example in lastpa
Find/view drive:
fdisk -l or $ mount
Encrypt the disk and backup the LUKS header:
Once encrypted, mount the encrypted LUKS:
cryptsetup --key-file /path/to/keyluks luksOpen /dev/disk/by-uuid/XXXXXXXXXXXXXXXXXXXXX Z5000LUKSA
Create pool:
# zpool create -f YOURPOOLNAME /dev/sdX zpool create -f Z5000LUKSAAA_zfs /dev/mapper/Z5000LUKSAAA
Create mount folder:
mkdir /media/Z5000LUKSAAA_zfs chmod 500 /media/Z5000LUKSAAA_zfs
Set features
zfs set compression=on Z5000LUKSAAA_zfs zfs set mountpoint=/media/Z5000LUKSAAA_zfs Z5000LUKSAAA_zfs zfs set atime=off K5000LUKSAAA_zfs # zfs set compression=off YOURPOOLNAME/YOURNAME # zfs set copies=2 YOURPOOLNAME/YOURNAME # to make visible the readonly snapshot folder .zfs/: # zfs set snapdir=visible YOURPOOLNAME/YOURNAME
Create filesystem
# zfs create YOURPOOLNAME/YOURNAME zfs create Z5000LUKSA_zfs/fs
If needed, add the drive to Sanoid for auto snapshots
/etc/sanoid/sanoid.conf
Mountpoints
Add new
zfs set mountpoint=/media/Z5000LUKS_zfs2 Z5000LUKS_zfs2/fs
Remove mountpoint
zfs set mountpoint=none Z5000LUKS_zfs2/fs
Install
If you need encryption you have to complile as explained in a section below. If don't need encription you can install it like this:
sudo apt install zfsutils-linux
Maybe also needed:
sudo apt install zfs-dkms sudo modprobe zfs
Auto snapshots
sanoid (syncoid)
sanoid also provides Syncoid
https://www.svennd.be/zfs-snapshots-of-proxmox-using-sanoid/
Install
apt-get install libcapture-tiny-perl
apt-get install libconfig-inifiles-perl git cd /opt git clone https://github.com/jimsalterjrs/sanoid ln -s /opt/sanoid/sanoid /usr/sbin/ ln -s /opt/sanoid/syncoid /usr/sbin/ mkdir -p /etc/sanoid cp /opt/sanoid/sanoid.conf /etc/sanoid/sanoid.conf cp /opt/sanoid/sanoid.defaults.conf /etc/sanoid/sanoid.defaults.conf
# Comentada linia 467 del /usr/sbin/syncoid (ara és linia 864 de v2.1.0):
pruneoldsyncsnaps($targethost,$targetfs,$newsyncsnap,$targetisroot,keys %{ $snaps{'target'}});
per:
if (!defined $args{'no-stream'}) { pruneoldsyncsnaps($targethost,$targetfs,$newsyncsnap,$targetisroot,keys %{ $snaps{'target'}}); } #afegit el if by marti
# per tal que no elimini els snapshots del desti, que podrien ser utils en el backup. En el backup hi ha menys disc usat ja que els snapshots intermitjos no es transfereixen (--no-stream)
/etc/sanoid/sanoid.conf
####################
# sanoid.conf file #
####################
[tank/subvol-104-disk-1]
use_template = production
#############################
# templates below this line #
#############################
[template_production]
# store hourly snapshots 36h
hourly = 36
# store 30 days of daily snaps
daily = 30
# store back 6 months of monthly
monthly = 6
# store back 3 yearly (remove manually if to large)
yearly = 3
# create new snapshots
autosnap = yes
# clean old snapshot
autoprune = yes
sudo crontab -e
*/5 * * * * /usr/sbin/sanoid --cron
or
0 * * * * /usr/sbin/sanoid --cron
Example syncoid
syncoid --identifier=Z8000LUKSA-Z8000LUKSB --keep-sync-snap --create-bookmark --debug --dumpsnaps --sshport=22 -c [email protected] --recursive Z8000LUKSA_zfs/ocell1 I8000LUKSA_zfs/ocell1
znapzend alternative to sanoid/syncoid
znapzend vs syncoid: https://github.com/jimsalterjrs/sanoid/issues/102
znapzend: https://github.com/oetiker/znapzend
manual: https://github.com/oetiker/znapzend/blob/master/doc/znapzendzetup.pod
Alternative: zfs-auto-snapshot
For Ubuntu 16.04:
https://romaco.ca/blog/2016/09/04/install-zfs-auto-snapshot-on-ubuntu-16-04-lts/ You can download the Ubuntu 16.04 package here, and install it by running:
wget https://furneaux.ca/w4ca8cqmwx234io/zfs-auto-snapshot-trustyport.deb dpkg -i zfs-auto-snapshot-trustyport.deb
To avoid too frequently
rm /etc/cron.d/zfs-auto-snapshot rm /etc/cron.hourly/zfs-auto-snapshot
Or configure frequency for the individual pools https://askubuntu.com/questions/322541/activate-zfs-snapshots
sudo zfs set com.sun:auto-snapshot=true tank/backup sudo zfs set com.sun:auto-snapshot:monthly=false tank/backup sudo zfs set com.sun:auto-snapshot:weekly=false tank/backup sudo zfs set com.sun:auto-snapshot:daily=true tank/backup sudo zfs set com.sun:auto-snapshot:hourly=false tank/backup sudo zfs set com.sun:auto-snapshot:frequent=false tank/backup
or disable
sudo zfs set com.sun:auto-snapshot=false tank/tmp
Other Alternatives:
http://wiki.complete.org/ZFSAutoSnapshots
- simplesnap - a simple and powerful cron-based network snapshot transfer tool by John Goerzen.
- Has a simple and secure wrapper so unfettered root access isn't needed.
- Designed to work with zfSnap or other snapshot-making tools on this page.
- Supports multiple backup sources, targets, and target sets (such as offsite media rotation)
- zfs-auto-snapshot - a simple cron-based service. Different names for different snapshots, with a simple count of how many to retain. Can exclude based on property.
- snapxfer, a wrapper around:
- zfSnap - a flexible snapshot-creation system. Can embed an expiry age in snapshot name, or override from command line.
- zxfer - a flexible snapshot-transfer system. Can use rsync or zfs snapshots. Used together with another snapshotting system. For remote transfers, requires root ssh authorized_keys or sudo access.
- zfs-snapshot-rotation-and-backup, a Python library and example scripts that use it. Expects ssh root access to remote.
- zsnap -- looks flexible and powerful, last commit 5 years ago. Requires Ruby.
- Suggested on ZFS on Linux mailing list:
- zetaback - feature-packed. Has a server-side agent to run on systems being backed up. Buggy config parser without sufficient validation on client.
- zrep - for zfs based replication (possibly remote) of filesystems. Can also be used just for backups. License prohibits public redistribution. Does failover too. Doesn't work on nested datasets. Designed to run once per minute. Same snapshot retention on both sides.
- zfs-auto-snapshot - as above, for snapshotting
Sync to another machine using syncoid (from sanoid)
syncoid website: https://github.com/jimsalterjrs/sanoid/
info: http://unicolet.blogspot.de/2016/05/from-0-to-zfs-replication-in-5m-with.html
comparative with rsync: https://arstechnica.com/information-technology/2015/12/rsync-net-zfs-replication-to-the-cloud-is-finally-here-and-its-fast/
sudo apt-get install mbuffer lzop pv git
installation explain in section above
Syncoid connecting without root
if it's run without root, syncoid uses sudo automatically
visudo
# Syncoid commands backup ALL=NOPASSWD: /sbin/zfs get * backup ALL=NOPASSWD: /sbin/zfs snapshot * backup ALL=NOPASSWD: /sbin/zfs send * # backup ALL=NOPASSWD: /sbin/zfs list * # We only want to destroy snapshots backup ALL=NOPASSWD: /sbin/zfs destroy *@syncoid_backup*
https://github.com/jimsalterjrs/sanoid/issues/82
Alternative: znapzend
https://github.com/jimsalterjrs/sanoid/issues/102
Native Encryption
Not yet supported with standard repository
Info: https://blog.heckel.xyz/2017/01/08/zfs-encryption-openzfs-zfs-on-linux/#Compile-and-install
Installation
The encryption is not in the ubuntu repository version of zfs, it needs to be compiled from source:
Dependencies:
sudo apt-get install zlib1g-dev uuid-dev libattr1-dev libblkid-dev libselinux-dev libudev-dev libssl-dev sudo apt-get install build-essential autoconf libtool gawk alien fakeroot linux-headers-$(uname -r) sudo apt-get install parted lsscsi ksh
mkdir zfssource cd zfssource git clone https://github.com/zfsonlinux/spl git clone https://github.com/zfsonlinux/zfs
Method 1: Custom packages
Info: https://github.com/zfsonlinux/zfs/wiki/Custom-Packages
sudo apt-get install build-essential autoconf libtool gawk alien fakeroot gdebi linux-headers-$(uname -r) sudo apt-get install zlib1g-dev uuid-dev libattr1-dev libblkid-dev libselinux-dev libudev-dev libssl-dev parted lsscsi wget ksh gdebi
cd spl git checkout master ./autogen.sh ./configure make deb rm *devel*.deb for file in *.deb; do sudo gdebi -q --non-interactive $file; done
cd ../zfs ./autogen.sh ./configure make deb rm *devel*.deb for file in *.deb; do sudo gdebi -q --non-interactive $file; done
Method 2 (not used)
Info oficial web: https://github.com/zfsonlinux/zfs/wiki/Building-ZFS
cd spl git checkout master ./autogen.sh ./configure --prefix=/usr # << Don't forget the --prefix make -s -j$(nproc) make install cd ..
cd zfs ./autogen.sh git checkout master ./configure --prefix=/usr # << Don't forget the --prefix make -s -j$(nproc) make install cd ..
To load the new modules:
# Remove existing modules (repeat until successful!) modprobe -r splat modprobe -r zavl modprobe -r zcommon modprobe -r zunicode modprobe -r znvpair modprobe -r icp modprobe -r spl modprobe -r zfs # Insert newly compiled modules (all of these must succeed!) cd /lib/modules/$(uname -r)/extra insmod avl/zavl.ko insmod unicode/zunicode.ko insmod spl/spl.ko insmod nvpair/znvpair.ko insmod zcommon/zcommon.ko insmod icp/icp.ko insmod zfs/zfs.ko
Alternatively:
sudo ./scripts/zfs.sh
zfs.sh: The freshly built kernel modules can be loaded using zfs.sh. This script can latter be used to unload the kernel modules with the -u option.
Enabling feature in pool
zpool set feature@encryption=enabled testpool
Creating a encrypted zfs
zfs create -o encryption=on -o keyformat=passphrase -o keylocation=prompt testpool/enc1 zfs create -o encryption=on -o keyformat=passphrase -o keylocation=prompt -o pbkdf2iters=500000 testpool/enc1
Reading encryption properties
zfs get -p encryption,keystatus,keysource,pbkdf2iters Z4000LUKSB_zfs
new:
zfs get -p encryption,keystatus,keyformat,keylocation,pbkdf2iters Z4000LUKSB_zfs
Read password from file
dd if=/dev/urandom of=/usb/key.dat bs=32 count=1 zfs change-key -o keylocation=file:///usb/key.dat -o keyformat=raw Pool
to return it back to password prompt:
zfs change-key -o keylocation=prompt -o keyformat=passphrase Pool
External key
use script in onenote.
or alternatively:
Es besteht ebenfalls die Möglichkeit ein entsprechendes Passwort oder (raw)key-File per GET Request von einem anderen Server zu holen.
Bsp.:
zfs create -o encryption=on -o keysource=raw,https://keys.example.com/mykey <pool/dataset>
info: https://techgoat.net/index.php?id=174
Autoload of key at boot
by default, even if keylocation=file:///root/zfs_key, zfs does not load the key automatically.
To load you need to do:
zfs unload-key -r banshee/encrypted zfs mount -l banshee/encrypted
or
zfs mount -l banshee/encrypted
So you need a service to load it. For example the one above Section External Key
Transfer encrypted volumes
Now it is possible. Before the problem was that Syncoid didn't support the raw send option (--sendoptions=w), which means that the received volume in the receiving system was unencrypted.
syncoid --no-rollback --no-stream --sshport=443 --sendoptions=w --compress=none --identifier=Z8000LUKSA_zfs-Z8000LUKSB_zfs --keep-sync-snap --create-bookmark --debug --dumpsnaps -c [email protected] --recursive ....
If you forget to use -w when zfs sending a dataset with its key loaded, the replication will work—but the target will be unencrypted!
Info: https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-to-openzfs-native-encryption/
Syncoid Info: https://kimono-koans.github.io/opinionated-guide/
Disk failure
Info: https://docs.joyent.com/private-cloud/troubleshooting/disk-replacement
Info: https://docs.oracle.com/cd/E19253-01/819-5461/gbbwl/index.html
Mount snapshots as readonly folders automatically
As of today, ZFS on Linux (ZoL) now also supports read-only access to snapshots via a invisible (not hidden, invisible) directory in the root of the dataset.
# ls -a /tank/test ./ ../ boot.tar text.tar text.tar.2 # cd /tank/test/.zfs/ # ls -a ./ ../ shares/ snapshot/ In the snapshot directory you'll find read-only copies of all your snapshots.
You can make .zfs visible via zfs set snapdir=visible tank/test.
Source: https://pthree.org/2012/12/19/zfs-administration-part-xii-snapshots-and-clones/
