0chaos.eu

/ /
Tags: linux freebsd luks zfs openzfs geli jails syslog git ssh http nginx
Published: 2023-11-16

Linux/FreeBSD guides

Collection of posts previously contained in different articles.

Table of Contents

0.1 – Encrypted Void Linux setup w/o encrypted boot

This is a guide for installing Void Linux on an EFI system with non encrypted boot and encrypted root. The guide is based on practical paranoid article and official Void Linux article.

Before you start

I wrote this guide to help myself installing Void Linux w/o encrypted boot. I haven’t checked it for grammar errors, so there might be some. I hope it will help you, and good luck. And I know that I use way to many pictures/terminal copy paste.

Disclaimer

I am not responsible for any actions you take! This guide was written 2/7/2020, so the info may be outdated. Please remember to use your own computer’s partition names.

Required items

You need to have a computer with safe boot disabled and a live copy of Void Linux, ether by live USB or CD.
I am using the Cinnamon version of Void Linux for installing it, this is so I don’t need to setup WIFI, nor use the TTY directly.

Setting up Void

After you have booted a live USB/CD, you open a terminal.

Partitioning the disk

The first step is setting the partitions up. I will be using a tool called cfdisk for this. You can start cfdisk by running cfdisk (<device path>) in the terminal, you may need to use sudo. Normally you don’t need to specify the device, but if the size seems wrong you should specify it. You can get a list of devices by writing lsblk in your terminal.
My output:

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0         7:0    0   801M  1 loop /run/initramfs/live
loop1         7:1    0 704.7M  1 loop 
loop2         7:2    0   2.3G  1 loop 
β”œβ”€live-rw   254:0    0   2.3G  0 dm   /
└─live-base 254:1    0   2.3G  1 dm   
loop3         7:3    0    32G  0 loop 
└─live-rw   254:0    0   2.3G  0 dm   /
sda           8:0    1  28.9G  0 disk 
β”œβ”€sda1        8:1    1     1M  0 part 
β”œβ”€sda2        8:2    1    50M  0 part 
└─sda3        8:3    1    28G  0 part /run/initramfs/isoscan
sdb           8:16   1  29.3G  0 disk 
β”œβ”€sdb1        8:17   1  29.3G  0 part 
└─sdb2        8:18   1   512K  0 part /run/media/anon/UEFI_NTFS
nvme0n1     259:0    0 953.9G  0 disk 
β”œβ”€nvme0n1p1 259:1    0   512M  0 part 
β”œβ”€nvme0n1p2 259:2    0   9.3G  0 part 
└─nvme0n1p3 259:3    0 944.1G  0 part 

I know that my SSD is 1TB, so the device name is nvme0n1, the path is /dev/, so my device path is /dev/nvme0n1.
After launching cfdisk, your interface should look a bit like this:

                               Disk: /dev/nvme0n1
           Size: 953.89 GiB, 1024209543168 bytes, 2000409264 sectors
          Label: gpt, identifier: C798685E-B053-4677-8B55-69FD721B9271

    Device               Start         End     Sectors    Size Type
>>  /dev/nvme0n1p1        2048     1050623     1048576    512M EFI System       
    /dev/nvme0n1p2     1050624    20582399    19531776    9.3G Linux filesystem
    /dev/nvme0n1p3    20582400  2000408575  1979826176  944.1G Linux filesystem





 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚ Partition name: EFI System Partition                                       β”‚
 β”‚ Partition UUID: 07FD5BA4-DF63-45BA-A409-897C38D0C089                       β”‚
 β”‚ Partition type: EFI System (C12A7328-F81F-11D2-BA4B-00A0C93EC93B)          β”‚
 β”‚Filesystem UUID: 72BA-9490                                                  β”‚
 β”‚     Filesystem: vfat                                                       β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
     [ Delete ]  [ Resize ]  [  Quit  ]  [  Type  ]  [  Help  ]  [  Write ]
     [  Dump  ]

The first step is to delete all the partitions, and then creating a new partition:

                               Disk: /dev/nvme0n1
           Size: 953.89 GiB, 1024209543168 bytes, 2000409264 sectors
          Label: gpt, identifier: C798685E-B053-4677-8B55-69FD721B9271

    Device             Start            End        Sectors       Size Type
>>  Free space          2048     2000409230     2000407183     953.9G           



 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚Filesystem UUID: 72BA-9490                                                  β”‚
 β”‚     Filesystem: vfat                                                       β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
           [   New  ]  [  Quit  ]  [  Help  ]  [  Write ]  [  Dump  ]

The partition should be 1G, which is more than enough.

                               Disk: /dev/nvme0n1
           Size: 953.89 GiB, 1024209543168 bytes, 2000409264 sectors
          Label: gpt, identifier: C798685E-B053-4677-8B55-69FD721B9271

    Device             Start            End        Sectors       Size Type
>>  Free space          2048     2000409230     2000407183     953.9G           


 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚Filesystem UUID: 72BA-9490                                                  β”‚
 β”‚     Filesystem: vfat                                                       β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
 Partition size: 1G    


     May be followed by M for MiB, G for GiB, T for TiB, or S for sectors.

You should then change the type to EFI, by selecting it with the >> and then pressing enter on [ Type ], you should then navigate you pointer up to the top, where EFI System is written, and then press enter.

                        β”Œ Select partition type ───────┐
                        β”‚ EFI System                   β”‚
                        β”‚ MBR partition scheme         β”‚
                        β”‚ Intel Fast Flash             β”‚
                        β”‚ BIOS boot                    β”‚
                        β”‚ Sony boot partition          β”‚
                        β”‚ Lenovo boot partition        β”‚
                        β”‚ PowerPC PReP boot            β”‚
                        β”‚ ONIE boot                    β”‚
                        β”‚ ONIE config                  β”‚
                        β”‚ Microsoft reserved           β”‚
                        β”‚ Microsoft basic data         β”‚
                        β”‚ Microsoft LDM metadata       β”‚
                        β”‚ Microsoft LDM data           β”‚
                        β”‚ Windows recovery environment β”‚
                        β”‚ IBM General Parallel Fs      β”‚
                        β”‚ Microsoft Storage Spaces     β”‚
                        β”‚ HP-UX data                   β”‚
                        β”‚ HP-UX service                β”‚
                        β”‚ Linux swap                   β”‚
                        β”‚ Linux filesystem             β”‚
                        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β†“β”€β”˜

                      C12A7328-F81F-11D2-BA4B-00A0C93EC93B

The next partition we create, is the boot partition. The boot partition should be 10G, which is also more than enough. The new partition should already have the default type of Linux Filesystem.
The last partition, is the root partition, we will give it the rest of the space.
When you are finished, it should look something like this:

                               Disk: /dev/nvme0n1
           Size: 953.89 GiB, 1024209543168 bytes, 2000409264 sectors
          Label: gpt, identifier: C798685E-B053-4677-8B55-69FD721B9271

    Device               Start         End     Sectors    Size Type
    /dev/nvme0n1p1        2048     2099199     2097152      1G EFI System
    /dev/nvme0n1p2     2099200    23070719    20971520     10G Linux filesystem
>>  /dev/nvme0n1p3    23070720  2000409230  1977338511  942.9G Linux filesystem 





 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚Partition UUID: 78D36147-ED9B-BB4B-8F96-6DFD28536D23                        β”‚
 β”‚Partition type: Linux filesystem (0FC63DAF-8483-4772-8E79-3D69D8477DE4)     β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
     [ Delete ]  [ Resize ]  [  Quit  ]  [  Type  ]  [  Help  ]  [  Write ]
     [  Dump  ]

You are now done with creating the partitions, and can now press [ Write ] and then [ Quit ].

Setting up LUKS

In this step we will create our luks partition and mount it.
We first format the partition and create a key. In this example we use 512 bit AES keys, and sha512, this is more secure, but could be slower (I have not tested other sizes). Please replace /dev/nvme0n1p3 with your root partition.

cryptsetup \
    --cipher aes-xts-plain64 \
    --key-size 512 \
    --hash sha512 \
    --iter-time 5000 \
    --use-random \
    luksFormat /dev/nvme0n1p3

You can then open the LUKS device. You can replace cryptroot with another name.

cryptsetup luksOpen /dev/nvme0n1p3 cryptroot

LVM

In my setup I create a 20GB swap and the rest is root.
First thing is the volume group. LUKS is mounted under /dev/mapper/. You can rename voidvm to anything you want.

vgcreate voidvm /dev/mapper/cryptroot

I then create a swap partition, named swap.

lvcreate --name swap -L 20G voidvm

And then root with the rest of the space.

lvcreate --name root -l 100%FREE voidvm

Filesystems

I will be using vfat on efi, ext4 on boot, and f2fs for root.
f2fs is only for flash drives, so please use ext4 or another filesystem if you are on HDDs.
Here I format my EFI, boot and root.

mkfs.vfat /dev/nvme0n1p1
mkfs.ext4 /dev/nvme0n1p2
mkfs.f2fs /dev/voidvm/root

We then create our swap:

mkswap /dev/voidvm/swap

Mount & chroot

We then mount all the partitions and some extra bits.

$ mount /dev/voidvm/root /mnt # Mount root
$ for dir in dev proc sys run; do mkdir -p /mnt/$dir ;\
    mount --rbind /$dir /mnt/$dir ;\
    mount --make-rslave /mnt/$dir ; done
$ mkdir -p /mnt/boot # Create mount point for boot
$ mount /dev/nvme0n1p2 /mnt/boot # Mount boot
$ mkdir -p /mnt/boot/efi # Create EFI mount point
$ mount /dev/nvme0n1p1 /mnt/boot/efi/ # Mount EFI

We then install the base system, this may take some time:

$ xbps-install -Sy -R https://alpha.de.repo.voidlinux.org/current \
    -r /mnt base-system cryptsetup grub-x86_64-efi lvm2

When it’s done, we can enter the chroot and finish up the configuration.

$ chroot /mnt
$ chown root:root /
$ chmod 755 /
$ passwd root
$ echo voidvm > /etc/hostname
$ echo "LANG=en_US.UTF-8" > /etc/locale.conf
$ echo "en_US.UTF-8 UTF-8" >> /etc/default/libc-locales
$ xbps-reconfigure -f glibc-locales
$ chsh -s /bin/bash

We can then add a user, replace zerocool with your username:

useradd \
    --create-home \
    --groups wheel,users,audio,video,storage,cdrom,input \
    --shell /bin/bash \
    zerocool

Fire the visudo command and uncomment the following line to allow sudo access to users in the wheel group.
%wheel ALL=(ALL) ALL

Fstab

Now this is the most dangerus part. You need to open fstab, and add some lines at the bottom, please use tabs, I don’t know if it make a difference to use tabs. You can use the vi editor to open the file, remember to search how to use the vi editor online, if you don’t know.
Your fstab should look a bit like this:

##
## See fstab(5).
##
## <file system>         <dir>           <type>  <options>               <dump>  <pass>
tmpfs                   /tmp            tmpfs   defaults,nosuid,nodev   0       0
/dev/voidvm/root        /               f2fs    defaults,noatime        0       0
/dev/nvme0n1p2          /boot           ext4    defaults,noatime,discard        0       2
/dev/nvme0n1p1          /boot/efi       vfat    defaults                0       0
/dev/voidvm/swap        swap            swap    defaults                0       0

This is another part of another guide that I don’t know what does, but it works with it.

# create a new file in /etc/dracut.conf.d/override.conf
# containing the following line
hostonly=yes
Grub

You should now edit /etc/default/grub, add rd.auto=1 rd.lvm .vg=voidvm rd.luks.allow-discards to GRUB_CMDLINE_LINUX_DEFAULT, the line should look like this.

GRUB_CMDLINE_LINUX_DEFAULT="loglevel=4 slub_debug=P page_poison=1 rd.auto=1 rd.lvm
.vg=voidvm rd.luks.allow-discards"

You can now install grub.

$ grub-install \
    --target=x86_64-efi \
    --efi-directory=/boot/efi \
    --boot-directory=/boot
Installing for x86_64-efi platform.
Installation finished. No error reported.

And then generate grub config.

$grub-mkconfig -o /boot/grub/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.7.12_1
Found initrd image: /boot/initramfs-5.7.12_1.img
done

Last step is to generate initramfs.

# 
# you can find it in the output
# of the grub command above
xbps-reconfigure -f linux5.4

All done, we are now ready to reboot into void linux.

# exit the chroot
$ exit

# unmount recursively from /mnt
$ umount -R /mnt

$ reboot

0.2 – FreeBSD encrypted ZFS guide

First time setup

Run this in a root shell:

gpart create -s gpt ada1
gpart create -s gpt ada2
gpart add -t freebsd-zfs -l ada1-vol0 ada1
gpart add -t freebsd-zfs -l ada2-vol0 ada2
mkdir /boot/keys
dd if=/dev/random of=/boot/keys/maria.key bs=4096 count=1
geli init -l 256 -P -K /boot/keys/maria.key /dev/gpt/ada1-vol0
geli init -l 256 -P -K /boot/keys/maria.key /dev/gpt/ada2-vol0
geli attach -p -k /boot/keys/maria.key /dev/gpt/ada1-vol0
geli attach -p -k /boot/keys/maria.key /dev/gpt/ada2-vol0
geli status
zpool create tank mirror gpt/ada1-vol0.eli gpt/ada2-vol0.eli
zpool list
zpool status

Recover from drive failure

Run this in a root shell:

name="ada2"
gpart create -s gpt "$name"
gpart add -t freebsd-zfs -l "${name}-vol0" "$name"
geli init -l 256 -P -K /boot/key/maria.key "/dev/gpt/${name}-vol0"
geli attach -p -k /boot/keys/maria.key "/dev/gpt/${name}-vol0"
geli status
zpool replace tank <old drive> "$name"

zfs_crypt service

File contents of /usr/local/rc.d/zfs_crypt:

#!/bin/sh

# PROVIDE: zfs_crypt
# BEFORE: LOGIN

. /etc/rc.subr

name="zfs_crypt"
rcvar="${name}_enable"
start_cmd="${name}_start"
stop_cmd=":"

zfs_crypt_start()
{
echo "Unlocking encrypted devices"
geli attach -p -k /boot/keys/maria.key /dev/gpt/ada1-vol0
geli attach -p -k /boot/keys/maria.key /dev/gpt/ada2-vol0

echo "Importing zfs"
zpool import tank
}

load_rc_config $name
run_rc_command "$1"

0.3 – FreeBSD syslogd in a jail

Configuration

I am using the host ip on all jails, and I allow raw sockets:

/etc/jail.conf

exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
allow.raw_sockets;

# Set defaults
ip4 = inherit;
persist;

I have two jails, jsyslog will be the syslogd server and jroot
will be the client.

jsyslog

The syslog server config, can be anything, that you like. I have removed all
the standard log files, and replaced them with my own multi directory file
structure.

/etc/syslog.conf

# System logs
security.*                                      /var/log2/sys/security.log
auth.info;authpriv.info                         /var/log2/sys/auth.log
*.notice;kern.debug;lpr.info;mail.crit;news.err /var/log2/sys/system.log
cron.*                                          /var/log2/sys/cron.log
*.crit                                          /var/log2/sys/critical.log

# Mail logs
mail.info                                       /var/log2/mail/info.log

I rotate all my logs when they hit 1 MB, and I delete, when there are 5 or 7
files. Flags:

-J Compress files with bzip.
-C Create files if they don’t exist.

/etc/newsyslog.conf

# System logs
# -----------
/var/log2/sys/security.log              640  7     1000 * JC
/var/log2/sys/auth.log                  640  7     1000 * JC
/var/log2/sys/system.log                640  5     1000 * JC
/var/log2/sys/cron.log                  640  7     1000 * JC
/var/log2/sys/critical.log              640  7     1000 * JC

# Mail
# ----
/var/log2/mail/info.log                 640  7     1000 * JC

Remember to start syslogd without any -s flags, and make sure port
514 is free (514 is default port).

jroot

The only thing jroot does is redirecting everything to the remote syslog
server.

/etc/syslog.conf

# Everything is commented out except this:
*.*                                             @loghost.chaos.local

I have an alias for loghost in my /etc/hosts file:

/etc/hosts

::1                     localhost localhost.my.domain
127.0.0.1               localhost localhost.my.domain

# This one is the important line
192.168.1.3             loghost.chaos.local loghost

Since no logs are stored in the jail, newsyslog shouldn’t do anything. You can
either disable it, or comment out /etc/newsyslog.conf. I’ve commented out
the config file, because newsyslog could be started by any program.

Remember to change the rc.syslog flags in /etc/rc.conf, multiple
syslogd daemons can’t listen on the same port, and are therefore required
to bind to different ports. I’m using theese flags:

/etc/rc.conf

syslogd_enable="YES"
syslogd_flags="-s -b :5141"

Testing if it works

If you enter the jroot jail, you can use logger(1) to test if the messages
arrive. If it doesn’t work, you always have the posibility of starting
syslogd manually, probably using the debug flag (-d; BSD extension).

0.4 – FreeBSD server housekeeping

Notes to myself about server housekeeping. New things may be introduced.

Updating system packages

pkg update
pkg upgrade

Apply security patches

freebsd-update fetch
freebsd-update install

Update system packages in all jails

# Get ID of each jail
JAILS=$(jls | awk 'NR!=1' | awk '{print $1}')
# Update package repo of each jail (3 in parallel)
echo $JAILS | xargs -P 3 -I _ pkg -j _ update
# Upgrade packages of each jail (sequentially)
echo $JAILS | xargs -I _ -o pkg -j _ upgrade

Applying security patches in all jails

# Get ID of each jail
JAILS=$(jls | awk 'NR!=1' | awk '{print $1}')
# Fetch patches
echo $JAILS | xargs -I _ -o jexec _ freebsd-update fetch 
# Install patches (sequentially)
echo $JAILS | xargs -I _ -o jexec _ freebsd-update install

0.5 – *nix server setup

This “guide” is just a collection of useful configuration.

SSH

sshd is often compiled with some things disabled and different defaults, but
this should work most places. I personally only use RSA and sha256/sha512.

# Requires OpenSSH 6.7+
# License: CC0

# Manually specify what sshd binds to
Port 22
AddressFamily any

# Only accept RSA (change path if necessary)
HostKey /etc/ssh/ssh_host_rsa_key

# Algorithms (only RSA & sha256/sha512)
KexAlgorithms diffie-hellman-group-exchange-sha256
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes256-ctr
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-256

# Only accept public keys
AuthenticationMethods publickey
PermitRootLogin prohibit-password

# Authentication
UsePAM no
LoginGraceTime 30s
MaxAuthTries 3
MaxSessions 10
IgnoreRhosts yes
AuthorizedKeysFile .ssh/authorized_keys

# Extra options
Compression no
RekeyLimit default 120s
X11Forwarding no

# Log each login to auth
SyslogFacility auth
LogLevel VERBOSE