These are my personal notes while performing the installation. Instructions might be incomplete, dangerous or wrong.

Problem: Local storage for K3s installation in VM

Cleanest solution: Mount NFS on host and use hostpath on pod

Also: Would be good to export all k3s state, maybe backups are better than full nfs export

dom0$ cat libvirt-net-private.xml
  <bridge name='virbr1' stp='on' delay='0'/>
  <ip address='' netmask=''>
      <range start='' end=''/>
virsh net-define ./libvirt-net-private.xml
virsh net-autostart private
virsh net-start private

Add to vbox

virsh edit k3s
    <interface type='bridge'>
      <source bridge='virbr1'/>
      <model type='virtio'/>
virsh shutdown k3s
virsh start k3s

# shows eth1
ip l

# configure network
auto eth1
iface eth1 inet static
/etc/init.d/networking restart

Ping does not work from k3s Check arp on dom0 (works):

$ arp -i virbr1            ether   52:54:00:f0:60:24   C                     virbr1

My guess is that firewall is preventing traffic.

dom0$ iptables -P INPUT ACCEPT

Ping works -> add iptable rules

Add to firewall definition:

iptables -A INPUT -i $IVMS -j ACCEPT

Add NFS share


UUID=7a8730cf-4185-4d56-b1e9-d053a7b17a50 /mnt/red1_vms 	btrfs 	rw,noatime,nodiratime,compress=lzo,space_cache,subvol=@vms 0 0
/mnt/red1_vms/storage/k3s				  /export/k3s_store	  none 	  bind 		   0 0


# Reload nfs
exportfs -ra
k3s$ mount /mnt/nfs
# Does not work connection refused
k3s$ apk add nfs-utils
k3s$ mount /mnt/nfs
# works now

Add correct fstab automount: /etc/fstab /mnt/k3s_store nfs auto,rw,_netdev 0 0

Startup automount

$ rc-update add nfsmount
$ rc-service nfsmount start
$ reboot

NFS permission denied on k3s

Checklist - Mounted with rw - Exported with rw - dom0 can write on it

Problem with mount directory is root owned and client-root is trying to write at it. With NFSs default setting root_squash (man exports), the root user is mapped to another uid.

We could disable the root uid remapping:


This handles the k3s root user the same as the dom0 root user. Another approach is to chmod a+rwx the desired directory and face the situation, that uid/gid 0 from the guest is mapped to nobody:nobody on the filesystem. Since many containers are running as root and security is not a real concern here, we disable root squashing with no_root_squash for now.

Side-note: Host directory sharing


<filesystem type='mount' accessmode='mapped'>
  <source dir='/mnt/red1_vms/storage/k3s'/>
  <target dir='k3s_store'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>


k3s_store /mnt/k3s_store            9p             trans=virtio    0       0

Does not work as expected, because file permissions are mapped. Only virtualized filesystem. Problems with k3s setup (I/O errors).