Btrfs image : a recipe


#1

A while ago, a poster showed interest in a btrfs image.
Here is my recipe for experimental disaster.

A foreword :

I want to use the btrfs storage driver on Docker.
Since I can’t have multiple partitions on a NBD, I “need” a btrfs root.

Creating a root image formated with btrfs.

make_btr_image.sh {

#!/bin/bash
set -ex

# from http://serverfault.com/a/547929
# if we can't find an agent, start one, and restart the script.
if [ -z "$SSH_AUTH_SOCK" ] ; then
  exec ssh-agent bash -c "ssh-add ; $0"
  exit
fi

IP=8.8.4.4
BS="4.1.6-std #249" # older kernels dont behave so well with btrfs
BASE="image:ubuntu-trusty-14-04-lts"

# now a good ressource for `scw create` examples is :
# https://github.com/scaleway/scaleway-cli/tree/master/examples
MACHINE=$(scw create --ip-address=$IP --bootscript="$BS" --volume=50GB "$BASE")

scw start $MACHINE
scw attach $MACHINE
# i like to look at the blinkendioden
# when it's finished blinking C-q
  
#scw exec $MACHINE "base64 -d | gzip -dc > img.sh" <<EOF ## doesnt seem to work
#you may need to ssh-keygen -f ~/.ssh/known_hosts -R $IP
ssh -v root@$IP "base64 -d | gzip -dc > img.sh" <<EOF
H4sIAMChZ1YCA5VWW2/bNhR+1684dYN1wyyrfdnDAg9IHafN2tiG7RYdhi2hKSpiLZEqSTnxiv73
faTkWxp364sSkzzfuX3n8vRJspAqWTCbR9FTGrFSEOmMyOX4Z6G1s86wishyIytHK2YkU46cptqK
HkQGtTFCuWLdDTLtA0u2riptnEiJGUGlVNAhuhC4yyXPoUQVayKpeFGnwhIJawEjWVExvmS3/oip
FJ/KncKSWhZpuhOXyjpWFDboDJfxFgCPdhhSwVSan01fDeenLWTGloLnBs51aWPOFjC4fgBxJ12u
a0fkJYgqI1eyELjq4fGFVBBbByn4Ke0mAoBAzJjj+ULfdzdmW8q0IW4Ec1LdkmPmVrjmcPeaEFuo
9tGluc9DKjJWF8Fa2EJKUxy3avofEN/buoTrXQ8PbwO6IEY+4tH7s+nl2WjebzPgs+wx7do6UUKW
59IJ7mojujR4QVaYlTAH5+TWVfCMmTLPorPp4HW/+RdY76zYhSozugwZKaT1mee6rLQSng7glL8I
uCu4NhhfTcaj4Wg+65dMqm6tcGx29q10Aae8OyBhw0G3NTs6H74cj+ez+fRscn1+Oe0npXJJvaiV
q2NXFckGxdvhVW8tbPmW4p99vGjy5tXs+nI0ePvufNh/xlnMhXEykxyRtF2EVCt8a1MgyJVji8Kf
yqp2srBxhVT64PM4zXkV80KGbOCN7RZyoayNy3UOLxTKq0slU3G66JJiSuO7SLcSSrjYaV0AW7kq
fFIY0CUDOwsNJdbm+NQpBFeyBK1An0AuGCVh3P0O7lkUXV1Op+NpP3eu+jVJSmmMNj1e6Drtofyk
Ej1o3MTNV6tF5M6H7y8Hw36SilUCtBc+lhMjKpRxhGKMoZHqYBfF61PaHd0alh6etUWFM1o4k9nG
OdB511g8+iAXfEkyo1QaUE6bNT1IMIl7ZNJGePMnPaE4pc7Jgycd+uvUp1RFROUSSBRX9PBRlEmv
8EKbkjnauHjfsi3au/uBrjTCsvcGjHn6EK6156b0byuNZkPxp6+03hyYltleCAbFb5uOEmcUj+lv
ce+MyLywj79/Ggxofx9z5eU2kqHeJad3IZ3kTG3dekPwg5jHsS9DOvGVvNdJTtpWgaO9wu2c7Eq1
g6u2gnC+XzOdjb6HZuJdw8JOFPkuJ30cP7/o9X75EqUaXqITa18SNSvot6/EE+F4IpV0iXPrE9nT
aBJGpgKySjys8gxt2ZIRn2rwKA2tUFdrTJpQ6ghDVYC1oWtVIKZx9FEvKKYrNCCUj0KHI9DWNSJu
PXseXVy+Hc6u5+PrwXjyR79zYM7seY9rlXW+BrzAsxymoFrjpVgD11rNJfM9MXTwta7RYzkPGUZA
bhLPhKTHaowaI/8R6bWXuznU/3O/QzsLNuDHjEDNhmbezMjRy3Nq+gIGmOZoTd5Jm9cu1XfqW3p8
R/HWxVUL2PaXY3pfGcZFVvuhWDcU9kM3RXvUSqG8G9K3Ffd/9O5Ej6kcNNeIc5pKJzVGcqsgDETv
ezPX/ksf5GNfhyxuxY9pnK2V3yAUUmVpKYwSBeo1rUHAb+YMYnHzPG6f7zTMc8zYds+SezxerMGP
Y9G48Q4+Hly63Wai94hRtTWJ9cufh94hBC3Bnve+L/hJF9aTsGiJnK0kfqDYfDtprT3mcru2JIbP
AqAfZyERkgvyxsvbGjsP8tVUBatATaxXfrSM5pM2Z8fQgbaL3RXamtXweTNmQ+IvJ54Rxm+GFsbD
DUHnoxlxpvATEc6wsyku0t4RHR7NBgVvmhzrypvrk1OEaobp26XpCAY6MHfFztSRcHfaLP1mKkyG
DNnDWBxztxFLdmIBbXgveB0WknbFtN/FzC0JNI8fIefjKoxwRgrfWdptsRSOYSFg9ON8+GH+0xE9
heasSFplG5Hv1vD7bDz6Lg3xR6tVUHP+ejABQfTyGF+xvflPaG/oA9LF/rXtpcmGVgFn4KdKM2zY
AhNps0zu83e7yoSp5/X5VyefDzS3I5BX7cUX/H0wAL+0N+3Aq7dLweEy8C97O9O0yA0AAA==
EOF

ssh -v root@$IP "cat img.sh" | less

# Uncomment me when you approve of the content !
#ssh -v root@$IP "bash img.sh"

scw stop $MACHINE
SNAPID=$(scw commit -v 1 $MACHINE)
scw tag $SNAPID trusty-btrfs
scw rm $MACHINE

}

The base64 stuff is where it’s happening:
- core is image_creation.sh from https://www.scaleway.com/docs/create-an-image-from-scratch/;
- add the apt-get install line;
- mkfs.btrfs insted of mkfs.ext4;

Resolving the dependancies is quite slow, but cdebootstrap doesn’t work for us:
it doesnt have the same options & seems to apply to debian only.

Booting this brand new world.

boot.sh {

#!/bin/bash

set -ex

# from https://serverfault.com/a/547929
# if we can't find an agent, start one, and restart the script.
if [ -z "$SSH_AUTH_SOCK" ] ; then
  exec ssh-agent bash -c "ssh-add ; $0"
  exit
fi

IP=8.8.4.4
BS="4.1.6-std #249" # older kernels dont behave so well with btrfs
BASE="image:ubuntu-trusty-14-04-lts"

MACHINE=$(scw create --env="INITRD_DROPBEAR=1 INITRD_VERBOSE=1" --ip-address=$IP\
                     --bootscript=$BS user/trusty-btrfs)
  
#you may need to ssh-keygen -f ~/.ssh/known_hosts -R $IP
ssh root@$IP "cat> run.sh" <<EOF
set -x

cd /run
kv=$(uname -r)
for m in fs/btrfs/btrfs.ko crypto/xor.ko lib/raid6/raid6_pq.ko lib/zlib_deflate/zlib_deflate.ko
do
  wget http://mirror.scaleway.com/kernel/${kv}/modules/${kv}/kernel/$m
done

for m in xor.ko zlib_deflate.ko raid6_pq.ko btrfs.ko
do
  insmod $m
done

# we have support for btrfs in kernel now
mount /dev/nbd0 /newroot

# get my keys to the newly mounted system
cp /root/.ssh -pr /newroot/root/

continue-boot
EOF

ssh root@$IP "sh run.sh"
sleep 10 # finish the boot

# the server key will change
ssh-keygen -f ~/.ssh/known_hosts -R $IP
ssh root@$IP mount

}

See https://mirror.scaleway.com/kernel/4.1.6-249/modules/4.1.6-249/modules.dep for dependancies and
/usr/local/sbin/oc-sync-kernel-modules for the whole story.

Some ressources I found helpful :

Some things I want to look into :


#2

Actually, it may be a better idea to use : https://github.com/boris-arzur/scaleway-docker

Boot it with :

#!/bin/bash

set -ex

# from https://serverfault.com/a/547929
# if we can't find an agent, start one, and restart the script.
if [ -z "$SSH_AUTH_SOCK" ]
then
  exec ssh-agent bash -c "ssh-add && bash $0"
fi

IP=8.8.4.4
BS=latest # older kernels dont behave so well with btrfs

MACHINE=$(scw create --env="INITRD_DROPBEAR=1 INITRD_VERBOSE=1" --ip-address=$IP\
                     --bootscript=$BS user/${the_name_you_gave_it})

#you may need to ssh-keygen -f ~/.ssh/known_hosts -R $IP
ssh root@$IP "cat> run.sh" <<EOF
set -x

cd /run
kv=\$(uname -r)
for m in fs/btrfs/btrfs.ko crypto/xor.ko lib/raid6/raid6_pq.ko lib/zlib_deflate/zlib_deflate.ko
do
  wget http://mirror.scaleway.com/kernel/\${kv}/modules/\${kv}/kernel/\$m
done

for m in xor.ko zlib_deflate.ko raid6_pq.ko btrfs.ko
do
  insmod \$m
done

# we have support for btrfs in kernel now
mount /dev/nbd0 /newroot

# get my keys to the newly mounted system
cp /root/.ssh -pr /newroot/root/

continue-boot
EOF

ssh root@$IP "sh run.sh"
sleep 10 # finish the boot

# the server key will change
ssh-keygen -f ~/.ssh/known_hosts -R $IP
ssh root@$IP mount