[OFFICIAL] Bootscripts (Kernels, Initrd, Cmdline)


#1

Online Labs’ Bootscripts

Bootscripts are composed of three elements:

  • A kernel (with an optional DTB)
  • An initrd: an archive containing the scripts needed to fetch the missing kernel modules and dynamically mount your NBD root volume, fully loaded in RAM before loading your volumes
  • A cmdline: passed to the kernel at startup, available for the initrd scripts and available from the server by cat /proc/cmdline

Today we introduce the bootscripts API.
You will now be able to change which bootscript your server will boot with.
You will always boot with the bootscript you attached to your server, if you set a null bootscript, then you will boot on the default one (in general the latest stable kernel).

We are working hard with http://free-electrons.com and Marvell’s teams to get stable and efficient kernels.
On new kernel releases, we will add a post on this topic will description, so stay tuned.


Kernels versions:

  • 3.2.x: Marvell’s kernel, not mainstream, less features, better integration with the hardware, today,
  • 3.17.x: Latest stable mainline kernel, we still have known bugs on this version
  • 3.18.x: Unstable next mainline kernel, with most of new features, but not stable

List bootscripts

$ curl -s https://api.cloud.online.net/bootscripts --header 'X-Auth-Token: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx' | python -m json.tool
{
    "bootscripts": [
        {
            "bootcmdargs": {
                "id": "bf308e86-dc34-43d5-bb37-cdd3b5596ca2",
                "value": "ip=dhcp boot=local root=/dev/nbd0"
            },
            "id": "025b67f9-10a5-4478-94a7-1c13eaa16356",
            "initrd": {
                "id": "fe70e4dc-fb87-47e8-bf61-4c75c6f5a61e",
                "path": "initrd/pimouss-uInitrd",
                "title": "pimouss-uInitrd"
            },
            "kernel": {
                "dtb": "dtb/pimouss-computing.dtb.3.17",
                "id": "2a04936b-dc53-4221-a0d7-a482af8b29d3",
                "path": "kernel/pimouss-uImage-3.17-90-perf",
                "title": "Pimouss 3.17-90-perf"
            },
            "public": true
        },
        {
            "bootcmdargs": {
                "id": "bf308e86-dc34-43d5-bb37-cdd3b5596ca2",
                "value": "ip=dhcp boot=local root=/dev/nbd0"
            },
            "id": "ed1cfe4e-0eaa-41bb-a09b-07a18aba63f6",
            "initrd": {
                "id": "fe70e4dc-fb87-47e8-bf61-4c75c6f5a61e",
                "path": "initrd/pimouss-uInitrd",
                "title": "pimouss-uInitrd"
            },
            "kernel": {
                "dtb": "dtb/pimouss-computing.dtb.3.18-rc1",
                "id": "9a3d761d-5858-41dd-8b7d-a1e592f777b4",
                "path": "kernel/pimouss-uImage-3.18-rc1-94-std",
                "title": "Pimouss 3.18-rc1-94-std"
            },
            "public": true
        },
        {
            "bootcmdargs": {
                "id": "bf308e86-dc34-43d5-bb37-cdd3b5596ca2",
                "value": "ip=dhcp boot=local root=/dev/nbd0"
            },
            "id": "82a32c38-d9ef-4dce-b534-d77df4253ac7",
            "initrd": {
                "id": "fe70e4dc-fb87-47e8-bf61-4c75c6f5a61e",
                "path": "initrd/pimouss-uInitrd",
                "title": "pimouss-uInitrd"
            },
            "kernel": {
                "dtb": "",
                "id": "103affda-0bb9-4dbb-850a-fb1946b668c4",
                "path": "kernel/pimouss-uImage-3.2.34",
                "title": "Pimouss 3.2.34 #34 SMP"
            },
            "public": true
        },
        {
            "bootcmdargs": {
                "id": "bf308e86-dc34-43d5-bb37-cdd3b5596ca2",
                "value": "ip=dhcp boot=local root=/dev/nbd0"
            },
            "id": "8e7dc70d-c75e-4553-af5f-6d8ee7ecffe1",
            "initrd": {
                "id": "fe70e4dc-fb87-47e8-bf61-4c75c6f5a61e",
                "path": "initrd/pimouss-uInitrd",
                "title": "pimouss-uInitrd"
            },
            "kernel": {
                "dtb": "dtb/pimouss-computing.dtb.3.17",
                "id": "9db9a963-adb6-424b-87d5-b38a54b9746d",
                "path": "kernel/pimouss-uImage-3.17-85-std",
                "title": "Pimouss 3.17-85-std"
            },
            "public": true
        }
    ]
}

Set a bootscript for a server:

$ curl -s https://api.cloud.online.net/servers/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx --header 'X-Auth-Token: yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyy' --header 'Content-Type: application/json' -X PATCH -d '{"bootscript": "zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzz"}'

xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx: server id
yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyy: API Token
zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzz: bootscript id


You can unset the bootscript and boot with the default value by settings the bootscript to null

$ curl -s https://api.cloud.online.net/servers/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx --header 'X-Auth-Token: yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyy' --header 'Content-Type: application/json' -X PATCH -d '{"bootscript": null}'

These calls will only change your server definition, you will need to reboot it to get a new kernel

Check your current version with uname

root@c1-10-1-15-34:~# uname -a
Linux c1-10-1-15-34 3.17.0-90 #7 SMP Mon Oct 20 13:54:37 CEST 2014 armv7l armv7l armv7l GNU/Linux
root@c1-10-1-15-34:~#

Current public bootscripts


3.17-90-perf

  • Id: 025b67f9-10a5-4478-94a7-1c13eaa16356
  • Kernel: mainline=3.17.0 build=90-perf
  • Initrd: standard
  • Cmdline: ip=dhcp boot=local root=/dev/nbd0

3.18-rc-94-std

  • Id: ed1cfe4e-0eaa-41bb-a09b-07a18aba63f6
  • Kernel: mainline=3.18-rc1 build=94-std
  • Initrd: standard
  • Cmdline: ip=dhcp boot=local root=/dev/nbd0

3.2.34 #34 SMP

  • Id: 82a32c38-d9ef-4dce-b534-d77df4253ac7
  • Kernel: Marvell=3.2.34 SMP, build=34
  • Initrd: Standard
  • Cmdline: ip=dhcp boot=local root=/dev/nbd0

3.17-85-std

  • Id: 8e7dc70d-c75e-4553-af5f-6d8ee7ecffe1
  • Kernel: mainline=3.17.0 build=85-std
  • Initrd: Standard
  • Cmdline: ip=dhcp boot=local root=/dev/nbd0

See the next messages for New bootscript announcements.

More info about the kernels:


nbd0 connection issues lead to stuck instance
[OFFICIAL] Issues with mainline Kernel
#4

You can also use the sdk to update your bootscript. Below an exemple to list bootscripts, and update a server.

from ocs_sdk.apis import ComputeAPI

AUTH_TOKEN = ''
SERVER_ID = ''
BOOTSCRIPT_ID = None  # or the bootscript id

api = ComputeAPI(auth_token=AUTH_TOKEN)

# List and print bootscripts
result = api.query().bootscripts.get()
for bootscript in result['bootscripts']:
    print bootscript

# Update the bootscript of SERVER_ID
api.query().servers(SERVER_ID).patch({'bootscript': BOOTSCRIPT_ID})

#5

New bootscript: 3.17-90-perf with initrd debug

  • Id: 6a840c8f-a73f-432b-acd2-16bd73fb6fec
  • Kernel: mainline=3.17.0 build=90-perf
  • Initrd: standard
  • Cmdline: ip=dhcp root=/dev/nbd0 boot=local initrd_debug=FULL
  • Comment: verbose initrd

#6

From what I understand, there is no way to create our own bootscripts, with a custom kernel, initrd and cmdline.
Do you think that this feature can be implemented before the end of the beta phase ?


#7

We can do it manually, what are your needs ?


#8

I would like to test the possibility of having a fully encrypted system, root included.
That means having dropbear embedded in the initrd as well as other specific configuration files.


#9

We can provide a new bootscript with an initrd that will drop a busybox shell before mounting the rootfs (but after connecting nbd device).
You won’t have all your scripts and luks binaries, but the initrd will contain a wget and tar, so you can download all the files you need.
If you can get a working POC, we can manage to create together a new initrd with all the needed binaries and libraries.


#11

New bootscript: 3.17-100-std

  • Id: 918a439a-d003-4867-9ac8-00ee93bb4e27
  • Kernel: mainline=3.17.0 build=100-std
  • Initrd: standard
  • Cmdline: ip=dhcp root=/dev/nbd0 boot=local
  • Comment: Latest kernel build with patch for network issues

Linked Kernel


[investigating] Ping: sendmsg: No buffer space available
#12

I’d love to be able to add custom args to the kernel like rootfstype and rootflags

 rootfstype=btrfs rootflags=subvol=@root

[investigating] Cloud-init support
#13

User-defiend kernel cmd lines are planned

For now we can create some kernel cmd lines, can you describe how to use a server with rootfstype=btrfs rootflags=subvol=@root, so we can make this cmd line public and add a documentation


#14

One more : cloud-config-url=http://my-cloud-manager.com/cloud.cfg for cloud-init

“rootfstype=btrfs rootflags=subvol=XXXXX” should be dynamic, the subvolume name @root is a private naming scheme, also, it’s interesting to change the subvolume name and use a snapshot for testing, roolback and recovery.


[investigating] Cloud-init support
#15

I think cloud-init is a standalone project and we will need to add more than just a custom cmd-line (probably a custom initrd script too)


Please continue the discussion on :


#16

I would LOVE to be able to get NetBSD running on your cloud servers now that NetBSD has ARM MP support. Would you be willing to work with me or someone else from the NetBSD community to make this happen?


#17

Yep !
We need :

  • a kernel that runs on Marvell’s ArmadaXP with nbd-client support
  • an initrd with armhf binaries and a custom script that fetch metadata server and mount root device

Do you have sources link + build instructions for one or both of them ?


#18

Ah, nbd is going to be the killer here. No chance of iscsi, I suppose? :slight_smile:


#19

You can setup an iscsi or nfs server on another linux-based server


#20

New bootscript: rescue

  • Id: a5485ac0-7611-44fb-8704-b409fcf71e00
  • Kernel: mainline=3.17.0 build=96-std
  • Initrd: standard
  • Cmdline: ip=dhcp boot=rescue rescue_image=http://212.47.225.66/rescue-images/rescue/rootfs-armhf-rescue-latest.tar.gz

This bootscript will create a ramdisk with the contents of a downloaded rootfs, you will have access to all your disks and will be able to perform debug/rescue actions


#21

New bootscript: rescue

  • Id: d28611ff-08bd-4bdd-9f73-084a0e1ec9dc
  • Kernel: mainline=3.17.0 build=119-std with aufs
  • Initrd: standard
  • Cmdline: ip=dhcp boot=local root=/dev/nbd0 USE_XNBD=1

Changes:

  • Attach nbd0 using xnbd-client instead of bbd-client -> improved stability
  • Signing built modules
  • Enabled MODVERSIONS
  • Removed MVSDIO (SD/MMC)
  • Can now force a module load/unload
  • Fixed the 1 load average of nolp.ko

Here are the commits:


And the full config file:


Nolp.ko makes system slow, Oops on rmmod
[OFFICIAL] Linux Kernel (new modules, optimizations, hacks, ...)
#22

The latest kernel available to boot is 3.18.rc1-94. Unfortunately there is no overlay.ko in rc1, and I would like to test overlayfs. Yes, I known there is 3.17-aufs available, but I really want to try overlayfs.

I can see that 3.18.0-124 kernel with overlayfs module exists on mirror.cloud.online.net. Does it mean I should expect it to be bootable soon? Or maybe there is some way I can try booting it now?


#23

Waking up this thread.
I’m actually missing tun/tap device feature in 4.5.7-* kernels.