[investigating] Cloud-init support


#1

Cloud-init support would be awesome to ease the adoption of the Online Labs IAAS.


What are the missing features to be compatible with cloud-init framework ?

  • compatible metadata
  • customisable kernel cmd lines
  • custom initrd ?

Related talks




[OFFICIAL] Bootscripts (Kernels, Initrd, Cmdline)
Metadata management for cloud-init
#2

We are using cloud-init to do a basic configuration of our cloud servers. cloud-init is a python tool that fetches metadata from the cloud vendor (ips, location, volumes, ssh keys) and does the server configuration at boot time, it can replace some of your OCS scripts.

Cloud-init is now a standard available on most cloud providers (EC2, OpenStack based, Digital Ocean …), each provider has it’s own API for retreiving metadata.

The custom configuration is done with a user-data script given when the cloud server is created, it can be a shell script, a cloud-init configuration. On EC2 and Openstack clouds, the user-data is at http://169.254.169.254/latest/user-data

With cloud init compatible metadata service and user-data provider your customers will be able to automate the servers customization without creating custom volumes, images or initrd scripts.


#3

What’s the status of this? Is there any support planned? Would be really helpful for automating infrastructure.


#4

@martn, actually the status is paused,

Do you have use cases / examples to help us achieve this new feature ?


#6

That’s sad, large auto-scaling clusters aren’t much fun without cloud-init :slight_smile:

Use-cases include:

  • use stock images and specify docker image to fetch and run in metadata
  • mounting volumes
  • distribution of one-time-tokens used to obtain temporary secrets from a “service storing secrets”
    (this is an important security factor in addition IP-level security groups and static secrets burned into an image, as those can be broken by an attacker, one-time-tokens can’t reused).

Our EC2 infrastructure heavily relies on user-data from cloud-init. And whilst we have worked around it in other environments where it’s not available, I would dare argue that it makes large auto-scaling clusters far more feasible.

Building reliable systems by having a provisioning node SSH into machines is very hard. Hence, why this is an important feature in modern cloud environments.

Note: cloud-init metadata can be provided by attaching a tiny network-attached storage device.
But it’s far easier to just make a special route for 169.254.169.254:80 to a dedicated meta-data server that looks up the private IP address of the caller and returns metadata from you database.

(at this point I’m still just playing with the idea of using scaleway, but cloud-init would certainly make adoption easier and reduce the need for specially engineered solutions for automatic provisioning).


#7

Cloud-init support has been added on most x86_64 images: https://blog.online.net/2018/07/05/introducing-scaleway-cloud-init-support/
How about amd64 images? Any roadmap?