Automatic installation on multi-node cluster with a single iso

[rohitntnx]

Hello,

I am trying to use kairos to install ubuntu image on multiple nodes. I have following constrains:

1. A single iso file should be able to cater to all the nodes of the cluster
2. Auroraboot can't be used to generate the iso since it depends on docker which I can't have. I am relying on xorisso to copy files such as config.yaml into the iso downloaded from kairos image repository.
3. I do not have a DHCP server running and hence require each server to have a static IP configuration.
4. Each node needs to have a unique hostname based on a node_id that is not recognized by kairos/cloud-init. I have a script that identifies the node and sets the hostname.

I am not able to execute custom scripts from config.yaml since any custom script that is copied in the iso's root directory is not available since /run/initramfs/live is not mounted.

Is there a way to copy arbitrary files from the iso to installer environment (temporary or permanent) so that they can be executed at different stages of the installation ?

Any help is appreciated.

base image name: kairos-ubuntu-24.04-standard-amd64-generic-v3.6.0-k3sv1.33.5+k3s1.iso

[jimmykarily]

2. Auroraboot can't be used to generate the iso since it depends on docker which I can't have. I am relying on xorisso to copy files such as config.yaml into the iso downloaded from kairos image repository.

auroraboot can also be used as a binary (it's built with go) and it doesn't require docker, at least not for all of its features. You may be able to build an ISO using the binary directly, I'm not 100% sure.

Is there a way to copy arbitrary files from the iso to installer environment (temporary or permanent) so that they can be executed at different stages of the installation ?

See my reply here: #3857 (comment)
You are probably doing it correctly already but you need to find where your files are. Hopefully that will be easy using find otherwise you can provide more details (e.g. the xorriso command you used to copy the files in).

[rohitntnx]

@mauromorales
/run/initramfs/live is not mounted so files are not available to be copied.

[mauromorales]

@rohitntnx I'm not sure I follow, could you explain what you are doing a bit further. I understand you are placing your files in the livecd with xorriso, and that if you boot the livecd the files are there but I don't think i understand what's happening after that

[rohitntnx]

Iso has the following files.(config.yaml test.sh)
In config.yaml, there is a command to execute test.sh in initramfs/boot/fs stage but it can't find it as /run/initramfs/live is not mounted. Furthermore I want test.sh to create a new file newconfig.yaml in /oem that will get applied on further stages.
This will allow me to insert business logic to configure multiple nodes using single ISO. The business logic will check for the serial number of the node and create newconfig.yaml accordingly.

[talss89]

Hi @rohitntnx, you may already have solved this, apologies if so.

... and further apologies if any of this sounds like 'teaching you to suck eggs' 🙄

• /run/initramfs/live will be available only when running from the ISO ([ -f "/run/cos/live_mode" ]).
• initramfs / boot / fs stages run on both live and installed systems.
• If you're not guarding those stages with a live mode check (so it executes again after install / reboot), you'll see errors relating to /run/initramfs/live not being mounted.
• Furthermore initramfs / boot / fs stages haven't yet mounted the COS_OEM partition (when running in live mode), so running test.sh which tries to write into /oem will work, but it'll write into the ramdisk - vanishing on reboot.

I'm making assumptions there, but perhaps that could be what's happening.

If you hook into before-install you should be able to run /run/initramfs/live/test.sh. Make sure you're writing into /run/cos/oem though, as that's where the installer mounts the COS_OEM partition during the live mode install phase.

Once rebooted, your newconfig.yaml will be in /oem.

TLDR;

stages:
  before-install:
    - name: "Do the thing"
      commands:
        - cp /run/initramfs/live/your-thing-on-iso /run/cos/oem/your-thing-on-disk
        # or, in your case:
        - /run/initramfs/live/test.sh

[rohitntnx]

Thanks!! I did exactly what you suggested, and it worked. It took me some time to figure out what filesystem view is available at what stages.
Meanwhile, I am facing another issue with the ordering of stages that I have asked about here: #3917

Maybe you can save me a few hours of debugging @talss89

[rohitntnx]

Thanks !