KVM: VM that gets a NIC on a config drive network keeps a data volume on a reserved device id and cannot start #14513

Problem

The device ids reserved for the CD-ROMs and the config drive ISO depend on whether the VM uses config drive. A VM without config drive can have a data volume on a device id that becomes reserved when a NIC on a config drive network is added later. The volume keeps its device id, and on q35 its target is the same as a CD-ROM, so the VM cannot start (XML error: target '...' duplicated).

Not caused by the device id fix: the volume was given that id while the VM did not use config drive, and on 4.23.0.0 the same flow fails with the 4.23 device ids (1, 2, 4) already with vm.iso.max.count=1, and with data volumes on 1, 2, 5 it fails with vm.iso.max.count=2 (target 'sde' duplicated between the empty CD-ROM and the config drive ISO). The notes of both fixes say that volumes already on a reserved device id are not changed.

Versions

Fails before the device id fixes:

• 4.22.1.x: not run for this flow (the 4.22 device id fix does not change the device ids of VMs without config drive, so the 3rd volume is on 4 there too).
• 4.23.0.0: data volume on deviceid 4 (the 3rd volume on 4.23): the VM cannot start (target 'sde' duplicated with the config drive ISO), with vm.iso.max.count=1 and 2. Data volumes on 1, 2, 5: starts with vm.iso.max.count=1, cannot start with 2.

Still fails after the device id fixes (volumes already on an id that becomes reserved are not changed):

• 4.22 build with the device id fix (4.22.2.0-SNAPSHOT): data volume on deviceid 4, the VM cannot start (target 'sde' duplicated with the config drive ISO), default vm.iso.max.count=1.
• main (24.0.0-SNAPSHOT): the 3rd volume is now on deviceid 5. The VM starts with vm.iso.max.count=1 (better than 4.23.0.0) and cannot start with vm.iso.max.count=2 (target 'sdf' duplicated with ISO 2 or its empty CD-ROM).

Environment:

KVM on OL 9, qemu-kvm 7.2.0, libvirt 9.0.0, machine type pc-q35-rhel9.2.0 (template settings rootDiskController=scsi, kvm.guest.os.machine.type=pc-q35-rhel9.2.0, UEFI=LEGACY), host.cdrom.max.count 2. NFS primary storage.

The steps to reproduce the bug

1. Create two L2 networks: N1 with DefaultL2NetworkOffering, N2 with DefaultL2NetworkOfferingConfigDrive.
2. (main only) Set vm.iso.max.count to 2.
3. Deploy a q35 scsi VM on N1 (stopped) and attach 3 data volumes. They get device ids 1, 2, 4 (4.22) or 1, 2, 5 (main).
4. Add a NIC on N2 to the VM (addNicToVirtualMachine).
5. Start the VM. It fails with 530 "Unable to orchestrate the start of VM instance", and the management server log has "Unable to start VM on Host ... XML error: target 'sde' (4.22) or 'sdf' (main) duplicated".
6. On 4.22: detach the 3rd volume and attach it again. It gets deviceid 5 and the VM starts. (Not tried on main for this flow.)

What to do about it?

Expected: a VM that starts using config drive after a NIC is added starts, with its data volumes not on the device ids of the CD-ROMs and the config drive ISO.