Upgrade from v1.6.x to v1.6.y

General information

An Upgrade button appears on the Dashboard screen whenever a new SUSE Virtualization version that you can upgrade to becomes available. For more information, see Start an upgrade.

For information about upgrading SUSE Virtualization in air-gapped environments, see Prepare an air-gapped upgrade.

Update the Harvester UI Extension on SUSE Rancher Prime v2.12

You must use a compatible version (v1.6.x) of the Harvester UI Extension to import SUSE Virtualization v1.6.x clusters on Rancher v2.12.

  1. On the Rancher UI, go to local → Apps → Repositories.

  2. Locate the repository named harvester, and then select ⋮ → Refresh.

  3. Go to the Extensions screen.

  4. Locate the extension named Harvester, and then click Update.

  5. Select a compatible version, and then click Update.

  6. Allow some time for the extension to be updated, and then refresh the screen.

Known issues

Upgrade is stuck in the "Pre-drained" state

In certain situations, the Instance Manager might fail to clean up an engine instance, even after the state of the engine CR has changed to "Stopped". The upgrade process becomes stuck in the "Pre-drained" state because the instance-manager pod cannot be deleted while the corresponding PodDisruptionBudget (PDB) still exists.

The workaround is to delete the instance-manager PDB after ensuring that all volumes are healthy.

Related issues: #8977 and #11605

Guest cluster is stuck in the "Updating" state

An RKE2 guest cluster may become stuck in the "Updating" state after SUSE Virtualization is upgraded. The following error message is displayed on the SUSE Virtualization UI:

Configuring etcd node(s) rke2-pool1-xdvfc-qf4vb: Node condition MemoryPressure is Unknown. Node condition DiskPressure is Unknown. Node condition PIDPressure is Unknown. Node condition Ready is Unknown, waiting for probes: calico, etcd, kube-apiserver, kube-controller-manager

The issue occurs when the guest node’s IP address changes after the upgrade, causing etcd to malfunction. It is likely that the underlying virtual machine was rebooted several times and received a new IP address from the DHCP server.

To address the issue, perform the following steps:

  1. On the Rancher UI, delete the error-causing node from the guest cluster.

  2. On the SUSE Virtualization UI, check the status of the underlying virtual machine.

  3. If necessary, restart the virtual machine.

The virtual machine is removed, and the guest cluster attempts to create a new node. Once the node is created, the status of the guest cluster changes to "Active".

Related issue: #8950

Stopped virtual machine is stuck in the "Starting" state

A SUSE Storage volume can flap between the "Detaching" and "Detached" states after a live migration. Because the volume is not ready, the associated virtual machine is unable to fully start.

The workaround is to clear the volume’s status.currentMigrationNodeID using the following command:

kubectl patch -n longhorn-system volume <volume> \
  --type=merge \
  --subresource status \
  -p '{"status":{"currentMigrationNodeID":""}}'

Related issues: #8949 and #11479

4. Upgrade to v1.6.1-rc2 stuck in the "Pre-drained" state

The upgrade process may become indefinitely stuck in the "Pre-drained" state if the following specific upgrade path is followed:

  • The cluster was first upgraded from v1.5.1 to v1.6.0.

  • A subsequent upgrade from v1.6.0 to v1.6.1-rc2 is attempted.

You may see the following error message when you check the SUSE Virtualization controller:

harvester-899b4df79-mzgkx apiserver time="2025-10-11T07:21:17Z" level=error msg="error syncing 'fleet-local/custom-a8796656aa4c-machine-plan': handler harvester-upgrade-secret-controller: jobs.batch \"hvst-upgrade-7zqp7-post-drain-hp-113-tink-system\" already exists, requeuing"
harvester-899b4df79-mzgkx apiserver time="2025-10-11T07:23:17Z" level=error msg="error syncing 'fleet-local/custom-a8796656aa4c-machine-plan': handler harvester-upgrade-secret-controller: jobs.batch \"hvst-upgrade-7zqp7-post-drain-hp-113-tink-system\" already exists, requeuing"
harvester-899b4df79-mzgkx apiserver time="2025-10-11T07:25:17Z" level=error msg="error syncing 'fleet-local/custom-a8796656aa4c-machine-plan': handler harvester-upgrade-secret-controller: jobs.batch \"hvst-upgrade-7zqp7-post-drain-hp-113-tink-system\" already exists, requeuing"

This is a rare synchronization failure: the job was already created but the upgrade object was not updated.

The workaround is to delete the existing post-drain job and then wait for the upgrade controller to recreate it.

Related issue: #9293

Unnecessary live migrations during the upgrade

SUSE Virtualization v1.6.x enables CPU and memory hot-plugging for virtual machines through KubeVirt’s LiveMigrate workload update strategy. However, upgrading the KubeVirt operator triggers simultaneous live migrations of all running virtual machines to immediately update their virt-launcher pods. This mass migration can overwhelm cluster resources and degrade performance.

To prevent this issue, temporarily disable the LiveMigrate workload update method before upgrading and re-enable it once the upgrade completes. Virtual machines will then migrate naturally during node upgrades, allowing the virt-launcher image to update gradually.

Starting with v1.8.0, this process is handled automatically. The workaround is only necessary when upgrading to a version earlier than v1.8.0.

Pre-upgrade workaround steps

  1. Disable the LiveMigrate workload update method.

       kubectl patch kubevirt kubevirt -n harvester-system --type=merge --patch-file=/dev/stdin <<EOF
   spec:
     workloadUpdateStrategy:
       workloadUpdateMethods: []
   EOF

  2. Edit the harvester ManagedChart resource to prevent drift warnings.

       kubectl edit managedchart.management.cattle.io harvester -n fleet-local

  3. Add the following entry to the spec.diff.comparePatches field:

       spec:
     diff:
       comparePatches:
       # Keep existing entries
       - apiVersion: kubevirt.io/v1
         jsonPointers:
         - /spec/workloadUpdateStrategy/workloadUpdateMethods
         kind: KubeVirt
         name: kubevirt

    You must preserve the existing entries.

  4. Verify that the ManagedChart resource is ready.

       kubectl get managedchart.management.cattle.io harvester -n fleet-local

With these changes, virtual machines will not be scheduled for live migration because of virt-launcher pod image mismatches during the upgrade.

CPU and memory hot-plugging is temporarily unavailable during the upgrade. For more information about decoupling this feature from workload update flags, see KubeVirt issue #17329.

Post-upgrade workaround steps

After the upgrade completes successfully, restore the LiveMigrate workload update method to re-enable CPU and memory hot-plugging.

  1. Re-enable the LiveMigrate workload update method.

       kubectl patch kubevirt kubevirt -n harvester-system --type=merge --patch-file=/dev/stdin <<EOF
   spec:
     workloadUpdateStrategy:
       workloadUpdateMethods:
       - LiveMigrate
   EOF

  2. Edit the harvester ManagedChart resource.

       kubectl edit managedchart.management.cattle.io harvester -n fleet-local

  3. Remove the entry you added to the spec.diff.comparePatches field.

  4. Verify that the LiveMigrate method has been restored.

       kubectl get kubevirt kubevirt -n harvester-system -o yaml | grep -A 3 workloadUpdateStrategy

    Expected output:

       workloadUpdateStrategy:
     workloadUpdateMethods:
     - LiveMigrate

Related issue: #10349