This is unreleased documentation for SUSE® Virtualization v1.9 (Dev).

Creating an RKE2 Kubernetes Cluster

You can now Provision new nodes and create a cluster using RKE2/K3s on top of the SUSE Virtualization cluster in SUSE Rancher Prime using the built-in Harvester Node Driver.

rke2-cluster

  • VLAN network is required for Harvester Node Driver.

  • Provisioning RKE2 Kubernetes clusters involves configuring the IP address of the underlying virtual machines. You can do this using a DHCP server on the VLAN network that the virtual machines are attached to. If such a server does not exist on the network, you can use the Managed DHCP feature to configure the IP address.

  • Harvester Node Driver only supports cloud images.

  • For the port requirements of the guest clusters deployed within SUSE Virtualization, please refer to the doc here.

  • For RKE2 with Harvester Cloud Provider support matrix, please refer to the website here.

Backward Compatibility Notice

Please note a known backward compatibility issue if you’re using the Harvester Cloud Provider version v0.2.2 or higher. If your SUSE Virtualization version is below v1.2.0 and you intend to use newer RKE2 versions (i.e., >= v1.26.6+rke2r1, v1.25.11+rke2r1, v1.24.15+rke2r1), it is essential to upgrade your SUSE Virtualization cluster to v1.2.0 or a higher version before proceeding with the upgrade of the guest Kubernetes cluster or Harvester Cloud Provider.

For a detailed support matrix, please refer to the Harvester CCM & CSI Driver with RKE2 Releases section of the official website.

Create your cloud credentials

  1. Click ☰ > Cluster Management.

  2. Click Cloud Credentials.

  3. Click Create.

  4. Click Harvester.

  5. Enter your cloud credential name

  6. Select "Imported Harvester Cluster".

  7. Click Create.

create-harvester-cloud-credentials

Create RKE2 Kubernetes cluster

Users can create a RKE2 Kubernetes cluster from the Cluster Management page via the Harvester Node Driver.

  1. Select Clusters menu.

  2. Click Create button.

  3. Select Harvester node driver.

  4. Select a Cloud Credential.

  5. Enter Cluster Name (required).

  6. Enter Namespace (required).

  7. Enter Image (required).

  8. Enter Network Name (required).

  9. Enter SSH User (required).

  10. (optional) Configure the Show Advanced > User Data to install the required packages of VM.

#cloud-config
package_update: true
packages:
  - qemu-guest-agent
runcmd:
  - - systemctl
    - enable
    - '--now'
    - qemu-guest-agent.service

  1. In the Basics section, use the Kubernetes Version drop-down menu to select your preferred Kubernetes distribution (RKE2 or K3s) and its corresponding version.

  2. Configure cluster-specific settings. (Loads dynamically based on your selection.)

  3. Click Create.

Calico and Canal networks require the iptables or xtables-nft package to be installed on the node. For more information, see Canal and IP exhaustion in the RKE2 documentation.

Machine Pools (Basic)

create-rke2-harvester-cluster-1

Machine Pools (Advanced)

create-rke2-harvester-cluster-2

Kubernetes Distribution & Version

create-rke2-harvester-cluster-rke2-k3s-selection

K3s Settings (Only applies if K3s is selected)

create-rke2-harvester-cluster-3

  • RKE2 v1.21.5+rke2r2 or above provides a built-in Harvester Cloud Provider and guest CSI driver integration.

  • Only imported SUSE Virtualization clusters are supported by the Harvester Node Driver.

Add node affinity

The Harvester Node Driver now supports scheduling a group of machines to particular nodes through the node affinity rules, which can provide high availability and better resource utilization.

Node affinity can be added to the machine pools during the cluster creation:

  1. Click the Show Advanced button and click the Add Node Selector affinity-add-node-selector

  2. Set priority to Required if you wish the scheduler to schedule the machines only when the rules are met.

  3. Click Add Rule to specify the node affinity rules, e.g., for the topology spread constraints use case, you can add the region and zone labels as follows:

    key: topology.kubernetes.io/region
operator: in list
values: us-east-1
---
key: topology.kubernetes.io/zone
operator: in list
values: us-east-1a
    affinity-add-rules

Add workload affinity

The workload affinity rules allow you to constrain which nodes your machines can be scheduled on based on the labels of workloads (VMs and Pods) already running on these nodes, instead of the node labels.

Workload affinity rules can be added to the machine pools during the cluster creation:

  1. Select Show Advanced and choose Add Workload Selector. affinity-add-workload-selector

  2. Select Type, Affinity or Anti-Affinity.

  3. Select Priority. Prefered means it’s an optional rule, and Required means a mandatory rule.

  4. Select the namespaces for the target workloads.

  5. Select Add Rule to specify the workload affinity rules.

  6. Set Topology Key to specify the label key that divides SUSE Virtualization hosts into different topologies.

See the Kubernetes Pod Affinity and Anti-Affinity Documentation for more details.

Update RKE2 Kubernetes cluster

  1. Click ☰ > Cluster Management.

  2. Select Clusters menu.

  3. Find the target cluster.

  4. Click the three-dot menu on the right and select Edit Config.

The non-grayed fields of the RKE2 machine pool represent the Harvester virtual machine configurations. Any modifications to these fields will trigger node reprovisioning.

rke2-harvester-fields

Do not change the first(primary) Network. By default, RKE2/K3s relies on it to boot the cluster. If different Machine Pools use different networks, the cluster may break.

Using Harvester RKE2 node driver in air gapped environment

RKE2 provisioning relies on the qemu-guest-agent package to get the IP of the virtual machine.

Calico and Canal require the iptables or xtables-nft package to be installed on the node.

However, it may not be feasible to install packages in an air gapped environment.

You can address the installation constraints with the following options:

  • Option 1. Use a VM image preconfigured with required packages (e.g., iptables, qemu-guest-agent).

  • Option 2. Go to Show Advanced > User Data to allow VMs to install the required packages via an HTTP(S) proxy.

Example user data in SUSE Virtualization node template:

#cloud-config
apt:
  http_proxy: http://192.168.0.1:3128
  https_proxy: http://192.168.0.1:3128

Guest cluster provisioning failure after re-registering SUSE Virtualization cluster

Guest cluster provisioning may fail with an error similar to the following:

clusters.management.cattle.io "<old-cluster-id>" not found

This can occur if you remove a SUSE Virtualization cluster from Rancher and later re-register the same cluster, leaving behind stale cloud credentials.

For more information, see Stale cloud credentials after cluster registration.