KDMP backups

KDMP backups are generic backups that Portworx Backup supports utilizing the KDMP driver. If you have deployed Portworx Backup without Portworx Enterprise, then a KDMP backup get triggered.

If you do not want to use cloud native drivers and prefer to use Portworx Backup's KDMP driver, you need to set the backup type as generic in kdmp-config ConfigMap that resides in kube-system namespace of your application cluster.

KDMP backup types

Portworx backup supports two variants of KDMP backups as outlined in the following sections:

1. KDMP with local snapshot

   In Portworx Backup, KDMP driver takes snapshots and later offloads or uploads these snapshots to S3 compliant object store.

   In this case Portworx Backup performs the following tasks:

   a. Takes available snapshot storageclass information
   b. Creates a snapshot
   c. Uses KDMP driver to attach the mount point of the PV on the worker node
   d. Copies the file system contents to object storage
   

   

2. Direct KDMP/Generic

   In this backup type, Portworx Backup utilizes the KDMP driver to attach the mount point of the PV on the worker node and copies the file system contents to object storage.

   

When a KDMP backup occurs

Portworx Backup triggers a KDMP backup in the following scenarios:

1. Portworx Backup without Portworx Enterprise:

   a. If the storage provider does not support CSI snapshots

   b. If the storage provider supports CSI snapshots and if you want to offload the backup to S3 object store along with the selection of volume snapshot class during creation of backup.

   KDMP backup matrix

   Following table provides KDMP backup matrix when the storage provider supports CSI snapshot:

Cloud storage provider	Storage type	Backup mechanism
AWS	EBS	Cloud-native driver
	EFS	KDMP with local snapshot
Azure	Disk	Cloud-native driver
	File share	Direct KDMP
IBM	IBM storage	KDMP with local snapshot (if you select Offload to S3 with CSI snapshot class option in the user interface) Native CSI backup (if you do not select Offload to S3 with CSI snapshot class) Direct KDMP (if you have set BACKUP_TYPE: "Generic" in the kdmp-config configmap
GCP	Google disk	Cloud-native driver
	Google file	Direct KDMP
OCP	OCP storage	Direct KDMP

2. Regardless of the storage provider and whether or not the CSI snapshot is supported by these storage providers, if you update the parameter BACKUP_TYPE: "Generic" in the kdmp-config ConfigMap. Refer to this topic to learn more on how to update kdmp-config ConfigMap.

3. For some on premises (for example, Flash Array) or on cloud storage providers without CSI snapshot support, direct KDMP backup gets triggered and the Portworx Backup user interface does not provide Offload to CSI snapshot backup location option. This is the default behavior of Portworx Backup.

4. Cross-cloud and cross-region backups and restores with Portworx Backup.

5. 3-2-1 backups

Create a KDMP backup

To trigger a KDMP backup for the storage provider that supports CSI snapshot class, ensure that you select the following options in the Create Backup window:

• CSI snapshot class

• Offload CSI snapshots to backup location

  
  note

  Offload CSI snapshots to backup location option appears in the Create Backup window only if the storage provider supports CSI snapshot mechanism.

KDMP-config ConfigMap

Exclude files and directories

Consider a scenario where you have directories or files (hidden, read-only, logs) with different extensions and you wish to exclude those from being included in the backup. Portworx Backup now allows you to exclude such directories or files from being backed up. To exclude or omit such directories and files from being included in the backup, you need to update the kdmp-config ConfigMap with the following steps:

1. Run the following command to edit the kdmp-config ConfigMap:

   kubectl edit cm kdmp-config -n kube-system

2. (Optional, execute the following command only if you want direct KDMP backup in cases where it is native CSI or cloud native backup)

   Add the following parameter in the data section:

   BACKUP_TYPE: "Generic"
   note

   You must add the above parameter on the source application cluster where you perform backup. It is not mandatory to add the parameter on the destination cluster where you restore.

3. Add the following key-value pair:

   KDMP_EXCLUDE_FILE_LIST: |
     <storageclassName1>=<dir-list>,<file-list1>,....
     <storageclassName2>=<dir-list>,<file-list1>,....

   Example

   apiVersion: v1
   data:
   ENABLE_PX_GENERIC_BACKUP: "true"
   KDMP_BACKUP_JOB_LIMIT: "5"
   KDMP_DELETE_JOB_LIMIT: "5"
   KDMP_EXCLUDE_FILE_LIST: |
     px-db=dir1,file1,dir2
     mysql=dir1,file1,dir2
   KDMP_KOPIAEXECUTOR_IMAGE: kopiaexecutor:1.2.10
   KDMP_KOPIAEXECUTOR_IMAGE_SECRET: ""
   KDMP_KOPIAEXECUTOR_LIMIT_CPU: "0.2"
   KDMP_KOPIAEXECUTOR_LIMIT_MEMORY: 1Gi
   KDMP_KOPIAEXECUTOR_REQUEST_CPU: "0.1"
   KDMP_KOPIAEXECUTOR_REQUEST_MEMORY: 700Mi
   KDMP_MAINTENANCE_JOB_LIMIT: "5"
   KDMP_RESTORE_JOB_LIMIT: "5"
   SNAPSHOT_TIMEOUT: ""
   kind: ConfigMap
   metadata:
   name: kdmp-config
   namespace: kube-system
   resourceVersion: "7120917"
   uid: bd1e505b-0191-414c-aedb-0b1d413d4002

   You can exclude directories and files from being backed up for both direct KDMP and KDMP with local snapshot backup types. For the latter, you cannot exclude any directories or files from being backed up if the local snapshot already exists even after kdmp-config ConfigMap is updated with the key-value pair specified above. If you want to restore with excluded directories or files, delete the existing local snapshot and then restore the backup.

KDMP backups and restores with large volume PVCs

KDMP job pods consume increased amounts of memory for the large volume PVC backup and restore operations to backup locations. As a result, you may see out of memory alerts or a failure of the kdmp job pods that run on each application or target cluster. During these scenarios, Portworx by Pure Storage recommends to increase the CPU and memory limit related parameters in the kdmp-config ConfigMap which resides in the kube-system namespace on the target cluster or application cluster.

Update the following parameters in kdmp-config ConfigMap to resolve OOM errors:

KDMP_KOPIAEXECUTOR_LIMIT_CPU: "0.4"
KDMP_KOPIAEXECUTOR_REQUEST_CPU: "0.2"
KDMP_KOPIAEXECUTOR_REQUEST_MEMORY: 2Gi
KDMP_KOPIAEXECUTOR_LIMIT_MEMORY: 4Gi

Related topics:

• Create backup

• Cross-region backups

• Backup CRDs

• Backup CSI snapshots

• Large resource backups

• Install from IBM cloud