gs_probackup对gs_basebackup的增强

• 支持PTRACK增量备份
• 支持增量恢复：基于页级增量恢复
• Merge合并：合并增量备份
• 数据备份有效性自动检测，无需实际的数据恢复
• 数据库实例checksum检测(需要打开checksum）
• 备份保留策略：按时间或数量保留WAL归档及备份
• 多线程并行化：支持backup、restore、merge等命令
• 支持远程操作：支持SSH方式远程备份或恢复实例
• 支持从standby备份，避免主库额外负载压力
• 数据存储在外部目录：与PGDATA保持独立
• 备份元数据管理：命令配置持久化到文件

备份支持FULL全备和PTRACK增量备份两种模式

系统环境：

[omm@opengaussdb ~]$ cat /etc/redhat-release

CentOS Linux release 7.6.1810 (Core)

集群环境（单机）：

[omm@opengaussdb ~]$ gs_om -t status --detail

[   Cluster State   ]

cluster_state   : Normal

redistributing  : No

current_az      : AZ_ALL

[  Datanode State   ]

node       node_ip         port      instance                   state

-----------------------------------------------------------------------

1  opengaussdb 192.192.103.134 15400  6001 /openGauss/data/dn   P Primary Normal

一、备份设置

1.初始化备份路径

gs_probackup init -B backup-path

例:

[omm@opengaussdb ~]$ gs_probackup  init -B /openGauss/backup/backup

INFO: Backup catalog '/openGauss/backup/backup' successfully inited

2.添加备份实例

gs_probackup add-instance -B backup-path

例：

[omm@opengaussdb ~]$ gs_probackup add-instance -B /openGauss/backup/backup -D /openGauss/data/dn --instance=instance_local

INFO: Instance 'instance_local' successfully inited

[omm@opengaussdb data]$ ls -ll /openGauss/backup/backup/backups/

total 0

drwx------ 4 omm dbgrp 59 Oct 30 11:56 instance_local

[omm@opengaussdb data]$ ls -ll /openGauss/backup/backup/wal/

total 0

drwx------ 2 omm dbgrp 6 Oct 30 11:30 instance_local

# 一个实例对应一个文件夹，instance_local是自定义的备份目录下的实例名，不同实例名称不同。

3.查看备份配置

[omm@opengaussdb ~]$ gs_probackup show-config -B /openGauss/backup/backup --instance instance_local --format json

{

    "pgdata": "/openGauss/data/dn",

    "system-identifier": "5932367421433466834",

    "pgdatabase": "omm",

    "archive-timeout": "5min",

    "log-level-console": "LOG",

    "log-level-file": "OFF",

    "log-filename": "pg_probackup.log",

    "log-rotation-size": "0TB",

    "log-rotation-age": "0d",

    "retention-redundancy": "0",

    "retention-window": "0",

    "wal-depth": "0",

    "compress-algorithm": "none",

    "compress-level": "1",

    "remote-proto": "ssh",

    "enable-dss": "false",

    "instance-id": "-1"

}

4.修改系统参数

 vi /etc/sysctl.conf 的以下参数

kernel.sem = 50100 128256000 50100 2560

sysctl -p生效

5.修改数据库参数

执行备份前需要开启数据库相关参数enable_cbm_tracking，否则增量备份失败，如下

[omm@opengaussdb ~]$ gs_guc reload -D /openGauss/data/dn/ -c "enable_cbm_tracking=on"

The gs_guc run with the following arguments: [gs_guc -D /openGauss/data/dn/ -c enable_cbm_tracking=on reload ].

NOTICE: Turn on cbm tracking function.

expected instance path: [/openGauss/data/dn/postgresql.conf]

gs_guc reload: enable_cbm_tracking=on: [/openGauss/data/dn/postgresql.conf]

server signaled

Total instances: 1. Failed instances: 0.

Success to perform gs_guc!

二、执行备份

1.执行全量备份

gs_probackup backup -B /openGauss/backup/backup --instance instance_local -b full -d postgres -p 15400

例：

[omm@opengaussdb ~]$ gs_probackup backup -B /openGauss/backup/backup --instance instance_local -b full -d postgres -p 15400

INFO: Backup start, gs_probackup version: 2.4.2, instance: instance_local, backup ID: T4XETD, backup mode: FULL, wal mode: STREAM, remote: false, compress-algorithm: none, compress-level: 1

LOG: Backup destination is initialized

LOG: This openGauss instance was initialized with data block checksums. Data block corruption will be detected

LOG: Database backup start

LOG: started streaming WAL at 0/66000000 (timeline 1)

[2025-10-30 11:53:38]: check identify system success

[2025-10-30 11:53:38]: send START_REPLICATION 0/66000000 success

[2025-10-30 11:53:38]: keepalive message is received

[2025-10-30 11:53:38]: keepalive message is received

INFO: PGDATA size: 1573MB

INFO: Start transferring data files

LOG: Creating page header map "/openGauss/backup/backup/backups/instance_local/T4XETD/page_header_map"

[2025-10-30 11:53:43]: keepalive message is received

INFO: Data files are transferred, time elapsed: 6s

INFO: wait for pg_stop_backup()

INFO: pg_stop backup() successfully executed

LOG: stop_lsn: 0/660001E8

LOG: Looking for LSN 0/660001E8 in segment: 000000010000000000000066

LOG: Found WAL segment: /openGauss/backup/backup/backups/instance_local/T4XETD/database/pg_xlog/000000010000000000000066

LOG: Thread [0]: Opening WAL segment "/openGauss/backup/backup/backups/instance_local/T4XETD/database/pg_xlog/000000010000000000000066"

LOG: Found LSN: 0/660001E8

LOG: finished streaming WAL at 0/67000000 (timeline 1)

LOG: Getting the Recovery Time from WAL

LOG: Thread [0]: Opening WAL segment "/openGauss/backup/backup/backups/instance_local/T4XETD/database/pg_xlog/000000010000000000000066"

INFO: Syncing backup files to disk

INFO: Backup files are synced, time elapsed: 19s

INFO: Validating backup T4XETD

INFO: Backup T4XETD data files are valid

INFO: Backup T4XETD resident size: 1590MB

INFO: Backup T4XETD completed

查看备份

gs_probackup show -B /openGauss/backup/backup/

[omm@opengaussdb ~]$  gs_probackup show -B /openGauss/backup/backup/

BACKUP INSTANCE 'instance_local'

=======================================================================

 Instance        Version  ID      Recovery Time           Mode  WAL Mode  TLI  Time    Data   WAL  Zratio  Start LSN   Stop LSN    Type  Status

=======================================================================

 instance_local  9.2      T4XETD  2025-10-30 11:53:44+08  FULL  STREAM    1/0   34s  1574MB  16MB    1.00  0/66000028  0/660001E8  FILE  OK  

2.增量备份

gs_probackup backup -B /openGauss/backup/backup --instance instance_local -b ptrack -d postgres -p 15400

例：

[omm@opengaussdb ~]$ gs_probackup backup -B /openGauss/backup/backup --instance instance_local -b ptrack -d postgres -p 15400

INFO: Backup start, gs_probackup version: 2.4.2, instance: instance_local, backup ID: T4XEYK, backup mode: PTRACK, wal mode: STREAM, remote: false, compress-algorithm: none, compress-level: 1

LOG: Backup destination is initialized

LOG: This openGauss instance was initialized with data block checksums. Data block corruption will be detected

LOG: Database backup start

LOG: Latest valid FULL backup: T4XETD

INFO: Parent backup: T4XETD

LOG: started streaming WAL at 0/68000000 (timeline 1)

[2025-10-30 11:56:46]: check identify system success

[2025-10-30 11:56:46]: send START_REPLICATION 0/68000000 success

[2025-10-30 11:56:46]: keepalive message is received

[2025-10-30 11:56:46]: keepalive message is received

INFO: PGDATA size: 1573MB

LOG: Current tli: 1

LOG: Parent start_lsn: 0/66000028

LOG: start_lsn: 0/68000028

INFO: Extracting pagemap of changed blocks

INFO: change bitmap start lsn location is 0/66000028

INFO: change bitmap end lsn location is 00000000/68000028

INFO: Pagemap successfully extracted, time elapsed: 0 sec

INFO: Start transferring data files

LOG: Creating page header map "/openGauss/backup/backup/backups/instance_local/T4XEYK/page_header_map"

INFO: Data files are transferred, time elapsed: 1s

INFO: wait for pg_stop_backup()

INFO: pg_stop backup() successfully executed

LOG: stop_lsn: 0/680001E8

LOG: Looking for LSN 0/680001E8 in segment: 000000010000000000000068

LOG: Found WAL segment: /openGauss/backup/backup/backups/instance_local/T4XEYK/database/pg_xlog/000000010000000000000068

LOG: Thread [0]: Opening WAL segment "/openGauss/backup/backup/backups/instance_local/T4XEYK/database/pg_xlog/000000010000000000000068"

LOG: Found LSN: 0/680001E8

LOG: finished streaming WAL at 0/69000000 (timeline 1)

LOG: Getting the Recovery Time from WAL

LOG: Thread [0]: Opening WAL segment "/openGauss/backup/backup/backups/instance_local/T4XEYK/database/pg_xlog/000000010000000000000068"

INFO: Syncing backup files to disk

INFO: Backup files are synced, time elapsed: 15s

INFO: Validating backup T4XEYK

INFO: Backup T4XEYK data files are valid

INFO: Backup T4XEYK resident size: 274MB

INFO: Backup T4XEYK completed

查看备份

[omm@opengaussdb ~]$  gs_probackup show -B /openGauss/backup/backup/

BACKUP INSTANCE 'instance_local'

=======================================================================

 Instance        Version  ID      Recovery Time           Mode    WAL Mode  TLI  Time    Data   WAL  Zratio  Start LSN   Stop LSN    Type  Status

=======================================================================

 instance_local  9.2      T4XEYK  2025-10-30 11:56:47+08  PTRACK  STREAM    1/1   22s   258MB  16MB    1.00  0/68000028  0/680001E8  FILE  OK

 instance_local  9.2      T4XETD  2025-10-30 11:53:44+08  FULL    STREAM    1/0   34s  1574MB  16MB    1.00  0/66000028  0/660001E8  FILE  OK

三、测试数据恢复

1.恢复到指定时间

[omm@opengaussdb ~]$  gs_probackup show -B /openGauss/backup/backup/

BACKUP INSTANCE 'instance_local'

==================================================================================================================================================

 Instance        Version  ID      Recovery Time           Mode    WAL Mode  TLI  Time    Data   WAL  Zratio  Start LSN   Stop LSN    Type  Status

==================================================================================================================================================

 instance_local  9.2      T4XEYK  2025-10-30 11:56:47+08  PTRACK  STREAM    1/1   22s   258MB  16MB    1.00  0/68000028  0/680001E8  FILE  OK

 instance_local  9.2      T4XETD  2025-10-30 11:53:44+08  FULL    STREAM    1/0   34s  1574MB  16MB    1.00  0/66000028  0/660001E8  FILE  OK  

1.1 停止数据库

[omm@opengaussdb ~]$ gs_ctl stop -D /openGauss/data/dn

[2025-10-30 12:31:00.230][8621][][gs_ctl]: gs_ctl stopped ,datadir is /openGauss/data/dn

waiting for server to shut down........................... done

server stopped

1.2 模拟数据丢失

移走数据目录

[omm@opengaussdb data]$ mv dn dnbak

1.3 恢复到指定时间点

指定要恢复到的时间，当前只能指定备份中的recovery-time。不支持随意指定时间点

[omm@opengaussdb ~]$ gs_probackup restore -B /openGauss/backup/backup/ --instance instance_local --recovery-target-time='2025-10-30 11:53:44'

LOG: Restore begin.

LOG: there is no file tablespace_map

LOG: check tablespace directories of backup T4XETD

LOG: check external directories of backup T4XETD

INFO: Validating backup T4XETD

INFO: Backup T4XETD data files are valid

LOG: Thread [1]: Opening WAL segment "/openGauss/backup/backup/backups/instance_local/T4XETD/database/pg_xlog/000000010000000000000066"

INFO: Backup validation completed successfully on time 2025-10-30 11:53:44+08, xid 52249 and LSN 0/660001E8

INFO: Backup T4XETD is valid.

INFO: Restoring the database from backup at 2025-10-30 11:53:37+08

LOG: there is no file tablespace_map

LOG: Restore directories and symlinks... in /openGauss/data/dn

INFO: Start restoring backup files. PGDATA size: 1589MB

LOG: Start thread 1

INFO: Backup files are restored. Transfered bytes: 1589MB, time elapsed: 5s

INFO: Restore incremental ratio (less is better): 100% (1589MB/1589MB)

INFO: Syncing restored files to disk

INFO: Restored backup files are synced, time elapsed: 1m:20s

LOG: ----------------------------------------

LOG: creating recovery.conf

INFO: Restore of backup T4XETD completed..

1.4 检查数据恢复成功

[omm@opengaussdb ~]$ gs_ctl start -D /openGauss/data/dn

[omm@opengaussdb ~]$ gsql postgres

gsql ((openGauss 5.0.3 build 89d144c2) compiled at 2024-07-31 20:59:31 commit 0 last mr  )

Non-SSL connection (SSL connection is recommended when requiring high-security)

Type "help" for help.

openGauss=# \c testdb test

Password for user test:

Non-SSL connection (SSL connection is recommended when requiring high-security)

You are now connected to database "testdb" as user "test".

testdb=> select count(*) from t1;

 count

-------

   200

(1 row)

2.恢复到最新时间点

2.1 停止数据库

[omm@opengaussdb ~]$ gs_ctl stop -D /openGauss/data/dn

[2025-10-30 13:36:21.893][16504][][gs_ctl]: gs_ctl stopped ,datadir is /openGauss/data/dn

waiting for server to shut down.................................... done

server stopped

2.2 模拟数据丢失

移走数据目录

[omm@opengaussdb data]$ mv dn dnbak

2.3 恢复到最新数据

恢复是不指定--recovery-target-time，数据库恢复将apply已有的wal文件，将数据恢复到wal日志中记录的最新数据。

[omm@opengaussdb ~]$ gs_probackup restore -B /openGauss/backup/backup/ --instance instance_local

LOG: Restore begin.

LOG: there is no file tablespace_map

LOG: check tablespace directories of backup T4XEYK

LOG: check external directories of backup T4XEYK

INFO: Validating parents for backup T4XEYK

INFO: Validating backup T4XETD

INFO: Backup T4XETD data files are valid

INFO: Validating backup T4XEYK

INFO: Backup T4XEYK data files are valid

LOG: Thread [1]: Opening WAL segment "/openGauss/backup/backup/backups/instance_local/T4XEYK/database/pg_xlog/000000010000000000000068"

INFO: Backup T4XEYK WAL segments are valid

INFO: Backup T4XEYK is valid.

INFO: Restoring the database from backup at 2025-10-30 11:56:44+08

LOG: there is no file tablespace_map

LOG: Restore directories and symlinks... in /openGauss/data/dn

INFO: Start restoring backup files. PGDATA size: 1589MB

LOG: Start thread 1

INFO: Backup files are restored. Transfered bytes: 1589MB, time elapsed: 3s

INFO: Restore incremental ratio (less is better): 100% (1589MB/1589MB)

INFO: Syncing restored files to disk

INFO: Restored backup files are synced, time elapsed: 1m:21s

INFO: Restore of backup T4XEYK completed.

2.4 检查数据恢复成功

[omm@opengaussdb ~]$ gsql postgres

gsql ((openGauss 5.0.3 build 89d144c2) compiled at 2024-07-31 20:59:31 commit 0 last mr  )

Non-SSL connection (SSL connection is recommended when requiring high-security)

Type "help" for help.

openGauss=# \c testdb test

Password for user test:

Non-SSL connection (SSL connection is recommended when requiring high-security)

You are now connected to database "testdb" as user "test".

testdb=> select count(*) from t1;

 count

-------

  1000

(1 row)

四、删除备份

1.查看备份

[omm@opengaussdb ~]$ gs_probackup show -B /openGauss/backup/backup/

BACKUP INSTANCE 'instance_local'

===========================================================

 Instance        Version  ID      Recovery Time           Mode    WAL Mode  TLI  Time    Data   WAL  Zratio  Start LSN   Stop LSN    Type  Status

===========================================================

 instance_local  9.2      T4XEYK  2025-10-30 11:56:47+08  PTRACK  STREAM    1/1   22s   258MB  16MB    1.00  0/68000028  0/680001E8  FILE  OK

 instance_local  9.2      T4XETD  2025-10-30 11:53:44+08  FULL    STREAM    1/0   34s  1574MB  16MB    1.00  0/66000028  0/660001E8  FILE  OK

2.删除备份

[omm@opengaussdb ~]$ gs_probackup delete -B /openGauss/backup/backup/ --instance=instance_local -i T4XEYK

LOG: Backups based on T4XEYK will be deleted

LOG: Backup T4XEYK will be deleted

INFO: Resident data size to free by delete of backup T4XEYK : 274MB

INFO: Delete: T4XEYK 2025-10-30 11:56:47+08

#-i 指定备份集的ID

3.查看删除后的备份

[omm@opengaussdb ~]$ gs_probackup show -B /openGauss/backup/backup/

BACKUP INSTANCE 'instance_local'

===========================================================

 Instance        Version  ID      Recovery Time           Mode  WAL Mode  TLI  Time    Data   WAL  Zratio  Start LSN   Stop LSN    Type  Status

===========================================================

 instance_local  9.2      T4XETD  2025-10-30 11:53:44+08  FULL  STREAM    1/0   34s  1574MB  16MB    1.00  0/66000028  0/660001E8  FILE  OK