Tag Archives: PostgreSQL 17

Incremental backups in PostgreSQL 17

PostgreSQL, , , , , , , , , , , , , , , , ,

Incremental Backups in PostgreSQL 17

Table of Contents

1. WAL Summarization (Similar to Oracle Block Change Tracking)

PostgreSQL 17 introduces a background process called the walsummarizer, which tracks which blocks in data files have changed over ranges of WAL (Write Ahead Log) records.
This feature enables efficient incremental backups by identifying only the blocks changed since the previous backup.
WAL summaries are stored in: PGDATA/pg_wal/summaries/

New Configuration Settings:
1. summarize_wal (boolean) – Enables or disables the summarizer. Default: off.
2. wal_summary_keep_time – Duration to retain summary files. Default: 10 days.

2. Incremental File System Backups with pg_basebackup

-- PostgreSQL 17 adds an --incremental option to pg_basebackup.
-- When WAL summarization is enabled, you can create backups that include only the changed blocks since the previous full or incremental backup.
-- A new utility (e.g., pg_combinebackup) is used to merge full and incremental backups during restoration.

3. New Functions and Utilities for WAL Summarization

 	pg_available_wal_summaries()
 	pg_wal_summary_contents()
 	pg_get_wal_summarizer_state()

Tool: pg_walsummary — A CLI utility to inspect WAL summary files.

4. How to Enable and Use summarize_wal

4.1. Check Current Status

postgres=# SHOW summarize_wal;
 summarize_wal
---------------
 off
(1 row)

postgres=#

[postgres@lxicbpgdsgv01 ~]$ ps -ef | grep postgres
postgres    1390       1  0 12:15 ?        00:00:00 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
postgres    1456    1390  0 12:15 ?        00:00:00 postgres: logger
postgres    1466    1390  0 12:15 ?        00:00:00 postgres: checkpointer
postgres    1467    1390  0 12:15 ?        00:00:00 postgres: background writer
postgres    1479    1390  0 12:15 ?        00:00:00 postgres: walwriter
postgres    1480    1390  0 12:15 ?        00:00:00 postgres: autovacuum launcher
postgres    1481    1390  0 12:15 ?        00:00:00 postgres: archiver
postgres    1484    1390  0 12:15 ?        00:00:00 postgres: logical replication launcher
root        3205    3150  0 12:48 pts/0    00:00:00 su - postgres
postgres    3212    3205  0 12:48 pts/0    00:00:00 -bash
postgres    3290    3212 99 12:49 pts/0    00:00:00 ps -ef
postgres    3291    3212  0 12:49 pts/0    00:00:00 grep --color=auto postgres
[postgres@lxicbpgdsgv01 ~]$

4.2. Enable WAL Summarization

postgres=# ALTER SYSTEM SET summarize_wal = 'on';
ALTER SYSTEM
postgres=#

4.3. Reload PostgreSQL Configuration

postgres=# SELECT pg_reload_conf();
 pg_reload_conf
----------------
 t
(1 row)

postgres=#

4.4. Confirm summarize_wal is Enabled

postgres=# SHOW summarize_wal;
 summarize_wal
---------------
 on <----
(1 row)

postgres=#

[postgres@lxicbpgdsgv01 ~]$ ps -ef | grep postgres
postgres    1390       1  0 12:15 ?        00:00:00 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
postgres    1456    1390  0 12:15 ?        00:00:00 postgres: logger
postgres    1466    1390  0 12:15 ?        00:00:00 postgres: checkpointer
postgres    1467    1390  0 12:15 ?        00:00:00 postgres: background writer
postgres    1479    1390  0 12:15 ?        00:00:00 postgres: walwriter
postgres    1480    1390  0 12:15 ?        00:00:00 postgres: autovacuum launcher
postgres    1481    1390  0 12:15 ?        00:00:00 postgres: archiver
postgres    1484    1390  0 12:15 ?        00:00:00 postgres: logical replication launcher
root        3205    3150  0 12:48 pts/0    00:00:00 su - postgres
postgres    3212    3205  0 12:48 pts/0    00:00:00 -bash
postgres    3445    1390  0 12:51 ?        00:00:00 postgres: walsummarizer <-------
postgres    3537    3212  0 12:53 pts/0    00:00:00 ps -ef
postgres    3538    3212  0 12:53 pts/0    00:00:00 grep --color=auto postgres
[postgres@lxicbpgdsgv01 ~]$

4.5. List Available WAL Summaries (Ordered by Start LSN)

postgres=# SELECT * FROM pg_available_wal_summaries() ORDER BY start_lsn;
 tli | start_lsn  |  end_lsn
-----+------------+------------
   1 | 0/95000028 | 0/9501BA38
   1 | 0/9501BA38 | 0/96000028
(2 rows)

postgres=#

postgres=# select pg_get_wal_summarizer_state();
  pg_get_wal_summarizer_state
--------------------------------
 (1,0/96000028,0/960000A0,3445)
(1 row)

postgres=#

postgres=# SELECT * FROM pg_available_wal_summaries() LIMIT 1;
 tli | start_lsn  |  end_lsn
-----+------------+------------
   1 | 0/95000028 | 0/9501BA38
(1 row)

postgres=#

postgres=# SELECT * FROM pg_wal_summary_contents(1, '0/95000028', '0/9501BA38');
 relfilenode | reltablespace | reldatabase | relforknumber | relblocknumber | is_limit_block
-------------+---------------+-------------+---------------+----------------+----------------
        1247 |         16542 |       16544 |             0 |             14 | f
        1249 |         16542 |       16544 |             0 |             54 | f
        1249 |         16542 |       16544 |             0 |             55 | f
        1259 |         16542 |       16544 |             0 |              1 | f
        2608 |         16542 |       16544 |             0 |             12 | f
        2610 |         16542 |       16544 |             0 |              1 | f
        2658 |         16542 |       16544 |             0 |             15 | f
        2659 |         16542 |       16544 |             0 |             10 | f
        2662 |         16542 |       16544 |             0 |              2 | f
        2663 |         16542 |       16544 |             0 |              1 | f
        2663 |         16542 |       16544 |             0 |              2 | f
        2673 |         16542 |       16544 |             0 |              4 | f
        2673 |         16542 |       16544 |             0 |              9 | f
        2674 |         16542 |       16544 |             0 |              7 | f
        2674 |         16542 |       16544 |             0 |              5 | f
        2678 |         16542 |       16544 |             0 |              1 | f
        2679 |         16542 |       16544 |             0 |              1 | f
        2703 |         16542 |       16544 |             0 |              2 | f
        2704 |         16542 |       16544 |             0 |              4 | f
        2704 |         16542 |       16544 |             0 |              1 | f
        3455 |         16542 |       16544 |             0 |              1 | f
       16558 |         16542 |       16544 |             0 |              0 | t
       16558 |         16542 |       16544 |             0 |              0 | f
       16561 |         16542 |       16544 |             0 |              0 | t
       16562 |         16542 |       16544 |             0 |              0 | t
       16562 |         16542 |       16544 |             0 |              0 | f
(26 rows)

postgres=#

5. Weekly Backup Strategy (PostgreSQL 17 / Oracle Style)

DayBackup TypeDescriptionBased On
SundayFull Backup (Level 0)Copies the entire database — all datafiles (similar to Oracle Level 0 backups).—
MondayIncremental BackupBacks up only datafile blocks changed since Sunday’s full backup.Sunday’s backup_manifest
TuesdayIncremental BackupBacks up only blocks changed since Monday’s incremental backup.Monday’s backup_manifest
WednesdayIncremental BackupBacks up only blocks changed since Tuesday’s incremental backup.Tuesday’s backup_manifest
ThursdayIncremental BackupBacks up only blocks changed since Wednesday’s incremental backup.Wednesday’s backup_manifest
FridayIncremental BackupBacks up only blocks changed since Thursday’s incremental backup.Thursday’s backup_manifest
SaturdayIncremental BackupBacks up only blocks changed since Friday’s incremental backup.Friday’s backup_manifest
 
# Sunday - Full backup
nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/Sun_Full -l "This is Sunday Full backup (Level 0)" -Ft -Xs -P -v > /pgBackup/pgsql17/backup/log/Sunday_Full.log 2>&1 &

# Monday - Incremental based on Sunday full
nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/Mon_Incr -l "This is Monday Incremental (Level 1)" -i /pgBackup/pgsql17/backup/Sun_Full/backup_manifest -Ft -Xs -P -v > /pgBackup/pgsql17/backup/log/Mon_Incr.log 2>&1 &

# Tuesday - Incremental based on Monday incremental
nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/Tue_Incr -l "This is Tuesday Incremental (Level 1)" -i /pgBackup/pgsql17/backup/Mon_Incr/backup_manifest -Ft -Xs -P -v > /pgBackup/pgsql17/backup/log/Tue_Incr.log 2>&1 &

# Wednesday - Incremental based on Tuesday incremental
nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/Wed_Incr -l "This is Wednesday Incremental (Level 1)" -i /pgBackup/pgsql17/backup/Tue_Incr/backup_manifest -Ft -Xs -P -v > /pgBackup/pgsql17/backup/log/Wed_Incr.log 2>&1 &

# Thursday - Incremental based on Wednesday incremental
nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/Thu_Incr -l "This is Thursday Cumulative" -i /pgBackup/pgsql17/backup/Wed_Incr/backup_manifest -Ft -Xs -P -v > /pgBackup/pgsql17/backup/log/Thu_Incr.log 2>&1 &

# Friday - Incremental based on Thursday incremental
nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/Fri_Incr -l "This is Friday Incremental (Level 1)" -i /pgBackup/pgsql17/backup/Thu_Incr/backup_manifest -Ft -Xs -P -v > /pgBackup/pgsql17/backup/log/Fri_Incr.log 2>&1 &

# Saturday - Incremental based on Friday incremental
nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/Sat_Incr -l "This is Saturday Incremental (Level 1)" -i /pgBackup/pgsql17/backup/Fri_Incr/backup_manifest -Ft -Xs -P -v > /pgBackup/pgsql17/backup/log/Sat_Incr.log 2>&1 &

6. Restore Incremental Path

A new utility (e.g., pg_combinebackup) is used to merge full and incremental backups during restoration.

To restore the full database as of Saturday, apply these backups in sequence:

  • Sunday (Full Backup)
  • Monday (Incremental)
  • Tuesday (Incremental)
  • Wednesday (Incremental)
  • Thursday (Incremental)
  • Friday (Incremental)
  • Saturday (Incremental)
# Incremental Restore Path
# Combine all backups to single directory /pgBackup/pgsql17/backup/combined_full_backup

# Extract first using tar command

for dir in Sun_Full Mon_Incr Tue_Incr Wed_Incr Thu_Incr Fri_Incr Sat_Incr
do
echo "Extracting in $dir ..."
cd /pgBackup/pgsql17/backup/$dir || continue
for f in *.tar; do
tar -xf "$f"
done
done

[postgres@lxicbpgdsgv01 ~]$ for dir in Sun_Full Mon_Incr Tue_Incr Wed_Incr Thu_Incr Fri_Incr Sat_Incr
do
echo "Extracting in $dir ..."
cd /pgBackup/pgsql17/backup/$dir || continue
for f in *.tar; do
tar -xf "$f"
done
done
Extracting in Sun_Full ...
tar: pg_stat: time stamp 2025-10-13 20:15:19 is 13627.701332157 s in the future
Extracting in Mon_Incr ...
tar: pg_stat: time stamp 2025-10-13 20:15:19 is 13627.308655658 s in the future
Extracting in Tue_Incr ...
tar: pg_stat: time stamp 2025-10-13 20:15:19 is 13626.906480756 s in the future
Extracting in Wed_Incr ...
tar: pg_stat: time stamp 2025-10-13 20:15:19 is 13626.534470296 s in the future
Extracting in Thu_Incr ...
tar: pg_stat: time stamp 2025-10-13 20:15:19 is 13626.217307586 s in the future
Extracting in Fri_Incr ...
tar: pg_stat: time stamp 2025-10-13 20:15:19 is 13626.03629929 s in the future
Extracting in Sat_Incr ...
tar: pg_stat: time stamp 2025-10-13 20:15:19 is 13625.670340241 s in the future
[postgres@lxicbpgdsgv01 Sat_Incr]$

# Combine backup to /pgBackup/pgsql17/backup/combined_full_backup

nohup pg_combinebackup /pgBackup/pgsql17/backup/Sun_Full \
/pgBackup/pgsql17/backup/Mon_Incr \
/pgBackup/pgsql17/backup/Tue_Incr \
/pgBackup/pgsql17/backup/Wed_Incr \
/pgBackup/pgsql17/backup/Thu_Incr \
/pgBackup/pgsql17/backup/Fri_Incr \
/pgBackup/pgsql17/backup/Sat_Incr \
-o /pgBackup/pgsql17/backup/combined_full_backup \
> /pgBackup/pgsql17/backup/log/restore_incremental.log 2>&1 &

[postgres@lxicbpgdsgv01 ~]$ ls -ltr /pgBackup/pgsql17/backup/combined_full_backup
total 39296
-rw-------. 1 postgres postgres 240 Oct 13 16:29 backup_label
-rw-------. 1 postgres postgres 1174528 Oct 13 16:29 16542.tar
-rw-------. 1 postgres postgres 16778752 Oct 13 16:29 pg_wal.tar
-rw-------. 1 postgres postgres 1171968 Oct 13 16:29 16545.tar
-rw-------. 1 postgres postgres 4002816 Oct 13 16:29 base.tar
drwx------. 4 postgres postgres 32 Oct 13 16:29 PG_17_202406281
-rw-------. 1 postgres postgres 68 Oct 13 16:29 tablespace_map
drwx------. 5 postgres postgres 33 Oct 13 16:29 base
drwx------. 2 postgres postgres 4096 Oct 13 16:29 global
drwx------. 2 postgres postgres 188 Oct 13 16:29 log
-rw-------. 1 postgres postgres 5600 Oct 13 16:29 pg_hba.conf
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_dynshmem
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_commit_ts
drwx------. 4 postgres postgres 68 Oct 13 16:29 pg_logical
-rw-------. 1 postgres postgres 2640 Oct 13 16:29 pg_ident.conf
drwx------. 4 postgres postgres 36 Oct 13 16:29 pg_multixact
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_twophase
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_tblspc
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_subtrans
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_stat_tmp
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_stat
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_snapshots
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_serial
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_replslot
drwx------. 2 postgres postgres 6 Oct 13 16:29 pg_notify
drwx------. 2 postgres postgres 18 Oct 13 16:29 pg_xact
-rw-------. 1 postgres postgres 3 Oct 13 16:29 PG_VERSION
-rw-------. 1 postgres postgres 1169 Oct 13 16:29 postgresql.conf.bkp_10sep2025
-rw-------. 1 postgres postgres 30702 Oct 13 16:29 postgresql.conf.bkp
-rw-------. 1 postgres postgres 109 Oct 13 16:29 postgresql.auto.conf
drwx------. 4 postgres postgres 45 Oct 13 16:29 pg_wal
-rw-------. 1 postgres postgres 1169 Oct 13 16:29 postgresql.conf
-rw-------. 1 postgres postgres 30 Oct 13 16:29 current_logfiles
-rw-------. 1 postgres postgres 227 Oct 13 16:29 backup_label.old
-rw-------. 1 postgres postgres 16777216 Oct 13 16:29 0000000100000000000000B2
-rw-------. 1 postgres postgres 238882 Oct 13 16:29 backup_manifest
[postgres@lxicbpgdsgv01 ~]$

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

Clone PostgreSQL Cluster from One Host to Another Using pg_basebackup (No Archive Mode)

PostgreSQL, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

How To Backup and Restore PostgreSQL DB Cluster from One Host to Another Using pg_basebackup (No Archive Mode)

Table of Contents

1. Goal
2. Environment (Source / Target)
3. Verify Existing Setup
4. Verify postgresql.conf
5. Verify pg_hba.conf
6. Verify Users
7. Directory Permissions
8. Take Backup
9. Verify Backup
10. Transfer Backup to Target
11. Prepare Target & Restore

                  11.1 Backup Target (Optional)
                  11.2 Stop PostgreSQL and Clean Directories
                  11.3 Restore Data and WAL
                  11.4 Set Permissions
                  11.5 Verify Restored Files
                  11.6 Create Symbolic Link for WAL

12. Start PostgreSQL on Target
13. Final Verification

In PostgreSQL, a cluster is a group of databases managed by one PostgreSQL server. This is just a standalone server, not like an Oracle RAC cluster.

1. Goal

1. Perform a consistent backup using pg_basebackup from lxicbpgdsgv01.
2. Restore the backup on lxicbpgdsgv02 without applying archived WALs.

2. Environment (Source / Target)

AspectSourceTargetDifference
Hostnamelxicbpgdsgv01lxicbpgdsgv02Different hostnames
IP Address192.168.2.51192.168.2.52Different IPs
OSRHEL 9RHEL 9Same
DB VersionPostgreSQL v17.6PostgreSQL v17.6Same
Archive modeNo ArchivelogNo ArchivelogSame
PGDATA/pgData/pgsql17/data/pgdata/pgsql17/dataDifferent path case (D vs d)
WAL Directory/pgWal/pgsql17/wal/pgwal/pgsql17/walDifferent path case (W vs w)
Tablespacepg_defaultpg_defaultSame
DatabasesDELL, ORCLNo DatabasesNeed to clone

 3. Verify Existing Setup

[postgres@lxicbpgdsgv01 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# \l+
                                                                                       List of databases
   Name    |  Owner   | Encoding | Locale Provider |   Collate   |    Ctype    | Locale | ICU Rules |   Access privileges   |  Size   | Tablespace |                Description
-----------+----------+----------+-----------------+-------------+-------------+--------+-----------+-----------------------+---------+------------+--------------------------------------------
 dell      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7763 kB | pg_default |
 orcl      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7907 kB | pg_default |
 postgres  | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 492 MB  | pg_default | default administrative connection database
 template0 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7545 kB | pg_default | unmodifiable empty database
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
 template1 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7723 kB | pg_default | default template for new databases
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
(5 rows)

postgres=#

postgres=# \c dell
You are now connected to database "dell" as user "postgres".
dell=#

dell=# \db
       List of tablespaces
    Name    |  Owner   | Location
------------+----------+----------
 pg_default | postgres |
 pg_global  | postgres |
(2 rows)

dell=# 
dell=# \dn test
 List of schemas
 Name |  Owner
------+----------
 test | postgres
(1 row)

dell=#
dell=# \echo :AUTOCOMMIT
on
dell=#

dell=# CREATE TABLE test.emp ( name TEXT, designation TEXT, project TEXT, company TEXT);
CREATE TABLE
dell=# INSERT INTO test.emp VALUES ('Sugi', 'DBA', 'Jetstar', 'iGATE');
INSERT 0 1
dell=# INSERT INTO test.emp VALUES ('Teja', 'DBA', 'RCM', 'iGATE');
INSERT 0 1
dell=# INSERT INTO test.emp VALUES ('RAJ', 'DBA', 'RCM', 'iGATE');
INSERT 0 1
dell=#
dell=# select * from test.emp;
 name | designation | project | company
------+-------------+---------+---------
 Sugi | DBA         | Jetstar | iGATE
 Teja | DBA         | RCM     | iGATE
 RAJ  | DBA         | RCM     | iGATE
(3 rows)

dell=#

dell=# \c orcl
You are now connected to database "orcl" as user "postgres".
orcl=#
orcl=# \dt
            List of relations
 Schema |    Name     | Type  |  Owner
--------+-------------+-------+----------
 public | sample_data | table | postgres
(1 row)

orcl=#
orcl=# select count(*) from sample_data;
 count
-------
  1000  <----- 
(1 row)

orcl=#

4. Verify postgresql.conf


postgres=# SHOW wal_level;
 wal_level
-----------
 replica  <---- Should be replica
(1 row)

postgres=#
postgres=# SHOW max_wal_senders;
 max_wal_senders
-----------------
 10 <----- 
(1 row)

postgres=# SHOW archive_mode;
 archive_mode
--------------
 off  <----- No archive log mode
(1 row)

postgres=# show archive_command;
 archive_command
-----------------
 (disabled)
(1 row)

postgres=#

5. Verify pg_hba.conf

# TYPE  DATABASE        USER            ADDRESS             METHOD
# Local connections for replication (for pg_basebackup run locally)
local   replication     all                                 trust

# Remote connections for replication (for pg_basebackup run remotely)
#host    replication     repl_user       192.168.2.52/32     scram-sha-256

6. Verify user permissions:REPLICATION or SUPERUSER required

postgres=# \du
                             List of roles
 Role name |                         Attributes
-----------+------------------------------------------------------------
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS

postgres=#

7. Directory Permissions for Backup

[postgres@lxicbpgdsgv01 ~]$ ls -ld /pgBackup/pgsql17/backup/
drwx------. 2 postgres postgres 6 Oct  9 18:46 /pgBackup/pgsql17/backup/
[postgres@lxicbpgdsgv01 ~]$

8. Take Backup

  • Want symlinks preserved → use (both -Fp & –waldir)-Fp --waldir=/pgWal/pgsql17/wal
  • Want single tar archive → use -Ft, but recreate symlinks after restore, for pg_wal
  • The –waldir option in pg_basebackup is supported only when using the plain format (-Fp), not with the tar format (-Ft).
[postgres@lxicbpgdsgv01 ~]$ nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025 -Ft -Xs -P > /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025.log 2>&1 &
[1] 4438
[postgres@lxicbpgdsgv01 ~]$

**** I have forgot use -v (verbose option to get extra output)
nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025 -Ft -Xs -P -v > /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025.log 2>&1 &

9. Verify Backup


[postgres@lxicbpgdsgv01 ~]$ cat /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025.log
nohup: ignoring input
waiting for checkpoint
133816/536026 kB (24%), 0/1 tablespace
299960/536026 kB (55%), 0/1 tablespace
447864/536026 kB (83%), 0/1 tablespace
536037/536037 kB (100%), 0/1 tablespace
536037/536037 kB (100%), 1/1 tablespace
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025
total 540M
-rw-------. 1 postgres postgres 524M Oct  9 18:56 base.tar  
-rw-------. 1 postgres postgres 223K Oct  9 18:56 backup_manifest
-rw-------. 1 postgres postgres  17M Oct  9 18:56 pg_wal.tar 
[postgres@lxicbpgdsgv01 ~]$

10. Transfer Backup to Target


[postgres@lxicbpgdsgv01 pg_basebackup_lxicbpgdsgv01_10sep2025]$ scp * 192.168.2.52:/pgbackup/pgsql17/backup/
postgres@192.168.2.52's password:  
backup_manifest                   100%  222KB  30.7MB/s   00:00
base.tar                          100%  523MB  74.0MB/s   00:07
pg_wal.tar                        100%   16MB  43.5MB/s   00:00
[postgres@lxicbpgdsgv01 pg_basebackup_lxicbpgdsgv01_10sep2025]$

11. Prepare Target & Restore

-- List Source Backup files 
[postgres@lxicbpgdsgv02 ~]$ ls -lrth /pgbackup/pgsql17/backup
total 540M
-rw-------. 1 postgres postgres 223K Oct  9 19:08 backup_manifest
-rw-------. 1 postgres postgres 524M Oct  9 19:08 base.tar
-rw-------. 1 postgres postgres  17M Oct  9 19:08 pg_wal.tar
[postgres@lxicbpgdsgv02 ~]$

11.1 Backup Target (Optional)


nohup pg_basebackup -U postgres -D /pgbackup/pgsql17/pg_basebackup_lxicbpgdsgv02_10sep2025 -Ft -Xs -P > /pgbackup/pgsql17/pg_basebackup_lxicbpgdsgv02_10sep2025.log 2>&1 &

-- Verify Tablespaces path -- It's pg_default in our case
-- Verify WAL File (Redo log files) location 

[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgdata/pgsql17/data/pg_wal
lrwxrwxrwx. 1 postgres postgres 18 Oct  9 17:37 /pgdata/pgsql17/data/pg_wal -> /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 ~]$

11.2 Stop PostgreSQL and Clean Directories

-- Stop PostgreSQL DB Cluster
[root@lxicbpgdsgv02 ~]# systemctl stop postgresql-17.service
[root@lxicbpgdsgv02 ~]#
[root@lxicbpgdsgv02 ~]# ps -ef | grep postgres
root       10259    7832  0 19:20 pts/0    00:00:00 grep --color=auto postgres
[root@lxicbpgdsgv02 ~]#

-- Remove all from Data directory 

[postgres@lxicbpgdsgv02 ~]$ rm -rf /pgdata/pgsql17/data/*
[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgdata/pgsql17/data/
total 0
[postgres@lxicbpgdsgv02 ~]$


-- Remove all from WAL directory 

[postgres@lxicbpgdsgv02 ~]$ rm -rf /pgwal/pgsql17/wal/*
[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgwal/pgsql17/wal/
total 0
[postgres@lxicbpgdsgv02 ~]$

11.3 Restore Data and WAL

-- List source backup files

[postgres@lxicbpgdsgv02 ~]$ cd /pgbackup/pgsql17/backup/
[postgres@lxicbpgdsgv02 backup]$ ls -ltr
total 552652
-rw-------. 1 postgres postgres    227724 Oct  9 19:08 backup_manifest
-rw-------. 1 postgres postgres 548901888 Oct  9 19:08 base.tar
-rw-------. 1 postgres postgres  16778752 Oct  9 19:08 pg_wal.tar
drwxr-xr-x. 3 postgres postgres        18 Oct  9 19:32 data_lxicbpgdsgv02_old
[postgres@lxicbpgdsgv02 backup]$

# Restore DATA Directory 
[postgres@lxicbpgdsgv02 ~]$ nohup tar -xvf /pgbackup/pgsql17/backup/base.tar -C /pgdata/pgsql17/data > /pgbackup/pgsql17/backup/base_restore.log 2>&1 &
[1] 11289
[postgres@lxicbpgdsgv02 ~]$


# Restore WAL Directory 

[postgres@lxicbpgdsgv02 ~]$ nohup tar -xvf /pgbackup/pgsql17/backup/pg_wal.tar -C /pgwal/pgsql17/wal > /pgbackup/pgsql17/backup/pg_wal_restore.log 2>&1 &
[1] 11304
[postgres@lxicbpgdsgv02 ~]$

-- OR -- 

# Restore base and WAL archives sequentially in a single command

nohup bash -c "tar -xvf /pgbackup/pgsql17/backup/base.tar -C /pgdata/pgsql17/data && tar -xvf /pgbackup/pgsql17/backup/pg_wal.tar -C /pgwal/pgsql17/wal" > /pgbackup/pgsql17/backup/fulltar_restore.log 2>&1 &

11.4 Set Permissions

[postgres@lxicbpgdsgv02 ~]$ chown -R postgres:postgres /pgdata/pgsql17/data
[postgres@lxicbpgdsgv02 ~]$ chmod 700 /pgdata/pgsql17/data
[postgres@lxicbpgdsgv02 ~]$
[postgres@lxicbpgdsgv02 ~]$ chown -R postgres:postgres /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 ~]$ chmod 700 /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 ~]$

11.5 Verify Restored Files

[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgdata/pgsql17/data
total 72
-rw-------. 1 postgres postgres    88 Sep 30 21:50 postgresql.auto.conf
drwx------. 2 postgres postgres    18 Sep 30 21:50 pg_xact
-rw-------. 1 postgres postgres     3 Sep 30 21:50 PG_VERSION
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_twophase
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_tblspc
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_subtrans
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_stat_tmp
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_snapshots
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_serial
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_notify
drwx------. 4 postgres postgres    36 Sep 30 21:50 pg_multixact
-rw-------. 1 postgres postgres  2640 Sep 30 21:50 pg_ident.conf
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_dynshmem
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_commit_ts
-rw-------. 1 postgres postgres 30702 Oct  7 18:20 postgresql.conf.bkp
drwx------. 2 postgres postgres    84 Oct  9 00:01 log
-rw-------. 1 postgres postgres  1169 Oct  9 16:31 postgresql.conf.bkp_10sep2025
-rw-------. 1 postgres postgres  1171 Oct  9 16:32 postgresql.conf
drwx------. 7 postgres postgres    59 Oct  9 18:08 base
drwx------. 2 postgres postgres     6 Oct  9 18:44 pg_stat
-rw-------. 1 postgres postgres    30 Oct  9 18:44 current_logfiles
-rw-------. 1 postgres postgres  5600 Oct  9 18:53 pg_hba.conf
-rw-------. 1 postgres postgres     0 Oct  9 18:56 tablespace_map
drwx------. 2 postgres postgres     6 Oct  9 18:56 pg_replslot
drwx------. 4 postgres postgres    68 Oct  9 18:56 pg_logical
-rw-------. 1 postgres postgres   227 Oct  9 18:56 backup_label
drwx------. 4 postgres postgres    45 Oct  9 20:06 pg_wal <--- Created as Directory, instead of symbolic link 
drwx------. 2 postgres postgres  4096 Oct  9 20:06 global
[postgres@lxicbpgdsgv02 ~]$
[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgwal/pgsql17/wal
total 16384
-rw-------. 1 postgres postgres 16777216 Oct  9 18:56 000000010000000000000045
[postgres@lxicbpgdsgv02 ~]$

Please note:

--- Want symlinks preserved → use -Fp
--- Want single tar archive → use -Ft, but recreate symlinks after restore, for pg_wal

11.6 Create Symbolic Link for WAL

[postgres@lxicbpgdsgv02 ~]$ ls -ld /pgdata/pgsql17/data/pg_wal
drwx------. 4 postgres postgres 45 Oct  9 20:06 /pgdata/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv02 ~]$
[postgres@lxicbpgdsgv02 ~]$ rm -rf /pgdata/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv02 ~]$ ln -s /pgwal/pgsql17/wal /pgdata/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv02 ~]$
[postgres@lxicbpgdsgv02 ~]$ ls -ld /pgdata/pgsql17/data/pg_wal
lrwxrwxrwx. 1 postgres postgres 18 Oct  9 21:04 /pgdata/pgsql17/data/pg_wal -> /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 ~]$ 
[postgres@lxicbpgdsgv02 ~]$ cd /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 wal]$ ll
total 16384
-rw-------. 1 postgres postgres 16777216 Oct  9 18:56 000000010000000000000045
[postgres@lxicbpgdsgv02 wal]$

12. Start PostgreSQL on Target — no recovery needed.

[root@lxicbpgdsgv02 ~]# systemctl start postgresql-17.service
[root@lxicbpgdsgv02 ~]# systemctl status postgresql-17.service
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Thu 2025-10-09 21:06:08 +08; 5s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 12375 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 12380 (postgres)
      Tasks: 7 (limit: 20496)
     Memory: 33.6M
        CPU: 114ms
     CGroup: /system.slice/postgresql-17.service
             ├─12380 /usr/pgsql-17/bin/postgres -D /pgdata/pgsql17/data
             ├─12381 "postgres: logger "
             ├─12382 "postgres: checkpointer "
             ├─12383 "postgres: background writer "
             ├─12385 "postgres: walwriter "
             ├─12386 "postgres: autovacuum launcher "
             └─12387 "postgres: logical replication launcher "

Oct 09 21:06:06 lxicbpgdsgv02.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 09 21:06:06 lxicbpgdsgv02.rajasekhar.com postgres[12380]: 2025-10-09 21:06:06.781 +08 [12380] LOG:  redirecting log output to logging collector process
Oct 09 21:06:06 lxicbpgdsgv02.rajasekhar.com postgres[12380]: 2025-10-09 21:06:06.781 +08 [12380] HINT:  Future log output will appear in directory "log".
Oct 09 21:06:08 lxicbpgdsgv02.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv02 ~]#

13. Final Verification

[postgres@lxicbpgdsgv02 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# \l+
                                                                                       List of databases
   Name    |  Owner   | Encoding | Locale Provider |   Collate   |    Ctype    | Locale | ICU Rules |   Access privileges   |  Size   | Tablespace |                Description
-----------+----------+----------+-----------------+-------------+-------------+--------+-----------+-----------------------+---------+------------+--------------------------------------------
 dell      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7609 kB | pg_default |
 orcl      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7753 kB | pg_default |
 postgres  | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 492 MB  | pg_default | default administrative connection database
 template0 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7545 kB | pg_default | unmodifiable empty database
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
 template1 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7569 kB | pg_default | default template for new databases
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
(5 rows)

postgres=# \db
       List of tablespaces
    Name    |  Owner   | Location
------------+----------+----------
 pg_default | postgres |
 pg_global  | postgres |
(2 rows)

postgres=#
postgres=# \c dell
You are now connected to database "dell" as user "postgres".
dell=# \dt test.*
        List of relations
 Schema | Name | Type  |  Owner
--------+------+-------+----------
 test   | emp  | table | postgres
(1 row)

dell=#

dell=# select * from test.emp;  <---- We can see Data 
 name | designation | project | company
------+-------------+---------+---------
 Sugi | DBA         | Jetstar | iGATE
 Teja | DBA         | RCM     | iGATE
 RAJ  | DBA         | RCM     | iGATE
(3 rows)

dell=#
dell=# \c orcl
You are now connected to database "orcl" as user "postgres".
orcl=# \dt
            List of relations
 Schema |    Name     | Type  |  Owner
--------+-------------+-------+----------
 public | sample_data | table | postgres
(1 row)

orcl=#
orcl=# select count(*) from sample_data;
 count
-------
  1000 <----
(1 row)

orcl=#

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

Change PostgreSQL WAL Directory Path (pg_wal)

PostgreSQL, , , , , , , , , , , , , , , ,

Table of Contents

0. Aim
1. Verify Existing pg_wal Directory
2. Create the New Directory
3. Stop PostgreSQL Service
4. Copy WAL Files to New Location
5. Backup Old WAL Directory
6. Create Symlink
7. Fix Permissions
8. Start PostgreSQL Service
9. Verify WAL Functionality
10. Remove Old WAL Directory (Optional)

0. Aim

To change the PostgreSQL 17 WAL File directory from its default location to new mount point

From : /var/lib/pgsql/17/data

TO : /pgData/pgsql17/data

1. Verify Existing pg_wal directory


[postgres@lxicbpgdsgv01 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# SELECT current_setting('data_directory') || '/pg_wal' AS wal_directory;
        wal_directory
-----------------------------
 /pgData/pgsql17/data/pg_wal  <----- 
(1 row)

postgres=# exit
[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgData/pgsql17/data/pg_wal
total 48M
drwx------. 2 postgres postgres   6 Sep 30 21:50 summaries
-rw-------. 1 postgres postgres 16M Oct  8 04:24 000000010000000000000008
-rw-------. 1 postgres postgres 16M Oct  8 16:08 000000010000000000000006
-rw-------. 1 postgres postgres 16M Oct  8 16:08 000000010000000000000007
drwx------. 2 postgres postgres  43 Oct  8 16:08 archive_status
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ du -sh /pgData/pgsql17/data/pg_wal
48M     /pgData/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv01 ~]$

2. Create the New Directory on a new disk

[root@lxicbpgdsgv01 ~]# mkdir -p /pgData/pgsql17/data
[root@lxicbpgdsgv01 ~]# chown postgres:postgres /pgData/pgsql17/data
[root@lxicbpgdsgv01 ~]# chmod 700 /pgData/pgsql17/data
[root@lxicbpgdsgv01 ~]#

3. Stop PostgreSQL Service

[root@lxicbpgdsgv01 ~]# systemctl stop postgresql-17.service
[root@lxicbpgdsgv01 ~]#
[root@lxicbpgdsgv01 ~]# ps -ef | grep postgres
root        6887    6721  0 16:08 pts/0    00:00:00 grep --color=auto postgres
[root@lxicbpgdsgv01 ~]#

4. Copy Existing WAL Files to New Location

[postgres@lxicbpgdsgv01 ~]$ nohup rsync -avh --progress /pgData/pgsql17/data/pg_wal/ /pgWal/pgsql17/wal/ > rsync_pgwal.log 2>&1 &
[1] 6943
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$
[1]+  Done                    nohup rsync -avh --progress /pgData/pgsql17/data/pg_wal/ /pgWal/pgsql17/wal/ > rsync_pgwal.log 2>&1
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ cat rsync_pgwal.log
nohup: ignoring input
sending incremental file list
./
000000010000000000000006
         16.78M 100%   81.89MB/s    0:00:00 (xfr#1, to-chk=5/7)
000000010000000000000007
         16.78M 100%   43.84MB/s    0:00:00 (xfr#2, to-chk=4/7)
000000010000000000000008
         16.78M 100%   31.07MB/s    0:00:00 (xfr#3, to-chk=3/7)
archive_status/
archive_status/000000010000000000000006.done
              0 100%    0.00kB/s    0:00:00 (xfr#4, to-chk=0/7)
summaries/

sent 50.34M bytes  received 107 bytes  33.56M bytes/sec
total size is 50.33M  speedup is 1.00
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgWal/pgsql17/wal/
total 48M
drwx------. 2 postgres postgres   6 Sep 30 21:50 summaries
-rw-------. 1 postgres postgres 16M Oct  8 04:24 000000010000000000000008
-rw-------. 1 postgres postgres 16M Oct  8 16:08 000000010000000000000006
-rw-------. 1 postgres postgres 16M Oct  8 16:08 000000010000000000000007
drwx------. 2 postgres postgres  43 Oct  8 16:08 archive_status
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ du -sh /pgWal/pgsql17/wal/
48M     /pgWal/pgsql17/wal/
[postgres@lxicbpgdsgv01 ~]$

5. Move old directory as backup

[postgres@lxicbpgdsgv01 ~]$ mv /pgData/pgsql17/data/pg_wal /pgData/pgsql17/data/pg_wal.bak
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ ls -ld /pgData/pgsql17/data/pg_wal
ls: cannot access '/pgData/pgsql17/data/pg_wal': No such file or directory
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ ls -ld /pgData/pgsql17/data/pg_wal.bak
drwx------. 4 postgres postgres 141 Oct  8 16:08 /pgData/pgsql17/data/pg_wal.bak
[postgres@lxicbpgdsgv01 ~]$

6. Create symlink

[postgres@lxicbpgdsgv01 ~]$ ln -s /pgWal/pgsql17/wal /pgData/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ ls -ltr /pgData/pgsql17/data/pg_wal
lrwxrwxrwx. 1 postgres postgres 18 Oct  8 16:16 /pgData/pgsql17/data/pg_wal -> /pgWal/pgsql17/wal
[postgres@lxicbpgdsgv01 ~]$

7. Fix permissions (if required)

[postgres@lxicbpgdsgv01 ~]$ chown -R postgres:postgres /pgWal/pgsql17/wal

8. Start PostgreSQL Service

[root@lxicbpgdsgv01 ~]# systemctl start postgresql-17.service
[root@lxicbpgdsgv01 ~]#
[root@lxicbpgdsgv01 ~]# systemctl status postgresql-17.service
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Wed 2025-10-08 16:20:46 +08; 7s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 7079 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 7084 (postgres)
      Tasks: 8 (limit: 15835)
     Memory: 18.1M
        CPU: 94ms
     CGroup: /system.slice/postgresql-17.service
             ├─7084 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
             ├─7086 "postgres: logger "
             ├─7087 "postgres: checkpointer "
             ├─7088 "postgres: background writer "
             ├─7090 "postgres: walwriter "
             ├─7091 "postgres: autovacuum launcher "
             ├─7092 "postgres: archiver "
             └─7093 "postgres: logical replication launcher "

Oct 08 16:20:46 lxicbpgdsgv01.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 08 16:20:46 lxicbpgdsgv01.rajasekhar.com postgres[7084]: 2025-10-08 16:20:46.162 +08 [7084] LOG:  redirecting log output to logging collector process
Oct 08 16:20:46 lxicbpgdsgv01.rajasekhar.com postgres[7084]: 2025-10-08 16:20:46.162 +08 [7084] HINT:  Future log output will appear in directory "log".
Oct 08 16:20:46 lxicbpgdsgv01.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv01 ~]#

9. Verify

-- Load WAL File generation (Testing)

postgres=# -- Create test table
DROP TABLE IF EXISTS wal_test;
CREATE TABLE wal_test (
    id serial PRIMARY KEY,
    data text
);

-- Generate WAL traffic
DO $$
DECLARE
    i integer;
BEGIN
    FOR i IN 1..50 LOOP
        -- INSERT: 10,000 rows
        INSERT INTO wal_test (data)
        SELECT repeat('wal_test_data_', 50)
        FROM generate_series(1, 10000);

        -- UPDATE: 5,000 rows using CTE with LIMIT
        WITH to_update AS (
            SELECT id FROM wal_test WHERE id % 2 = 0 LIMIT 5000
        )
        UPDATE wal_test
        SET data = data || '_updated'
        WHERE id IN (SELECT id FROM to_update);

        -- DELETE: 5,000 rows using CTE with LIMIT
        WITH to_delete AS (
            SELECT id FROM wal_test WHERE id % 3 = 0 LIMIT 5000
        )
        DELETE FROM wal_test
        WHERE id IN (SELECT id FROM to_delete);

        -- Commit to flush WAL
        COMMIT;

        -- Optional pause to slow down the loop
        PERFORM pg_sleep(0.5);
    END LOOP;
END$$;
DROP TABLE
CREATE TABLE
DO
postgres=# exit
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ ls -lrth  /pgWal/pgsql17/wal
total 752M
drwx------. 2 postgres postgres    6 Sep 30 21:50 summaries
-rw-------. 1 postgres postgres  16M Oct  8 16:34 000000010000000000000009
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000A
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000B
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000C
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000D
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000E
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000F
-rw-------. 1 postgres postgres  16M Oct  8 16:34 000000010000000000000010
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000011
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000012
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000013
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000014
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000015
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000016
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000017
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000018
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000019
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001A
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001B
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001C
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001D
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001E
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001F
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000020
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000021
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000022
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000023
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000024
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000025
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000026
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000027
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000028
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000029
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002A
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002B
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002C
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002D
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002E
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002F
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000030
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000031
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000032
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000033
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000034
-rw-------. 1 postgres postgres  16M Oct  8 16:36 000000010000000000000035
drwx------. 2 postgres postgres 4.0K Oct  8 16:36 archive_status
-rw-------. 1 postgres postgres  16M Oct  8 16:36 xlogtemp.7289
[postgres@lxicbpgdsgv01 ~]$

10. Remove Old WAL Directory (Optional, later)

[postgres@lxicbpgdsgv01 ~]$ ls -ld /pgData/pgsql17/data/pg_wal.bak
drwx------. 4 postgres postgres 141 Oct  8 16:08 /pgData/pgsql17/data/pg_wal.bak
[postgres@lxicbpgdsgv01 ~]$ rm -rf /pgData/pgsql17/data/pg_wal.bak
[postgres@lxicbpgdsgv01 ~]$

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

WAL Files

PostgreSQL, , , , , , , , , , , , ,

Understanding WAL Files in PostgreSQL – For Oracle DBAs

What is WAL?

WAL (Write-Ahead Logging) ensures data durability and crash recovery. Every change is first written to WAL before being flushed to data files. 

1. WAL Location: WAL files are stored in $PGDATA/pg_wal/.
2. Fixed Size: Each WAL segment is pre-allocated with a fixed size (typically 16MB), set at cluster initialization (initdb).
3. Writing & Switching: WAL segments are written sequentially. PostgreSQL switches to the next segment when the current one is full or a manual switch is triggered (e.g., via pg_switch_wal()).
4. WAL and Archive File Size: When archived, the exact same 16MB WAL segment file is copied as raw binary to the archive destination. Therefore, archive files always match the WAL segment size.
5. Archiving: If archive_mode = on, PostgreSQL uses the archive_command to copy completed WAL segments to the archive destination (e.g., /pgArch/...). Only full segments are archived.
6. Recycling: If archiving is disabled, PostgreSQL reuses old WAL segments once they’re no longer needed for recovery or replication.

WAL Files = Redo Log Files
WAL Archives = Archive Log Files

PostgreSQL vs Oracle Comparison

AspectOraclePostgreSQL
Redo Log FilesCalled Redo Log FilesCalled WAL Files
Archive Log FilesCalled Archive Log FilesCalled WAL Archives
Redo Log File Size~200MB (default)16MB (default and fixed)
Redo Log Files CountMinimum 2 or more redo log groups
(Best practice: 3)		No fixed number
Controlled by: min_wal_size / wal_segment_size
Example: 80MB / 16MB = 5 WAL files
Archive Log File Size~200MB (same as redo log file size)16MB (same as WAL file size)
Log File StorageRedo log files stored in redo log groupsWAL files stored in pg_wal directory
Archiving MechanismArchive logs created if archiving is enabledArchived via archive_command if archive_mode = on
Overwrite BehaviorRedo logs are overwritten when full (if archive not enabled)WAL segments are recycled/overwritten when full (if archive_mode is off)
Force Switch BehaviorALTER SYSTEM SWITCH LOGFILE;SELECT pg_switch_wal();

PostgreSQL WAL Configuration Parameters

ParameterMeaningDefault
wal_segment_sizeSize of each WAL file16MB
min_wal_sizeMinimum total WAL size retained80MB (5 WAL files)
Common in production: 8GB
max_wal_sizeMax WAL before triggering a checkpoint1GB
Common in production: 16GB

Check Configuration

[postgres@lxicbpgdsgv01 ~]$ psql -c "SHOW wal_segment_size;"
wal_segment_size
------------------
16MB  <----- Each redo log file size
(1 row)

[postgres@lxicbpgdsgv01 ~]$


[postgres@lxicbpgdsgv01 ~]$ psql -c "SHOW min_wal_size;"
 min_wal_size
--------------
 80MB <---- 80MB/16MB = 5 <--- Total 5 redo log files with each 16MB
(1 row)

[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ psql -c "SHOW max_wal_size;"
 max_wal_size
--------------
 1GB  <---- 1024MB
(1 row)

[postgres@lxicbpgdsgv01 ~]$

View WAL files (Redo log files)

postgres=# SELECT current_setting('data_directory') || '/pg_wal' AS wal_directory;
        wal_directory
-----------------------------
 /pgData/pgsql17/data/pg_wal  <---- Redo log file directory
(1 row)

postgres=# 
[postgres@lxicbpgdsgv01 ~]$ ls -lh /pgData/pgsql17/data/pg_wal | grep -E '^-.* [0-9]+ .* [0-9A-F]{24}$'
-rw-------. 1 postgres postgres 16M Oct 8 01:33 000000010000000000000005
-rw-------. 1 postgres postgres 16M Oct 8 01:28 000000010000000000000006
-rw-------. 1 postgres postgres 16M Oct 8 01:28 000000010000000000000007
[postgres@lxicbpgdsgv01 ~]$

min_wal_size = 80MB means PostgreSQL will try to keep at least 5 WAL segment files on disk (since 80MB ÷ 16MB = 5 files).

However, in our case, only 3 WAL files are currently present. This is likely because the system is under low load in the testing environment, so fewer WAL files are needed at the moment.

---- Lets Generate Load

postgres=# -- Create test table
DROP TABLE IF EXISTS wal_test;
CREATE TABLE wal_test (
    id serial PRIMARY KEY,
    data text
);

-- Generate WAL traffic
DO $$
DECLARE
    i integer;
BEGIN
    FOR i IN 1..50 LOOP
        -- INSERT: 10,000 rows
        INSERT INTO wal_test (data)
        SELECT repeat('wal_test_data_', 50)
        FROM generate_series(1, 10000);

        -- UPDATE: 5,000 rows using CTE with LIMIT
        WITH to_update AS (
            SELECT id FROM wal_test WHERE id % 2 = 0 LIMIT 5000
        )
        UPDATE wal_test
        SET data = data || '_updated'
        WHERE id IN (SELECT id FROM to_update);

        -- DELETE: 5,000 rows using CTE with LIMIT
        WITH to_delete AS (
            SELECT id FROM wal_test WHERE id % 3 = 0 LIMIT 5000
        )
        DELETE FROM wal_test
        WHERE id IN (SELECT id FROM to_delete);

        -- Commit to flush WAL
        COMMIT;

        -- Optional pause to slow down the loop
        PERFORM pg_sleep(0.5);
    END LOOP;
END$$;
DROP TABLE
CREATE TABLE
DO
postgres=# exit
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ 
ls -lrth /pgData/pgsql17/data/pg_wal
total 752M
drwx------. 2 postgres postgres    6 Sep 30 21:50 summaries
-rw-------. 1 postgres postgres  16M Oct  8 16:34 000000010000000000000009
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000A
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000B
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000C
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000D
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000E
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000F
-rw-------. 1 postgres postgres  16M Oct  8 16:34 000000010000000000000010
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000011
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000012
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000013
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000014
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000015
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000016
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000017
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000018
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000019
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001A
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001B
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001C
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001D
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001E
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001F
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000020
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000021
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000022
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000023
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000024
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000025
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000026
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000027
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000028
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000029
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002A
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002B
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002C
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002D
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002E
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002F
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000030
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000031
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000032
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000033
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000034
-rw-------. 1 postgres postgres  16M Oct  8 16:36 000000010000000000000035
drwx------. 2 postgres postgres 4.0K Oct  8 16:36 archive_status
-rw-------. 1 postgres postgres  16M Oct  8 16:36 xlogtemp.7289
[postgres@lxicbpgdsgv01 ~]$ 

-- Now We can see Many WAL Files (Redo log files) generated, count increased from 3 WAL files to 50+ WAL files

View Archived WAL Files (Archive log files)

postgres=# SHOW archive_command;
        archive_command
-------------------------------
 cp %p /pgArch/pgsql17/arch/%f
(1 row)

postgres=#

[postgres@lxicbpgdsgv01 ~]$ 
[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgArch/pgsql17/arch/
total 976M
-rw-------. 1 postgres postgres 16M Oct  7 23:50 000000010000000000000001
-rw-------. 1 postgres postgres 16M Oct  7 23:51 000000010000000000000002
-rw-------. 1 postgres postgres 16M Oct  8 01:28 000000010000000000000003
-rw-------. 1 postgres postgres 16M Oct  8 01:28 000000010000000000000004
-rw-------. 1 postgres postgres 16M Oct  8 04:24 000000010000000000000005
-rw-------. 1 postgres postgres 16M Oct  8 16:08 000000010000000000000006
-rw-------. 1 postgres postgres 16M Oct  8 16:24 000000010000000000000007
-rw-------. 1 postgres postgres 16M Oct  8 16:25 000000010000000000000008
-rw-------. 1 postgres postgres 16M Oct  8 16:34 000000010000000000000009
-rw-------. 1 postgres postgres 16M Oct  8 16:34 00000001000000000000000A
-rw-------. 1 postgres postgres 16M Oct  8 16:34 00000001000000000000000B
-rw-------. 1 postgres postgres 16M Oct  8 16:34 00000001000000000000000C
-rw-------. 1 postgres postgres 16M Oct  8 16:34 00000001000000000000000D
-rw-------. 1 postgres postgres 16M Oct  8 16:34 00000001000000000000000E
-rw-------. 1 postgres postgres 16M Oct  8 16:34 00000001000000000000000F
-rw-------. 1 postgres postgres 16M Oct  8 16:34 000000010000000000000010
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000011
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000012
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000013
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000014
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000015
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000016
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000017
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000018
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000019
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000001A
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000001B
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000001C
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000001D
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000001E
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000001F
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000020
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000021
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000022
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000023
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000024
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000025
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000026
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000027
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000028
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000029
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000002A
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000002B
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000002C
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000002D
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000002E
-rw-------. 1 postgres postgres 16M Oct  8 16:35 00000001000000000000002F
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000030
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000031
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000032
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000033
-rw-------. 1 postgres postgres 16M Oct  8 16:35 000000010000000000000034
-rw-------. 1 postgres postgres 16M Oct  8 16:36 000000010000000000000035
-rw-------. 1 postgres postgres 16M Oct  8 16:36 000000010000000000000036
-rw-------. 1 postgres postgres 16M Oct  8 16:36 000000010000000000000037
-rw-------. 1 postgres postgres 16M Oct  8 16:36 000000010000000000000038
-rw-------. 1 postgres postgres 16M Oct  8 16:36 000000010000000000000039
-rw-------. 1 postgres postgres 16M Oct  8 16:36 00000001000000000000003A
-rw-------. 1 postgres postgres 16M Oct  8 16:36 00000001000000000000003B
-rw-------. 1 postgres postgres 16M Oct  8 16:36 00000001000000000000003C
-rw-------. 1 postgres postgres 16M Oct  8 16:36 00000001000000000000003D
[postgres@lxicbpgdsgv01 ~]$

-- WAL and Archive File Size: When archived, the exact same 16MB WAL segment file is copied as raw binary to the archive destination. Therefore, archive files always match the WAL segment size (redo log size).

Best Practices

  • Use a dedicated mount point for pg_wal (WAL directory) – eg. /pgWal
  • Set min_wal_size to higher value in production (e.g. 8GB)
  • Keep WAL archive files on a separate mount point for better I/O and space isolation – eg. /pgArch
  • Monitor WAL archive directory regularly to avoid disk full errors

Note:

  • WAL file size is fixed at cluster init (default: 16MB). You cannot change it for an existing cluster.
  • PostgreSQL WAL Size Be Can Increased only at NEW cluster creation time via initdb (64MB)
    /usr/pgsql-17/bin/initdb –wal-segsize=64 -D /pgData/pgsql17/data

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

Enable Archive Mode (WAL Archiving)

PostgreSQL, , , , , , , , , , , , , , ,

Enable Archive Mode in PostgreSQL 17

Table of Contents

0. Importance of Archive Log Files
1. Verify Existing Archive Mode
2. Ensure Archive Directory Exists
3. Edit postgresql.conf
4. Restart PostgreSQL
5. Verify the Configuration
6. Test Archiving Works

0. Importance of Archive Log Files

Archive log files, also known as WAL archives (Write-Ahead Log archives) , play a critical role in PostgreSQL for data protection, recovery, and replication.

A. Point-in-Time Recovery (PITR): Restore the database to a specific point in time, useful for accidental data changes or loss.
B. Continuous Backup: Works with base backups to enable robust disaster recovery.
C. Streaming Replication Support: Helps standby servers catch up if they fall behind.
D. Disaster Recovery: Enables full recovery after hardware or data corruption.
E. Data Audit & Analysis: Allows decoding WAL logs for auditing and compliance.

🛑 Important Note:

If WAL archiving is disabled and a backup is taken, you can only restore to the exact backup time, not to any point after.

1. Verify Existing Archive Mode

[postgres@lxicbpgdsgv01 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# SHOW archive_mode;
 archive_mode 
--------------
 off <--- it's off
(1 row)

postgres=# SHOW archive_command;
 archive_command 
-----------------
 (disabled) <----
(1 row)
[root@lxicbpgdsgv01 ~]# systemctl status postgresql-17
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Tue 2025-10-07 18:33:05 +08; 2min 26s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 5911 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 5916 (postgres)
      Tasks: 7 (limit: 15700)
     Memory: 21.3M
        CPU: 177ms
     CGroup: /system.slice/postgresql-17.service
             ├─5916 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
             ├─5917 "postgres: logger "
             ├─5918 "postgres: checkpointer "
             ├─5919 "postgres: background writer "
             ├─5921 "postgres: walwriter "
             ├─5922 "postgres: autovacuum launcher "
             └─5923 "postgres: logical replication launcher "

Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com postgres[5916]: 2025-10-07 18:33:05.392 +08 [5916] LOG:  redirecting log output to logging collector process
Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com postgres[5916]: 2025-10-07 18:33:05.392 +08 [5916] HINT:  Future log output will appear in directory "log".
Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv01 ~]#

Currently, there is no active archive process running in the background.

2. Ensure Archive Directory Exists

[root@lxicbpgdsgv01 ~]# mkdir -p /pgArch/pgsql17/arch/
[root@lxicbpgdsgv01 ~]# chown postgres:postgres /pgArch/pgsql17/arch/
[root@lxicbpgdsgv01 ~]# chmod 700 /pgArch/pgsql17/arch/

3. Edit postgresql.conf

[postgres@lxicbpgdsgv01 ~]$ vi /pgData/pgsql17/data/postgresql.conf

Update or add the following lines:

archive_mode = on
archive_command = 'cp %p /pgArch/pgsql17/arch/%f'

Explanation:

  • %p = Full path of WAL file
  • %f = WAL file name
  • cp %p /pgArch/pgsql17/arch/%f = Command to copy the WAL file to archive directory

4. Restart PostgreSQL

[root@lxicbpgdsgv01 ~]# systemctl status postgresql-17
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Tue 2025-10-07 23:48:29 +08; 18s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 7868 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 7873 (postgres)
      Tasks: 8 (limit: 15700)
     Memory: 19.0M
        CPU: 96ms
     CGroup: /system.slice/postgresql-17.service
             ├─7873 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
             ├─7874 "postgres: logger "
             ├─7875 "postgres: checkpointer "
             ├─7876 "postgres: background writer "
             ├─7878 "postgres: walwriter "
             ├─7879 "postgres: autovacuum launcher "
             ├─7880 "postgres: archiver "  <------
             └─7881 "postgres: logical replication launcher "

Oct 07 23:48:29 lxicbpgdsgv01.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 07 23:48:29 lxicbpgdsgv01.rajasekhar.com postgres[7873]: 2025-10-07 23:48:29.503 +08 [7873] LOG:  redirecting log output to logging collector process
Oct 07 23:48:29 lxicbpgdsgv01.rajasekhar.com postgres[7873]: 2025-10-07 23:48:29.503 +08 [7873] HINT:  Future log output will appear in directory "log".
Oct 07 23:48:29 lxicbpgdsgv01.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv01 ~]#

Make sure the archiver process is running.

5. Verify the Configuration

[postgres@lxicbpgdsgv01 ~]$ psql -c "SHOW archive_mode;"
 archive_mode 
--------------
 on <-----------
(1 row)

[postgres@lxicbpgdsgv01 ~]$ psql -c "SHOW archive_command;"
        archive_command         
-------------------------------
 cp %p /pgArch/pgsql17/arch/%f  <-----------------
(1 row)

6. Test Archiving Works

[postgres@lxicbpgdsgv01 ~]$ psql -c "SELECT pg_switch_wal();"

Check archive folder for new WAL files:

[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgArch/pgsql17/arch/
total 16M
-rw-------. 1 postgres postgres 16M Oct  7 23:50 000000010000000000000001

[postgres@lxicbpgdsgv01 ~]$ psql -c "SELECT pg_switch_wal();"
[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgArch/pgsql17/arch/
total 32M
-rw-------. 1 postgres postgres 16M Oct  7 23:50 000000010000000000000001
-rw-------. 1 postgres postgres 16M Oct  7 23:51 000000010000000000000002

 Archiving is working successfully.

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

Change Data Directory

PostgreSQL, , , , , , , , , , , , , , , , , , , , , , , ,

PostgreSQL 17 – Change Data Directory

0. Overview
1. Stop PostgreSQL Service
2. Create the New Directory
3. Copy Existing Data to New Location
4. Move existing data directory
5. Update PostgreSQL Configuration to Use New Data Directory
6. Reload systemd and start PostgreSQL
7. Edit postgres .bash_profile
8. Verify New Data Directory is in Use
9. Remove Old Data Directory (Optional – A week after)

0. Overview

To change the PostgreSQL 17 data directory from its default location

From: /var/lib/pgsql/17/data To: /pgData/pgsql17/data

 

1. Stop PostgreSQL Service

[postgres@lxicbpgdsgv01 ~]$ psql -c "SHOW data_directory;"
     data_directory
------------------------
 /var/lib/pgsql/17/data <---- existing data directory
(1 row)

[postgres@lxicbpgdsgv01 ~]$

[root@lxicbpgdsgv01 ~]# systemctl stop postgresql-17
[root@lxicbpgdsgv01 ~]# 
[root@lxicbpgdsgv01 ~]# ps -ef | grep postgres
root        5759    4079  0 18:09 pts/1    00:00:00 grep --color=auto postgres
[root@lxicbpgdsgv01 ~]#

2. Create the New Directory

[root@lxicbpgdsgv01 ~]# mkdir -p /pgData/pgsql17/data
[root@lxicbpgdsgv01 ~]# chown postgres:postgres /pgData/pgsql17/data
[root@lxicbpgdsgv01 ~]# chmod 700 /pgData/pgsql17/data
[root@lxicbpgdsgv01 ~]#

3. Copy Existing Data to New Location

[postgres@lxicbpgdsgv01 ~]$ nohup rsync -av --progress /var/lib/pgsql/17/data/ /pgData/pgsql17/data/ > rsync_pgdata.log 2>&1 &
[1] 5863
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ tail -f rsync_pgdata.log
pg_wal/000000010000000000000002
     16,777,216 100%   17.68MB/s    0:00:00 (xfr#1621, to-chk=3/1652)
pg_wal/archive_status/
pg_wal/summaries/
pg_xact/
pg_xact/0000
          8,192 100%    8.84kB/s    0:00:00 (xfr#1622, to-chk=0/1652)

sent 73,827,441 bytes  received 31,005 bytes  49,238,964.00 bytes/sec
total size is 73,717,583  speedup is 1.00
^C[1]+  Done                    nohup rsync -av --progress /var/lib/pgsql/17/data/ /pgData/pgsql17/data/ > rsync_pgdata.log 2>&1

[postgres@lxicbpgdsgv01 ~]$

4. Move existing data directory

[postgres@lxicbpgdsgv01 ~]$ mv /var/lib/pgsql/17/data /var/lib/pgsql/17/data_bkp_10072025
[postgres@lxicbpgdsgv01 ~]$

5. Update PostgreSQL Configuration to Use New Data Directory

[postgres@lxicbpgdsgv01 ~]$ cat /usr/lib/systemd/system/postgresql-17.service | grep -i "Environment=PGDATA"
Environment=PGDATA=/var/lib/pgsql/17/data/  <-----
[postgres@lxicbpgdsgv01 ~]$

[root@lxicbpgdsgv01 ~]# ls -ltr /usr/lib/systemd/system/postgresql-17.service
-rw-r--r--. 1 root root 1788 Aug 13 17:39 /usr/lib/systemd/system/postgresql-17.service
[root@lxicbpgdsgv01 ~]#

[root@lxicbpgdsgv01 ~]# sed -i 's|^Environment=PGDATA=.*|Environment=PGDATA=/pgData/pgsql17/data/|' /usr/lib/systemd/system/postgresql-17.service
[root@lxicbpgdsgv01 ~]#
[root@lxicbpgdsgv01 ~]# cat /usr/lib/systemd/system/postgresql-17.service | grep -i "Environment=PGDATA"
Environment=PGDATA=/pgData/pgsql17/data/ <------
[root@lxicbpgdsgv01 ~]#

6. Reload systemd and start PostgreSQL

[root@lxicbpgdsgv01 ~]# systemctl daemon-reload
[root@lxicbpgdsgv01 ~]#

[root@lxicbpgdsgv01 ~]# systemctl start postgresql-17
[root@lxicbpgdsgv01 ~]#

[root@lxicbpgdsgv01 ~]# systemctl cat postgresql-17 | grep -i Environment=PGDATA
Environment=PGDATA=/pgData/pgsql17/data/ <-----
[root@lxicbpgdsgv01 ~]#

[root@lxicbpgdsgv01 ~]# systemctl status postgresql-17
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Tue 2025-10-07 18:33:05 +08; 2min 26s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 5911 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 5916 (postgres)
      Tasks: 7 (limit: 15700)
     Memory: 21.3M
        CPU: 177ms
     CGroup: /system.slice/postgresql-17.service
             ├─5916 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
             ├─5917 "postgres: logger "
             ├─5918 "postgres: checkpointer "
             ├─5919 "postgres: background writer "
             ├─5921 "postgres: walwriter "
             ├─5922 "postgres: autovacuum launcher "
             └─5923 "postgres: logical replication launcher "

Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com postgres[5916]: 2025-10-07 18:33:05.392 +08 [5916] LOG:  redirecting log output to logging collector process
Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com postgres[5916]: 2025-10-07 18:33:05.392 +08 [5916] HINT:  Future log output will appear in directory "log".
Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.

7. Edit postgres .bash_profile

[postgres@lxicbpgdsgv01 ~]$ cat .bash_profile | grep -i PGDATA=
PGDATA=/var/lib/pgsql/17/data
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ sed -i 's|^PGDATA=.*|PGDATA=/pgData/pgsql17/data|' ~/.bash_profile
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ cat .bash_profile | grep -i PGDATA=
PGDATA=/pgData/pgsql17/data <-----
[postgres@lxicbpgdsgv01 ~]$ . .bash_profile <-- reload the profile
[postgres@lxicbpgdsgv01 ~]$

8. Verify New Data Directory is in Use

[postgres@lxicbpgdsgv01 ~]$ psql -c "SHOW data_directory;"
    data_directory
----------------------
 /pgData/pgsql17/data  <---- new data directory
(1 row)

[postgres@lxicbpgdsgv01 ~]$

9. Remove Old Data Directory (Optional – A week after)

[postgres@lxicbpgdsgv01 ~]$ ls -ld /var/lib/pgsql/17/data_bkp_10072025
drwx------. 20 postgres postgres 4096 Oct  7 18:20 /var/lib/pgsql/17/data_bkp_10072025
[postgres@lxicbpgdsgv01 ~]$ rm -rf /var/lib/pgsql/17/data_bkp_10072025
[postgres@lxicbpgdsgv01 ~]$

 

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

pg_dump & pg_restore

PostgreSQL, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

PostgreSQL pg_dump and pg_restore Guide

Table of Contents
___________________________________________________________________________________________________

Backup:

0. pg_dump / pg_restore –help
1. Backup Output Formats
2. Full Database Backup

3. Schema Level Backup
4. Table Level Backup
5. Object Level Backup (PostgreSQL v17 Feature)

Restore:

6. Pre-requisites for Restore
7. Full Database Restore
8. Schema Level Restore

9. Table Level Restore

___________________________________________________________________________________________________

0. pg_dump/pg_restore –help

Click to expand pg_dump –help
[postgres@pg17 ~]$ pg_dump --help
pg_dump dumps a database as a text file or to other formats.

Usage:
  pg_dump [OPTION]... [DBNAME]

General options:
  -f, --file=FILENAME          output file or directory name
  -F, --format=c|d|t|p         output file format (custom, directory, tar,
                               plain text (default))
  -j, --jobs=NUM               use this many parallel jobs to dump
  -v, --verbose                verbose mode
  -V, --version                output version information, then exit
  -Z, --compress=0-9           compression level for compressed formats
  --lock-wait-timeout=TIMEOUT  fail after waiting TIMEOUT for a table lock
  --no-sync                    do not wait for changes to be written safely to disk
  -?, --help                   show this help, then exit

Options controlling the output content:
  -a, --data-only              dump only the data, not the schema
  -b, --blobs                  include large objects in dump
  -B, --no-blobs               exclude large objects in dump
  -c, --clean                  clean (drop) database objects before recreating
  -C, --create                 include commands to create database in dump
  -e, --extension=PATTERN      dump the specified extension(s) only
  -E, --encoding=ENCODING      dump the data in encoding ENCODING
  -n, --schema=PATTERN         dump the specified schema(s) only
  -N, --exclude-schema=PATTERN do NOT dump the specified schema(s)
  -O, --no-owner               skip restoration of object ownership in
                               plain-text format
  -s, --schema-only            dump only the schema, no data
  -S, --superuser=NAME         superuser user name to use in plain-text format
  -t, --table=PATTERN          dump the specified table(s) only
  -T, --exclude-table=PATTERN  do NOT dump the specified table(s)
  -x, --no-privileges          do not dump privileges (grant/revoke)
  --binary-upgrade             for use by upgrade utilities only
  --column-inserts             dump data as INSERT commands with column names
  --disable-dollar-quoting     disable dollar quoting, use SQL standard quoting
  --disable-triggers           disable triggers during data-only restore
  --enable-row-security        enable row security (dump only content user has
                               access to)
  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)
  --extra-float-digits=NUM     override default setting for extra_float_digits
  --if-exists                  use IF EXISTS when dropping objects
  --include-foreign-data=PATTERN
                               include data of foreign tables on foreign
                               servers matching PATTERN
  --inserts                    dump data as INSERT commands, rather than COPY
  --load-via-partition-root    load partitions via the root table
  --no-comments                do not dump comments
  --no-publications            do not dump publications
  --no-security-labels         do not dump security label assignments
  --no-subscriptions           do not dump subscriptions
  --no-table-access-method     do not dump table access methods
  --no-tablespaces             do not dump tablespace assignments
  --no-toast-compression       do not dump TOAST compression methods
  --no-unlogged-table-data     do not dump unlogged table data
  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands
  --quote-all-identifiers      quote all identifiers, even if not key words
  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts
  --section=SECTION            dump named section (pre-data, data, or post-data)
  --serializable-deferrable    wait until the dump can run without anomalies
  --snapshot=SNAPSHOT          use given snapshot for the dump
  --strict-names               require table and/or schema include patterns to
                               match at least one entity each
  --use-set-session-authorization
                               use SET SESSION AUTHORIZATION commands instead of
                               ALTER OWNER commands to set ownership

Connection options:
  -d, --dbname=DBNAME      database to dump
  -h, --host=HOSTNAME      database server host or socket directory
  -p, --port=PORT          database server port number
  -U, --username=NAME      connect as specified database user
  -w, --no-password        never prompt for password
  -W, --password           force password prompt (should happen automatically)
  --role=ROLENAME          do SET ROLE before dump

If no database name is supplied, then the PGDATABASE environment
variable value is used.

Report bugs to <pgsql-bugs@lists.postgresql.org>.
PostgreSQL home page: <https://www.postgresql.org/>
[postgres@pg17 ~]$
Click to expand pg_restore –help
[postgres@pg17 ~]$ pg_restore --help
pg_restore restores a PostgreSQL database from an archive created by pg_dump.

Usage:
  pg_restore [OPTION]... [FILE]

General options:
  -d, --dbname=NAME        connect to database name
  -f, --file=FILENAME      output file name (- for stdout)
  -F, --format=c|d|t       backup file format (should be automatic)
  -l, --list               print summarized TOC of the archive
  -v, --verbose            verbose mode
  -V, --version            output version information, then exit
  -?, --help               show this help, then exit

Options controlling the restore:
  -a, --data-only              restore only the data, no schema
  -c, --clean                  clean (drop) database objects before recreating
  -C, --create                 create the target database
  -e, --exit-on-error          exit on error, default is to continue
  -I, --index=NAME             restore named index
  -j, --jobs=NUM               use this many parallel jobs to restore
  -L, --use-list=FILENAME      use table of contents from this file for
                               selecting/ordering output
  -n, --schema=NAME            restore only objects in this schema
  -N, --exclude-schema=NAME    do not restore objects in this schema
  -O, --no-owner               skip restoration of object ownership
  -P, --function=NAME(args)    restore named function
  -s, --schema-only            restore only the schema, no data
  -S, --superuser=NAME         superuser user name to use for disabling triggers
  -t, --table=NAME             restore named relation (table, view, etc.)
  -T, --trigger=NAME           restore named trigger
  -x, --no-privileges          skip restoration of access privileges (grant/revoke)
  -1, --single-transaction     restore as a single transaction
  --disable-triggers           disable triggers during data-only restore
  --enable-row-security        enable row security
  --if-exists                  use IF EXISTS when dropping objects
  --no-comments                do not restore comments
  --no-data-for-failed-tables  do not restore data of tables that could not be
                               created
  --no-publications            do not restore publications
  --no-security-labels         do not restore security labels
  --no-subscriptions           do not restore subscriptions
  --no-table-access-method     do not restore table access methods
  --no-tablespaces             do not restore tablespace assignments
  --section=SECTION            restore named section (pre-data, data, or post-data)
  --strict-names               require table and/or schema include patterns to
                               match at least one entity each
  --use-set-session-authorization
                               use SET SESSION AUTHORIZATION commands instead of
                               ALTER OWNER commands to set ownership

Connection options:
  -h, --host=HOSTNAME      database server host or socket directory
  -p, --port=PORT          database server port number
  -U, --username=NAME      connect as specified database user
  -w, --no-password        never prompt for password
  -W, --password           force password prompt (should happen automatically)
  --role=ROLENAME          do SET ROLE before restore

The options -I, -n, -N, -P, -t, -T, and --section can be combined and specified
multiple times to select multiple objects.

If no input file name is supplied, then standard input is used.

Report bugs to <pgsql-bugs@lists.postgresql.org>.
PostgreSQL home page: <https://www.postgresql.org/>
[postgres@pg17 ~]$

1. Backup Output Formats

A. Plain Text Format
# Using -Fp explicitly:
pg_dump -U postgres -d your_db_name -Fp -f /path/to/your_db.sql

# Shell redirect
pg_dump -U postgres -d your_db_name > /path/to/your_db.sql

B. Tar Format
pg_dump -U postgres -d your_db_name -Ft -f /path/to/your_db.tar

C. Directory Format and Parallel Backup
-- Parallel backup is only supported with -Fd
-- pg_dump will create the directory if it doesn't exist.

# Without parallelism
pg_dump -U postgres -d your_db_name -Fd -f /path/to/backup_dir/

# With parallelism
pg_dump -U postgres -d your_db_name -Fd -j8 -f /path/to/backup_dir/

D. Custom Format (Compressed binary format by default)
pg_dump -U postgres -d your_db_name -Fc -f /path/to/your_db.dump


2. Full Database Backup


A. Plain Text Format

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v > /pgBackup/pgsql15/backup/blpua/database_blpua_full.sql 2> /pgBackup/pgsql15/backup/blpua/log/database_blpua_full.log &

-- OR --

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v -Fp -f /pgBackup/pgsql15/backup/blpua/database_blpua_full1.sql > /pgBackup/pgsql15/backup/blpua/log/database_blpua_full1.log 2>&1 &

-- OR -- 

[postgres@pg17 ~]$ pg_dump -U postgres -d blpua | split -b 1G - /pgBackup/pgsql15/backup/blpua/database_blpua_split.dmp
[postgres@pg17 ~]$
[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/database_blpua_s*
-rw-r--r--. 1 postgres postgres 1.0G Sep 30 11:03 /pgBackup/pgsql15/backup/blpua/database_blpua_split.dmpaa
-rw-r--r--. 1 postgres postgres 650M Sep 30 11:03 /pgBackup/pgsql15/backup/blpua/database_blpua_split.dmpab
[postgres@pg17 ~]$

-- OR --

[postgres@pg17 ~]$ nohup sh -c "pg_dump -U postgres -d blpua -v | gzip > /pgBackup/pgsql15/backup/blpua/database_blpua.gz" > /pgBackup/pgsql15/backup/blpua/log/database_blpua.log 2>&1 &

-- OR --

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v > /pgBackup/pgsql15/backup/blpua/database_blpua.sql 2> /pgBackup/pgsql15/backup/blpua/log/database_blpua.log & gzip /pgBackup/pgsql15/backup/blpua/database_blpua.sql


B. Tar Format

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v -Ft -f /pgBackup/pgsql15/backup/blpua/database_blpua.tar > /pgBackup/pgsql15/backup/blpua/log/blpua_tar_dump.log 2>&1 &


C. Directory Format

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v -Fd -j4 -f /pgBackup/pgsql15/backup/blpua_dir > /pgBackup/pgsql15/backup/blpua/log/blpua_dump.log 2>&1 &


D. Custome Format

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -Fc -v -f /pgBackup/pgsql15/backup/blpua/database_blpua_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/database_blpua_$(date +%Y_%m_%d).log 2>&1 &
[postgres@pg17 ~]$ 
[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/database_blpua*.dmp
-rw-r--r--. 1 postgres postgres 402M Sep 29 04:58 /pgBackup/pgsql15/backup/blpua/database_blpua_2025_09_29.dmp

[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/log/database_blpua*.log
-rw-r--r--. 1 postgres postgres 2.5K Sep 29 04:58 /pgBackup/pgsql15/backup/blpua/log/database_blpua_2025_09_29.log
[postgres@pg17 ~]$


3. Schema Level Backup

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -n blp -Fc -v -f /pgBackup/pgsql15/backup/blpua/schema_blp_blpua_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua_$(date +%Y_%m_%d).log 2>&1 &
[1] 5472
[postgres@pg17 ~]$

[postgres@pg17 ~]$ jobs -l
[1]+  5472 Running                 nohup pg_dump -U postgres -d blpua -n blp -Fc -v -f /pgBackup/pgsql15/backup/blpua/schema_blp_blpua_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua_$(date +%Y_%m_%d).log 2>&1 &
[postgres@pg17 ~]$

[postgres@pg17 ~]$ jobs -l
[1]+  5472 Done                    nohup pg_dump -U postgres -d blpua -n blp -Fc -v -f /pgBackup/pgsql15/backup/blpua/schema_blp_blpua_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua_$(date +%Y_%m_%d).log 2>&1
[postgres@pg17 ~]$


[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/schema_blp_blpua*
-rw-r--r--. 1 postgres postgres 402M Sep 30 02:09 /pgBackup/pgsql15/backup/blpua/schema_blp_blpua_2025_09_30.dmp
[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua*
-rw-r--r--. 1 postgres postgres 2.5K Sep 30 02:09 /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua_2025_09_30.log
[postgres@pg17 ~]$


4. Table Level Backup

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -t blp.employees -Fc -v -f /pgBackup/pgsql15/backup/blpua/table_blp_employees_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/table_blp_employees_$(date +%Y_%m_%d).log 2>&1 &
[1] 5652
[postgres@pg17 ~]$ 

[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/table_blp_employees_*
-rw-r--r--. 1 postgres postgres 62M Sep 30 02:22 /pgBackup/pgsql15/backup/blpua/table_blp_employees_2025_09_30.dmp
[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/log/table_blp_employees_*
-rw-r--r--. 1 postgres postgres 2.0K Sep 30 02:22 /pgBackup/pgsql15/backup/blpua/log/table_blp_employees_2025_09_30.log
[postgres@pg17 ~]$


5. Object Level Backup (PostgreSQL v17 Feature)

++ filter is only for pg_dump, not pg_restore

[postgres@lxicbpgdsgv01 ~]$ cat include_tables.par
include table demo.table_1
include table demo.table_2
include table demo.table_3
[postgres@lxicbpgdsgv01 ~]$


[postgres@lxicbpgdsgv01 ~]$ pg_dump -d testdb_source --filter=include_tables.par > include_tables.sql

[postgres@lxicbpgdsgv01 ~]$ ls -lrth include_tables.sql
-rw-r--r--. 1 postgres postgres 4.5K Sep 30 22:19 include_tables.sql
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ cat include_tables.sql | grep -i "CREATE TABLE"
CREATE TABLE demo.table_1 (
CREATE TABLE demo.table_2 (
CREATE TABLE demo.table_3 (
[postgres@lxicbpgdsgv01 ~]$


[postgres@lxicbpgdsgv01 ~]$ cat exclude_tables.par
exclude table demo.table_1
exclude table demo.table_2
exclude table demo.table_3
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ pg_dump -d testdb_source --filter=exclude_tables.par > exclude_tables.sql
[postgres@lxicbpgdsgv01 ~]$ ls -ltr exclude_tables.sql
-rw-r--r--. 1 postgres postgres 10931 Sep 30 22:23 exclude_tables.sql
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ cat exclude_tables.sql | grep -i "CREATE TABLE"
CREATE TABLE demo.table_10 (
CREATE TABLE demo.table_4 (
CREATE TABLE demo.table_5 (
CREATE TABLE demo.table_6 (
CREATE TABLE demo.table_7 (
CREATE TABLE demo.table_8 (
CREATE TABLE demo.table_9 (
[postgres@lxicbpgdsgv01 ~]$


6. Pre-requisites for restore

postgres=# CREATE TABLESPACE TEST_TB LOCATION '/pgTb/pgsql15/test';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE TEST_IX LOCATION '/pgIx/pgsql15/test';
CREATE TABLESPACE
postgres=#
postgres=# CREATE DATABASE TEST TABLESPACE TEST_TB;
CREATE DATABASE
postgres=#

[postgres@pg17 blpua]$ scp *.dmp postgres@192.168.2.32:/pgBackup/pgsql15/backup/test
postgres@192.168.2.32's password:
database_blpua_2025_09_29.dmp                           100%  402MB  39.9MB/s   00:10
schema_blp_blpua_2025_09_30.dmp                         100%  402MB  43.8MB/s   00:09
table_blp_employees_2025_09_30.dmp                      100%   61MB  47.0MB/s   00:01
[postgres@pg17 blpua]$


7. Full Database Restore

++ The source database uses two custom tablespaces: blpua_tbs01 and blpua_ixtbs01.
++ On the target system, I have two different tablespaces: test_tb and test_ix.
++ Unlike Oracle, PostgreSQL does not provide a direct equivalent of REMAP_TABLESPACE.
++ Therefore, to simplify the restore and ensure all objects are placed into the default tablespace (test_tb).
++ Using --no-owner --no-tablespaces in pg_restore — even if not specified, data still gets written to the default database tablespace. These options just help avoid errors in the log.
++ This ensures that all objects are restored to the default tablespace of the target database, regardless of their original tablespace assignments.
++ Later, we can move all indexes to the desired target tablespace using the ALTER INDEX ... SET TABLESPACE command.
postgres=# \c test
You are now connected to database "test" as user "postgres".
test=# ALTER INDEX blp.idx_employee_name SET TABLESPACE test_ix;  <--  This command physically relocates the index to the test_ix tablespace.
ALTER INDEX
test=#


[postgres@lxtrdpgdsgv01 ~]$ nohup pg_restore -U postgres -d test --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/database_blpua_2025_09_29.dmp > /pgBackup/pgsql15/backup/test/log/restore_blpua_test_$(date +%Y_%m_%d).log 2>&1 &
[1] 5300
[postgres@lxtrdpgdsgv01 ~]$

[postgres@lxtrdpgdsgv01 ~]$ cat /pgBackup/pgsql15/backup/test/log/restore_blpua_test_2025_09_30.log
nohup: ignoring input
pg_restore: connecting to database for restore
pg_restore: creating SCHEMA "blp"
pg_restore: creating TYPE "blp.address"
pg_restore: creating TYPE "blp.employment_status"
pg_restore: creating DOMAIN "blp.positive_integer"
pg_restore: creating FUNCTION "blp.get_salary(integer)"
pg_restore: creating FUNCTION "blp.log_update()"
pg_restore: creating PROCEDURE "blp.raise_salary(integer, numeric)"
pg_restore: creating SEQUENCE "blp.emp_id_seq"
pg_restore: creating TABLE "blp.employees"
pg_restore: creating MATERIALIZED VIEW "blp.emp_summary"
pg_restore: creating SEQUENCE "blp.employees_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.employees_id_seq"
pg_restore: creating VIEW "blp.high_paid_employees"
pg_restore: creating TABLE "blp.metrics"
pg_restore: creating TABLE "blp.metrics_high"
pg_restore: creating SEQUENCE "blp.metrics_id_seq"
pg_restore: creating TABLE "blp.metrics_low"
pg_restore: creating TABLE "blp.metrics_mid"
pg_restore: creating TABLE "blp.metrics_rest"
pg_restore: creating TABLE "blp.sales"
pg_restore: creating TABLE "blp.sales_2021"
pg_restore: creating TABLE "blp.sales_2022"
pg_restore: creating TABLE "blp.sales_2023"
pg_restore: creating TABLE "blp.sales_2024"
pg_restore: creating SEQUENCE "blp.sales_id_seq"
pg_restore: creating TABLE "blp.test_data"
pg_restore: creating SEQUENCE "blp.test_data_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.test_data_id_seq"
pg_restore: creating TABLE ATTACH "blp.metrics_high"
pg_restore: creating TABLE ATTACH "blp.metrics_low"
pg_restore: creating TABLE ATTACH "blp.metrics_mid"
pg_restore: creating TABLE ATTACH "blp.metrics_rest"
pg_restore: creating TABLE ATTACH "blp.sales_2021"
pg_restore: creating TABLE ATTACH "blp.sales_2022"
pg_restore: creating TABLE ATTACH "blp.sales_2023"
pg_restore: creating TABLE ATTACH "blp.sales_2024"
pg_restore: creating DEFAULT "blp.employees id"
pg_restore: creating DEFAULT "blp.test_data id"
pg_restore: processing data for table "blp.employees"
pg_restore: processing data for table "blp.metrics_high"
pg_restore: processing data for table "blp.metrics_low"
pg_restore: processing data for table "blp.metrics_mid"
pg_restore: processing data for table "blp.metrics_rest"
pg_restore: processing data for table "blp.sales_2021"
pg_restore: processing data for table "blp.sales_2022"
pg_restore: processing data for table "blp.sales_2023"
pg_restore: processing data for table "blp.sales_2024"
pg_restore: processing data for table "blp.test_data"
pg_restore: executing SEQUENCE SET emp_id_seq
pg_restore: executing SEQUENCE SET employees_id_seq
pg_restore: executing SEQUENCE SET metrics_id_seq
pg_restore: executing SEQUENCE SET sales_id_seq
pg_restore: executing SEQUENCE SET test_data_id_seq
pg_restore: creating CONSTRAINT "blp.employees employees_pkey"
pg_restore: creating CONSTRAINT "blp.metrics metrics_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_high metrics_high_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_low metrics_low_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_mid metrics_mid_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_rest metrics_rest_pkey"
pg_restore: creating CONSTRAINT "blp.sales sales_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2021 sales_2021_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2022 sales_2022_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2023 sales_2023_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2024 sales_2024_pkey"
pg_restore: creating CONSTRAINT "blp.test_data test_data_pkey"
pg_restore: creating INDEX "blp.idx_employee_name"
pg_restore: creating INDEX ATTACH "blp.metrics_high_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_low_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_mid_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_rest_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2021_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2022_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2023_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2024_pkey"
pg_restore: creating TRIGGER "blp.employees employee_update_trigger"
pg_restore: creating MATERIALIZED VIEW DATA "blp.emp_summary"
[postgres@lxtrdpgdsgv01 ~]$


8. Schema Level Restore

++ pg_restore does not support schema remapping like Oracle’s REMAP_SCHEMA.
++ I want to restore schema blp --> trd. 
++ Option A: Restore Dump as it is, Then Rename <--- Best for simplicity
++ Option B: Split Dump into Schema + Data, Modify Both <--- This is more complex, but possible if you must use schema remapping.
++ This is schema dump file : /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
++ Using --no-owner --no-tablespaces in pg_restore — even if not specified, data still gets written to the default database tablespace. These options just help avoid errors in the log.


A. To Restore schema from own schema’s backup

postgres=# CREATE TABLESPACE ORCL_TB LOCATION '/pgTb/pgsql15/orcl';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE ORCL_IX LOCATION '/pgIx/pgsql15/orcl';
CREATE TABLESPACE
postgres=#
postgres=# CREATE DATABASE ORCL TABLESPACE ORCL_TB;
CREATE DATABASE
postgres=#

-- With NO parallelism


[postgres@lxtrdpgdsgv01 ~]$ nohup pg_restore -U postgres -d orcl --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp > /pgBackup/pgsql15/backup/test/log/restore_blpua_test_$(date +%Y_%m_%d).log 2>&1 &
[1] 7344
[postgres@lxtrdpgdsgv01 ~]$

postgres=# \c orcl
You are now connected to database "orcl" as user "postgres".
orcl=# ALTER SCHEMA blp RENAME TO trd;
ALTER SCHEMA
orcl=# \dn
      List of schemas
  Name  |       Owner
--------+-------------------
 public | pg_database_owner
 trd    | postgres
(2 rows)

orcl=#


[postgres@lxtrdpgdsgv01 ~]$ cat /pgBackup/pgsql15/backup/test/log/restore_blpua_test_2025_09_30.log
nohup: ignoring input
pg_restore: connecting to database for restore
pg_restore: creating SCHEMA "blp"
pg_restore: creating TYPE "blp.address"
pg_restore: creating TYPE "blp.employment_status"
pg_restore: creating DOMAIN "blp.positive_integer"
pg_restore: creating FUNCTION "blp.get_salary(integer)"
pg_restore: creating FUNCTION "blp.log_update()"
pg_restore: creating PROCEDURE "blp.raise_salary(integer, numeric)"
pg_restore: creating SEQUENCE "blp.emp_id_seq"
pg_restore: creating TABLE "blp.employees"
pg_restore: creating MATERIALIZED VIEW "blp.emp_summary"
pg_restore: creating SEQUENCE "blp.employees_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.employees_id_seq"
pg_restore: creating VIEW "blp.high_paid_employees"
pg_restore: creating TABLE "blp.metrics"
pg_restore: creating TABLE "blp.metrics_high"
pg_restore: creating SEQUENCE "blp.metrics_id_seq"
pg_restore: creating TABLE "blp.metrics_low"
pg_restore: creating TABLE "blp.metrics_mid"
pg_restore: creating TABLE "blp.metrics_rest"
pg_restore: creating TABLE "blp.sales"
pg_restore: creating TABLE "blp.sales_2021"
pg_restore: creating TABLE "blp.sales_2022"
pg_restore: creating TABLE "blp.sales_2023"
pg_restore: creating TABLE "blp.sales_2024"
pg_restore: creating SEQUENCE "blp.sales_id_seq"
pg_restore: creating TABLE "blp.test_data"
pg_restore: creating SEQUENCE "blp.test_data_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.test_data_id_seq"
pg_restore: creating TABLE ATTACH "blp.metrics_high"
pg_restore: creating TABLE ATTACH "blp.metrics_low"
pg_restore: creating TABLE ATTACH "blp.metrics_mid"
pg_restore: creating TABLE ATTACH "blp.metrics_rest"
pg_restore: creating TABLE ATTACH "blp.sales_2021"
pg_restore: creating TABLE ATTACH "blp.sales_2022"
pg_restore: creating TABLE ATTACH "blp.sales_2023"
pg_restore: creating TABLE ATTACH "blp.sales_2024"
pg_restore: creating DEFAULT "blp.employees id"
pg_restore: creating DEFAULT "blp.test_data id"
pg_restore: processing data for table "blp.employees"
pg_restore: processing data for table "blp.metrics_high"
pg_restore: processing data for table "blp.metrics_low"
pg_restore: processing data for table "blp.metrics_mid"
pg_restore: processing data for table "blp.metrics_rest"
pg_restore: processing data for table "blp.sales_2021"
pg_restore: processing data for table "blp.sales_2022"
pg_restore: processing data for table "blp.sales_2023"
pg_restore: processing data for table "blp.sales_2024"
pg_restore: processing data for table "blp.test_data"
pg_restore: executing SEQUENCE SET emp_id_seq
pg_restore: executing SEQUENCE SET employees_id_seq
pg_restore: executing SEQUENCE SET metrics_id_seq
pg_restore: executing SEQUENCE SET sales_id_seq
pg_restore: executing SEQUENCE SET test_data_id_seq
pg_restore: creating CONSTRAINT "blp.employees employees_pkey"
pg_restore: creating CONSTRAINT "blp.metrics metrics_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_high metrics_high_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_low metrics_low_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_mid metrics_mid_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_rest metrics_rest_pkey"
pg_restore: creating CONSTRAINT "blp.sales sales_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2021 sales_2021_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2022 sales_2022_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2023 sales_2023_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2024 sales_2024_pkey"
pg_restore: creating CONSTRAINT "blp.test_data test_data_pkey"
pg_restore: creating INDEX "blp.idx_employee_name"
pg_restore: creating INDEX ATTACH "blp.metrics_high_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_low_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_mid_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_rest_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2021_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2022_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2023_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2024_pkey"
pg_restore: creating TRIGGER "blp.employees employee_update_trigger"
pg_restore: creating MATERIALIZED VIEW DATA "blp.emp_summary"
[postgres@lxtrdpgdsgv01 ~]$


--- OR ---

-- With parallelism 4

nohup pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp > /pgBackup/pgsql15/backup/test/log/restore_blpua_test_$(date +%Y_%m_%d).log1 2>&1 &

[postgres@lxtrdpgdsgv01 ~]$ ps -ef | grep pg_restore
postgres    7089    5102  0 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7091    7089  2 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7092    7089  3 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7093    7089  2 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7094    7089  2 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7126    5736  0 05:49 pts/1    00:00:00 grep --color=auto pg_restore
[postgres@lxtrdpgdsgv01 ~]$


B. To Restore only a specific schema from Full Database Backup

Step 1: Create schema 

gebua=# CREATE SCHEMA IF NOT EXISTS blp;

Step 2: Restore Objects and Data

nohup pg_restore -U postgres -d gebua -n blp --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/database_blpua_2025_09_29.dmp > /pgBackup/pgsql15/backup/test/log/restore_blpua_test_$(date +%Y_%m_%d).log_full 2>&1 &


9. Table Level Restore


A. Restore Table from own table’s backup

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -l /pgBackup/pgsql15/backup/test/table_blp_employees_2025_09_30.dmp
;
; Archive created at 2025-09-30 02:22:21 EDT
;     dbname: blpua
;     TOC Entries: 13
;     Compression: -1
;     Dump Version: 1.14-0
;     Format: CUSTOM
;     Integer: 4 bytes
;     Offset: 8 bytes
;     Dumped from database version: 15.13
;     Dumped by pg_dump version: 15.13
;
;
; Selected TOC Entries:
;
229; 1259 92805 TABLE blp employees postgres
228; 1259 92804 SEQUENCE blp employees_id_seq postgres
4338; 0 0 SEQUENCE OWNED BY blp employees_id_seq postgres
4181; 2604 92808 DEFAULT blp employees id postgres
4331; 0 92805 TABLE DATA blp employees postgres
4339; 0 0 SEQUENCE SET blp employees_id_seq postgres
4183; 2606 92812 CONSTRAINT blp employees employees_pkey postgres
4184; 1259 92824 INDEX blp idx_employee_name postgres
4185; 2620 92829 TRIGGER blp employees employee_update_trigger postgres
[postgres@lxtrdpgdsgv01 ~]$


Step 1: Create schema 

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=# CREATE SCHEMA IF NOT EXISTS blp;
CREATE SCHEMA
edpua=#


Step 2: Restore Objects and Data

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -d edpua --no-tablespaces -v /pgBackup/pgsql15/backup/test/table_blp_employees_2025_09_30.dmp
pg_restore: connecting to database for restore
pg_restore: creating TABLE "blp.employees"
pg_restore: creating SEQUENCE "blp.employees_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.employees_id_seq"
pg_restore: creating DEFAULT "blp.employees id"
pg_restore: processing data for table "blp.employees"
pg_restore: executing SEQUENCE SET employees_id_seq
pg_restore: creating CONSTRAINT "blp.employees employees_pkey"
pg_restore: creating INDEX "blp.idx_employee_name"
pg_restore: creating TRIGGER "blp.employees employee_update_trigger"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 4185; 2620 92829 TRIGGER employees employee_update_trigger postgres
pg_restore: error: could not execute query: ERROR:  function blp.log_update() does not exist
Command was: CREATE TRIGGER employee_update_trigger AFTER UPDATE ON blp.employees FOR EACH ROW EXECUTE FUNCTION blp.log_update();


pg_restore: warning: errors ignored on restore: 1
[postgres@lxtrdpgdsgv01 ~]$

++ PostgreSQL does not have a --ignore-errors flag, The above erros ignorable.


B. Restore Single Table data-only from Schema backup

edpua=# TRUNCATE TABLE blp.employees;
TRUNCATE TABLE
edpua=#

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -d edpua --data-only -t employees -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
pg_restore: connecting to database for restore
pg_restore: processing data for table "blp.employees"
[postgres@lxtrdpgdsgv01 ~]$

edpua=# select count(*) from blp.employees;
 5000000  <-----

edpua=#


C. Restore Single Table from Schema backup

Step 1: Create schema 

edpua=# CREATE SCHEMA blp;

Step 2: Restore Objects and Data

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -d edpua --clean -t employees -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
pg_restore: connecting to database for restore
pg_restore: dropping TABLE employees
pg_restore: creating TABLE "blp.employees"
pg_restore: processing data for table "blp.employees"
[postgres@lxtrdpgdsgv01 ~]$
[postgres@lxtrdpgdsgv01 ~]$ psql
psql (15.14)
Type "help" for help.

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=# select count(*) from blp.employees;
  count
---------
 5000000 <----
(1 row)

edpua=#


D. Restore Single Table from Schema backup (add both -n and -t)

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -d edpua --clean -n blp -t employees -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
pg_restore: connecting to database for restore
pg_restore: dropping TABLE employees
pg_restore: creating TABLE "blp.employees"
pg_restore: processing data for table "blp.employees"
[postgres@lxtrdpgdsgv01 ~]$
[postgres@lxtrdpgdsgv01 ~]$ psql
psql (15.14)
Type "help" for help.

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=# select count(*) from blp.employees;
  count
---------
 5000000 <-----
(1 row)

edpua=#


E. Restoring Using a Plain SQL File (COPY-based)

Step 1: Extracted table data from custom-format dump

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -n blp -t employees -f /pgBackup/pgsql15/backup/test/blp_employees.sql /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
[postgres@lxtrdpgdsgv01 ~]$
[postgres@lxtrdpgdsgv01 ~]$ ls -lrth /pgBackup/pgsql15/backup/test/blp_employees.sql
-rw-r--r--. 1 postgres postgres 210M Sep 30 09:17 /pgBackup/pgsql15/backup/test/blp_employees.sql
[postgres@lxtrdpgdsgv01 ~]$

[postgres@lxtrdpgdsgv01 ~]$ head -50 /pgBackup/pgsql15/backup/test/blp_employees.sql
--
-- PostgreSQL database dump
--

\restrict TthhVsS8JvVxrplYNApzHUalZpygXUqFg2pIvbSH8GTkbJ6WBNyW0JLnuuvkPdt

-- Dumped from database version 15.13
-- Dumped by pg_dump version 15.13

SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = off;

SET default_tablespace = '';

SET default_table_access_method = heap;

--
-- Name: employees; Type: TABLE; Schema: blp; Owner: postgres
--

CREATE TABLE blp.employees (
    id integer NOT NULL,
    name text,
    salary numeric(10,2),
    hired_on date
);


ALTER TABLE blp.employees OWNER TO postgres;

--
-- Data for Name: employees; Type: TABLE DATA; Schema: blp; Owner: postgres
--

COPY blp.employees (id, name, salary, hired_on) FROM stdin;
1       Employee_1      70969.47        2005-06-13
2       Employee_2      94341.15        2006-12-09
3       Employee_3      101657.57       2012-07-05
4       Employee_4      54701.92        2010-01-05
5       Employee_5      104229.43       2001-06-28
6       Employee_6      95842.57        2010-10-06
7       Employee_7      50618.08        2015-06-02
[postgres@lxtrdpgdsgv01 ~]$

Step 2: Restore using psql command

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=#
edpua=# drop table blp.employees;
DROP TABLE
edpua=# exit
[postgres@lxtrdpgdsgv01 ~]$


psql -U postgres -d edpua < /pgBackup/pgsql15/backup/test/blp_employees.sql

[postgres@lxtrdpgdsgv01 ~]$ psql -U postgres -d edpua < /pgBackup/pgsql15/backup/test/blp_employees.sql
SET
SET
SET
SET
SET
 set_config
------------

(1 row)

SET
SET
SET
SET
SET
SET
CREATE TABLE
ALTER TABLE
COPY 5000000
[postgres@lxtrdpgdsgv01 ~]$

[postgres@lxtrdpgdsgv01 ~]$ psql
psql (15.14)
Type "help" for help.

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=# SELECT COUNT(*) FROM blp.employees;
  count
---------
 5000000 <-----
(1 row)

edpua=#

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/