Tag Archives: PostgreSQL

Backup and Restore on Same Host Using pg_basebackup

PostgreSQL, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

How to Backup and Restore PostgreSQL DB Cluster on the Same Host Using pg_basebackup

Table of Contents

1. Environment
2. Verify Existing DB Setup
3. Pre-requisites
4. Take Backup
5. Prepare & Restore
     5.1 Stop PostgreSQL and Clean Directories
     5.2 Remove all from Data & WAL directories
     5.3 Restore Data and WAL
         A. Copy Backup files to $PGDATA
         B. Copy WAL files from $PGDATA/pg_wal to /pgWal/pgsql17/wal/
         C. Remove the existing $PGDATA/pg_wal directory
         D. Create a symbolic link pointing $PGDATA/pg_wal to a separate WAL directory
         E. Set Permissions
6. Start PostgreSQL
7. Final Verification

1. Environment

ASPECTEnv
Hostnamelxicbpgdsgv01
IP Address192.168.2.51
OSRHEL 9
DB VersionPostgreSQL v17.6
Archive modearchive_mode=off
pgData/pgData/pgsql17/data
WAL Directory/pgWal/pgsql17/wal
Tablespacepg_default
DatabasesDELL, ORCL

2. Verify Existing DB Setup

[postgres@lxicbpgdsgv01 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# \l+
                                                                                       List of databases
   Name    |  Owner   | Encoding | Locale Provider |   Collate   |    Ctype    | Locale | ICU Rules |   Access privileges   |  Size   | Tablespace |                Description
-----------+----------+----------+-----------------+-------------+-------------+--------+-----------+-----------------------+---------+------------+--------------------------------------------
 dell      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7763 kB | pg_default |
 orcl      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7907 kB | pg_default |
 postgres  | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 492 MB  | pg_default | default administrative connection database
 template0 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7545 kB | pg_default | unmodifiable empty database
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
 template1 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7723 kB | pg_default | default template for new databases
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
(5 rows)

postgres=# \db
       List of tablespaces
    Name    |  Owner   | Location
------------+----------+----------
 pg_default | postgres |
 pg_global  | postgres |
(2 rows)

postgres=# \c dell
You are now connected to database "dell" as user "postgres".
dell=# select * from test.emp;
 name | designation | project | company
------+-------------+---------+---------
 Sugi | DBA         | Jetstar | iGATE
 Teja | DBA         | RCM     | iGATE
 RAJ  | DBA         | RCM     | iGATE
(3 rows)

dell=#

3. Pre-requisites

  • Verify postgresql.conf
postgres=# SHOW wal_level;
 wal_level
-----------
 replica <----
(1 row)

postgres=#
postgres=# SHOW max_wal_senders;
 max_wal_senders
-----------------
 10
(1 row)

postgres=# SHOW archive_mode;
 archive_mode
--------------
 off
(1 row)

postgres=#
  • Verify pg_hba.conf
# TYPE  DATABASE        USER            ADDRESS             METHOD
# Local connections for replication (for pg_basebackup run locally)
local   replication     all                                 trust

# Remote connections for replication (for pg_basebackup run remotely)
#host    replication     repl_user       192.168.2.52/32     scram-sha-256
  • Verify user permissions:REPLICATION or SUPERUSER required
postgres=# \du
                             List of roles
 Role name |                         Attributes
-----------+------------------------------------------------------------
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS

postgres=#
  • Verify free space for backup
[postgres@lxicbpgdsgv01 ~]$ du -sh /pgData/pgsql17/data/
524M    /pgData/pgsql17/data/
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ du -sh /pgWal/pgsql17/wal/
801M    /pgWal/pgsql17/wal/
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ df -h /pgBackup/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdd1       100G  746M  100G   1% /pgBackup <----
[postgres@lxicbpgdsgv01 ~]$ mkdir -p /pgBackup/pgsql17/backup/basebackup_10OCT2025
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ mkdir -p /pgBackup/pgsql17/backup/log/
[postgres@lxicbpgdsgv01 ~]$

4. Take Backup

[postgres@lxicbpgdsgv01 ~]$ ls -ltrh /pgWal/pgsql17/wal | wc -l
53 <---- 
[postgres@lxicbpgdsgv01 ~]$

pg_basebackup does not include all files from the pg_wal directory in the backup. Instead, it selectively includes only the WAL files required to make the base backup consistent at the point in time the backup was taken.

-- If you want to store the base backup and WAL files in separate backup directories. 

nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/basebackup_10OCT2025 --waldir=/pgBackup/pgsql17/backup/wal_backup -Fp -Xs -P -v > /pgBackup/pgsql17/backup/log/basebackup_10OCT2025.log 2>&1 &

The --waldir option in pg_basebackup is supported only when using the plain format (-Fp), not with the tar format (-Ft).

If we want symlinks preserved → use (both -Fp & --waldir use together) -Fp --waldir=/pgWal/pgsql17/wal

-- OR --

The directory mush be empty: /pgBackup/pgsql17/backup/basebackup_10OCT2025

[postgres@lxicbpgdsgv01 ~]$ nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/basebackup_10OCT2025 -Fp -Xs -P -v > /pgBackup/pgsql17/backup/log/basebackup_10OCT2025.log 2>&1 &
[1] 4973
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ cat /pgBackup/pgsql17/backup/log/basebackup_10OCT2025.log
nohup: ignoring input
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 0/85000028 on timeline 1
pg_basebackup: starting background WAL receiver
pg_basebackup: created temporary replication slot "pg_basebackup_4975"
279413/536037 kB (52%), 0/1 tablespace (...asebackup_10OCT2025/base/5/16533)
536047/536047 kB (100%), 0/1 tablespace (...ckup_10OCT2025/global/pg_control)
536047/536047 kB (100%), 1/1 tablespace
pg_basebackup: write-ahead log end point: 0/85000158
pg_basebackup: waiting for background process to finish streaming ...
pg_basebackup: syncing data to disk ...
pg_basebackup: renaming backup_manifest.tmp to backup_manifest
pg_basebackup: base backup completed  <-----
[postgres@lxicbpgdsgv01 ~]$

5. Prepare & Restore

5.1 Stop PostgreSQL and Clean Directories

[root@lxicbpgdsgv01 ~]# systemctl stop postgresql-17.service
[root@lxicbpgdsgv01 ~]# 
[root@lxicbpgdsgv01 ~]# ps -ef | grep postgres
root        5057    3151  0 18:57 pts/0    00:00:00 grep --color=auto postgres
[root@lxicbpgdsgv01 ~]#

5.2 Remove all from Data & WAL directories

-- Remove all from PGDATA directory

[root@lxicbpgdsgv01 ~]# rm -rf /pgData/pgsql17/data/*
[root@lxicbpgdsgv01 ~]# ls -ltr /pgData/pgsql17/data/
total 0
[root@lxicbpgdsgv01 ~]#

-- Remove all from WAL directory 

[root@lxicbpgdsgv01 ~]# rm -rf /pgWal/pgsql17/wal/*
[root@lxicbpgdsgv01 ~]# ls -ltr /pgWal/pgsql17/wal/
total 0
[root@lxicbpgdsgv01 ~]#

5.3 Restore Data and WAL

A. Copy Backup files to $PGDATA
Since we did not use the -Fp format with the --waldir option, all required WAL files will be included in the backup under the default path: PGDATA/pg_wal.

[postgres@lxicbpgdsgv01 ~]$ ls -ltr /pgData/pgsql17/data/
total 0
[postgres@lxicbpgdsgv01 ~]$ cd /pgBackup/pgsql17/backup/basebackup_10OCT2025/
[postgres@lxicbpgdsgv01 basebackup_10OCT2025]$ cp -Rp * /pgData/pgsql17/data/
[postgres@lxicbpgdsgv01 basebackup_10OCT2025]$
[postgres@lxicbpgdsgv01 basebackup_10OCT2025]$ ls -ltr /pgData/pgsql17/data/
total 296
-rw-------. 1 postgres postgres    227 Oct 10 18:41 backup_label
drwx------. 4 postgres postgres     77 Oct 10 18:41 pg_wal
drwx------. 7 postgres postgres     59 Oct 10 18:41 base
drwx------. 4 postgres postgres     68 Oct 10 18:41 pg_logical
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_dynshmem
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_commit_ts
drwx------. 2 postgres postgres    110 Oct 10 18:41 log
-rw-------. 1 postgres postgres   1169 Oct 10 18:41 postgresql.conf.bkp_10sep2025
-rw-------. 1 postgres postgres  30702 Oct 10 18:41 postgresql.conf.bkp
-rw-------. 1 postgres postgres     88 Oct 10 18:41 postgresql.auto.conf
drwx------. 2 postgres postgres     18 Oct 10 18:41 pg_xact
-rw-------. 1 postgres postgres      3 Oct 10 18:41 PG_VERSION
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_twophase
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_tblspc
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_subtrans
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_stat_tmp
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_stat
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_snapshots
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_serial
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_replslot
drwx------. 2 postgres postgres      6 Oct 10 18:41 pg_notify
drwx------. 4 postgres postgres     36 Oct 10 18:41 pg_multixact
-rw-------. 1 postgres postgres   2640 Oct 10 18:41 pg_ident.conf
-rw-------. 1 postgres postgres   5600 Oct 10 18:41 pg_hba.conf
-rw-------. 1 postgres postgres   1171 Oct 10 18:41 postgresql.conf
drwx------. 2 postgres postgres   4096 Oct 10 18:41 global
-rw-------. 1 postgres postgres     30 Oct 10 18:41 current_logfiles
-rw-------. 1 postgres postgres 227736 Oct 10 18:41 backup_manifest
[postgres@lxicbpgdsgv01 basebackup_10OCT2025]$
B. Copy WAL files from $PGDATA/pg_wal to /pgWal/pgsql17/wal/
[postgres@lxicbpgdsgv01 ~]$ ls -ltr /pgWal/pgsql17/wal/
total 0
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ cd /pgData/pgsql17/data/pg_wal/
[postgres@lxicbpgdsgv01 pg_wal]$ cp -Rp * /pgWal/pgsql17/wal
[postgres@lxicbpgdsgv01 pg_wal]$
[postgres@lxicbpgdsgv01 pg_wal]$ ls -ltr /pgWal/pgsql17/wal
total 16384
drwx------. 2 postgres postgres        6 Oct 10 18:41 summaries
drwx------. 2 postgres postgres        6 Oct 10 18:41 archive_status
-rw-------. 1 postgres postgres 16777216 Oct 10 18:41 000000010000000000000085
[postgres@lxicbpgdsgv01 pg_wal]$ cd
[postgres@lxicbpgdsgv01 ~]$
C. Remove the existing $PGDATA/pg_wal directory
[postgres@lxicbpgdsgv01 ~]$ rm -rf /pgData/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv01 ~]$
D. Create a symbolic link pointing $PGDATA/pg_wal to a separate WAL directory
[postgres@lxicbpgdsgv01 ~]$ ln -s /pgWal/pgsql17/wal /pgData/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ ls -ld /pgData/pgsql17/data/pg_wal
lrwxrwxrwx. 1 postgres postgres 18 Oct 10 19:43 /pgData/pgsql17/data/pg_wal -> /pgWal/pgsql17/wal
[postgres@lxicbpgdsgv01 ~]$
E. Set Permissions
[postgres@lxicbpgdsgv01 ~]$ chown -R postgres:postgres /pgData/pgsql17/data
[postgres@lxicbpgdsgv01 ~]$ chmod 700 /pgData/pgsql17/data
[postgres@lxicbpgdsgv01 ~]$ chown -R postgres:postgres /pgWal/pgsql17/wal
[postgres@lxicbpgdsgv01 ~]$ chmod 700 /pgWal/pgsql17/wal

6. Start PostgreSQL

[root@lxicbpgdsgv01 ~]# systemctl start postgresql-17.service
[root@lxicbpgdsgv01 ~]# systemctl status postgresql-17.service
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Fri 2025-10-10 19:45:21 +08; 5s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 5230 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 5235 (postgres)
      Tasks: 7 (limit: 15835)
     Memory: 34.2M
        CPU: 94ms
     CGroup: /system.slice/postgresql-17.service
             ├─5235 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
             ├─5236 "postgres: logger "
             ├─5237 "postgres: checkpointer "
             ├─5238 "postgres: background writer "
             ├─5240 "postgres: walwriter "
             ├─5241 "postgres: autovacuum launcher "
             └─5242 "postgres: logical replication launcher "

Oct 10 19:45:21 lxicbpgdsgv01.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 10 19:45:21 lxicbpgdsgv01.rajasekhar.com postgres[5235]: 2025-10-10 19:45:21.177 +08 [5235] LOG:  redirecting log output to logging collector process
Oct 10 19:45:21 lxicbpgdsgv01.rajasekhar.com postgres[5235]: 2025-10-10 19:45:21.177 +08 [5235] HINT:  Future log output will appear in directory "log".
Oct 10 19:45:21 lxicbpgdsgv01.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv01 ~]#

7. Final Verification

[postgres@lxicbpgdsgv01 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# \l+
                                                                                       List of databases
   Name    |  Owner   | Encoding | Locale Provider |   Collate   |    Ctype    | Locale | ICU Rules |   Access privileges   |  Size   | Tablespace |                Description
-----------+----------+----------+-----------------+-------------+-------------+--------+-----------+-----------------------+---------+------------+--------------------------------------------
 dell      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7609 kB | pg_default |
 orcl      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7753 kB | pg_default |
 postgres  | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 492 MB  | pg_default | default administrative connection database
 template0 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7545 kB | pg_default | unmodifiable empty database
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
 template1 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7569 kB | pg_default | default template for new databases
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
(5 rows)

postgres=# \db
       List of tablespaces
    Name    |  Owner   | Location
------------+----------+----------
 pg_default | postgres |
 pg_global  | postgres |
(2 rows)

postgres=# \c dell
You are now connected to database "dell" as user "postgres".
dell=# select * from test.emp;
 name | designation | project | company
------+-------------+---------+---------
 Sugi | DBA         | Jetstar | iGATE
 Teja | DBA         | RCM     | iGATE
 RAJ  | DBA         | RCM     | iGATE
(3 rows)

dell=#

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

Clone PostgreSQL Cluster from One Host to Another Using pg_basebackup (No Archive Mode)

PostgreSQL, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

How To Backup and Restore PostgreSQL DB Cluster from One Host to Another Using pg_basebackup (No Archive Mode)

Table of Contents

1. Goal
2. Environment (Source / Target)
3. Verify Existing Setup
4. Verify postgresql.conf
5. Verify pg_hba.conf
6. Verify Users
7. Directory Permissions
8. Take Backup
9. Verify Backup
10. Transfer Backup to Target
11. Prepare Target & Restore

                  11.1 Backup Target (Optional)
                  11.2 Stop PostgreSQL and Clean Directories
                  11.3 Restore Data and WAL
                  11.4 Set Permissions
                  11.5 Verify Restored Files
                  11.6 Create Symbolic Link for WAL

12. Start PostgreSQL on Target
13. Final Verification

In PostgreSQL, a cluster is a group of databases managed by one PostgreSQL server. This is just a standalone server, not like an Oracle RAC cluster.

1. Goal

1. Perform a consistent backup using pg_basebackup from lxicbpgdsgv01.
2. Restore the backup on lxicbpgdsgv02 without applying archived WALs.

2. Environment (Source / Target)

AspectSourceTargetDifference
Hostnamelxicbpgdsgv01lxicbpgdsgv02Different hostnames
IP Address192.168.2.51192.168.2.52Different IPs
OSRHEL 9RHEL 9Same
DB VersionPostgreSQL v17.6PostgreSQL v17.6Same
Archive modeNo ArchivelogNo ArchivelogSame
PGDATA/pgData/pgsql17/data/pgdata/pgsql17/dataDifferent path case (D vs d)
WAL Directory/pgWal/pgsql17/wal/pgwal/pgsql17/walDifferent path case (W vs w)
Tablespacepg_defaultpg_defaultSame
DatabasesDELL, ORCLNo DatabasesNeed to clone

 3. Verify Existing Setup

[postgres@lxicbpgdsgv01 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# \l+
                                                                                       List of databases
   Name    |  Owner   | Encoding | Locale Provider |   Collate   |    Ctype    | Locale | ICU Rules |   Access privileges   |  Size   | Tablespace |                Description
-----------+----------+----------+-----------------+-------------+-------------+--------+-----------+-----------------------+---------+------------+--------------------------------------------
 dell      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7763 kB | pg_default |
 orcl      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7907 kB | pg_default |
 postgres  | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 492 MB  | pg_default | default administrative connection database
 template0 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7545 kB | pg_default | unmodifiable empty database
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
 template1 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7723 kB | pg_default | default template for new databases
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
(5 rows)

postgres=#

postgres=# \c dell
You are now connected to database "dell" as user "postgres".
dell=#

dell=# \db
       List of tablespaces
    Name    |  Owner   | Location
------------+----------+----------
 pg_default | postgres |
 pg_global  | postgres |
(2 rows)

dell=# 
dell=# \dn test
 List of schemas
 Name |  Owner
------+----------
 test | postgres
(1 row)

dell=#
dell=# \echo :AUTOCOMMIT
on
dell=#

dell=# CREATE TABLE test.emp ( name TEXT, designation TEXT, project TEXT, company TEXT);
CREATE TABLE
dell=# INSERT INTO test.emp VALUES ('Sugi', 'DBA', 'Jetstar', 'iGATE');
INSERT 0 1
dell=# INSERT INTO test.emp VALUES ('Teja', 'DBA', 'RCM', 'iGATE');
INSERT 0 1
dell=# INSERT INTO test.emp VALUES ('RAJ', 'DBA', 'RCM', 'iGATE');
INSERT 0 1
dell=#
dell=# select * from test.emp;
 name | designation | project | company
------+-------------+---------+---------
 Sugi | DBA         | Jetstar | iGATE
 Teja | DBA         | RCM     | iGATE
 RAJ  | DBA         | RCM     | iGATE
(3 rows)

dell=#

dell=# \c orcl
You are now connected to database "orcl" as user "postgres".
orcl=#
orcl=# \dt
            List of relations
 Schema |    Name     | Type  |  Owner
--------+-------------+-------+----------
 public | sample_data | table | postgres
(1 row)

orcl=#
orcl=# select count(*) from sample_data;
 count
-------
  1000  <----- 
(1 row)

orcl=#

4. Verify postgresql.conf


postgres=# SHOW wal_level;
 wal_level
-----------
 replica  <---- Should be replica
(1 row)

postgres=#
postgres=# SHOW max_wal_senders;
 max_wal_senders
-----------------
 10 <----- 
(1 row)

postgres=# SHOW archive_mode;
 archive_mode
--------------
 off  <----- No archive log mode
(1 row)

postgres=# show archive_command;
 archive_command
-----------------
 (disabled)
(1 row)

postgres=#

5. Verify pg_hba.conf

# TYPE  DATABASE        USER            ADDRESS             METHOD
# Local connections for replication (for pg_basebackup run locally)
local   replication     all                                 trust

# Remote connections for replication (for pg_basebackup run remotely)
#host    replication     repl_user       192.168.2.52/32     scram-sha-256

6. Verify user permissions:REPLICATION or SUPERUSER required

postgres=# \du
                             List of roles
 Role name |                         Attributes
-----------+------------------------------------------------------------
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS

postgres=#

7. Directory Permissions for Backup

[postgres@lxicbpgdsgv01 ~]$ ls -ld /pgBackup/pgsql17/backup/
drwx------. 2 postgres postgres 6 Oct  9 18:46 /pgBackup/pgsql17/backup/
[postgres@lxicbpgdsgv01 ~]$

8. Take Backup

  • Want symlinks preserved → use (both -Fp & –waldir)-Fp --waldir=/pgWal/pgsql17/wal
  • Want single tar archive → use -Ft, but recreate symlinks after restore, for pg_wal
  • The –waldir option in pg_basebackup is supported only when using the plain format (-Fp), not with the tar format (-Ft).
[postgres@lxicbpgdsgv01 ~]$ nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025 -Ft -Xs -P > /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025.log 2>&1 &
[1] 4438
[postgres@lxicbpgdsgv01 ~]$

**** I have forgot use -v (verbose option to get extra output)
nohup pg_basebackup -U postgres -D /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025 -Ft -Xs -P -v > /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025.log 2>&1 &

9. Verify Backup


[postgres@lxicbpgdsgv01 ~]$ cat /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025.log
nohup: ignoring input
waiting for checkpoint
133816/536026 kB (24%), 0/1 tablespace
299960/536026 kB (55%), 0/1 tablespace
447864/536026 kB (83%), 0/1 tablespace
536037/536037 kB (100%), 0/1 tablespace
536037/536037 kB (100%), 1/1 tablespace
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgBackup/pgsql17/backup/pg_basebackup_lxicbpgdsgv01_10sep2025
total 540M
-rw-------. 1 postgres postgres 524M Oct  9 18:56 base.tar  
-rw-------. 1 postgres postgres 223K Oct  9 18:56 backup_manifest
-rw-------. 1 postgres postgres  17M Oct  9 18:56 pg_wal.tar 
[postgres@lxicbpgdsgv01 ~]$

10. Transfer Backup to Target


[postgres@lxicbpgdsgv01 pg_basebackup_lxicbpgdsgv01_10sep2025]$ scp * 192.168.2.52:/pgbackup/pgsql17/backup/
postgres@192.168.2.52's password:  
backup_manifest                   100%  222KB  30.7MB/s   00:00
base.tar                          100%  523MB  74.0MB/s   00:07
pg_wal.tar                        100%   16MB  43.5MB/s   00:00
[postgres@lxicbpgdsgv01 pg_basebackup_lxicbpgdsgv01_10sep2025]$

11. Prepare Target & Restore

-- List Source Backup files 
[postgres@lxicbpgdsgv02 ~]$ ls -lrth /pgbackup/pgsql17/backup
total 540M
-rw-------. 1 postgres postgres 223K Oct  9 19:08 backup_manifest
-rw-------. 1 postgres postgres 524M Oct  9 19:08 base.tar
-rw-------. 1 postgres postgres  17M Oct  9 19:08 pg_wal.tar
[postgres@lxicbpgdsgv02 ~]$

11.1 Backup Target (Optional)


nohup pg_basebackup -U postgres -D /pgbackup/pgsql17/pg_basebackup_lxicbpgdsgv02_10sep2025 -Ft -Xs -P > /pgbackup/pgsql17/pg_basebackup_lxicbpgdsgv02_10sep2025.log 2>&1 &

-- Verify Tablespaces path -- It's pg_default in our case
-- Verify WAL File (Redo log files) location 

[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgdata/pgsql17/data/pg_wal
lrwxrwxrwx. 1 postgres postgres 18 Oct  9 17:37 /pgdata/pgsql17/data/pg_wal -> /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 ~]$

11.2 Stop PostgreSQL and Clean Directories

-- Stop PostgreSQL DB Cluster
[root@lxicbpgdsgv02 ~]# systemctl stop postgresql-17.service
[root@lxicbpgdsgv02 ~]#
[root@lxicbpgdsgv02 ~]# ps -ef | grep postgres
root       10259    7832  0 19:20 pts/0    00:00:00 grep --color=auto postgres
[root@lxicbpgdsgv02 ~]#

-- Remove all from Data directory 

[postgres@lxicbpgdsgv02 ~]$ rm -rf /pgdata/pgsql17/data/*
[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgdata/pgsql17/data/
total 0
[postgres@lxicbpgdsgv02 ~]$


-- Remove all from WAL directory 

[postgres@lxicbpgdsgv02 ~]$ rm -rf /pgwal/pgsql17/wal/*
[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgwal/pgsql17/wal/
total 0
[postgres@lxicbpgdsgv02 ~]$

11.3 Restore Data and WAL

-- List source backup files

[postgres@lxicbpgdsgv02 ~]$ cd /pgbackup/pgsql17/backup/
[postgres@lxicbpgdsgv02 backup]$ ls -ltr
total 552652
-rw-------. 1 postgres postgres    227724 Oct  9 19:08 backup_manifest
-rw-------. 1 postgres postgres 548901888 Oct  9 19:08 base.tar
-rw-------. 1 postgres postgres  16778752 Oct  9 19:08 pg_wal.tar
drwxr-xr-x. 3 postgres postgres        18 Oct  9 19:32 data_lxicbpgdsgv02_old
[postgres@lxicbpgdsgv02 backup]$

# Restore DATA Directory 
[postgres@lxicbpgdsgv02 ~]$ nohup tar -xvf /pgbackup/pgsql17/backup/base.tar -C /pgdata/pgsql17/data > /pgbackup/pgsql17/backup/base_restore.log 2>&1 &
[1] 11289
[postgres@lxicbpgdsgv02 ~]$


# Restore WAL Directory 

[postgres@lxicbpgdsgv02 ~]$ nohup tar -xvf /pgbackup/pgsql17/backup/pg_wal.tar -C /pgwal/pgsql17/wal > /pgbackup/pgsql17/backup/pg_wal_restore.log 2>&1 &
[1] 11304
[postgres@lxicbpgdsgv02 ~]$

-- OR -- 

# Restore base and WAL archives sequentially in a single command

nohup bash -c "tar -xvf /pgbackup/pgsql17/backup/base.tar -C /pgdata/pgsql17/data && tar -xvf /pgbackup/pgsql17/backup/pg_wal.tar -C /pgwal/pgsql17/wal" > /pgbackup/pgsql17/backup/fulltar_restore.log 2>&1 &

11.4 Set Permissions

[postgres@lxicbpgdsgv02 ~]$ chown -R postgres:postgres /pgdata/pgsql17/data
[postgres@lxicbpgdsgv02 ~]$ chmod 700 /pgdata/pgsql17/data
[postgres@lxicbpgdsgv02 ~]$
[postgres@lxicbpgdsgv02 ~]$ chown -R postgres:postgres /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 ~]$ chmod 700 /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 ~]$

11.5 Verify Restored Files

[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgdata/pgsql17/data
total 72
-rw-------. 1 postgres postgres    88 Sep 30 21:50 postgresql.auto.conf
drwx------. 2 postgres postgres    18 Sep 30 21:50 pg_xact
-rw-------. 1 postgres postgres     3 Sep 30 21:50 PG_VERSION
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_twophase
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_tblspc
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_subtrans
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_stat_tmp
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_snapshots
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_serial
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_notify
drwx------. 4 postgres postgres    36 Sep 30 21:50 pg_multixact
-rw-------. 1 postgres postgres  2640 Sep 30 21:50 pg_ident.conf
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_dynshmem
drwx------. 2 postgres postgres     6 Sep 30 21:50 pg_commit_ts
-rw-------. 1 postgres postgres 30702 Oct  7 18:20 postgresql.conf.bkp
drwx------. 2 postgres postgres    84 Oct  9 00:01 log
-rw-------. 1 postgres postgres  1169 Oct  9 16:31 postgresql.conf.bkp_10sep2025
-rw-------. 1 postgres postgres  1171 Oct  9 16:32 postgresql.conf
drwx------. 7 postgres postgres    59 Oct  9 18:08 base
drwx------. 2 postgres postgres     6 Oct  9 18:44 pg_stat
-rw-------. 1 postgres postgres    30 Oct  9 18:44 current_logfiles
-rw-------. 1 postgres postgres  5600 Oct  9 18:53 pg_hba.conf
-rw-------. 1 postgres postgres     0 Oct  9 18:56 tablespace_map
drwx------. 2 postgres postgres     6 Oct  9 18:56 pg_replslot
drwx------. 4 postgres postgres    68 Oct  9 18:56 pg_logical
-rw-------. 1 postgres postgres   227 Oct  9 18:56 backup_label
drwx------. 4 postgres postgres    45 Oct  9 20:06 pg_wal <--- Created as Directory, instead of symbolic link 
drwx------. 2 postgres postgres  4096 Oct  9 20:06 global
[postgres@lxicbpgdsgv02 ~]$
[postgres@lxicbpgdsgv02 ~]$ ls -ltr /pgwal/pgsql17/wal
total 16384
-rw-------. 1 postgres postgres 16777216 Oct  9 18:56 000000010000000000000045
[postgres@lxicbpgdsgv02 ~]$

Please note:

--- Want symlinks preserved → use -Fp
--- Want single tar archive → use -Ft, but recreate symlinks after restore, for pg_wal

11.6 Create Symbolic Link for WAL

[postgres@lxicbpgdsgv02 ~]$ ls -ld /pgdata/pgsql17/data/pg_wal
drwx------. 4 postgres postgres 45 Oct  9 20:06 /pgdata/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv02 ~]$
[postgres@lxicbpgdsgv02 ~]$ rm -rf /pgdata/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv02 ~]$ ln -s /pgwal/pgsql17/wal /pgdata/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv02 ~]$
[postgres@lxicbpgdsgv02 ~]$ ls -ld /pgdata/pgsql17/data/pg_wal
lrwxrwxrwx. 1 postgres postgres 18 Oct  9 21:04 /pgdata/pgsql17/data/pg_wal -> /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 ~]$ 
[postgres@lxicbpgdsgv02 ~]$ cd /pgwal/pgsql17/wal
[postgres@lxicbpgdsgv02 wal]$ ll
total 16384
-rw-------. 1 postgres postgres 16777216 Oct  9 18:56 000000010000000000000045
[postgres@lxicbpgdsgv02 wal]$

12. Start PostgreSQL on Target — no recovery needed.

[root@lxicbpgdsgv02 ~]# systemctl start postgresql-17.service
[root@lxicbpgdsgv02 ~]# systemctl status postgresql-17.service
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Thu 2025-10-09 21:06:08 +08; 5s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 12375 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 12380 (postgres)
      Tasks: 7 (limit: 20496)
     Memory: 33.6M
        CPU: 114ms
     CGroup: /system.slice/postgresql-17.service
             ├─12380 /usr/pgsql-17/bin/postgres -D /pgdata/pgsql17/data
             ├─12381 "postgres: logger "
             ├─12382 "postgres: checkpointer "
             ├─12383 "postgres: background writer "
             ├─12385 "postgres: walwriter "
             ├─12386 "postgres: autovacuum launcher "
             └─12387 "postgres: logical replication launcher "

Oct 09 21:06:06 lxicbpgdsgv02.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 09 21:06:06 lxicbpgdsgv02.rajasekhar.com postgres[12380]: 2025-10-09 21:06:06.781 +08 [12380] LOG:  redirecting log output to logging collector process
Oct 09 21:06:06 lxicbpgdsgv02.rajasekhar.com postgres[12380]: 2025-10-09 21:06:06.781 +08 [12380] HINT:  Future log output will appear in directory "log".
Oct 09 21:06:08 lxicbpgdsgv02.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv02 ~]#

13. Final Verification

[postgres@lxicbpgdsgv02 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# \l+
                                                                                       List of databases
   Name    |  Owner   | Encoding | Locale Provider |   Collate   |    Ctype    | Locale | ICU Rules |   Access privileges   |  Size   | Tablespace |                Description
-----------+----------+----------+-----------------+-------------+-------------+--------+-----------+-----------------------+---------+------------+--------------------------------------------
 dell      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7609 kB | pg_default |
 orcl      | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 7753 kB | pg_default |
 postgres  | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           |                       | 492 MB  | pg_default | default administrative connection database
 template0 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7545 kB | pg_default | unmodifiable empty database
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
 template1 | postgres | UTF8     | libc            | en_SG.UTF-8 | en_SG.UTF-8 |        |           | =c/postgres          +| 7569 kB | pg_default | default template for new databases
           |          |          |                 |             |             |        |           | postgres=CTc/postgres |         |            |
(5 rows)

postgres=# \db
       List of tablespaces
    Name    |  Owner   | Location
------------+----------+----------
 pg_default | postgres |
 pg_global  | postgres |
(2 rows)

postgres=#
postgres=# \c dell
You are now connected to database "dell" as user "postgres".
dell=# \dt test.*
        List of relations
 Schema | Name | Type  |  Owner
--------+------+-------+----------
 test   | emp  | table | postgres
(1 row)

dell=#

dell=# select * from test.emp;  <---- We can see Data 
 name | designation | project | company
------+-------------+---------+---------
 Sugi | DBA         | Jetstar | iGATE
 Teja | DBA         | RCM     | iGATE
 RAJ  | DBA         | RCM     | iGATE
(3 rows)

dell=#
dell=# \c orcl
You are now connected to database "orcl" as user "postgres".
orcl=# \dt
            List of relations
 Schema |    Name     | Type  |  Owner
--------+-------------+-------+----------
 public | sample_data | table | postgres
(1 row)

orcl=#
orcl=# select count(*) from sample_data;
 count
-------
  1000 <----
(1 row)

orcl=#

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

Disable ARCHIVELOG Mode

PostgreSQL, , , , , , , , , , , , , ,

How to Disable ARCHIVELOG Mode

Table of Contents

1. Verify Existing Archive Mode
2. Edit the archive settings
3. Restart PostgreSQL
4. Verify Current Mode
5. Verify WAL Archiving Behavior

1. Verify Existing Archive Mode

postgres=# SHOW archive_mode;
 archive_mode
--------------
 on  <------ 
(1 row)

postgres=#

postgres=# SHOW archive_command;
        archive_command
-------------------------------
 cp %p /pgArch/pgsql17/arch/%f  <----- 
(1 row)

postgres=#

2. Edit the archive settings


[postgres@lxicbpgdsgv01 ~]$ cp /pgData/pgsql17/data/postgresql.conf /pgData/pgsql17/data/postgresql.conf.bkp_10sep2025
[postgres@lxicbpgdsgv01 ~]$ vi /pgData/pgsql17/data/postgresql.conf

#archive_mode = on
#archive_command = 'cp %p /pgArch/pgsql17/arch/%f'

3. Restart PostgreSQL

[root@lxicbpgdsgv01 ~]# systemctl stop postgresql-17.service
[root@lxicbpgdsgv01 ~]#
[root@lxicbpgdsgv01 ~]# systemctl start  postgresql-17.service
[root@lxicbpgdsgv01 ~]#
[root@lxicbpgdsgv01 ~]# systemctl status postgresql-17.service
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Thu 2025-10-09 16:34:01 +08; 3s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 3492 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 3497 (postgres)
      Tasks: 7 (limit: 15835)
     Memory: 17.6M
        CPU: 92ms
     CGroup: /system.slice/postgresql-17.service
             ├─3497 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
             ├─3498 "postgres: logger "
             ├─3499 "postgres: checkpointer "
             ├─3500 "postgres: background writer "
             ├─3502 "postgres: walwriter "
             ├─3503 "postgres: autovacuum launcher "
             └─3504 "postgres: logical replication launcher "

Oct 09 16:34:01 lxicbpgdsgv01.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 09 16:34:01 lxicbpgdsgv01.rajasekhar.com postgres[3497]: 2025-10-09 16:34:01.929 +08 [3497] LOG:  redirecting log output to logging collector process
Oct 09 16:34:01 lxicbpgdsgv01.rajasekhar.com postgres[3497]: 2025-10-09 16:34:01.929 +08 [3497] HINT:  Future log output will appear in directory "log".
Oct 09 16:34:01 lxicbpgdsgv01.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv01 ~]#

4. Verify Current Mode

[postgres@lxicbpgdsgv01 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# SHOW archive_mode;
 archive_mode
--------------
 off  <------ it's disabled
(1 row)

postgres=# SHOW archive_command;
 archive_command
-----------------
 (disabled) <-------
(1 row)

postgres=#

5. Verify WAL Archiving Behavior


postgres=# CHECKPOINT;
CHECKPOINT
postgres=#
postgres=# CHECKPOINT;
CHECKPOINT
postgres=# CHECKPOINT;
CHECKPOINT
postgres=#
postgres=# exit
postgres=# SELECT pg_switch_wal();
 pg_switch_wal
---------------
 0/44000000
(1 row)

postgres=# SELECT pg_switch_wal();
 pg_switch_wal
---------------
 0/44000000
(1 row)

postgres=# SELECT pg_switch_wal();
 pg_switch_wal
---------------
 0/44000000
(1 row)

postgres=#
[postgres@lxicbpgdsgv01 ~]$ ls -ltr /pgArch/pgsql17/arch/
total 0  <---- Archivelogs not generating
[postgres@lxicbpgdsgv01 ~]$

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

Change PostgreSQL WAL Directory Path (pg_wal)

PostgreSQL, , , , , , , , , , , , , , , ,

Table of Contents

0. Aim
1. Verify Existing pg_wal Directory
2. Create the New Directory
3. Stop PostgreSQL Service
4. Copy WAL Files to New Location
5. Backup Old WAL Directory
6. Create Symlink
7. Fix Permissions
8. Start PostgreSQL Service
9. Verify WAL Functionality
10. Remove Old WAL Directory (Optional)

0. Aim

To change the PostgreSQL 17 WAL File directory from its default location to new mount point

From : /var/lib/pgsql/17/data

TO : /pgData/pgsql17/data

1. Verify Existing pg_wal directory


[postgres@lxicbpgdsgv01 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# SELECT current_setting('data_directory') || '/pg_wal' AS wal_directory;
        wal_directory
-----------------------------
 /pgData/pgsql17/data/pg_wal  <----- 
(1 row)

postgres=# exit
[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgData/pgsql17/data/pg_wal
total 48M
drwx------. 2 postgres postgres   6 Sep 30 21:50 summaries
-rw-------. 1 postgres postgres 16M Oct  8 04:24 000000010000000000000008
-rw-------. 1 postgres postgres 16M Oct  8 16:08 000000010000000000000006
-rw-------. 1 postgres postgres 16M Oct  8 16:08 000000010000000000000007
drwx------. 2 postgres postgres  43 Oct  8 16:08 archive_status
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ du -sh /pgData/pgsql17/data/pg_wal
48M     /pgData/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv01 ~]$

2. Create the New Directory on a new disk

[root@lxicbpgdsgv01 ~]# mkdir -p /pgData/pgsql17/data
[root@lxicbpgdsgv01 ~]# chown postgres:postgres /pgData/pgsql17/data
[root@lxicbpgdsgv01 ~]# chmod 700 /pgData/pgsql17/data
[root@lxicbpgdsgv01 ~]#

3. Stop PostgreSQL Service

[root@lxicbpgdsgv01 ~]# systemctl stop postgresql-17.service
[root@lxicbpgdsgv01 ~]#
[root@lxicbpgdsgv01 ~]# ps -ef | grep postgres
root        6887    6721  0 16:08 pts/0    00:00:00 grep --color=auto postgres
[root@lxicbpgdsgv01 ~]#

4. Copy Existing WAL Files to New Location

[postgres@lxicbpgdsgv01 ~]$ nohup rsync -avh --progress /pgData/pgsql17/data/pg_wal/ /pgWal/pgsql17/wal/ > rsync_pgwal.log 2>&1 &
[1] 6943
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$
[1]+  Done                    nohup rsync -avh --progress /pgData/pgsql17/data/pg_wal/ /pgWal/pgsql17/wal/ > rsync_pgwal.log 2>&1
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ cat rsync_pgwal.log
nohup: ignoring input
sending incremental file list
./
000000010000000000000006
         16.78M 100%   81.89MB/s    0:00:00 (xfr#1, to-chk=5/7)
000000010000000000000007
         16.78M 100%   43.84MB/s    0:00:00 (xfr#2, to-chk=4/7)
000000010000000000000008
         16.78M 100%   31.07MB/s    0:00:00 (xfr#3, to-chk=3/7)
archive_status/
archive_status/000000010000000000000006.done
              0 100%    0.00kB/s    0:00:00 (xfr#4, to-chk=0/7)
summaries/

sent 50.34M bytes  received 107 bytes  33.56M bytes/sec
total size is 50.33M  speedup is 1.00
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgWal/pgsql17/wal/
total 48M
drwx------. 2 postgres postgres   6 Sep 30 21:50 summaries
-rw-------. 1 postgres postgres 16M Oct  8 04:24 000000010000000000000008
-rw-------. 1 postgres postgres 16M Oct  8 16:08 000000010000000000000006
-rw-------. 1 postgres postgres 16M Oct  8 16:08 000000010000000000000007
drwx------. 2 postgres postgres  43 Oct  8 16:08 archive_status
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ du -sh /pgWal/pgsql17/wal/
48M     /pgWal/pgsql17/wal/
[postgres@lxicbpgdsgv01 ~]$

5. Move old directory as backup

[postgres@lxicbpgdsgv01 ~]$ mv /pgData/pgsql17/data/pg_wal /pgData/pgsql17/data/pg_wal.bak
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ ls -ld /pgData/pgsql17/data/pg_wal
ls: cannot access '/pgData/pgsql17/data/pg_wal': No such file or directory
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ ls -ld /pgData/pgsql17/data/pg_wal.bak
drwx------. 4 postgres postgres 141 Oct  8 16:08 /pgData/pgsql17/data/pg_wal.bak
[postgres@lxicbpgdsgv01 ~]$

6. Create symlink

[postgres@lxicbpgdsgv01 ~]$ ln -s /pgWal/pgsql17/wal /pgData/pgsql17/data/pg_wal
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ ls -ltr /pgData/pgsql17/data/pg_wal
lrwxrwxrwx. 1 postgres postgres 18 Oct  8 16:16 /pgData/pgsql17/data/pg_wal -> /pgWal/pgsql17/wal
[postgres@lxicbpgdsgv01 ~]$

7. Fix permissions (if required)

[postgres@lxicbpgdsgv01 ~]$ chown -R postgres:postgres /pgWal/pgsql17/wal

8. Start PostgreSQL Service

[root@lxicbpgdsgv01 ~]# systemctl start postgresql-17.service
[root@lxicbpgdsgv01 ~]#
[root@lxicbpgdsgv01 ~]# systemctl status postgresql-17.service
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Wed 2025-10-08 16:20:46 +08; 7s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 7079 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 7084 (postgres)
      Tasks: 8 (limit: 15835)
     Memory: 18.1M
        CPU: 94ms
     CGroup: /system.slice/postgresql-17.service
             ├─7084 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
             ├─7086 "postgres: logger "
             ├─7087 "postgres: checkpointer "
             ├─7088 "postgres: background writer "
             ├─7090 "postgres: walwriter "
             ├─7091 "postgres: autovacuum launcher "
             ├─7092 "postgres: archiver "
             └─7093 "postgres: logical replication launcher "

Oct 08 16:20:46 lxicbpgdsgv01.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 08 16:20:46 lxicbpgdsgv01.rajasekhar.com postgres[7084]: 2025-10-08 16:20:46.162 +08 [7084] LOG:  redirecting log output to logging collector process
Oct 08 16:20:46 lxicbpgdsgv01.rajasekhar.com postgres[7084]: 2025-10-08 16:20:46.162 +08 [7084] HINT:  Future log output will appear in directory "log".
Oct 08 16:20:46 lxicbpgdsgv01.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv01 ~]#

9. Verify

-- Load WAL File generation (Testing)

postgres=# -- Create test table
DROP TABLE IF EXISTS wal_test;
CREATE TABLE wal_test (
    id serial PRIMARY KEY,
    data text
);

-- Generate WAL traffic
DO $$
DECLARE
    i integer;
BEGIN
    FOR i IN 1..50 LOOP
        -- INSERT: 10,000 rows
        INSERT INTO wal_test (data)
        SELECT repeat('wal_test_data_', 50)
        FROM generate_series(1, 10000);

        -- UPDATE: 5,000 rows using CTE with LIMIT
        WITH to_update AS (
            SELECT id FROM wal_test WHERE id % 2 = 0 LIMIT 5000
        )
        UPDATE wal_test
        SET data = data || '_updated'
        WHERE id IN (SELECT id FROM to_update);

        -- DELETE: 5,000 rows using CTE with LIMIT
        WITH to_delete AS (
            SELECT id FROM wal_test WHERE id % 3 = 0 LIMIT 5000
        )
        DELETE FROM wal_test
        WHERE id IN (SELECT id FROM to_delete);

        -- Commit to flush WAL
        COMMIT;

        -- Optional pause to slow down the loop
        PERFORM pg_sleep(0.5);
    END LOOP;
END$$;
DROP TABLE
CREATE TABLE
DO
postgres=# exit
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ ls -lrth  /pgWal/pgsql17/wal
total 752M
drwx------. 2 postgres postgres    6 Sep 30 21:50 summaries
-rw-------. 1 postgres postgres  16M Oct  8 16:34 000000010000000000000009
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000A
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000B
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000C
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000D
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000E
-rw-------. 1 postgres postgres  16M Oct  8 16:34 00000001000000000000000F
-rw-------. 1 postgres postgres  16M Oct  8 16:34 000000010000000000000010
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000011
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000012
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000013
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000014
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000015
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000016
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000017
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000018
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000019
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001A
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001B
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001C
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001D
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001E
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000001F
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000020
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000021
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000022
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000023
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000024
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000025
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000026
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000027
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000028
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000029
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002A
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002B
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002C
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002D
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002E
-rw-------. 1 postgres postgres  16M Oct  8 16:35 00000001000000000000002F
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000030
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000031
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000032
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000033
-rw-------. 1 postgres postgres  16M Oct  8 16:35 000000010000000000000034
-rw-------. 1 postgres postgres  16M Oct  8 16:36 000000010000000000000035
drwx------. 2 postgres postgres 4.0K Oct  8 16:36 archive_status
-rw-------. 1 postgres postgres  16M Oct  8 16:36 xlogtemp.7289
[postgres@lxicbpgdsgv01 ~]$

10. Remove Old WAL Directory (Optional, later)

[postgres@lxicbpgdsgv01 ~]$ ls -ld /pgData/pgsql17/data/pg_wal.bak
drwx------. 4 postgres postgres 141 Oct  8 16:08 /pgData/pgsql17/data/pg_wal.bak
[postgres@lxicbpgdsgv01 ~]$ rm -rf /pgData/pgsql17/data/pg_wal.bak
[postgres@lxicbpgdsgv01 ~]$

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

Enable Archive Mode (WAL Archiving)

PostgreSQL, , , , , , , , , , , , , , ,

Enable Archive Mode in PostgreSQL 17

Table of Contents

0. Importance of Archive Log Files
1. Verify Existing Archive Mode
2. Ensure Archive Directory Exists
3. Edit postgresql.conf
4. Restart PostgreSQL
5. Verify the Configuration
6. Test Archiving Works

0. Importance of Archive Log Files

Archive log files, also known as WAL archives (Write-Ahead Log archives) , play a critical role in PostgreSQL for data protection, recovery, and replication.

A. Point-in-Time Recovery (PITR): Restore the database to a specific point in time, useful for accidental data changes or loss.
B. Continuous Backup: Works with base backups to enable robust disaster recovery.
C. Streaming Replication Support: Helps standby servers catch up if they fall behind.
D. Disaster Recovery: Enables full recovery after hardware or data corruption.
E. Data Audit & Analysis: Allows decoding WAL logs for auditing and compliance.

🛑 Important Note:

If WAL archiving is disabled and a backup is taken, you can only restore to the exact backup time, not to any point after.

1. Verify Existing Archive Mode

[postgres@lxicbpgdsgv01 ~]$ psql
psql (17.6)
Type "help" for help.

postgres=# SHOW archive_mode;
 archive_mode 
--------------
 off <--- it's off
(1 row)

postgres=# SHOW archive_command;
 archive_command 
-----------------
 (disabled) <----
(1 row)
[root@lxicbpgdsgv01 ~]# systemctl status postgresql-17
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Tue 2025-10-07 18:33:05 +08; 2min 26s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 5911 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 5916 (postgres)
      Tasks: 7 (limit: 15700)
     Memory: 21.3M
        CPU: 177ms
     CGroup: /system.slice/postgresql-17.service
             ├─5916 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
             ├─5917 "postgres: logger "
             ├─5918 "postgres: checkpointer "
             ├─5919 "postgres: background writer "
             ├─5921 "postgres: walwriter "
             ├─5922 "postgres: autovacuum launcher "
             └─5923 "postgres: logical replication launcher "

Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com postgres[5916]: 2025-10-07 18:33:05.392 +08 [5916] LOG:  redirecting log output to logging collector process
Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com postgres[5916]: 2025-10-07 18:33:05.392 +08 [5916] HINT:  Future log output will appear in directory "log".
Oct 07 18:33:05 lxicbpgdsgv01.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv01 ~]#

Currently, there is no active archive process running in the background.

2. Ensure Archive Directory Exists

[root@lxicbpgdsgv01 ~]# mkdir -p /pgArch/pgsql17/arch/
[root@lxicbpgdsgv01 ~]# chown postgres:postgres /pgArch/pgsql17/arch/
[root@lxicbpgdsgv01 ~]# chmod 700 /pgArch/pgsql17/arch/

3. Edit postgresql.conf

[postgres@lxicbpgdsgv01 ~]$ vi /pgData/pgsql17/data/postgresql.conf

Update or add the following lines:

archive_mode = on
archive_command = 'cp %p /pgArch/pgsql17/arch/%f'

Explanation:

  • %p = Full path of WAL file
  • %f = WAL file name
  • cp %p /pgArch/pgsql17/arch/%f = Command to copy the WAL file to archive directory

4. Restart PostgreSQL

[root@lxicbpgdsgv01 ~]# systemctl status postgresql-17
● postgresql-17.service - PostgreSQL 17 database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql-17.service; enabled; preset: disabled)
     Active: active (running) since Tue 2025-10-07 23:48:29 +08; 18s ago
       Docs: https://www.postgresql.org/docs/17/static/
    Process: 7868 ExecStartPre=/usr/pgsql-17/bin/postgresql-17-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
   Main PID: 7873 (postgres)
      Tasks: 8 (limit: 15700)
     Memory: 19.0M
        CPU: 96ms
     CGroup: /system.slice/postgresql-17.service
             ├─7873 /usr/pgsql-17/bin/postgres -D /pgData/pgsql17/data/
             ├─7874 "postgres: logger "
             ├─7875 "postgres: checkpointer "
             ├─7876 "postgres: background writer "
             ├─7878 "postgres: walwriter "
             ├─7879 "postgres: autovacuum launcher "
             ├─7880 "postgres: archiver "  <------
             └─7881 "postgres: logical replication launcher "

Oct 07 23:48:29 lxicbpgdsgv01.rajasekhar.com systemd[1]: Starting PostgreSQL 17 database server...
Oct 07 23:48:29 lxicbpgdsgv01.rajasekhar.com postgres[7873]: 2025-10-07 23:48:29.503 +08 [7873] LOG:  redirecting log output to logging collector process
Oct 07 23:48:29 lxicbpgdsgv01.rajasekhar.com postgres[7873]: 2025-10-07 23:48:29.503 +08 [7873] HINT:  Future log output will appear in directory "log".
Oct 07 23:48:29 lxicbpgdsgv01.rajasekhar.com systemd[1]: Started PostgreSQL 17 database server.
[root@lxicbpgdsgv01 ~]#

Make sure the archiver process is running.

5. Verify the Configuration

[postgres@lxicbpgdsgv01 ~]$ psql -c "SHOW archive_mode;"
 archive_mode 
--------------
 on <-----------
(1 row)

[postgres@lxicbpgdsgv01 ~]$ psql -c "SHOW archive_command;"
        archive_command         
-------------------------------
 cp %p /pgArch/pgsql17/arch/%f  <-----------------
(1 row)

6. Test Archiving Works

[postgres@lxicbpgdsgv01 ~]$ psql -c "SELECT pg_switch_wal();"

Check archive folder for new WAL files:

[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgArch/pgsql17/arch/
total 16M
-rw-------. 1 postgres postgres 16M Oct  7 23:50 000000010000000000000001

[postgres@lxicbpgdsgv01 ~]$ psql -c "SELECT pg_switch_wal();"
[postgres@lxicbpgdsgv01 ~]$ ls -lrth /pgArch/pgsql17/arch/
total 32M
-rw-------. 1 postgres postgres 16M Oct  7 23:50 000000010000000000000001
-rw-------. 1 postgres postgres 16M Oct  7 23:51 000000010000000000000002

 Archiving is working successfully.

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

PostgreSQL Backup and Restore Using pg_dumpall

PostgreSQL, , , , , , , , , , , , , , , , , , , , , , ,

PostgreSQL Backup and Restore Using pg_dumpall and psql

Table of Contents

0. Introduction

Backup:

1. Backup All Databases
2. Backup Users/Roles Definition
3. Backup Tablespaces Definition
4. Backup Schema Only (No Data)
5. Backup Data Only (No Schema)
6. Backup Data as INSERT Commands
7. Backup Global Objects Only (No Databases)

Restore:

8. Restore on Same Host

9. Restore to Another Host (Different Tablespace Paths)

Click to expand pg_dumpall –help
[postgres@lxtrdpgdsgv01 ~]$ pg_dumpall --help
pg_dumpall extracts a PostgreSQL database cluster into an SQL script file.

Usage:
  pg_dumpall [OPTION]...

General options:
  -f, --file=FILENAME          output file name
  -v, --verbose                verbose mode
  -V, --version                output version information, then exit
  --lock-wait-timeout=TIMEOUT  fail after waiting TIMEOUT for a table lock
  -?, --help                   show this help, then exit

Options controlling the output content:
  -a, --data-only              dump only the data, not the schema
  -c, --clean                  clean (drop) databases before recreating
  -E, --encoding=ENCODING      dump the data in encoding ENCODING
  -g, --globals-only           dump only global objects, no databases
  -O, --no-owner               skip restoration of object ownership
  -r, --roles-only             dump only roles, no databases or tablespaces
  -s, --schema-only            dump only the schema, no data
  -S, --superuser=NAME         superuser user name to use in the dump
  -t, --tablespaces-only       dump only tablespaces, no databases or roles
  -x, --no-privileges          do not dump privileges (grant/revoke)
  --binary-upgrade             for use by upgrade utilities only
  --column-inserts             dump data as INSERT commands with column names
  --disable-dollar-quoting     disable dollar quoting, use SQL standard quoting
  --disable-triggers           disable triggers during data-only restore
  --exclude-database=PATTERN   exclude databases whose name matches PATTERN
  --extra-float-digits=NUM     override default setting for extra_float_digits
  --if-exists                  use IF EXISTS when dropping objects
  --inserts                    dump data as INSERT commands, rather than COPY
  --load-via-partition-root    load partitions via the root table
  --no-comments                do not dump comments
  --no-publications            do not dump publications
  --no-role-passwords          do not dump passwords for roles
  --no-security-labels         do not dump security label assignments
  --no-subscriptions           do not dump subscriptions
  --no-sync                    do not wait for changes to be written safely to disk
  --no-table-access-method     do not dump table access methods
  --no-tablespaces             do not dump tablespace assignments
  --no-toast-compression       do not dump TOAST compression methods
  --no-unlogged-table-data     do not dump unlogged table data
  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands
  --quote-all-identifiers      quote all identifiers, even if not key words
  --restrict-key=RESTRICT_KEY  use provided string as psql \restrict key
  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts
  --use-set-session-authorization
                               use SET SESSION AUTHORIZATION commands instead of
                               ALTER OWNER commands to set ownership

Connection options:
  -d, --dbname=CONNSTR     connect using connection string
  -h, --host=HOSTNAME      database server host or socket directory
  -l, --database=DBNAME    alternative default database
  -p, --port=PORT          database server port number
  -U, --username=NAME      connect as specified database user
  -w, --no-password        never prompt for password
  -W, --password           force password prompt (should happen automatically)
  --role=ROLENAME          do SET ROLE before dump

If -f/--file is not used, then the SQL script will be written to the standard
output.

Report bugs to <pgsql-bugs@lists.postgresql.org>.
PostgreSQL home page: <https://www.postgresql.org/>
[postgres@lxtrdpgdsgv01 ~]$

 

0. Introduction

Note: pg_dumpall does not support custom format backups.

The pg_dumpall utility is used to back up an entire PostgreSQL environment, including:

* Roles and users
* Tablespaces
* All databases (schemas and data)

It is especially useful for:

* Full cluster migrations
* Disaster recovery
* Environment replication across dev, QA, and prod

When restoring to a different host, you'll need to:

* Update tablespace paths (using tools like sed)
* Pre-create required tablespace directories

This ensures compatibility and successful restoration across different environments.

Backup


1. Backup ALL databases

[postgres@lxtrdpgdsgv01 ~]$ nohup pg_dumpall -U postgres -v -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql > /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.log 2>&1 &
[1] 5610
[postgres@lxtrdpgdsgv01 ~]$ 
[postgres@lxtrdpgdsgv01 ~]$ cat /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.log | grep -i /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql
pg_dumpall: running ""/usr/pgsql-15/bin/pg_dump"  -v -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql  -Fa 'user=postgres dbname=template1'"
pg_dumpall: running ""/usr/pgsql-15/bin/pg_dump"  -v -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql --create -Fa 'user=postgres dbname=edpua'"
pg_dumpall: running ""/usr/pgsql-15/bin/pg_dump"  -v -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql --create -Fa 'user=postgres dbname=gebua'"
pg_dumpall: running ""/usr/pgsql-15/bin/pg_dump"  -v -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql --create -Fa 'user=postgres dbname=orcl'"
pg_dumpall: running ""/usr/pgsql-15/bin/pg_dump"  -v -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql  -Fa 'user=postgres dbname=postgres'"
[postgres@lxtrdpgdsgv01 ~]$

[postgres@lxtrdpgdsgv01 ~]$ psql
psql (15.14)
Type "help" for help.

postgres=# \c orcl
You are now connected to database "orcl" as user "postgres".
orcl=# SELECT
orcl-#     schemaname || '.' || relname AS table_name,
orcl-#     pg_size_pretty(pg_total_relation_size(relid)) AS total_size,
orcl-#     pg_size_pretty(pg_relation_size(relid)) AS table_size,
orcl-#     pg_size_pretty(pg_total_relation_size(relid) - pg_relation_size(relid)) AS index_size
orcl-# FROM
orcl-#     pg_catalog.pg_statio_user_tables
orcl-# ORDER BY
orcl-#     pg_total_relation_size(relid) DESC;
    table_name    | total_size | table_size | index_size
------------------+------------+------------+------------
 trd.metrics_high | 799 MB     | 498 MB     | 301 MB
 trd.metrics_mid  | 638 MB     | 398 MB     | 240 MB
 trd.employees    | 493 MB     | 322 MB     | 172 MB
 trd.sales_2023   | 399 MB     | 249 MB     | 150 MB
 trd.sales_2024   | 399 MB     | 249 MB     | 150 MB
 trd.sales_2022   | 399 MB     | 249 MB     | 150 MB
 trd.sales_2021   | 398 MB     | 248 MB     | 150 MB
 trd.metrics_low  | 159 MB     | 99 MB      | 60 MB
 trd.test_data    | 71 MB      | 50 MB      | 21 MB
 trd.emp_summary  | 24 kB      | 8192 bytes | 16 kB
 trd.metrics_rest | 8192 bytes | 0 bytes    | 8192 bytes
(11 rows)

orcl=#


2. Backup users/roles definition

[postgres@lxtrdpgdsgv01 ~]$ nohup pg_dumpall -U postgres --roles-only -v -f /pgBackup/pgsql15/backup/roles.sql > /pgBackup/pgsql15/backup/roles.log 2>&1 &
[1] 5205
[postgres@lxtrdpgdsgv01 ~]$


3. Backup tablespaces definition

[postgres@lxtrdpgdsgv01 ~]$ nohup pg_dumpall -U postgres --tablespaces-only -v -f /pgBackup/pgsql15/backup/tablespaces.sql > /pgBackup/pgsql15/backup/tablespaces.log 2>&1 &
[1] 5183
[postgres@lxtrdpgdsgv01 ~]$


4. Backup dump only the schema, no data

[postgres@lxtrdpgdsgv01 ~]$ nohup pg_dumpall -U postgres --schema-only -v -f /pgBackup/pgsql15/backup/schemas.sql > /pgBackup/pgsql15/backup/schemas.log 2>&1 &
[1] 4890
[postgres@lxtrdpgdsgv01 ~]$


5. Backup dump only the data, not the schema

[postgres@lxtrdpgdsgv01 ~]$ nohup pg_dumpall -U postgres --data-only -v -f /pgBackup/pgsql15/backup/dataonly.sql > /pgBackup/pgsql15/backup/dataonly.log 2>&1 &
[1] 5233
[postgres@lxtrdpgdsgv01 ~]$


6. Backup dump data as INSERT commands, rather than COPY

[postgres@lxtrdpgdsgv01 ~]$ nohup pg_dumpall -U postgres --inserts -v -f /pgBackup/pgsql15/backup/inserts.sql > /pgBackup/pgsql15/backup/inserts.log 2>&1 &
[1] 5274
[postgres@lxtrdpgdsgv01 ~]$


7. Backup dump only global objects, no databases

[postgres@lxtrdpgdsgv01 ~]$ nohup pg_dumpall -U postgres --globals-only -v -f /pgBackup/pgsql15/backup/globals.sql > /pgBackup/pgsql15/backup/globals.log 2>&1 &
[1] 5331
[postgres@lxtrdpgdsgv01 ~]$

Restore


8. Restore on same host


A. Drop Existing Databases (Optional)

 

[postgres@lxtrdpgdsgv01 ~]$ psql
psql (15.14)
Type "help" for help.

postgres=# \l
                                                 List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    | ICU Locale | Locale Provider |   Access privileges
-----------+----------+----------+-------------+-------------+------------+-----------------+-----------------------
 edpua     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 gebua     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 orcl      | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            | =c/postgres          +
           |          |          |             |             |            |                 | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            | postgres=CTc/postgres+
           |          |          |             |             |            |                 | =c/postgres
(6 rows)

postgres=# drop database edpua;
DROP DATABASE
postgres=# drop database gebua;
DROP DATABASE
postgres=# drop database orcl;
DROP DATABASE
postgres=# \l
                                                 List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    | ICU Locale | Locale Provider |   Access privileges
-----------+----------+----------+-------------+-------------+------------+-----------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            | =c/postgres          +
           |          |          |             |             |            |                 | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            | postgres=CTc/postgres+
           |          |          |             |             |            |                 | =c/postgres
(3 rows)

postgres=#

postgres=# \db
             List of tablespaces
    Name    |  Owner   |      Location
------------+----------+---------------------
 edpua_ix   | postgres | /pgIx/pgsql15/edpua
 edpua_tb   | postgres | /pgTb/pgsql15/edpua
 gebua_ix   | postgres | /pgIx/pgsql15/gebua
 gebua_tb   | postgres | /pgTb/pgsql15/gebua
 orcl_ix    | postgres | /pgIx/pgsql15/orcl
 orcl_tb    | postgres | /pgTb/pgsql15/orcl
 pg_default | postgres |
 pg_global  | postgres |
(8 rows)

postgres=#


B. Restore Full Backup

[postgres@lxtrdpgdsgv01 ~]$ nohup psql -U postgres -X -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql -d postgres > /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025_restore.log
[1] 4132
[postgres@lxtrdpgdsgv01 ~]$ 

-- OR --

nohup psql -U postgres -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql > /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025_restore.log 2>&1 &

-- Log: backup_all_databases_10OCT2025_restore.log


C. Verification

[postgres@lxtrdpgdsgv01 ~]$ psql
psql (15.14)
Type "help" for help.

postgres=# \l
                                                 List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    | ICU Locale | Locale Provider |   Access privileges
-----------+----------+----------+-------------+-------------+------------+-----------------+-----------------------
 edpua     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 gebua     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 orcl      | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            | =c/postgres          +
           |          |          |             |             |            |                 | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            | postgres=CTc/postgres+
           |          |          |             |             |            |                 | =c/postgres
(6 rows)

postgres=#


9. Restore to Another Host (Different Tablespace Paths)

A. Copy Backup File

[postgres@lxtrdpgdsgv01 backup]$ scp backup_all_databases_10OCT2025.sql 192.168.2.31:/pgBackup/pgsql15/backup/
postgres@192.168.2.31's password:
backup_all_databases_10OCT2025.sql                 100% 3556MB  51.5MB/s   01:09
[postgres@lxtrdpgdsgv01 backup]$ 

[postgres@pg17 backup]$ ls -ltr backup_all_databases_10OCT2025.sql
-rw-r--r--. 1 postgres postgres 3729147646 Oct  6 06:17 backup_all_databases_10OCT2025.sql
[postgres@pg17 backup]$


B. Extract Tablespace/Database Info

[postgres@lxtrdpgdsgv01 backup]$ grep -i "CREATE TABLESPACE" /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql
CREATE TABLESPACE orcl_tb OWNER postgres LOCATION '/pgTb/pgsql15/orcl';
CREATE TABLESPACE orcl_ix OWNER postgres LOCATION '/pgIx/pgsql15/orcl';
CREATE TABLESPACE gebua_tb OWNER postgres LOCATION '/pgTb/pgsql15/gebua';
CREATE TABLESPACE gebua_ix OWNER postgres LOCATION '/pgIx/pgsql15/gebua';
CREATE TABLESPACE edpua_tb OWNER postgres LOCATION '/pgTb/pgsql15/edpua';
CREATE TABLESPACE edpua_ix OWNER postgres LOCATION '/pgIx/pgsql15/edpua';
[postgres@lxtrdpgdsgv01 backup]$
[postgres@lxtrdpgdsgv01 backup]$ grep -i "CREATE DATABASE" /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql
CREATE DATABASE edpua WITH TEMPLATE = template0 ENCODING = 'UTF8' LOCALE_PROVIDER = libc LOCALE = 'en_US.UTF-8' TABLESPACE = edpua_tb;
CREATE DATABASE gebua WITH TEMPLATE = template0 ENCODING = 'UTF8' LOCALE_PROVIDER = libc LOCALE = 'en_US.UTF-8' TABLESPACE = gebua_tb;
CREATE DATABASE orcl WITH TEMPLATE = template0 ENCODING = 'UTF8' LOCALE_PROVIDER = libc LOCALE = 'en_US.UTF-8' TABLESPACE = orcl_tb;
[postgres@lxtrdpgdsgv01 backup]$


C. Create Tablespace Directories on Target

[postgres@pg17 ~]$ psql
psql (15.13)
Type "help" for help.

postgres=# \l
                                                 List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    | ICU Locale | Locale Provider |   Access privileges
-----------+----------+----------+-------------+-------------+------------+-----------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            | =c/postgres          +
           |          |          |             |             |            |                 | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |            | libc            | postgres=CTc/postgres+
           |          |          |             |             |            |                 | =c/postgres
(3 rows)

postgres=#
postgres=# \db
       List of tablespaces
    Name    |  Owner   | Location
------------+----------+----------
 pg_default | postgres |
 pg_global  | postgres |
(2 rows)

postgres=#


# Create Directory 

[postgres@pg17 ~]$ mkdir -p /pgData/pgsql15/orcl_tb
[postgres@pg17 ~]$ mkdir -p /pgData/pgsql15/orcl_ix
[postgres@pg17 ~]$ mkdir -p /pgData/pgsql15/gebua_tb
[postgres@pg17 ~]$ mkdir -p /pgData/pgsql15/gebua_ix
[postgres@pg17 ~]$ mkdir -p /pgData/pgsql15/edpua_tb
[postgres@pg17 ~]$ mkdir -p /pgData/pgsql15/edpua_ix


D. Create Tablespaces in PostgreSQL

postgres=# CREATE TABLESPACE orcl_tb OWNER postgres LOCATION  '/pgData/pgsql15/orcl_tb';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE orcl_ix OWNER postgres LOCATION  '/pgData/pgsql15/orcl_ix';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE gebua_tb OWNER postgres LOCATION '/pgData/pgsql15/gebua_tb';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE gebua_ix OWNER postgres LOCATION '/pgData/pgsql15/gebua_ix';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE edpua_tb OWNER postgres LOCATION '/pgData/pgsql15/edpua_tb';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE edpua_ix OWNER postgres LOCATION '/pgData/pgsql15/edpua_ix';
CREATE TABLESPACE
postgres=#


-- OR --- 
 You can modify the tablespace paths directly in the backup file; however, this approach is not recommended for large backup files due to  reliability concerns.

cd /pgBackup/pgsql15/backup/
# Replace ORCL paths
sed -i 's|/pgTb/pgsql15/orcl|/pgData/pgsql15/orcl_tb|g' backup_all_databases_10OCT2025.sql
sed -i 's|/pgIx/pgsql15/orcl|/pgData/pgsql15/orcl_ix|g' backup_all_databases_10OCT2025.sql

# Replace GEBUA paths
sed -i 's|/pgTb/pgsql15/gebua|/pgData/pgsql15/gebua_tb|g' backup_all_databases_10OCT2025.sql
sed -i 's|/pgIx/pgsql15/gebua|/pgData/pgsql15/gebua_ix|g' backup_all_databases_10OCT2025.sql

# Replace EDPUA paths
sed -i 's|/pgTb/pgsql15/edpua|/pgData/pgsql15/edpua_tb|g' backup_all_databases_10OCT2025.sql
sed -i 's|/pgIx/pgsql15/edpua|/pgData/pgsql15/edpua_ix|g' backup_all_databases_10OCT2025.sql


[postgres@pg17 ~]$ psql
psql (15.13)
Type "help" for help.

postgres=# \db
               List of tablespaces
    Name    |  Owner   |         Location
------------+----------+--------------------------
 edpua_ix   | postgres | /pgData/pgsql15/edpua_ix
 edpua_tb   | postgres | /pgData/pgsql15/edpua_tb
 gebua_ix   | postgres | /pgData/pgsql15/gebua_ix
 gebua_tb   | postgres | /pgData/pgsql15/gebua_tb
 orcl_ix    | postgres | /pgData/pgsql15/orcl_ix
 orcl_tb    | postgres | /pgData/pgsql15/orcl_tb
 pg_default | postgres |
 pg_global  | postgres |
(8 rows)

postgres=#


E. Optional: Backup on Target Host

[postgres@pg17 ~]$ nohup pg_dumpall -U postgres -v -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025_pg17.sql > /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025_pg17.log 2>&1 &
[1] 4246
[postgres@pg17 ~]$


F. Restore on Target Host

[postgres@pg17 ~]$ nohup psql -U postgres -X -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql -d postgres > /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025_restore_pg17.log 2>&1 &
[1] 5213
[postgres@pg17 ~]$ 

-- OR --

nohup psql -U postgres -f /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025.sql > /pgBackup/pgsql15/backup/backup_all_databases_10OCT2025_restore_pg17.log 2>&1 &

-- Log: backup_all_databases_10OCT2025_restore_pg17.log


G. Final Verification

[postgres@pg17 ~]$ psql
psql (15.13)
Type "help" for help.

postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | ICU Locale | Locale Provider | Access privileges
-----------+----------+----------+-------------+-------------+------------+-----------------+-----------------------
 edpua | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | | libc |
gebua | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | | libc |
orcl | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | | libc |
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | | libc |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | | libc | =c/postgres +
| | | | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | | libc | postgres=CTc/postgres+
| | | | | | | =c/postgres
(6 rows)

postgres=#

[postgres@pg17 ~]$ psql
psql (15.13)
Type "help" for help.

postgres=# \c orcl
You are now connected to database "orcl" as user "postgres".
orcl=# SELECT
orcl-# schemaname || '.' || relname AS table_name,
orcl-# pg_size_pretty(pg_total_relation_size(relid)) AS total_size,
orcl-# pg_size_pretty(pg_relation_size(relid)) AS table_size,
orcl-# pg_size_pretty(pg_total_relation_size(relid) - pg_relation_size(relid)) AS index_size
orcl-# FROM
orcl-# pg_catalog.pg_statio_user_tables
orcl-# ORDER BY
orcl-# pg_total_relation_size(relid) DESC;
 table_name | total_size | table_size | index_size
------------------+------------+------------+------------
trd.metrics_high | 799 MB | 498 MB | 301 MB
trd.metrics_mid | 638 MB | 398 MB | 240 MB
trd.employees | 493 MB | 322 MB | 172 MB
trd.sales_2023 | 399 MB | 249 MB | 150 MB
trd.sales_2024 | 399 MB | 249 MB | 150 MB
trd.sales_2022 | 399 MB | 249 MB | 150 MB
trd.sales_2021 | 398 MB | 248 MB | 150 MB
trd.metrics_low | 159 MB | 99 MB | 60 MB
trd.test_data | 71 MB | 50 MB | 21 MB
trd.emp_summary | 24 kB | 8192 bytes | 16 kB
trd.metrics_rest | 8192 bytes | 0 bytes | 8192 bytes
(11 rows)

orcl=#

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

pg_dump & pg_restore

PostgreSQL, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

PostgreSQL pg_dump and pg_restore Guide

Table of Contents
___________________________________________________________________________________________________

Backup:

0. pg_dump / pg_restore –help
1. Backup Output Formats
2. Full Database Backup

3. Schema Level Backup
4. Table Level Backup
5. Object Level Backup (PostgreSQL v17 Feature)

Restore:

6. Pre-requisites for Restore
7. Full Database Restore
8. Schema Level Restore

9. Table Level Restore

___________________________________________________________________________________________________

0. pg_dump/pg_restore –help

Click to expand pg_dump –help
[postgres@pg17 ~]$ pg_dump --help
pg_dump dumps a database as a text file or to other formats.

Usage:
  pg_dump [OPTION]... [DBNAME]

General options:
  -f, --file=FILENAME          output file or directory name
  -F, --format=c|d|t|p         output file format (custom, directory, tar,
                               plain text (default))
  -j, --jobs=NUM               use this many parallel jobs to dump
  -v, --verbose                verbose mode
  -V, --version                output version information, then exit
  -Z, --compress=0-9           compression level for compressed formats
  --lock-wait-timeout=TIMEOUT  fail after waiting TIMEOUT for a table lock
  --no-sync                    do not wait for changes to be written safely to disk
  -?, --help                   show this help, then exit

Options controlling the output content:
  -a, --data-only              dump only the data, not the schema
  -b, --blobs                  include large objects in dump
  -B, --no-blobs               exclude large objects in dump
  -c, --clean                  clean (drop) database objects before recreating
  -C, --create                 include commands to create database in dump
  -e, --extension=PATTERN      dump the specified extension(s) only
  -E, --encoding=ENCODING      dump the data in encoding ENCODING
  -n, --schema=PATTERN         dump the specified schema(s) only
  -N, --exclude-schema=PATTERN do NOT dump the specified schema(s)
  -O, --no-owner               skip restoration of object ownership in
                               plain-text format
  -s, --schema-only            dump only the schema, no data
  -S, --superuser=NAME         superuser user name to use in plain-text format
  -t, --table=PATTERN          dump the specified table(s) only
  -T, --exclude-table=PATTERN  do NOT dump the specified table(s)
  -x, --no-privileges          do not dump privileges (grant/revoke)
  --binary-upgrade             for use by upgrade utilities only
  --column-inserts             dump data as INSERT commands with column names
  --disable-dollar-quoting     disable dollar quoting, use SQL standard quoting
  --disable-triggers           disable triggers during data-only restore
  --enable-row-security        enable row security (dump only content user has
                               access to)
  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)
  --extra-float-digits=NUM     override default setting for extra_float_digits
  --if-exists                  use IF EXISTS when dropping objects
  --include-foreign-data=PATTERN
                               include data of foreign tables on foreign
                               servers matching PATTERN
  --inserts                    dump data as INSERT commands, rather than COPY
  --load-via-partition-root    load partitions via the root table
  --no-comments                do not dump comments
  --no-publications            do not dump publications
  --no-security-labels         do not dump security label assignments
  --no-subscriptions           do not dump subscriptions
  --no-table-access-method     do not dump table access methods
  --no-tablespaces             do not dump tablespace assignments
  --no-toast-compression       do not dump TOAST compression methods
  --no-unlogged-table-data     do not dump unlogged table data
  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands
  --quote-all-identifiers      quote all identifiers, even if not key words
  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts
  --section=SECTION            dump named section (pre-data, data, or post-data)
  --serializable-deferrable    wait until the dump can run without anomalies
  --snapshot=SNAPSHOT          use given snapshot for the dump
  --strict-names               require table and/or schema include patterns to
                               match at least one entity each
  --use-set-session-authorization
                               use SET SESSION AUTHORIZATION commands instead of
                               ALTER OWNER commands to set ownership

Connection options:
  -d, --dbname=DBNAME      database to dump
  -h, --host=HOSTNAME      database server host or socket directory
  -p, --port=PORT          database server port number
  -U, --username=NAME      connect as specified database user
  -w, --no-password        never prompt for password
  -W, --password           force password prompt (should happen automatically)
  --role=ROLENAME          do SET ROLE before dump

If no database name is supplied, then the PGDATABASE environment
variable value is used.

Report bugs to <pgsql-bugs@lists.postgresql.org>.
PostgreSQL home page: <https://www.postgresql.org/>
[postgres@pg17 ~]$
Click to expand pg_restore –help
[postgres@pg17 ~]$ pg_restore --help
pg_restore restores a PostgreSQL database from an archive created by pg_dump.

Usage:
  pg_restore [OPTION]... [FILE]

General options:
  -d, --dbname=NAME        connect to database name
  -f, --file=FILENAME      output file name (- for stdout)
  -F, --format=c|d|t       backup file format (should be automatic)
  -l, --list               print summarized TOC of the archive
  -v, --verbose            verbose mode
  -V, --version            output version information, then exit
  -?, --help               show this help, then exit

Options controlling the restore:
  -a, --data-only              restore only the data, no schema
  -c, --clean                  clean (drop) database objects before recreating
  -C, --create                 create the target database
  -e, --exit-on-error          exit on error, default is to continue
  -I, --index=NAME             restore named index
  -j, --jobs=NUM               use this many parallel jobs to restore
  -L, --use-list=FILENAME      use table of contents from this file for
                               selecting/ordering output
  -n, --schema=NAME            restore only objects in this schema
  -N, --exclude-schema=NAME    do not restore objects in this schema
  -O, --no-owner               skip restoration of object ownership
  -P, --function=NAME(args)    restore named function
  -s, --schema-only            restore only the schema, no data
  -S, --superuser=NAME         superuser user name to use for disabling triggers
  -t, --table=NAME             restore named relation (table, view, etc.)
  -T, --trigger=NAME           restore named trigger
  -x, --no-privileges          skip restoration of access privileges (grant/revoke)
  -1, --single-transaction     restore as a single transaction
  --disable-triggers           disable triggers during data-only restore
  --enable-row-security        enable row security
  --if-exists                  use IF EXISTS when dropping objects
  --no-comments                do not restore comments
  --no-data-for-failed-tables  do not restore data of tables that could not be
                               created
  --no-publications            do not restore publications
  --no-security-labels         do not restore security labels
  --no-subscriptions           do not restore subscriptions
  --no-table-access-method     do not restore table access methods
  --no-tablespaces             do not restore tablespace assignments
  --section=SECTION            restore named section (pre-data, data, or post-data)
  --strict-names               require table and/or schema include patterns to
                               match at least one entity each
  --use-set-session-authorization
                               use SET SESSION AUTHORIZATION commands instead of
                               ALTER OWNER commands to set ownership

Connection options:
  -h, --host=HOSTNAME      database server host or socket directory
  -p, --port=PORT          database server port number
  -U, --username=NAME      connect as specified database user
  -w, --no-password        never prompt for password
  -W, --password           force password prompt (should happen automatically)
  --role=ROLENAME          do SET ROLE before restore

The options -I, -n, -N, -P, -t, -T, and --section can be combined and specified
multiple times to select multiple objects.

If no input file name is supplied, then standard input is used.

Report bugs to <pgsql-bugs@lists.postgresql.org>.
PostgreSQL home page: <https://www.postgresql.org/>
[postgres@pg17 ~]$

1. Backup Output Formats

A. Plain Text Format
# Using -Fp explicitly:
pg_dump -U postgres -d your_db_name -Fp -f /path/to/your_db.sql

# Shell redirect
pg_dump -U postgres -d your_db_name > /path/to/your_db.sql

B. Tar Format
pg_dump -U postgres -d your_db_name -Ft -f /path/to/your_db.tar

C. Directory Format and Parallel Backup
-- Parallel backup is only supported with -Fd
-- pg_dump will create the directory if it doesn't exist.

# Without parallelism
pg_dump -U postgres -d your_db_name -Fd -f /path/to/backup_dir/

# With parallelism
pg_dump -U postgres -d your_db_name -Fd -j8 -f /path/to/backup_dir/

D. Custom Format (Compressed binary format by default)
pg_dump -U postgres -d your_db_name -Fc -f /path/to/your_db.dump


2. Full Database Backup


A. Plain Text Format

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v > /pgBackup/pgsql15/backup/blpua/database_blpua_full.sql 2> /pgBackup/pgsql15/backup/blpua/log/database_blpua_full.log &

-- OR --

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v -Fp -f /pgBackup/pgsql15/backup/blpua/database_blpua_full1.sql > /pgBackup/pgsql15/backup/blpua/log/database_blpua_full1.log 2>&1 &

-- OR -- 

[postgres@pg17 ~]$ pg_dump -U postgres -d blpua | split -b 1G - /pgBackup/pgsql15/backup/blpua/database_blpua_split.dmp
[postgres@pg17 ~]$
[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/database_blpua_s*
-rw-r--r--. 1 postgres postgres 1.0G Sep 30 11:03 /pgBackup/pgsql15/backup/blpua/database_blpua_split.dmpaa
-rw-r--r--. 1 postgres postgres 650M Sep 30 11:03 /pgBackup/pgsql15/backup/blpua/database_blpua_split.dmpab
[postgres@pg17 ~]$

-- OR --

[postgres@pg17 ~]$ nohup sh -c "pg_dump -U postgres -d blpua -v | gzip > /pgBackup/pgsql15/backup/blpua/database_blpua.gz" > /pgBackup/pgsql15/backup/blpua/log/database_blpua.log 2>&1 &

-- OR --

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v > /pgBackup/pgsql15/backup/blpua/database_blpua.sql 2> /pgBackup/pgsql15/backup/blpua/log/database_blpua.log & gzip /pgBackup/pgsql15/backup/blpua/database_blpua.sql


B. Tar Format

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v -Ft -f /pgBackup/pgsql15/backup/blpua/database_blpua.tar > /pgBackup/pgsql15/backup/blpua/log/blpua_tar_dump.log 2>&1 &


C. Directory Format

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -v -Fd -j4 -f /pgBackup/pgsql15/backup/blpua_dir > /pgBackup/pgsql15/backup/blpua/log/blpua_dump.log 2>&1 &


D. Custome Format

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -Fc -v -f /pgBackup/pgsql15/backup/blpua/database_blpua_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/database_blpua_$(date +%Y_%m_%d).log 2>&1 &
[postgres@pg17 ~]$ 
[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/database_blpua*.dmp
-rw-r--r--. 1 postgres postgres 402M Sep 29 04:58 /pgBackup/pgsql15/backup/blpua/database_blpua_2025_09_29.dmp

[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/log/database_blpua*.log
-rw-r--r--. 1 postgres postgres 2.5K Sep 29 04:58 /pgBackup/pgsql15/backup/blpua/log/database_blpua_2025_09_29.log
[postgres@pg17 ~]$


3. Schema Level Backup

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -n blp -Fc -v -f /pgBackup/pgsql15/backup/blpua/schema_blp_blpua_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua_$(date +%Y_%m_%d).log 2>&1 &
[1] 5472
[postgres@pg17 ~]$

[postgres@pg17 ~]$ jobs -l
[1]+  5472 Running                 nohup pg_dump -U postgres -d blpua -n blp -Fc -v -f /pgBackup/pgsql15/backup/blpua/schema_blp_blpua_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua_$(date +%Y_%m_%d).log 2>&1 &
[postgres@pg17 ~]$

[postgres@pg17 ~]$ jobs -l
[1]+  5472 Done                    nohup pg_dump -U postgres -d blpua -n blp -Fc -v -f /pgBackup/pgsql15/backup/blpua/schema_blp_blpua_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua_$(date +%Y_%m_%d).log 2>&1
[postgres@pg17 ~]$


[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/schema_blp_blpua*
-rw-r--r--. 1 postgres postgres 402M Sep 30 02:09 /pgBackup/pgsql15/backup/blpua/schema_blp_blpua_2025_09_30.dmp
[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua*
-rw-r--r--. 1 postgres postgres 2.5K Sep 30 02:09 /pgBackup/pgsql15/backup/blpua/log/schema_blp_blpua_2025_09_30.log
[postgres@pg17 ~]$


4. Table Level Backup

[postgres@pg17 ~]$ nohup pg_dump -U postgres -d blpua -t blp.employees -Fc -v -f /pgBackup/pgsql15/backup/blpua/table_blp_employees_$(date +%Y_%m_%d).dmp > /pgBackup/pgsql15/backup/blpua/log/table_blp_employees_$(date +%Y_%m_%d).log 2>&1 &
[1] 5652
[postgres@pg17 ~]$ 

[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/table_blp_employees_*
-rw-r--r--. 1 postgres postgres 62M Sep 30 02:22 /pgBackup/pgsql15/backup/blpua/table_blp_employees_2025_09_30.dmp
[postgres@pg17 ~]$ ls -lrth /pgBackup/pgsql15/backup/blpua/log/table_blp_employees_*
-rw-r--r--. 1 postgres postgres 2.0K Sep 30 02:22 /pgBackup/pgsql15/backup/blpua/log/table_blp_employees_2025_09_30.log
[postgres@pg17 ~]$


5. Object Level Backup (PostgreSQL v17 Feature)

++ filter is only for pg_dump, not pg_restore

[postgres@lxicbpgdsgv01 ~]$ cat include_tables.par
include table demo.table_1
include table demo.table_2
include table demo.table_3
[postgres@lxicbpgdsgv01 ~]$


[postgres@lxicbpgdsgv01 ~]$ pg_dump -d testdb_source --filter=include_tables.par > include_tables.sql

[postgres@lxicbpgdsgv01 ~]$ ls -lrth include_tables.sql
-rw-r--r--. 1 postgres postgres 4.5K Sep 30 22:19 include_tables.sql
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ cat include_tables.sql | grep -i "CREATE TABLE"
CREATE TABLE demo.table_1 (
CREATE TABLE demo.table_2 (
CREATE TABLE demo.table_3 (
[postgres@lxicbpgdsgv01 ~]$


[postgres@lxicbpgdsgv01 ~]$ cat exclude_tables.par
exclude table demo.table_1
exclude table demo.table_2
exclude table demo.table_3
[postgres@lxicbpgdsgv01 ~]$

[postgres@lxicbpgdsgv01 ~]$ pg_dump -d testdb_source --filter=exclude_tables.par > exclude_tables.sql
[postgres@lxicbpgdsgv01 ~]$ ls -ltr exclude_tables.sql
-rw-r--r--. 1 postgres postgres 10931 Sep 30 22:23 exclude_tables.sql
[postgres@lxicbpgdsgv01 ~]$
[postgres@lxicbpgdsgv01 ~]$ cat exclude_tables.sql | grep -i "CREATE TABLE"
CREATE TABLE demo.table_10 (
CREATE TABLE demo.table_4 (
CREATE TABLE demo.table_5 (
CREATE TABLE demo.table_6 (
CREATE TABLE demo.table_7 (
CREATE TABLE demo.table_8 (
CREATE TABLE demo.table_9 (
[postgres@lxicbpgdsgv01 ~]$


6. Pre-requisites for restore

postgres=# CREATE TABLESPACE TEST_TB LOCATION '/pgTb/pgsql15/test';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE TEST_IX LOCATION '/pgIx/pgsql15/test';
CREATE TABLESPACE
postgres=#
postgres=# CREATE DATABASE TEST TABLESPACE TEST_TB;
CREATE DATABASE
postgres=#

[postgres@pg17 blpua]$ scp *.dmp postgres@192.168.2.32:/pgBackup/pgsql15/backup/test
postgres@192.168.2.32's password:
database_blpua_2025_09_29.dmp                           100%  402MB  39.9MB/s   00:10
schema_blp_blpua_2025_09_30.dmp                         100%  402MB  43.8MB/s   00:09
table_blp_employees_2025_09_30.dmp                      100%   61MB  47.0MB/s   00:01
[postgres@pg17 blpua]$


7. Full Database Restore

++ The source database uses two custom tablespaces: blpua_tbs01 and blpua_ixtbs01.
++ On the target system, I have two different tablespaces: test_tb and test_ix.
++ Unlike Oracle, PostgreSQL does not provide a direct equivalent of REMAP_TABLESPACE.
++ Therefore, to simplify the restore and ensure all objects are placed into the default tablespace (test_tb).
++ Using --no-owner --no-tablespaces in pg_restore — even if not specified, data still gets written to the default database tablespace. These options just help avoid errors in the log.
++ This ensures that all objects are restored to the default tablespace of the target database, regardless of their original tablespace assignments.
++ Later, we can move all indexes to the desired target tablespace using the ALTER INDEX ... SET TABLESPACE command.
postgres=# \c test
You are now connected to database "test" as user "postgres".
test=# ALTER INDEX blp.idx_employee_name SET TABLESPACE test_ix;  <--  This command physically relocates the index to the test_ix tablespace.
ALTER INDEX
test=#


[postgres@lxtrdpgdsgv01 ~]$ nohup pg_restore -U postgres -d test --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/database_blpua_2025_09_29.dmp > /pgBackup/pgsql15/backup/test/log/restore_blpua_test_$(date +%Y_%m_%d).log 2>&1 &
[1] 5300
[postgres@lxtrdpgdsgv01 ~]$

[postgres@lxtrdpgdsgv01 ~]$ cat /pgBackup/pgsql15/backup/test/log/restore_blpua_test_2025_09_30.log
nohup: ignoring input
pg_restore: connecting to database for restore
pg_restore: creating SCHEMA "blp"
pg_restore: creating TYPE "blp.address"
pg_restore: creating TYPE "blp.employment_status"
pg_restore: creating DOMAIN "blp.positive_integer"
pg_restore: creating FUNCTION "blp.get_salary(integer)"
pg_restore: creating FUNCTION "blp.log_update()"
pg_restore: creating PROCEDURE "blp.raise_salary(integer, numeric)"
pg_restore: creating SEQUENCE "blp.emp_id_seq"
pg_restore: creating TABLE "blp.employees"
pg_restore: creating MATERIALIZED VIEW "blp.emp_summary"
pg_restore: creating SEQUENCE "blp.employees_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.employees_id_seq"
pg_restore: creating VIEW "blp.high_paid_employees"
pg_restore: creating TABLE "blp.metrics"
pg_restore: creating TABLE "blp.metrics_high"
pg_restore: creating SEQUENCE "blp.metrics_id_seq"
pg_restore: creating TABLE "blp.metrics_low"
pg_restore: creating TABLE "blp.metrics_mid"
pg_restore: creating TABLE "blp.metrics_rest"
pg_restore: creating TABLE "blp.sales"
pg_restore: creating TABLE "blp.sales_2021"
pg_restore: creating TABLE "blp.sales_2022"
pg_restore: creating TABLE "blp.sales_2023"
pg_restore: creating TABLE "blp.sales_2024"
pg_restore: creating SEQUENCE "blp.sales_id_seq"
pg_restore: creating TABLE "blp.test_data"
pg_restore: creating SEQUENCE "blp.test_data_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.test_data_id_seq"
pg_restore: creating TABLE ATTACH "blp.metrics_high"
pg_restore: creating TABLE ATTACH "blp.metrics_low"
pg_restore: creating TABLE ATTACH "blp.metrics_mid"
pg_restore: creating TABLE ATTACH "blp.metrics_rest"
pg_restore: creating TABLE ATTACH "blp.sales_2021"
pg_restore: creating TABLE ATTACH "blp.sales_2022"
pg_restore: creating TABLE ATTACH "blp.sales_2023"
pg_restore: creating TABLE ATTACH "blp.sales_2024"
pg_restore: creating DEFAULT "blp.employees id"
pg_restore: creating DEFAULT "blp.test_data id"
pg_restore: processing data for table "blp.employees"
pg_restore: processing data for table "blp.metrics_high"
pg_restore: processing data for table "blp.metrics_low"
pg_restore: processing data for table "blp.metrics_mid"
pg_restore: processing data for table "blp.metrics_rest"
pg_restore: processing data for table "blp.sales_2021"
pg_restore: processing data for table "blp.sales_2022"
pg_restore: processing data for table "blp.sales_2023"
pg_restore: processing data for table "blp.sales_2024"
pg_restore: processing data for table "blp.test_data"
pg_restore: executing SEQUENCE SET emp_id_seq
pg_restore: executing SEQUENCE SET employees_id_seq
pg_restore: executing SEQUENCE SET metrics_id_seq
pg_restore: executing SEQUENCE SET sales_id_seq
pg_restore: executing SEQUENCE SET test_data_id_seq
pg_restore: creating CONSTRAINT "blp.employees employees_pkey"
pg_restore: creating CONSTRAINT "blp.metrics metrics_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_high metrics_high_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_low metrics_low_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_mid metrics_mid_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_rest metrics_rest_pkey"
pg_restore: creating CONSTRAINT "blp.sales sales_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2021 sales_2021_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2022 sales_2022_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2023 sales_2023_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2024 sales_2024_pkey"
pg_restore: creating CONSTRAINT "blp.test_data test_data_pkey"
pg_restore: creating INDEX "blp.idx_employee_name"
pg_restore: creating INDEX ATTACH "blp.metrics_high_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_low_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_mid_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_rest_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2021_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2022_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2023_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2024_pkey"
pg_restore: creating TRIGGER "blp.employees employee_update_trigger"
pg_restore: creating MATERIALIZED VIEW DATA "blp.emp_summary"
[postgres@lxtrdpgdsgv01 ~]$


8. Schema Level Restore

++ pg_restore does not support schema remapping like Oracle’s REMAP_SCHEMA.
++ I want to restore schema blp --> trd. 
++ Option A: Restore Dump as it is, Then Rename <--- Best for simplicity
++ Option B: Split Dump into Schema + Data, Modify Both <--- This is more complex, but possible if you must use schema remapping.
++ This is schema dump file : /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
++ Using --no-owner --no-tablespaces in pg_restore — even if not specified, data still gets written to the default database tablespace. These options just help avoid errors in the log.


A. To Restore schema from own schema’s backup

postgres=# CREATE TABLESPACE ORCL_TB LOCATION '/pgTb/pgsql15/orcl';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE ORCL_IX LOCATION '/pgIx/pgsql15/orcl';
CREATE TABLESPACE
postgres=#
postgres=# CREATE DATABASE ORCL TABLESPACE ORCL_TB;
CREATE DATABASE
postgres=#

-- With NO parallelism


[postgres@lxtrdpgdsgv01 ~]$ nohup pg_restore -U postgres -d orcl --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp > /pgBackup/pgsql15/backup/test/log/restore_blpua_test_$(date +%Y_%m_%d).log 2>&1 &
[1] 7344
[postgres@lxtrdpgdsgv01 ~]$

postgres=# \c orcl
You are now connected to database "orcl" as user "postgres".
orcl=# ALTER SCHEMA blp RENAME TO trd;
ALTER SCHEMA
orcl=# \dn
      List of schemas
  Name  |       Owner
--------+-------------------
 public | pg_database_owner
 trd    | postgres
(2 rows)

orcl=#


[postgres@lxtrdpgdsgv01 ~]$ cat /pgBackup/pgsql15/backup/test/log/restore_blpua_test_2025_09_30.log
nohup: ignoring input
pg_restore: connecting to database for restore
pg_restore: creating SCHEMA "blp"
pg_restore: creating TYPE "blp.address"
pg_restore: creating TYPE "blp.employment_status"
pg_restore: creating DOMAIN "blp.positive_integer"
pg_restore: creating FUNCTION "blp.get_salary(integer)"
pg_restore: creating FUNCTION "blp.log_update()"
pg_restore: creating PROCEDURE "blp.raise_salary(integer, numeric)"
pg_restore: creating SEQUENCE "blp.emp_id_seq"
pg_restore: creating TABLE "blp.employees"
pg_restore: creating MATERIALIZED VIEW "blp.emp_summary"
pg_restore: creating SEQUENCE "blp.employees_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.employees_id_seq"
pg_restore: creating VIEW "blp.high_paid_employees"
pg_restore: creating TABLE "blp.metrics"
pg_restore: creating TABLE "blp.metrics_high"
pg_restore: creating SEQUENCE "blp.metrics_id_seq"
pg_restore: creating TABLE "blp.metrics_low"
pg_restore: creating TABLE "blp.metrics_mid"
pg_restore: creating TABLE "blp.metrics_rest"
pg_restore: creating TABLE "blp.sales"
pg_restore: creating TABLE "blp.sales_2021"
pg_restore: creating TABLE "blp.sales_2022"
pg_restore: creating TABLE "blp.sales_2023"
pg_restore: creating TABLE "blp.sales_2024"
pg_restore: creating SEQUENCE "blp.sales_id_seq"
pg_restore: creating TABLE "blp.test_data"
pg_restore: creating SEQUENCE "blp.test_data_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.test_data_id_seq"
pg_restore: creating TABLE ATTACH "blp.metrics_high"
pg_restore: creating TABLE ATTACH "blp.metrics_low"
pg_restore: creating TABLE ATTACH "blp.metrics_mid"
pg_restore: creating TABLE ATTACH "blp.metrics_rest"
pg_restore: creating TABLE ATTACH "blp.sales_2021"
pg_restore: creating TABLE ATTACH "blp.sales_2022"
pg_restore: creating TABLE ATTACH "blp.sales_2023"
pg_restore: creating TABLE ATTACH "blp.sales_2024"
pg_restore: creating DEFAULT "blp.employees id"
pg_restore: creating DEFAULT "blp.test_data id"
pg_restore: processing data for table "blp.employees"
pg_restore: processing data for table "blp.metrics_high"
pg_restore: processing data for table "blp.metrics_low"
pg_restore: processing data for table "blp.metrics_mid"
pg_restore: processing data for table "blp.metrics_rest"
pg_restore: processing data for table "blp.sales_2021"
pg_restore: processing data for table "blp.sales_2022"
pg_restore: processing data for table "blp.sales_2023"
pg_restore: processing data for table "blp.sales_2024"
pg_restore: processing data for table "blp.test_data"
pg_restore: executing SEQUENCE SET emp_id_seq
pg_restore: executing SEQUENCE SET employees_id_seq
pg_restore: executing SEQUENCE SET metrics_id_seq
pg_restore: executing SEQUENCE SET sales_id_seq
pg_restore: executing SEQUENCE SET test_data_id_seq
pg_restore: creating CONSTRAINT "blp.employees employees_pkey"
pg_restore: creating CONSTRAINT "blp.metrics metrics_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_high metrics_high_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_low metrics_low_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_mid metrics_mid_pkey"
pg_restore: creating CONSTRAINT "blp.metrics_rest metrics_rest_pkey"
pg_restore: creating CONSTRAINT "blp.sales sales_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2021 sales_2021_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2022 sales_2022_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2023 sales_2023_pkey"
pg_restore: creating CONSTRAINT "blp.sales_2024 sales_2024_pkey"
pg_restore: creating CONSTRAINT "blp.test_data test_data_pkey"
pg_restore: creating INDEX "blp.idx_employee_name"
pg_restore: creating INDEX ATTACH "blp.metrics_high_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_low_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_mid_pkey"
pg_restore: creating INDEX ATTACH "blp.metrics_rest_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2021_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2022_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2023_pkey"
pg_restore: creating INDEX ATTACH "blp.sales_2024_pkey"
pg_restore: creating TRIGGER "blp.employees employee_update_trigger"
pg_restore: creating MATERIALIZED VIEW DATA "blp.emp_summary"
[postgres@lxtrdpgdsgv01 ~]$


--- OR ---

-- With parallelism 4

nohup pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp > /pgBackup/pgsql15/backup/test/log/restore_blpua_test_$(date +%Y_%m_%d).log1 2>&1 &

[postgres@lxtrdpgdsgv01 ~]$ ps -ef | grep pg_restore
postgres    7089    5102  0 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7091    7089  2 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7092    7089  3 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7093    7089  2 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7094    7089  2 05:49 pts/0    00:00:00 pg_restore -U postgres -d orcl -j4 --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
postgres    7126    5736  0 05:49 pts/1    00:00:00 grep --color=auto pg_restore
[postgres@lxtrdpgdsgv01 ~]$


B. To Restore only a specific schema from Full Database Backup

Step 1: Create schema 

gebua=# CREATE SCHEMA IF NOT EXISTS blp;

Step 2: Restore Objects and Data

nohup pg_restore -U postgres -d gebua -n blp --no-owner --no-tablespaces -v /pgBackup/pgsql15/backup/test/database_blpua_2025_09_29.dmp > /pgBackup/pgsql15/backup/test/log/restore_blpua_test_$(date +%Y_%m_%d).log_full 2>&1 &


9. Table Level Restore


A. Restore Table from own table’s backup

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -l /pgBackup/pgsql15/backup/test/table_blp_employees_2025_09_30.dmp
;
; Archive created at 2025-09-30 02:22:21 EDT
;     dbname: blpua
;     TOC Entries: 13
;     Compression: -1
;     Dump Version: 1.14-0
;     Format: CUSTOM
;     Integer: 4 bytes
;     Offset: 8 bytes
;     Dumped from database version: 15.13
;     Dumped by pg_dump version: 15.13
;
;
; Selected TOC Entries:
;
229; 1259 92805 TABLE blp employees postgres
228; 1259 92804 SEQUENCE blp employees_id_seq postgres
4338; 0 0 SEQUENCE OWNED BY blp employees_id_seq postgres
4181; 2604 92808 DEFAULT blp employees id postgres
4331; 0 92805 TABLE DATA blp employees postgres
4339; 0 0 SEQUENCE SET blp employees_id_seq postgres
4183; 2606 92812 CONSTRAINT blp employees employees_pkey postgres
4184; 1259 92824 INDEX blp idx_employee_name postgres
4185; 2620 92829 TRIGGER blp employees employee_update_trigger postgres
[postgres@lxtrdpgdsgv01 ~]$


Step 1: Create schema 

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=# CREATE SCHEMA IF NOT EXISTS blp;
CREATE SCHEMA
edpua=#


Step 2: Restore Objects and Data

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -d edpua --no-tablespaces -v /pgBackup/pgsql15/backup/test/table_blp_employees_2025_09_30.dmp
pg_restore: connecting to database for restore
pg_restore: creating TABLE "blp.employees"
pg_restore: creating SEQUENCE "blp.employees_id_seq"
pg_restore: creating SEQUENCE OWNED BY "blp.employees_id_seq"
pg_restore: creating DEFAULT "blp.employees id"
pg_restore: processing data for table "blp.employees"
pg_restore: executing SEQUENCE SET employees_id_seq
pg_restore: creating CONSTRAINT "blp.employees employees_pkey"
pg_restore: creating INDEX "blp.idx_employee_name"
pg_restore: creating TRIGGER "blp.employees employee_update_trigger"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 4185; 2620 92829 TRIGGER employees employee_update_trigger postgres
pg_restore: error: could not execute query: ERROR:  function blp.log_update() does not exist
Command was: CREATE TRIGGER employee_update_trigger AFTER UPDATE ON blp.employees FOR EACH ROW EXECUTE FUNCTION blp.log_update();


pg_restore: warning: errors ignored on restore: 1
[postgres@lxtrdpgdsgv01 ~]$

++ PostgreSQL does not have a --ignore-errors flag, The above erros ignorable.


B. Restore Single Table data-only from Schema backup

edpua=# TRUNCATE TABLE blp.employees;
TRUNCATE TABLE
edpua=#

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -d edpua --data-only -t employees -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
pg_restore: connecting to database for restore
pg_restore: processing data for table "blp.employees"
[postgres@lxtrdpgdsgv01 ~]$

edpua=# select count(*) from blp.employees;
 5000000  <-----

edpua=#


C. Restore Single Table from Schema backup

Step 1: Create schema 

edpua=# CREATE SCHEMA blp;

Step 2: Restore Objects and Data

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -d edpua --clean -t employees -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
pg_restore: connecting to database for restore
pg_restore: dropping TABLE employees
pg_restore: creating TABLE "blp.employees"
pg_restore: processing data for table "blp.employees"
[postgres@lxtrdpgdsgv01 ~]$
[postgres@lxtrdpgdsgv01 ~]$ psql
psql (15.14)
Type "help" for help.

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=# select count(*) from blp.employees;
  count
---------
 5000000 <----
(1 row)

edpua=#


D. Restore Single Table from Schema backup (add both -n and -t)

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -d edpua --clean -n blp -t employees -v /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
pg_restore: connecting to database for restore
pg_restore: dropping TABLE employees
pg_restore: creating TABLE "blp.employees"
pg_restore: processing data for table "blp.employees"
[postgres@lxtrdpgdsgv01 ~]$
[postgres@lxtrdpgdsgv01 ~]$ psql
psql (15.14)
Type "help" for help.

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=# select count(*) from blp.employees;
  count
---------
 5000000 <-----
(1 row)

edpua=#


E. Restoring Using a Plain SQL File (COPY-based)

Step 1: Extracted table data from custom-format dump

[postgres@lxtrdpgdsgv01 ~]$ pg_restore -U postgres -n blp -t employees -f /pgBackup/pgsql15/backup/test/blp_employees.sql /pgBackup/pgsql15/backup/test/schema_blp_blpua_2025_09_30.dmp
[postgres@lxtrdpgdsgv01 ~]$
[postgres@lxtrdpgdsgv01 ~]$ ls -lrth /pgBackup/pgsql15/backup/test/blp_employees.sql
-rw-r--r--. 1 postgres postgres 210M Sep 30 09:17 /pgBackup/pgsql15/backup/test/blp_employees.sql
[postgres@lxtrdpgdsgv01 ~]$

[postgres@lxtrdpgdsgv01 ~]$ head -50 /pgBackup/pgsql15/backup/test/blp_employees.sql
--
-- PostgreSQL database dump
--

\restrict TthhVsS8JvVxrplYNApzHUalZpygXUqFg2pIvbSH8GTkbJ6WBNyW0JLnuuvkPdt

-- Dumped from database version 15.13
-- Dumped by pg_dump version 15.13

SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = off;

SET default_tablespace = '';

SET default_table_access_method = heap;

--
-- Name: employees; Type: TABLE; Schema: blp; Owner: postgres
--

CREATE TABLE blp.employees (
    id integer NOT NULL,
    name text,
    salary numeric(10,2),
    hired_on date
);


ALTER TABLE blp.employees OWNER TO postgres;

--
-- Data for Name: employees; Type: TABLE DATA; Schema: blp; Owner: postgres
--

COPY blp.employees (id, name, salary, hired_on) FROM stdin;
1       Employee_1      70969.47        2005-06-13
2       Employee_2      94341.15        2006-12-09
3       Employee_3      101657.57       2012-07-05
4       Employee_4      54701.92        2010-01-05
5       Employee_5      104229.43       2001-06-28
6       Employee_6      95842.57        2010-10-06
7       Employee_7      50618.08        2015-06-02
[postgres@lxtrdpgdsgv01 ~]$

Step 2: Restore using psql command

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=#
edpua=# drop table blp.employees;
DROP TABLE
edpua=# exit
[postgres@lxtrdpgdsgv01 ~]$


psql -U postgres -d edpua < /pgBackup/pgsql15/backup/test/blp_employees.sql

[postgres@lxtrdpgdsgv01 ~]$ psql -U postgres -d edpua < /pgBackup/pgsql15/backup/test/blp_employees.sql
SET
SET
SET
SET
SET
 set_config
------------

(1 row)

SET
SET
SET
SET
SET
SET
CREATE TABLE
ALTER TABLE
COPY 5000000
[postgres@lxtrdpgdsgv01 ~]$

[postgres@lxtrdpgdsgv01 ~]$ psql
psql (15.14)
Type "help" for help.

postgres=# \c edpua
You are now connected to database "edpua" as user "postgres".
edpua=# SELECT COUNT(*) FROM blp.employees;
  count
---------
 5000000 <-----
(1 row)

edpua=#

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

PostgreSQL Tablespace

Uncategorized, , , , , , , , , , , , , ,

PostgreSQL Tablespace Management

Table of Contents

___________________________________________________________________________________________________

0. Introduction
1. How big can a PostgreSQL tablespace grow?
2. Create a Tablespace
3. Create a Database with default tablespace
4. Create Tables on Data tablspace
5. Crate Index on Index tablespaces
6. Create a Dedicated Tablespace for Temp
7. Assign Temp Tablespace to a Database
8. Check Tablespace OIDs
9. Move Table Between Tablespaces
10. Verify Database Directory (Filesystem)
11. Rename tablespace
12. Drop Tablespace
13. Change ownership for Tablespace
14. Verify Symbolic Links
15. How to find tablespace size
___________________________________________________________________________________________________


0. Introduction

What is a Tablespace?

--> Tablespace is a logical structure in which objects are stored.

--> A tablespace is simply a physical storage location on disk where PostgreSQL stores database objects (tables, indexes, etc.). 

--> By default database objects are stored in the current default tablespace of the database.

List tablespaces:

\db+ or select * from pg_tablespace;

 

Types of Tablespaces: Default Tablespaces and Non-Default Tablespaces
I. Default Tablespaces

These come built-in with PostgreSQL:

pg_global – for shared system data
pg_default – for regular user data (tables, indexes, etc.)
Featurepg_defaultpg_global
PurposeStores regular user dataStores global system data
Used by default?Yes, for tables/indexesNo, only used internally
Can store user tables?✅ Yes❌ No
Location$PGDATA/base/$PGDATA/global/
Droppable?❌ No❌ No
II. Non-Default Tablespaces :

--> These are created by users to store data in custom locations.
--> Useful for managing storage better (e.g., putting large tables on faster disks).


1. How big can a PostgreSQL tablespace grow?

There is no fixed maximum size for a tablespace in PostgreSQL itself.
The limit depends on your operating system and filesystem where the tablespace directory is located.

For example:

On modern filesystems like ext4 or XFS, a single file can be several terabytes (TB) or even petabytes (PB).
PostgreSQL breaks large tables into multiple files (each typically up to 1 GB) internally, so extremely large tablespaces are supported.
Your disk/storage capacity is the main practical limit.


2. Create a Tablespace

[root@pg17 ~]# mkdir -p /pgData/pgsql15/edbua_tbs01
[root@pg17 ~]# mkdir -p /pgData/pgsql15/edbua_tbs02
[root@pg17 ~]# mkdir -p /pgData/pgsql15/edbua_ixtbs01
[root@pg17 ~]# mkdir -p /pgData/pgsql15/edbua_temptbs01
[root@pg17 ~]# chown postgres:postgres /pgData/pgsql15/edbua*
[root@pg17 ~]# chmod 700 /pgData/pgsql15/edbua*
[root@pg17 ~]#

[root@pg17 ~]# su - postgres
[postgres@pg17 ~]$ psql
psql (15.13)
Type "help" for help.

postgres=# CREATE TABLESPACE edbua_tbs01 LOCATION '/pgData/pgsql15/edbua_tbs01';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE edbua_tbs02 LOCATION '/pgData/pgsql15/edbua_tbs02';
CREATE TABLESPACE
postgres=# CREATE TABLESPACE edbua_ixtbs01 LOCATION '/pgData/pgsql15/edbua_ixtbs01';
CREATE TABLESPACE
postgres=#


3. Create a Database with default tablespace

postgres=# CREATE DATABASE edbua TABLESPACE edbua_tbs01;
CREATE DATABASE
postgres=# 
postgres=# SELECT oid, datname, dattablespace, datistemplate, datdba FROM pg_database WHERE datname = 'edbua';
  oid  | datname | dattablespace | datistemplate | datdba
-------+---------+---------------+---------------+--------
 92666 | edbua   |         92662 | f             |     10
(1 row)

postgres=#


4. Create Tables on Data tablspace

postgres=# \c edbua
You are now connected to database "edbua" as user "postgres".
edbua=# CREATE TABLE emp01 (id int, name text) TABLESPACE edbua_tbs01;
CREATE TABLE
edbua=# CREATE TABLE emp02 (id int, name text) TABLESPACE edbua_tbs01;
CREATE TABLE
edbua=# CREATE TABLE emp03 (id int, name text) TABLESPACE edbua_tbs01;
CREATE TABLE
edbua=#


5. Create Index on Index tablespaces

edbua=# CREATE INDEX emp01_ix ON emp01(name) TABLESPACE edbua_ixtbs01;
CREATE INDEX
edbua=# CREATE INDEX emp02_ix ON emp02(name) TABLESPACE edbua_ixtbs01;
CREATE INDEX
edbua=# CREATE INDEX emp03_ix ON emp03(name) TABLESPACE edbua_ixtbs01;
CREATE INDEX
edbua=# SELECT c.oid, c.relname, c.relkind, t.spcname AS tablespace, c.relfilenode
edbua-# FROM pg_class c
edbua-# LEFT JOIN pg_tablespace t ON c.reltablespace = t.oid
edbua-# WHERE c.relname IN ('emp01','emp02','emp03','emp01_ix','emp02_ix','emp03_ix');
  oid  | relname  | relkind |  tablespace   | relfilenode
-------+----------+---------+---------------+-------------
 92685 | emp01    | r       |               |       92685
 92690 | emp02    | r       |               |       92690
 92695 | emp03    | r       |               |       92695
 92700 | emp01_ix | i       | edbua_ixtbs01 |       92700
 92701 | emp02_ix | i       | edbua_ixtbs01 |       92701
 92702 | emp03_ix | i       | edbua_ixtbs01 |       92702
(6 rows)

edbua=#

If a table or index is created in the database’s default tablespace, PostgreSQL does not explicitly record the tablespace in pg_class.reltablespace.

But when it’s created in a non-default tablespace, PostgreSQL records that tablespace OID in pg_class.reltablespace.

If you want to always see the tablespace name (even when it's the default)

edbua=# SELECT
edbua-#     c.oid,
edbua-#     c.relname,
edbua-#     c.relkind,
edbua-#     COALESCE(t1.spcname, t2.spcname) AS tablespace,
edbua-#     c.relfilenode
edbua-# FROM pg_class c
edbua-# LEFT JOIN pg_tablespace t1 ON c.reltablespace = t1.oid
edbua-# LEFT JOIN pg_database d ON d.datname = current_database()
edbua-# LEFT JOIN pg_tablespace t2 ON d.dattablespace = t2.oid
edbua-# WHERE c.relname IN ('emp01','emp02','emp03','emp01_ix','emp02_ix','emp03_ix');
  oid  | relname  | relkind |  tablespace   | relfilenode
-------+----------+---------+---------------+-------------
 92685 | emp01    | r       | edbua_tbs01   |       92685
 92690 | emp02    | r       | edbua_tbs01   |       92690
 92695 | emp03    | r       | edbua_tbs01   |       92695
 92700 | emp01_ix | i       | edbua_ixtbs01 |       92700
 92701 | emp02_ix | i       | edbua_ixtbs01 |       92701
 92702 | emp03_ix | i       | edbua_ixtbs01 |       92702
(6 rows)

edbua=#

pg_tblspc/<tbs_oid>/PG_15_<catalog_version>/<db_oid>/<relfilenode>

edbua=# SELECT relname, pg_relation_filepath(oid) AS filepath
edbua-# FROM pg_class
edbua-# WHERE relname IN ('emp01','emp02','emp03','emp01_ix','emp02_ix','emp03_ix');
 relname  |                  filepath
----------+---------------------------------------------
 emp01    | pg_tblspc/92662/PG_15_202209061/92666/92685
 emp02    | pg_tblspc/92662/PG_15_202209061/92666/92690
 emp03    | pg_tblspc/92662/PG_15_202209061/92666/92695
 emp01_ix | pg_tblspc/92664/PG_15_202209061/92666/92700
 emp02_ix | pg_tblspc/92664/PG_15_202209061/92666/92701
 emp03_ix | pg_tblspc/92664/PG_15_202209061/92666/92702
(6 rows)

edbua=#


6. Create a Dedicated Tablespace for Temp

postgres=# CREATE TABLESPACE edbua_temptbs01 LOCATION '/pgData/pgsql15/edbua_temptbs01';
CREATE TABLESPACE
postgres=#


7. Assign Temp Tablespace to a Database

postgres=# ALTER DATABASE edbua SET temp_tablespaces = edbua_temptbs01;
ALTER DATABASE
postgres=#

postgres=# \c edbua
You are now connected to database "edbua" as user "postgres".
edbua=# 

edbua=# CREATE TEMP TABLE emp_tmp (id int, name text);
CREATE TABLE
edbua=#
edbua=# SELECT c.oid, c.relname, t.spcname, pg_relation_filepath(c.oid)
edbua-# FROM pg_class c
edbua-# LEFT JOIN pg_tablespace t ON c.reltablespace = t.oid
edbua-# WHERE c.relname = 'emp_tmp';
  oid  | relname |     spcname     |              pg_relation_filepath
-------+---------+-----------------+------------------------------------------------
 92705 | emp_tmp | edbua_temptbs01 | pg_tblspc/92665/PG_15_202209061/92666/t4_92705
(1 row)

edbua=#

pg_tblspc/<tbs_oid>/PG_15_<catalog_version>/<db_oid>/<relfilenode>


8. Check Tablespace OIDs

edbua=# SELECT oid, spcname AS tablespace_name, pg_tablespace_location(oid) AS location FROM pg_tablespace ORDER BY oid;
  oid  | tablespace_name |            location
-------+-----------------+---------------------------------
  1663 | pg_default      |
  1664 | pg_global       |
 24581 | dell_ts_data    | /u01/data/dell_ts_data
 24582 | dell_ts_index   | /u01/data/dell_ts_index
 24594 | dell_tbs        | /pgData/pgsql15/data
 24597 | trd_tb01        | /pgData/pgsql15/data/trd_tb01
 24598 | trd_ix01        | /pgData/pgsql15/data/trd_ix01
 32787 | geb_tb          | /pgData/pgsql15/pgtb/geb_tb
 32788 | geb_ix          | /pgData/pgsql15/pgix/geb_ix
 92662 | edbua_tbs01     | /pgData/pgsql15/edbua_tbs01
 92663 | edbua_tbs02     | /pgData/pgsql15/edbua_tbs02
 92664 | edbua_ixtbs01   | /pgData/pgsql15/edbua_ixtbs01
 92665 | edbua_temptbs01 | /pgData/pgsql15/edbua_temptbs01
(13 rows)

edbua=#


9. Move Table Between Tablespaces

edbua=# ALTER TABLE emp01 SET TABLESPACE edbua_tbs02;
ALTER TABLE
edbua=# ALTER TABLE emp02 SET TABLESPACE edbua_tbs02;
ALTER TABLE
edbua=# ALTER TABLE emp03 SET TABLESPACE edbua_tbs02;
ALTER TABLE
edbua=#

pg_tblspc/<tbs_oid>/PG_15_<catalog_version>/<db_oid>/<relfilenode>

edbua=# select pg_relation_filepath('emp01');
            pg_relation_filepath
---------------------------------------------
 pg_tblspc/92663/PG_15_202209061/92666/92710
(1 row)

edbua=# 

edbua=# SELECT relname, pg_relation_filepath(oid) AS filepath
FROM pg_class
WHERE relname IN ('emp01','emp02','emp03','emp01_ix','emp02_ix','emp03_ix');
 relname  |                  filepath
----------+---------------------------------------------
 emp01    | pg_tblspc/92663/PG_15_202209061/92666/92710
 emp02    | pg_tblspc/92663/PG_15_202209061/92666/92713
 emp03    | pg_tblspc/92663/PG_15_202209061/92666/92716
 emp01_ix | pg_tblspc/92664/PG_15_202209061/92666/92700
 emp02_ix | pg_tblspc/92664/PG_15_202209061/92666/92701
 emp03_ix | pg_tblspc/92664/PG_15_202209061/92666/92702
(6 rows)

edbua=#

Important: Indexes do not move automatically when you move a table.
They remain in their original tablespace (edbua_ixtbs01 in this case).
If you want indexes in edbua_tbs02, you must explicitly run:

edbua=# ALTER INDEX emp01_ix SET TABLESPACE edbua_tbs02;
ALTER INDEX
edbua=# ALTER INDEX emp02_ix SET TABLESPACE edbua_tbs02;
ALTER INDEX
edbua=# ALTER INDEX emp03_ix SET TABLESPACE edbua_tbs02;
ALTER INDEX
edbua=#
edbua=# SELECT c.relname, t.spcname AS tablespace, pg_relation_filepath(c.oid)
edbua-# FROM pg_class c
edbua-# LEFT JOIN pg_tablespace t ON c.reltablespace = t.oid
edbua-# WHERE c.relname IN ('emp01','emp02','emp03','emp01_ix','emp02_ix','emp03_ix');
 relname  | tablespace  |            pg_relation_filepath
----------+-------------+---------------------------------------------
 emp01_ix | edbua_tbs02 | pg_tblspc/92663/PG_15_202209061/92666/92719
 emp02_ix | edbua_tbs02 | pg_tblspc/92663/PG_15_202209061/92666/92720
 emp03_ix | edbua_tbs02 | pg_tblspc/92663/PG_15_202209061/92666/92721
 emp01    | edbua_tbs02 | pg_tblspc/92663/PG_15_202209061/92666/92710
 emp02    | edbua_tbs02 | pg_tblspc/92663/PG_15_202209061/92666/92713
 emp03    | edbua_tbs02 | pg_tblspc/92663/PG_15_202209061/92666/92716
(6 rows)

edbua=#

edbua=# SELECT
edbua-#     c.relname,
edbua-#     COALESCE(t1.spcname, t2.spcname) AS tablespace,
edbua-#     pg_catalog.pg_tablespace_location(COALESCE(c.reltablespace, d.dattablespace)) AS tablespace_path,
edbua-#     current_setting('data_directory') AS data_directory,
edbua-#     CASE
edbua-#         WHEN c.reltablespace = 0 THEN
edbua-#             current_setting('data_directory') || '/base/' || d.oid || '/' || c.relfilenode
edbua-#         ELSE
edbua-#             pg_catalog.pg_tablespace_location(c.reltablespace) || '/PG_' ||
edbua-#             split_part(split_part(version(), ' ', 2), '.', 1) || '_' ||
edbua-#             '202209061/' || d.oid || '/' || c.relfilenode
edbua-#     END AS full_path_guess
edbua-# FROM pg_class c
edbua-# LEFT JOIN pg_tablespace t1 ON c.reltablespace = t1.oid
edbua-# LEFT JOIN pg_database d ON d.datname = current_database()
edbua-# LEFT JOIN pg_tablespace t2 ON d.dattablespace = t2.oid
edbua-# WHERE c.relname IN ('emp01','emp02','emp03','emp01_ix','emp02_ix','emp03_ix');
 relname  | tablespace  |       tablespace_path       |    data_directory    |                     full_path_guess
----------+-------------+-----------------------------+----------------------+---------------------------------------------------------
 emp01_ix | edbua_tbs02 | /pgData/pgsql15/edbua_tbs02 | /pgData/pgsql15/data | /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92719
 emp02_ix | edbua_tbs02 | /pgData/pgsql15/edbua_tbs02 | /pgData/pgsql15/data | /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92720
 emp03_ix | edbua_tbs02 | /pgData/pgsql15/edbua_tbs02 | /pgData/pgsql15/data | /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92721
 emp01    | edbua_tbs02 | /pgData/pgsql15/edbua_tbs02 | /pgData/pgsql15/data | /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92710
 emp02    | edbua_tbs02 | /pgData/pgsql15/edbua_tbs02 | /pgData/pgsql15/data | /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92713
 emp03    | edbua_tbs02 | /pgData/pgsql15/edbua_tbs02 | /pgData/pgsql15/data | /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92716
(6 rows)

edbua=#


10. Verify Database Directory (Filesystem)

edbua=# SELECT oid, datname FROM pg_database WHERE datname = current_database();
  oid  | datname
-------+---------
 92666 | edbua
(1 row)

edbua=#

[root@pg17 ~]# ls -ltr /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/*
-rw-------. 1 postgres postgres    0 Sep 23 00:14 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92710
-rw-------. 1 postgres postgres    0 Sep 23 00:14 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92711
-rw-------. 1 postgres postgres 8192 Sep 23 00:14 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92712
-rw-------. 1 postgres postgres    0 Sep 23 00:14 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92713
-rw-------. 1 postgres postgres    0 Sep 23 00:14 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92714
-rw-------. 1 postgres postgres 8192 Sep 23 00:14 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92715
-rw-------. 1 postgres postgres    0 Sep 23 00:14 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92716
-rw-------. 1 postgres postgres    0 Sep 23 00:14 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92717
-rw-------. 1 postgres postgres 8192 Sep 23 00:14 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92718
-rw-------. 1 postgres postgres 8192 Sep 23 00:52 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92719
-rw-------. 1 postgres postgres 8192 Sep 23 00:52 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92720
-rw-------. 1 postgres postgres 8192 Sep 23 00:52 /pgData/pgsql15/edbua_tbs02/PG_15_202209061/92666/92721
[root@pg17 ~]#


Issue: Why are there 12 files when I have only 6 objects?

created only: 3 tables: emp01, emp02, emp03 & 3 indexes: emp01_ix, emp02_ix, emp03_ix

But in the tablespace directory, we're seeing 12 files: 3 tables + 3 indexes + 3 TOAST tables + 3 TOAST indexes

edbua=# SELECT
edbua-#     c.relname,
edbua-#     c.relfilenode,
edbua-#     c.relkind,
edbua-#     n.nspname AS schema,
edbua-#     t.spcname AS tablespace
edbua-# FROM
edbua-#     pg_class c
edbua-# JOIN
edbua-#     pg_namespace n ON n.oid = c.relnamespace
edbua-# LEFT JOIN
edbua-#     pg_tablespace t ON c.reltablespace = t.oid
edbua-# WHERE
edbua-#     c.reltablespace = (SELECT oid FROM pg_tablespace WHERE spcname = 'edbua_tbs02')
edbua-# ORDER BY
edbua-#     c.relfilenode;
       relname        | relfilenode | relkind |  schema  | tablespace
----------------------+-------------+---------+----------+-------------
 emp01                |       92710 | r       | public   | edbua_tbs02
 pg_toast_92685       |       92711 | t       | pg_toast | edbua_tbs02
 pg_toast_92685_index |       92712 | i       | pg_toast | edbua_tbs02
 emp02                |       92713 | r       | public   | edbua_tbs02
 pg_toast_92690       |       92714 | t       | pg_toast | edbua_tbs02
 pg_toast_92690_index |       92715 | i       | pg_toast | edbua_tbs02
 emp03                |       92716 | r       | public   | edbua_tbs02
 pg_toast_92695       |       92717 | t       | pg_toast | edbua_tbs02
 pg_toast_92695_index |       92718 | i       | pg_toast | edbua_tbs02
 emp01_ix             |       92719 | i       | public   | edbua_tbs02
 emp02_ix             |       92720 | i       | public   | edbua_tbs02
 emp03_ix             |       92721 | i       | public   | edbua_tbs02
(12 rows)

edbua=#


11. Rename tablespace

edbua=# ALTER TABLESPACE edbua_tbs02 RENAME TO edbua_tbs03;
ALTER TABLESPACE
edbua=#

edbua=# \db+ edbua*
                                                List of tablespaces
      Name       |  Owner   |            Location             | Access privileges | Options |  Size   | Description
-----------------+----------+---------------------------------+-------------------+---------+---------+-------------
 edbua_ixtbs01   | postgres | /pgData/pgsql15/edbua_ixtbs01   |                   |         | 6 bytes |
 edbua_tbs01     | postgres | /pgData/pgsql15/edbua_tbs01     |                   |         | 7789 kB |
 edbua_tbs03     | postgres | /pgData/pgsql15/edbua_tbs02     |                   |         | 48 kB   |
 edbua_temptbs01 | postgres | /pgData/pgsql15/edbua_temptbs01 |                   |         | 6 bytes |
(4 rows)

edbua=#


12. Drop Tablespace

edbua=# drop tablespace edbua_tbs03;
ERROR:  tablespace "edbua_tbs03" is not empty
edbua=#

edbua=# SELECT
edbua-#   CASE
edbua-#     WHEN c.relkind = 'r' THEN
edbua-#       format('ALTER TABLE %I.%I SET TABLESPACE edbua_tbs01;', n.nspname, c.relname)
edbua-#     WHEN c.relkind = 'i' THEN
edbua-#       format('ALTER INDEX %I.%I SET TABLESPACE edbua_tbs01;', n.nspname, c.relname)
edbua-#     WHEN c.relkind = 'S' THEN
edbua-#       format('ALTER SEQUENCE %I.%I SET TABLESPACE edbua_tbs01;', n.nspname, c.relname)
edbua-#   END AS alter_statement
edbua-# FROM pg_class c
edbua-# JOIN pg_namespace n ON n.oid = c.relnamespace
edbua-# WHERE c.reltablespace = (SELECT oid FROM pg_tablespace WHERE spcname = 'edbua_tbs03')
edbua-#   AND c.relkind IN ('r', 'i', 'S')
edbua-#   AND n.nspname NOT IN ('pg_catalog', 'pg_toast', 'information_schema')
edbua-# ORDER BY n.nspname, c.relname;
                     alter_statement
---------------------------------------------------------
 ALTER TABLE public.emp01 SET TABLESPACE edbua_tbs01;
 ALTER INDEX public.emp01_ix SET TABLESPACE edbua_tbs01;
 ALTER TABLE public.emp02 SET TABLESPACE edbua_tbs01;
 ALTER INDEX public.emp02_ix SET TABLESPACE edbua_tbs01;
 ALTER TABLE public.emp03 SET TABLESPACE edbua_tbs01;
 ALTER INDEX public.emp03_ix SET TABLESPACE edbua_tbs01;
(6 rows)

edbua=#

edbua=# ALTER TABLE public.emp01 SET TABLESPACE edbua_tbs01;
 ALTER INDEX public.emp02_ix SET TABLESPACE edbua_tbs01;
 ALTER TABLE public.emp03 SET TABLESPACE edbua_tbs01;
 ALTER INDEX public.emp03_ix SET TABLESPACE edbua_tbs01;ALTER TABLE
edbua=#  ALTER INDEX public.emp01_ix SET TABLESPACE edbua_tbs01;
ALTER INDEX
edbua=#  ALTER TABLE public.emp02 SET TABLESPACE edbua_tbs01;
ALTER TABLE
edbua=#  ALTER INDEX public.emp02_ix SET TABLESPACE edbua_tbs01;
ALTER INDEX
edbua=#  ALTER TABLE public.emp03 SET TABLESPACE edbua_tbs01;
ALTER TABLE
edbua=#  ALTER INDEX public.emp03_ix SET TABLESPACE edbua_tbs01;
ALTER INDEX
edbua=#

edbua=# drop tablespace edbua_tbs03;
DROP TABLESPACE
edbua=# \db+ edbua*
                                                List of tablespaces
      Name       |  Owner   |            Location             | Access privileges | Options |  Size   | Description
-----------------+----------+---------------------------------+-------------------+---------+---------+-------------
 edbua_ixtbs01   | postgres | /pgData/pgsql15/edbua_ixtbs01   |                   |         | 6 bytes |
 edbua_tbs01     | postgres | /pgData/pgsql15/edbua_tbs01     |                   |         | 7837 kB |
 edbua_temptbs01 | postgres | /pgData/pgsql15/edbua_temptbs01 |                   |         | 6 bytes |
(3 rows)

edbua=#


13. Change ownership for Tablespace

edbua=# ALTER TABLESPACE edbua_tbs01 OWNER TO teja;
ALTER TABLESPACE
edbua=# \db+ edbua*
                                                List of tablespaces
      Name       |  Owner   |            Location             | Access privileges | Options |  Size   | Description
-----------------+----------+---------------------------------+-------------------+---------+---------+-------------
 edbua_ixtbs01   | postgres | /pgData/pgsql15/edbua_ixtbs01   |                   |         | 6 bytes |
 edbua_tbs01     | teja     | /pgData/pgsql15/edbua_tbs01     |                   |         | 7837 kB |
 edbua_temptbs01 | postgres | /pgData/pgsql15/edbua_temptbs01 |                   |         | 6 bytes |
(3 rows)

edbua=# ALTER TABLESPACE edbua_tbs01 OWNER TO postgres;
ALTER TABLESPACE
edbua=# \db+ edbua*
                                                List of tablespaces
      Name       |  Owner   |            Location             | Access privileges | Options |  Size   | Description
-----------------+----------+---------------------------------+-------------------+---------+---------+-------------
 edbua_ixtbs01   | postgres | /pgData/pgsql15/edbua_ixtbs01   |                   |         | 6 bytes |
 edbua_tbs01     | postgres | /pgData/pgsql15/edbua_tbs01     |                   |         | 7837 kB |
 edbua_temptbs01 | postgres | /pgData/pgsql15/edbua_temptbs01 |                   |         | 6 bytes |
(3 rows)

edbua=#


14. Verify Symbolic Links

edbua=# show data_directory;
    data_directory
----------------------
 /pgData/pgsql15/data  <----
(1 row)

edbua=#

[root@pg17 pg_tblspc]# ls -ltr /pgData/pgsql15/data/pg_tblspc/*
lrwxrwxrwx. 1 postgres postgres 22 Aug 18 06:36 /pgData/pgsql15/data/pg_tblspc/24581 -> /u01/data/dell_ts_data
lrwxrwxrwx. 1 postgres postgres 23 Aug 18 06:36 /pgData/pgsql15/data/pg_tblspc/24582 -> /u01/data/dell_ts_index
lrwxrwxrwx. 1 postgres postgres 20 Sep  3 09:52 /pgData/pgsql15/data/pg_tblspc/24594 -> /pgData/pgsql15/data
lrwxrwxrwx. 1 postgres postgres 29 Sep  3 11:55 /pgData/pgsql15/data/pg_tblspc/24597 -> /pgData/pgsql15/data/trd_tb01
lrwxrwxrwx. 1 postgres postgres 29 Sep  3 11:55 /pgData/pgsql15/data/pg_tblspc/24598 -> /pgData/pgsql15/data/trd_ix01
lrwxrwxrwx. 1 postgres postgres 27 Sep  4 05:59 /pgData/pgsql15/data/pg_tblspc/32787 -> /pgData/pgsql15/pgtb/geb_tb
lrwxrwxrwx. 1 postgres postgres 27 Sep  4 06:03 /pgData/pgsql15/data/pg_tblspc/32788 -> /pgData/pgsql15/pgix/geb_ix
lrwxrwxrwx. 1 postgres postgres 27 Sep 20 07:20 /pgData/pgsql15/data/pg_tblspc/92662 -> /pgData/pgsql15/edbua_tbs01
lrwxrwxrwx. 1 postgres postgres 29 Sep 20 07:21 /pgData/pgsql15/data/pg_tblspc/92664 -> /pgData/pgsql15/edbua_ixtbs01
lrwxrwxrwx. 1 postgres postgres 31 Sep 20 07:21 /pgData/pgsql15/data/pg_tblspc/92665 -> /pgData/pgsql15/edbua_temptbs01
[root@pg17 pg_tblspc]#


15. How to find tablespace size

edbua=# SELECT spcname, pg_size_pretty(pg_tablespace_size(spcname)) AS size FROM pg_tablespace;
     spcname     |  size
-----------------+---------
 pg_default      | 1660 MB
 pg_global       | 571 kB
 dell_ts_data    | 7761 kB
 dell_ts_index   | 16 kB
 dell_tbs        | 0 bytes
 trd_tb01        | 1615 MB
 trd_ix01        | 2648 kB
 geb_tb          | 13 GB
 geb_ix          | 8950 MB
 edbua_ixtbs01   | 6 bytes
 edbua_temptbs01 | 6 bytes
 edbua_tbs01     | 7837 kB
(12 rows)

edbua=#

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
WhatsApp :
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

PostgreSQL pg_hba.conf

Uncategorized, , , , , , , , ,

 

PostgreSQL pg_hba.conf Guide

Table of Contents

1. What is pg_hba.conf?
2. Authentication Methods in pg_hba.conf
3. Create Database and User
4. Contents of pg_hba.conf
5. Play with peer
6. Play with trust
7. Play with reject
8. Restrict by User
9. Restrict by Network
10. Verify logs


1. What is pg_hba.conf?


Location: usually inside PostgreSQL data directory (/var/lib/pgsql/<version>/data/pg_hba.conf or /pgData/pgsql15/data/pg_hba.conf depending on your setup).

HBA = Host-Based Authentication.

Format:

# TYPE  DATABASE  USER  ADDRESS        METHOD
TYPE: local, host, hostssl, hostnossl

DATABASE: which DBs (e.g. all, postgres, mydb)
USER	: which roles (e.g. all, myuser)
ADDRESS	: client IP/CIDR (127.0.0.1/32, 192.168.2.0/24)

METHOD	: authentication method (trust, md5, scram-sha-256, peer, reject, cert)

OPTIONS	: optional settings (e.g., clientcert=1)


2. Authentication Methods in pg_hba.conf

trust: 

No password needed. Anyone who can connect to the server is trusted.
Use: testing only.
Not secure in production.


Example: host    all    all    127.0.0.1/32    trust

md5

Password required, stored as MD5 hash. Legacy support.
Weaker security, avoid if possible.


Example: host    all    all    127.0.0.1/32    md5

scram-sha-256

Password required, stored as salted SCRAM-SHA-256 hash.
Stronger than MD5.
Recommended method for production.

Example: host    all    all    127.0.0.1/32    scram-sha-256


peer
 
Works for local connections only.
The OS user name must match the PostgreSQL role.
Good for local scripts/services under same user.


Example:
local   all    all                     peer

reject
 
Explicitly denies access.
Useful for blocking unwanted connections (like a firewall rule).

Example:
host    all    all    0.0.0.0/0       reject

cert

Requires SSL/TLS client certificate.
PostgreSQL role must match certificate username.
Very secure for enterprise / production with SSL.
 
Example:
hostssl all   all    192.168.1.0/24   cert clientcert=1


3. Create Database and User

postgres=# CREATE USER teja WITH PASSWORD 'teja123';
CREATE ROLE
postgres=# CREATE DATABASE orcl OWNER teja;
CREATE DATABASE
postgres=#


4. Contents of pg_hba.conf

[postgres@pg17 ~]$ cat /pgData/pgsql15/data/pg_hba.conf
# TYPE  DATABASE         USER            ADDRESS                METHOD

# Allow local peer access for postgres
local   all             postgres                                peer

# Allow local password access for all
local   all             all                                     scram-sha-256

[postgres@pg17 ~]$


5. Play with peer

Works for local connections only. The OS user name must match the PostgreSQL role.

[postgres@pg17 ~]$ cat /pgData/pgsql15/data/pg_hba.conf
# TYPE  DATABASE         USER            ADDRESS                METHOD

# Allow local peer access for postgres

local   all             postgres                                peer


# Allow local password access for all
local   all             all                                     scram-sha-256

[postgres@pg17 ~]$

[postgres@pg17 ~]$ psql -U teja -d orcl
 Password for user teja: <---- it's asking password 

psql (15.13)
Type "help" for help.

orcl=> exit
[postgres@pg17 ~]$

 Password Not asking for user postgres 
[postgres@pg17 ~]$ psql -U postgres -d orcl
psql (15.13)
Type "help" for help.

orcl=#


6. Play with trust

No password needed. Anyone who can connect to the server is trusted.
Use: testing only. Not secure in production.

Change pg_hba.conf:

[postgres@pg17 ~]$ cat /pgData/pgsql15/data/pg_hba.conf
# TYPE  DATABASE         USER            ADDRESS                METHOD

# Allow local peer access for postgres
local   all             postgres                                peer

# Allow local password access for all

local   all             all                                     trust

[postgres@pg17 ~]$

[postgres@pg17 ~]$ /usr/pgsql-15/bin/pg_ctl reload -D /pgData/pgsql15/data/
server signaled
[postgres@pg17 ~]$ psql -U teja -d orcl  It won’t ask for a password. 
psql (15.13)
Type "help" for help.

orcl=>


7. Play with reject

[postgres@pg17 ~]$ cat /pgData/pgsql15/data/pg_hba.conf
# TYPE  DATABASE         USER            ADDRESS                METHOD

# Allow local peer access for postgres
local   all             postgres                                peer

# Allow local password access for all

local   all             all                                     reject

[postgres@pg17 ~]$

[postgres@pg17 ~]$ /usr/pgsql-15/bin/pg_ctl reload -D /pgData/pgsql15/data/
server signaled
[postgres@pg17 ~]$ psql -U teja -d orcl

psql: error: connection to server on socket "/run/postgresql/.s.PGSQL.5432" failed: FATAL:  pg_hba.conf rejects connection for host "[local]", user "teja", database "orcl", no encryption

[postgres@pg17 ~]$


8. Restrict by User

Allow only user teja, block everyone else:

[postgres@pg17 ~]$ cat /pgData/pgsql15/data/pg_hba.conf
# TYPE  DATABASE         USER            ADDRESS                METHOD

# Allow local peer access for postgres
local   all             postgres                                peer

# Allow local password access for all
local   all             all                                     trust

# Allow IPv4 localhost

host    orcl            teja             192.168.2.31/32         scram-sha-256
host    orcl            all              192.168.2.31/32         reject


[postgres@pg17 ~]$

[postgres@pg17 ~]$ /usr/pgsql-15/bin/pg_ctl reload -D /pgData/pgsql15/data/
server signaled
[postgres@pg17 ~]$ psql -U teja -d orcl -h 192.168.2.31
Password for user teja:
psql (15.13)
Type "help" for help.

orcl=> \conninfo
You are connected to database "orcl" as user "teja" on host "192.168.2.31" at port "5432".
orcl=>
orcl=> exit
[postgres@pg17 ~]$


[postgres@pg17 ~]$ psql -U postgres -d orcl -h 192.168.2.31

psql: error: connection to server at "192.168.2.31", port 5432 failed: FATAL:  pg_hba.conf rejects connection for host "192.168.2.31", user "postgres", database "orcl", no encryption

[postgres@pg17 ~]$


Notice: Only user Teja able to connect where as user postgres not able to connect


9. Restrict by Network

We want to block connections from IP 192.168.2.0/24 

[postgres@pg17 ~]$ cat /pgData/pgsql15/data/pg_hba.conf
# TYPE  DATABASE         USER            ADDRESS                METHOD

# Allow local peer access for postgres
local   all             postgres                                peer

# Allow local password access for all
local   all             all                                     trust

# Allow IPv4 localhost

host    all             all             192.168.2.0/24          reject

[postgres@pg17 ~]$

[postgres@pg17 ~]$ /usr/pgsql-15/bin/pg_ctl reload -D /pgData/pgsql15/data/
server signaled
[postgres@pg17 ~]$ 
[postgres@pg17 ~]$ psql -U teja -d orcl -h 192.168.2.31

psql: error: connection to server at "192.168.2.31", port 5432 failed: FATAL:  pg_hba.conf rejects connection for host "192.168.2.31", user "teja", database "orcl", no encryption

[postgres@pg17 ~]$


10. Verify logs

postgres=# SELECT * FROM pg_hba_file_rules;
 line_number | type  | database | user_name  |   address    |     netmask     |  auth_method  | options | error
-------------+-------+----------+------------+--------------+-----------------+---------------+---------+-------
           4 | local | {all}    | {postgres} |              |                 | peer          |         |
           7 | local | {all}    | {all}      |              |                 | trust         |         |
          11 | host  | {orcl}   | {teja}     | 192.168.2.31 | 255.255.255.255 | scram-sha-256 |         |
          12 | host  | {orcl}   | {all}      | 192.168.2.31 | 255.255.255.255 | reject        |         |
(4 rows)

postgres=#

[postgres@pg17 ~]$ tail -f /pgData/pgsql15/data/log/postgresql-Tue.log
2025-09-16 04:08:37.244 EDT [8294] LOG:  database system was shut down at 2025-09-16 04:08:32 EDT
2025-09-16 04:08:37.251 EDT [8289] LOG:  database system is ready to accept connections
2025-09-16 04:11:27.459 EDT [8289] LOG:  received SIGHUP, reloading configuration files
2025-09-16 04:11:31.842 EDT [8401] FATAL:  pg_hba.conf rejects connection for host "[local]", user "teja", database "orcl", no encryption
2025-09-16 04:13:37.568 EDT [8292] LOG:  checkpoint starting: time
2025-09-16 04:13:37.573 EDT [8292] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.002 s, total=0.006 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB
2025-09-16 04:16:15.863 EDT [8289] LOG:  received SIGHUP, reloading configuration files
2025-09-16 04:16:52.318 EDT [8474] FATAL:  pg_hba.conf rejects connection for host "192.168.2.31", user "teja", database "orcl", no encryption
2025-09-16 04:27:15.932 EDT [8289] LOG:  received SIGHUP, reloading configuration files
2025-09-16 04:27:50.593 EDT [8613] FATAL:  pg_hba.conf rejects connection for host "192.168.2.31", user "postgres", database "orcl", no encryption

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
WhatsApp : 
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

PostgreSQL User Management

Uncategorized, , , , , , , , ,

PostgreSQL User Management

Create Users, Groups and Schema Paths for PostgreSQL

Table of Contents

Step 1: Create a User
Step 2: Change User Password
Step 3: Grant Database Access
Step 4: Expire Password
Step 5: Set Password Never Expire
Step 6: Lock Account
Step 7: Unlock Account
Step 8: Create Schema
Step 9: Create Roles & Users
Step 10: Assign Ownership
Step 11: Grant Schema Privileges
Step 12: Grant RW Privileges
Step 13: Grant RO Privileges
Step 14: Assign Roles to Users
Step 15: Testing
Step 16: Set Schema Search Path
Step 17: Groups
Step 18: PostgreSQL — Table-View Privileges

Note: Users are just roles with LOGIN privilege; roles without LOGIN cannot connect.



Step 1: Create a User

-- Option 1: Using CREATE USER
CREATE USER john WITH PASSWORD 'mypassword';

-- Option 2: Using CREATE ROLE with LOGIN
CREATE ROLE john LOGIN PASSWORD 'mypassword';

postgres=# CREATE USER john WITH PASSWORD 'mypassword';
CREATE ROLE
postgres=# CREATE ROLE john LOGIN PASSWORD 'mypassword';
ERROR:  role "john" already exists



Step 2: Change User Password

Passwords can be changed by either an admin or the user.

By Admin:

postgres=# ALTER USER john WITH PASSWORD 'newpassword';
ALTER ROLE
postgres=#

-- OR --

postgres=# \password john
Enter new password for user "john":
Enter it again:
postgres=#


By User (self-service): From the psql prompt:

[postgres@pg17 ~]$ psql -h 192.168.2.31 -U john -d mydb -W
Password:

mydb=> \conninfo
You are connected to database "mydb" as user "john" on host "192.168.2.31" at port "5432".

mydb=> \password
Enter new password for user "john":
Enter it again:
mydb=>



Step 3: Grant Database Access

To allow a user to connect to a database:

postgres=# GRANT CONNECT ON DATABASE mydb TO john;
GRANT
postgres=#


Verify user login:

-- Connect as user
[postgres@pg17 ~]$ psql -h 192.168.2.31 -U john -d mydb -W
Password:

-- Check current user
mydb=> SELECT CURRENT_USER;
 current_user
--------------
 john
(1 row)

mydb=>
mydb=> select session_user;
 session_user
--------------
 john
(1 row)

-- Connection info
mydb=> \conninfo
You are connected to database "mydb" as user "john" on host "192.168.2.31" at port "5432".
mydb=>



Step 4: Expire User Password

postgres=# ALTER USER john VALID UNTIL '2025-09-11';
ALTER ROLE
postgres=#
postgres=# \du+
                                          List of roles
 Role name |                         Attributes                         | Member of | Description
-----------+------------------------------------------------------------+-----------+-------------
 john      | Password valid until 2025-09-11 00:00:00-04                | {}        |
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}        |

postgres=#



Step 5: Set Password to Never Expire

postgres=# \du
                                   List of roles
 Role name |                         Attributes                         | Member of
-----------+------------------------------------------------------------+-----------
 john      | Password valid until 2025-09-11 00:00:00-04                | {}
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}


-- Without changing existing password

postgres=# ALTER USER john VALID UNTIL 'infinity';
ALTER ROLE
postgres=#

-- With changing existing password

postgres=# ALTER USER john WITH PASSWORD 'newpassword' VALID UNTIL 'infinity';
ALTER ROLE
postgres=#
postgres=# \du
                                   List of roles
 Role name |                         Attributes                         | Member of
-----------+------------------------------------------------------------+-----------
 john      | Password valid until infinity                              | {}
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}


postgres=#

-- Set to future date

postgres=# ALTER USER john VALID UNTIL '2025-12-31';
ALTER ROLE
postgres=# \du+
                                          List of roles
 Role name |                         Attributes                         | Member of | Description
-----------+------------------------------------------------------------+-----------+-------------
 john      | Password valid until 2025-12-31 00:00:00-05                | {}        |
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}        |

postgres=#



Step 6: Lock User Account

postgres=# ALTER USER john NOLOGIN;
ALTER ROLE
postgres=# \du+
                                          List of roles
 Role name |                         Attributes                         | Member of | Description
-----------+------------------------------------------------------------+-----------+-------------
 john      | Cannot login                                              +| {}        |
           | Password valid until infinity                              |           |
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}        |

postgres=#



Step 7: Unlock User Account

postgres=# ALTER USER john LOGIN;
ALTER ROLE
postgres=# \du+
                                          List of roles
 Role name |                         Attributes                         | Member of | Description
-----------+------------------------------------------------------------+-----------+-------------
 john      | Password valid until infinity                              | {}        |
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}        |

postgres=#



Step 8: Create Schema

[postgres@pg17 ~]$ psql
psql (15.13)
Type "help" for help.

postgres=# \c mydb
You are now connected to database "mydb" as user "postgres".
mydb=#
mydb=# CREATE SCHEMA BLP;
CREATE SCHEMA
mydb=# \dn
      List of schemas
  Name  |       Owner
--------+-------------------
 blp    | postgres
 public | pg_database_owner
(2 rows)

mydb=#



Step 9: Create Roles & Users

postgres=# CREATE USER "BLP" WITH PASSWORD 'blp';
CREATE ROLE
postgres=# 
postgres=# CREATE ROLE blp_rw NOLOGIN;
CREATE ROLE
postgres=# CREATE ROLE blp_ro NOLOGIN;
CREATE ROLE
postgres=#

postgres=# CREATE USER alice WITH PASSWORD 'alice123';
CREATE ROLE
postgres=# CREATE USER bob WITH PASSWORD 'bob123';
CREATE ROLE
postgres=# CREATE USER charlie WITH PASSWORD 'charlie123';
CREATE ROLE
postgres=#

postgres=# \du+
                                          List of roles
 Role name |                         Attributes                         | Member of | Description
-----------+------------------------------------------------------------+-----------+-------------
 BLP       |                                                            | {}        |
 alice     |                                                            | {}        |
 blp_ro    | Cannot login                                               | {}        |
 blp_rw    | Cannot login                                               | {}        |
 bob       |                                                            | {}        |
 charlie   |                                                            | {}        |
 john      | Password valid until 2025-12-31 00:00:00-05                | {}        |
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}        |
 trduser   |                                                            | {}        |

postgres=#



Step 10: Assign Ownership on Schema

-- Please do NOT grant this privillege, Owner can drop the schema, change privileges, and has full control over all objects inside.

postgres=# \c mydb
You are now connected to database "mydb" as user "postgres".
mydb=#
mydb=# \dn
      List of schemas
  Name  |       Owner
--------+-------------------
 blp    | postgres
 public | pg_database_owner
(2 rows)

mydb=# ALTER SCHEMA BLP OWNER TO "BLP";
ALTER SCHEMA
mydb=#

mydb=# \dn+
                                       List of schemas
  Name  |       Owner       |           Access privileges            |      Description
--------+-------------------+----------------------------------------+------------------------
 blp    | BLP               |                                        |
 public | pg_database_owner | pg_database_owner=UC/pg_database_owner+| standard public schema
        |                   | =U/pg_database_owner                   |
(2 rows)

mydb=#



Step 11: Grant Schema Privileges to Owner

-- Full control on schema: usage + create
GRANT USAGE, CREATE ON SCHEMA blp TO "BLP";

mydb=# \dn+
                                       List of schemas
  Name  |       Owner       |           Access privileges            |      Description
--------+-------------------+----------------------------------------+------------------------
 blp    | BLP               |                                        |
 public | pg_database_owner | pg_database_owner=UC/pg_database_owner+| standard public schema
        |                   | =U/pg_database_owner                   |
(2 rows)

mydb=# GRANT USAGE, CREATE ON SCHEMA blp TO "BLP";
GRANT
mydb=# \dn+
                                       List of schemas
  Name  |       Owner       |           Access privileges            |      Description
--------+-------------------+----------------------------------------+------------------------
 blp    | BLP               | BLP=UC/BLP                             |
 public | pg_database_owner | pg_database_owner=UC/pg_database_owner+| standard public schema
        |                   | =U/pg_database_owner                   |
(2 rows)

mydb=#



Step 12: Grant RW Privileges

USAGE → allows the role to see the schema and its objects.

-- Grant schema access without CREATE
GRANT USAGE ON SCHEMA BLP TO blp_rw;

-- Grant DML on all existing tables
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA BLP TO blp_rw;

-- Future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA BLP GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO blp_rw;

mydb=# GRANT USAGE ON SCHEMA BLP TO blp_rw;
GRANT
mydb=# GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA BLP TO blp_rw;
GRANT
mydb=# ALTER DEFAULT PRIVILEGES IN SCHEMA BLP GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO blp_rw;
ALTER DEFAULT PRIVILEGES
mydb=#
mydb=# \dn+
                                       List of schemas
  Name  |       Owner       |           Access privileges            |      Description
--------+-------------------+----------------------------------------+------------------------
 blp    | BLP               | BLP=UC/BLP                            +|
        |                   | blp_rw=U/BLP                           |
 public | pg_database_owner | pg_database_owner=UC/pg_database_owner+| standard public schema
        |                   | =U/pg_database_owner                   |
(2 rows)

mydb=#



Step 13: Grant RO Privileges

-- Grant schema access without CREATE
GRANT USAGE ON SCHEMA BLP TO BLP_RO;

-- Grant SELECT on all existing tables
GRANT SELECT ON ALL TABLES IN SCHEMA BLP TO BLP_RO;

-- Future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA BLP GRANT SELECT ON TABLES TO BLP_RO;

mydb=# GRANT USAGE ON SCHEMA BLP TO BLP_RO;
GRANT
mydb=# GRANT SELECT ON ALL TABLES IN SCHEMA BLP TO BLP_RO;
GRANT
mydb=# ALTER DEFAULT PRIVILEGES IN SCHEMA BLP GRANT SELECT ON TABLES TO BLP_RO;
ALTER DEFAULT PRIVILEGES
mydb=#
mydb=# \dn+
                                       List of schemas
  Name  |       Owner       |           Access privileges            |      Description
--------+-------------------+----------------------------------------+------------------------
 blp    | BLP               | BLP=UC/BLP                            +|
        |                   | blp_rw=U/BLP                          +|
        |                   | blp_ro=U/BLP                           |
 public | pg_database_owner | pg_database_owner=UC/pg_database_owner+| standard public schema
        |                   | =U/pg_database_owner                   |
(2 rows)

mydb=#



Step 14: Assign Roles to Users

mydb=# GRANT BLP_RW TO ALICE;
GRANT ROLE
mydb=# 
mydb=# GRANT BLP_RO TO BOB,CHARLIE;
GRANT ROLE
mydb=#

mydb=# \du+
                                          List of roles
 Role name |                         Attributes                         | Member of | Description
-----------+------------------------------------------------------------+-----------+-------------
 BLP       |                                                            | {}        |
 alice     |                                                            | {blp_rw}  |
 blp_ro    | Cannot login                                               | {}        |
 blp_rw    | Cannot login                                               | {}        |
 bob       |                                                            | {blp_ro}  |
 charlie   |                                                            | {blp_ro}  |
 john      | Password valid until 2025-12-31 00:00:00-05                | {}        |
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}        |
 trduser   |                                                            | {}        |

mydb=#



Step 15: Testing

-- Login to BLP user on mydb database and create table on blp schema. 

[postgres@pg17 ~]$ psql -h 192.168.2.31 -U BLP -d mydb -W
Password:
psql (15.13)
Type "help" for help.

mydb=> \conninfo
You are connected to database "mydb" as user "BLP" on host "192.168.2.31" at port "5432".
mydb=> CREATE TABLE blp.employees (
mydb(>     emp_id SERIAL PRIMARY KEY,
mydb(>     first_name VARCHAR(50),
mydb(>     last_name VARCHAR(50),
mydb(>     hire_date DATE,
mydb(>     salary NUMERIC(10,2)
mydb(> );
CREATE TABLE
mydb=>

mydb=> INSERT INTO blp.employees (first_name, last_name, hire_date, salary) VALUES
mydb-> ('John', 'Doe', '2023-01-15', 5000.00),
mydb-> ('Jane', 'Smith', '2022-11-20', 6000.00),
mydb-> ('Alice', 'Johnson', '2024-03-01', 5500.00);
INSERT 0 3
mydb=>
mydb=> select * from blp.employees;
 emp_id | first_name | last_name | hire_date  | salary
--------+------------+-----------+------------+---------
      1 | John       | Doe       | 2023-01-15 | 5000.00
      2 | Jane       | Smith     | 2022-11-20 | 6000.00
      3 | Alice      | Johnson   | 2024-03-01 | 5500.00
(3 rows)

mydb=>

mydb=> drop table blp.employees;
DROP TABLE
mydb=>

-- Login to alice user on mydb database and update table on blp schema. 



[root@pg17 ~]# psql -h 192.168.2.31 -U alice -d mydb -W
Password:
psql (15.13)
Type "help" for help.

mydb=> \conninfo
You are connected to database "mydb" as user "alice" on host "192.168.2.31" at port "5432".
mydb=>

mydb=> \du+ alice
                  List of roles
 Role name | Attributes | Member of | Description
-----------+------------+-----------+-------------
 alice     |            | {blp_rw}  |

mydb=>


mydb=> \dt+ blp.*
                                      List of relations
 Schema |   Name    | Type  | Owner | Persistence | Access method |    Size    | Description
--------+-----------+-------+-------+-------------+---------------+------------+-------------
 blp    | employees | table | BLP   | permanent   | heap          | 8192 bytes |
(1 row)

mydb=> select * from blp.employees;
 emp_id | first_name | last_name | hire_date  | salary
--------+------------+-----------+------------+---------
      1 | John       | Doe       | 2023-01-15 | 5000.00
      2 | Jane       | Smith     | 2022-11-20 | 6000.00
      3 | Alice      | Johnson   | 2024-03-01 | 5500.00
(3 rows)

mydb=> UPDATE blp.employees
SET salary = CASE
                WHEN first_name = 'John' THEN 7000.00
                WHEN first_name = 'Alice' THEN 6500.00
             END
WHERE first_name IN ('John', 'Alice');
UPDATE 2
mydb=> select * from blp.employees;
 emp_id | first_name | last_name | hire_date  | salary
--------+------------+-----------+------------+---------
      2 | Jane       | Smith     | 2022-11-20 | 6000.00
      1 | John       | Doe       | 2023-01-15 | 7000.00
      3 | Alice      | Johnson   | 2024-03-01 | 6500.00
(3 rows)

mydb=>

-- Note, we have granted only DML privilleges, hence create and alter table command failing

mydb=> CREATE TABLE blp.departments (
mydb(>     dept_id SERIAL PRIMARY KEY,
mydb(>     dept_name VARCHAR(100) NOT NULL,
mydb(>     location VARCHAR(100)
mydb(> );
ERROR:  permission denied for schema blp
LINE 1: CREATE TABLE blp.departments (
                     ^
mydb=>

mydb=> ALTER TABLE blp.employees
mydb-> ADD COLUMN department VARCHAR(50);
ERROR:  must be owner of table employees
mydb=>

-- Login to bob user on mydb database and select table on blp schema. 

[postgres@pg17 ~]$ psql -h 192.168.2.31 -U bob -d mydb -W
Password:
psql (15.13)
Type "help" for help.

mydb=> \conninfo
You are connected to database "mydb" as user "bob" on host "192.168.2.31" at port "5432".
mydb=>
mydb=> \du+ bob
                  List of roles
 Role name | Attributes | Member of | Description
-----------+------------+-----------+-------------
 bob       |            | {blp_ro}  |

mydb-> \dt+ blp.*
                                      List of relations
 Schema |   Name    | Type  | Owner | Persistence | Access method |    Size    | Description
--------+-----------+-------+-------+-------------+---------------+------------+-------------
 blp    | employees | table | BLP   | permanent   | heap          | 8192 bytes |
(1 row)

mydb=> select * from blp.employees;
 emp_id | first_name | last_name | hire_date  | salary
--------+------------+-----------+------------+---------
      2 | Jane       | Smith     | 2022-11-20 | 6000.00
      1 | John       | Doe       | 2023-01-15 | 7000.00
      3 | Alice      | Johnson   | 2024-03-01 | 6500.00
(3 rows)

mydb=>

mydb=> INSERT INTO blp.employees (first_name, last_name, hire_date, salary) VALUES
mydb-> ('Bob', 'Williams', '2024-04-01', 5800.00),
mydb-> ('Clara', 'Brown', '2024-05-10', 6200.00),
mydb-> ('David', 'Lee', '2024-06-15', 5300.00);
ERROR:  permission denied for table employees
mydb=>



Step 16: Set Schema Search Path

[postgres@pg17 ~]$ psql -h 192.168.2.31 -U BLP -d mydb -W
Password:
psql (15.13)
Type "help" for help.

mydb=> 
mydb=> \conninfo
You are connected to database "mydb" as user "BLP" on host "192.168.2.31" at port "5432".
mydb=>
mydb=> \dt+ blp.*
                                      List of relations
 Schema |   Name    | Type  | Owner | Persistence | Access method |    Size    | Description
--------+-----------+-------+-------+-------------+---------------+------------+-------------
 blp    | employees | table | BLP   | permanent   | heap          | 8192 bytes |
(1 row)

mydb=> 

mydb=> select * from employees;
ERROR:  relation "employees" does not exist
LINE 1: select * from employees;
mydb=>

mydb=> SET search_path to BLP; -- Temporarily for this session
SET
mydb=> select * from employees;
 emp_id | first_name | last_name | hire_date  | salary
--------+------------+-----------+------------+---------
      2 | Jane       | Smith     | 2022-11-20 | 6000.00
      1 | John       | Doe       | 2023-01-15 | 7000.00
      3 | Alice      | Johnson   | 2024-03-01 | 6500.00
(3 rows)

mydb=>

-- Make the Schema Default for the user BLP -- Permenant
ALTER ROLE blp_owner SET search_path = BLP;


Step 17: Groups

postgres=# CREATE GROUP app_users;
postgres=# ALTER GROUP app_users ADD USER alice;
postgres=# ALTER GROUP app_users ADD USER bob;
postgres=# ALTER GROUP app_users ADD USER charlie;
postgres=# ALTER GROUP app_users DROP USER charlie;
postgres=# ALTER GROUP app_users RENAME TO appusers;
postgres=# DROP GROUP appusers;

mydb=> select * from pg_group;
          groname          | grosysid |    grolist
---------------------------+----------+---------------
 pg_database_owner         |     6171 | {}
 pg_read_all_data          |     6181 | {}
 pg_write_all_data         |     6182 | {}
 pg_monitor                |     3373 | {}
 pg_read_all_settings      |     3374 | {3373}
 pg_read_all_stats         |     3375 | {3373}
 pg_stat_scan_tables       |     3377 | {3373}
 pg_read_server_files      |     4569 | {}
 pg_write_server_files     |     4570 | {}
 pg_execute_server_program |     4571 | {}
 pg_signal_backend         |     4200 | {}
 pg_checkpoint             |     4544 | {}
 blp_rw                    |    84444 | {84447}
 blp_ro                    |    84445 | {84448,84449}
 app_users                 |    84468 | {}
(15 rows)

mydb=>



Step 18: PostgreSQL Object-Level Privilege Summary

mydb=> \z blp.employees
                               Access privileges
 Schema |   Name    | Type  | Access privileges | Column privileges | Policies
--------+-----------+-------+-------------------+-------------------+----------
 blp    | employees | table | BLP=arwdDxt/BLP  +|                   |
        |           |       | blp_ro=r/BLP     +|                   |
        |           |       | blp_rw=arwd/BLP   |                   |
(1 row)

Example:

 

ShortFull PrivilegeExample GRANT
rSELECTGRANT SELECT ON employees TO hr_user;
aINSERTGRANT INSERT ON employees TO hr_user;
wUPDATEGRANT UPDATE ON employees TO hr_user;
dDELETEGRANT DELETE ON employees TO hr_user;
DTRUNCATEGRANT TRUNCATE ON employees TO hr_user;
xREFERENCESGRANT REFERENCES ON employees TO hr_user;
tTRIGGERGRANT TRIGGER ON employees TO hr_user;
RRULEGRANT RULE ON employees TO hr_user; (rarely used)

 

Tip: the shorthand string you see in \z (for example arwdDxt) can be expanded by mapping each letter to the rows above, then converting them into one or more GRANT statements.

 

 

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
WhatsApp : 
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/