Tag Archives: PostgreSQL architecture

WAL Commit Process

PostgreSQL WAL Commit Process

1. The Modification (In-Memory)

When a user executes a data-modifying query (like INSERT, UPDATE, or DELETE):

  • The change is made to the table or index data inside the Shared Buffers (RAM). The page in memory is now marked as “dirty.”

  • Simultaneously, a record of this exact change is constructed and written sequentially into the WAL Buffers (also in RAM).

2. The COMMIT Command Issued

When the client sends the COMMIT command, PostgreSQL must guarantee that this change will survive a sudden power outage or crash before it can tell the user “Success.”

3. Flushing to Disk (XLogFlush)

To ensure durability without the massive overhead of writing entire data pages to disk immediately, Postgres uses the Write-Ahead Logging protocol:

  • The internal function XLogFlush() is called.

  • It identifies the exact position (Log Sequence Number, or LSN) of the commit record in the WAL Buffer.

  • It issues a synchronous write to flush all WAL buffers up to that LSN out of RAM and into the current 16MB WAL segment file on permanent storage.

  • An fsync() system call is issued to ensure the OS cache actually commits the data to physical disk platters or flash memory.

4. Acknowledgment to the Client

Once the operating system confirms that the WAL record is safely written to the physical storage, the transaction status is updated to “committed” in the commit log (CLOG), and PostgreSQL sends a success acknowledgment back to the client application.

Crucial Architectural Concepts

  • Write-Ahead Rule: The core rule of WAL is that changes to data pages must not be written to permanent database files on disk until the log records describing those changes have been flushed to stable storage. If the server crashes, Postgres reads the WAL from the last checkpoint forward and reapplies the changes (“redoes” them).

  • Asynchronous Commit Alternative: If you set the configuration parameter synchronous_commit = off, Postgres will acknowledge the client’s COMMIT before the WAL buffer is flushed to disk (relying on the WAL Writer background process to flush it within roughly 3 times wal_writer_delay). This massively increases write throughput but introduces a risk of losing up to a split-second of recent transactions if the server suddenly loses power.

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.
Thank you
Rajasekhar Amudala

Types of Shutdown in PostgreSQL

Types of Shutdown in PostgreSQL

PostgreSQL Shutdown TypeOracleBehavior
Smart (SIGTERM)SHUTDOWN NORMALThe server stops accepting new connections but allows existing sessions to continue running normally. PostgreSQL shuts down only after all active sessions disconnect and their transactions complete successfully.
Fast (SIGINT – Default)SHUTDOWN IMMEDIATEThe server stops accepting new connections and immediately terminates all active sessions. Any uncommitted transactions are rolled back, and PostgreSQL performs a clean shutdown gracefully.
Immediate (SIGQUIT)SHUTDOWN ABORTThe server terminates all processes instantly without performing a proper shutdown. No cleanup or checkpoint occurs, so PostgreSQL performs crash recovery by replaying WAL files during the next startup.
Shutdown TypeSignalCommand ExampleBehavior
SmartSIGTERMpg_ctl stop -D /pgData -m smartStops new connections, waits for existing sessions to finish normally, then shuts down.
Fast (Default)SIGINTpg_ctl stop -D /pgdata -m fastStops new connections, terminates active sessions, rolls back uncommitted transactions, and shuts down cleanly.
ImmediateSIGQUITpg_ctl stop -D /pgdata -m immediateStops immediately without proper shutdown. Recovery from WAL files happens during next startup.

Difference Between Reload and Restart

  • When we change server configuration parameters, PostgreSQL needs to read those changes before they become active.
  • Reload applies the new configuration without stopping or restarting the database service. Existing connections and database activity continue normally.
  • Some configuration parameter changes cannot be applied through reload and will only take effect after a restart.
  • Restart stops the PostgreSQL server gracefully, ends all running activities, releases resources, closes open files, and then starts the server again with the new configuration.

 

Smart Shutdown (SIGTERM):

[postgres@pgdb02 ~]$ pg_ctl stop -D /pgData/pgsql17/data -m smart
waiting for server to shut down.... done
server stopped
[postgres@pgdb02 ~]$

2026-07-03 14:26:59.741 +08 [3476] LOG:  received smart shutdown request
2026-07-03 14:26:59.752 +08 [3476] LOG:  background worker "logical replication launcher" (PID 3483) exited with exit code 1
2026-07-03 14:26:59.753 +08 [3478] LOG:  shutting down
2026-07-03 14:26:59.755 +08 [3478] LOG:  checkpoint starting: shutdown immediate
2026-07-03 14:26:59.765 +08 [3478] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.003 s, total=0.012 s; sync files=2, longest=0.003 s, average=0.002 s; distance=0 kB, estimate=0 kB; lsn=1/105465F0, redo lsn=1/105465F0
2026-07-03 14:26:59.771 +08 [3476] LOG:  database system is shut down


[postgres@pgdb02 ~]$ pg_ctl start -D /pgData/pgsql17/data
waiting for server to start....2026-07-03 14:27:18.394 +08 [3487] LOG:  redirecting log output to logging collector process
2026-07-03 14:27:18.394 +08 [3487] HINT:  Future log output will appear in directory "log".
 done
server started
[postgres@pgdb02 ~]$

2026-07-03 14:27:18.394 +08 [3487] LOG:  starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-14), 64-bit
2026-07-03 14:27:18.396 +08 [3487] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2026-07-03 14:27:18.396 +08 [3487] LOG:  listening on IPv6 address "::", port 5432
2026-07-03 14:27:18.399 +08 [3487] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"
2026-07-03 14:27:18.404 +08 [3487] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2026-07-03 14:27:18.413 +08 [3491] LOG:  database system was shut down at 2026-07-03 14:26:59 +08
2026-07-03 14:27:18.425 +08 [3487] LOG:  database system is ready to accept connections

Fast Shutdown (SIGINT – Default):

[postgres@pgdb02 ~]$ pg_ctl stop -D /pgData/pgsql17/data -m fast
waiting for server to shut down.... done
server stopped
[postgres@pgdb02 ~]$

2026-07-03 14:28:26.839 +08 [3487] LOG:  received fast shutdown request
2026-07-03 14:28:26.844 +08 [3487] LOG:  aborting any active transactions
2026-07-03 14:28:26.854 +08 [3487] LOG:  background worker "logical replication launcher" (PID 3494) exited with exit code 1
2026-07-03 14:28:26.860 +08 [3489] LOG:  shutting down
2026-07-03 14:28:26.862 +08 [3489] LOG:  checkpoint starting: shutdown immediate
2026-07-03 14:28:26.874 +08 [3489] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.005 s, total=0.015 s; sync files=2, longest=0.004 s, average=0.003 s; distance=0 kB, estimate=0 kB; lsn=1/105466A0, redo lsn=1/105466A0
2026-07-03 14:28:26.882 +08 [3487] LOG:  database system is shut down

[postgres@pgdb02 ~]$ pg_ctl start -D /pgData/pgsql17/data
waiting for server to start....2026-07-03 14:29:00.574 +08 [3505] LOG:  redirecting log output to logging collector process
2026-07-03 14:29:00.574 +08 [3505] HINT:  Future log output will appear in directory "log".
 done
server started
[postgres@pgdb02 ~]$

2026-07-03 14:29:00.575 +08 [3505] LOG:  starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-14), 64-bit
2026-07-03 14:29:00.579 +08 [3505] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2026-07-03 14:29:00.580 +08 [3505] LOG:  listening on IPv6 address "::", port 5432
2026-07-03 14:29:00.582 +08 [3505] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"
2026-07-03 14:29:00.587 +08 [3505] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2026-07-03 14:29:00.595 +08 [3509] LOG:  database system was shut down at 2026-07-03 14:28:26 +08
2026-07-03 14:29:00.607 +08 [3505] LOG:  database system is ready to accept connections

Immediate Shutdown (SIGQUIT):

[postgres@pgdb02 ~]$ pg_ctl stop -D /pgData/pgsql17/data -m immediate
waiting for server to shut down.... done
server stopped
[postgres@pgdb02 ~]$

2026-07-03 14:29:47.471 +08 [3505] LOG:  received immediate shutdown request
2026-07-03 14:29:47.485 +08 [3505] LOG:  database system is shut down


[postgres@pgdb02 ~]$ pg_ctl start -D /pgData/pgsql17/data
waiting for server to start....2026-07-03 14:30:17.965 +08 [3520] LOG:  redirecting log output to logging collector process
2026-07-03 14:30:17.965 +08 [3520] HINT:  Future log output will appear in directory "log".
. done
server started
[postgres@pgdb02 ~]$

2026-07-03 14:30:17.966 +08 [3520] LOG:  starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-14), 64-bit
2026-07-03 14:30:17.966 +08 [3520] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2026-07-03 14:30:17.966 +08 [3520] LOG:  listening on IPv6 address "::", port 5432
2026-07-03 14:30:17.970 +08 [3520] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"
2026-07-03 14:30:17.974 +08 [3520] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2026-07-03 14:30:17.983 +08 [3524] LOG:  database system was interrupted; last known up at 2026-07-03 14:29:00 +08
2026-07-03 14:30:19.211 +08 [3524] LOG:  database system was not properly shut down; automatic recovery in progress
2026-07-03 14:30:19.216 +08 [3524] LOG:  redo starts at 1/10546718
2026-07-03 14:30:19.216 +08 [3524] LOG:  invalid record length at 1/10546750: expected at least 24, got 0
2026-07-03 14:30:19.217 +08 [3524] LOG:  redo done at 1/10546718 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
2026-07-03 14:30:19.221 +08 [3522] LOG:  checkpoint starting: end-of-recovery immediate wait
2026-07-03 14:30:19.231 +08 [3522] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.003 s, total=0.012 s; sync files=2, longest=0.002 s, average=0.002 s; distance=0 kB, estimate=0 kB; lsn=1/10546750, redo lsn=1/10546750
2026-07-03 14:30:19.239 +08 [3520] LOG:  database system is ready to accept connections

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

How SELECT, INSERT, UPDATE, and DELETE Work Internally in PostgreSQL

 

How SELECT, INSERT, UPDATE, and DELETE work internally in PostgreSQL?

PostgreSQL uses MVCC (Multi-Version Concurrency Control), which means every modification creates a new version of the row instead of overwriting the old one. This is the core concept behind all DML operations.

1. Common Foundation (Applies to All Operations)

  • Heap: Data is stored in heap files (tables). Each row is a tuple.
  • Tuple Header: Every row contains:
    • xmin: Transaction ID that created the row
    • xmax: Transaction ID that deleted/updated the row (null if active)
    • Other visibility information
  • WAL (Write-Ahead Log): All changes are first written to WAL for durability before being applied to data files.
  • Shared Buffers: Data pages are read into memory before being modified.
  • Visibility Rules: A row is visible to a transaction based on its snapshot (which transactions it can see).

2. How SELECT Works

          Visibility Check: For each row, PostgreSQL checks:

    • The row’s xmin must be committed and visible to the current transaction.
    • The row’s xmax must be either null or not visible to the current transaction.

 


3. How INSERT Works

  1. Transaction Starts: A new transaction ID (xid) is assigned.
  2. Buffer Allocation: PostgreSQL finds a free slot in a heap page (or allocates a new page).
  3. Tuple Creation: A new row is created with:
    • xmin = current transaction ID
    • xmax = null
  4. WAL Logging: The change is written to the WAL buffer first.
  5. Page Modification: The new tuple is inserted into the shared buffer.
  6. Commit: On commit, the WAL is flushed to disk (fsync), making the change durable.

4. How UPDATE Works

UPDATE in PostgreSQL does not modify the existing row. It creates a new version of the row.

  1. Find the row using the same visibility rules as SELECT.
  2. Mark old row as dead: Set xmax = current transaction ID on the old tuple.
  3. Insert new row: Create a completely new tuple with:
    • xmin = current transaction ID
    • xmax = null
    • Updated column values
  4. Update indexes (if needed): New index entries point to the new tuple.
  5. WAL Logging: Both the old row’s xmax change and the new tuple are logged in WAL.
  6. Commit: Changes become visible to other transactions after commit.

Important: The old row version remains in the table until VACUUM cleans it up.


5. How DELETE Works

DELETE also uses MVCC — it doesn’t remove the row immediately.

  1. Locate the row using visibility rules.
  2. Mark as deleted: Set xmax = current transaction ID on the tuple.
  3. WAL Logging: The change is logged.
  4. Commit: The row becomes invisible to new transactions.
  5. Physical Removal: The row is not physically deleted from disk until VACUUM runs.

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

PostgreSQL 17 DBA Free Training Roadmap

PostgreSQL DBA Free Coaching

Free PostgreSQL DBA Training in Telugu for Beginners | Live Ongoing Batch | June 15 - July 31, 2026.
తెలుగులో నేర్చుకోండి · 5 Weeks · Online · Beginner Friendly · 100% Free

DBA Training Program
// 5-week structured training · 19 sessions · hands-on labs
5
Weeks
19
Sessions
100+
Topics
Future Free
Week 01 Lab Setup, Linux & PostgreSQL Fundamentals
Session 01 Lab Setup & Storage Configuration
Pre-requisites
  • Install VirtualBox on Windows (Do your self)
  • Install PgAdmin4 on Windows (Do your self)
  • Configure Oracle Linux 9.6 VM (will share link)
Linux VM Setup
  • Create Oracle Linux 9.6 VM
  • VM Hardware Sizing for PostgreSQL
  • Add Multiple Virtual Disks
Storage Configuration
  • Create Partitions
  • Create Filesystems
  • Mount Filesystems
  • Configure /etc/fstab
PostgreSQL Directory Layout
  • Create Mount Points: /pgData, /pgWal, /pgArch, /pgBackup, /pgLog
Session 02 Linux Administration for DBAs
  • Basic Linux Commands
  • File and Directory Management
  • User and Group Management
  • Permissions and Ownership
  • Process Management
  • Service Management using systemctl
  • Network Commands
  • Disk and Memory Monitoring
Session 03 PostgreSQL 17 Installation
Overview
  • PostgreSQL Overview and Versions
Installation Methods
  • RPM Installation
  • DNF/YUM Installation
  • Source Code Installation
Database Initialization
  • initdb
  • Custom WAL Location
  • PostgreSQL Service Configuration
Validation
  • Start PostgreSQL
  • Connect using psql
  • Connect using PgAdmin4
Session 04 PostgreSQL Architecture
Memory Architecture
  • Shared Buffers
  • WAL Buffers
  • Work Memory
  • Maintenance Work Memory
Process Architecture
  • Postmaster
  • Backend Processes
  • Checkpointer
  • Background Writer
  • WAL Writer
  • Autovacuum
Physical Storage Layout
  • Base Directory
  • Global Directory
  • WAL Directory
  • Tablespaces
Session 05 PostgreSQL Configuration Files
  • postgresql.conf
  • postgresql.auto.conf
  • pg_hba.conf
  • pg_ident.conf
  • Reload vs Restart
Week 02 Administration & Security
Session 06 Startup, Shutdown
  • PostgreSQL Startup Process
  • Smart Shutdown
  • Fast Shutdown (default)
  • Immediate Shutdown
Session 07 Database Administration
  • Creating Databases
  • Creating Schemas
  • Schema Search Path
  • Roles and Users
  • Access Control
Session 08 Tablespaces Management
  • pg_default
  • pg_global
  • Custom Tablespaces
  • Move Objects Between Tablespaces
  • Tablespace Monitoring
Session 09 Security
  • pg_hba.conf
  • Authentication Methods
  • pg_ident.conf
  • .pgpass
  • SSL/TLS Setup
Session 10 Vacuum and Analyze
  • MVCC Concepts
  • Dead Tuples
  • VACUUM
  • VACUUM FULL
  • ANALYZE
  • Autovacuum
Week 03 Backup & Recovery
Session 11 WAL Archiving
  • WAL Fundamentals
  • Archive Mode
  • Archive Command
Session 12 Logical Backup and Restore
  • pg_dump
  • pg_dumpall
  • Backup Formats
  • pg_restore
  • Restore using psql
Session 13 Physical Backup and Restore
  • pg_basebackup
  • Full Cluster Backup
  • Database Refresh on New Server
Session 14 PITR — Point-in-Time Recovery
  • PITR Concepts and Architecture
  • PITR Demonstration (hands-on)
Session 15 Full Recovery & Disaster Recovery
  • Full Database Recovery
  • Disaster Recovery Scenarios
Week 04 Replication, Failover
Session 16 Streaming Replication
  • Primary Configuration
  • Replica Configuration
  • Base Backup for Replica
  • Verify Replication
Session 17 Replication Administration
  • Replication Slots
  • Monitoring Replication
  • Async Replication
  • Sync Replication
  • Convert from ASYNC to SYNC
Session 18 Failover and pg_rewind
  • Manual Failover
  • Promote Replica
  • pg_rewind
  • Rejoin Old Primary as Replica
Week 05 Upgrades
Session 19 PostgreSQL Upgrades & Final Lab
  • Minor Version Upgrade
  • Major Version Upgrades
  • pg_upgrade
  • Upgrade using pg_dump/pg_restore
  • Upgrade Validation
  • Final End-to-End Lab
Future Batch Advanced Topics — Free Training
// Topics reserved for the next advanced batch · access is free
EXPLAIN / EXPLAIN ANALYZE Index Types & Tuning Table Partitioning Performance Tuning PostgreSQL Extensions PgBouncer pgmetrics pgcollector pgbadger Repmgr Patroni Maintenance Operations Oracle → PostgreSQL (Ora2Pg) Advanced Data Migration Logical Replication Multi-Node HA Architectures

PostgreSQL DBA Free Training Roadmap · Click any session card to expand topics

PostgreSQL

PostgreSQL DBA Step by Step Learning

#PostgreSQL DBA Topics
1How to Install PostgreSQL ON Linux?
2How to Install PostgreSQL on Linux 7 using source code?
3How to START/STOP PostgreSQL ON Linux?
4How to Create Database in PostgreSQL?
5PostgreSQL User Management
6PostgreSQL pg_hba.conf Guide
7PostgreSQL Change Data Directory
8Understanding WAL Files in PostgreSQL – For Oracle DBAs
9Change PostgreSQL WAL Directory Path (pg_wal)
10Enable Archive Mode in PostgreSQL 17
11How to Disable ARCHIVELOG Mode
12PostgreSQL Tablespace Management
13PostgreSQL pg_dump and pg_restore Guide
14PostgreSQL Backup and Restore Using pg_dumpall and psql
15pg_basebackup – Backup, Restore, and Recovery
16Backup & Restore PostgreSQL DB Cluster to Another Host (No Archive Mode)
17Backup & Restore PostgreSQL DB Cluster on Same Host
18Restore PostgreSQL to New Host using pg_basebackup + WAL Archives
19PostgreSQL PITR – Point in Time Recovery
20Configure Streaming Replication in PostgreSQL
21Manual Failover in PostgreSQL Streaming Replication
22Convert Asynchronous Replication to Synchronous Replication

 

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

PSQL

PostgreSQL DBA Step by Step Learning

#PostgreSQL DBA Topics
1How to Install PostgreSQL ON Linux?
2How to Install PostgreSQL on Linux 7 using source code?
3How to START/STOP PostgreSQL ON Linux?
4How to Create Database in PostgreSQL?
5PostgreSQL User Management
6PostgreSQL pg_hba.conf Guide
7PostgreSQL Change Data Directory
8Understanding WAL Files in PostgreSQL – For Oracle DBAs
9Change PostgreSQL WAL Directory Path (pg_wal)
10Enable Archive Mode in PostgreSQL 17
11How to Disable ARCHIVELOG Mode
12PostgreSQL Tablespace Management
13PostgreSQL pg_dump and pg_restore Guide
14PostgreSQL Backup and Restore Using pg_dumpall and psql
15pg_basebackup – Backup, Restore, and Recovery
16Backup & Restore PostgreSQL DB Cluster to Another Host (No Archive Mode)
17Backup & Restore PostgreSQL DB Cluster on Same Host
18Restore PostgreSQL to New Host using pg_basebackup + WAL Archives
19PostgreSQL PITR – Point in Time Recovery
20Configure Streaming Replication in PostgreSQL
21Manual Failover in PostgreSQL Streaming Replication
22Convert Asynchronous Replication to Synchronous Replication

 

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/