Table of Contents

1. Overview and Golden Rules

Environment used in this guide:

Replica set name: rs0. The service is mongod.service, managed with systemctl only. Its stop action runs mongod --shutdown, which is a clean shutdown.

💡 Golden rules:
• Do not stop the primary directly. Use a planned rs.stepDown() so the primary hands over its role on purpose.
• Confirm a new primary exists before you stop the old one.
• Both secondaries must be healthy and caught up before the step down.
• Always use a clean stop (systemctl stop mongod). Never use kill -9.
• Use systemctl only. Do not mix it with mongoctl.sh or manual mongod starts.

2. Pre-Checks Before Any Shutdown

Step 1 – Connect and check the replica set health. All members must be healthy before you start maintenance. Otherwise, a planned stop can leave you without a majority.

/app/mongodb/7.0.43/bin/mongosh --host localhost -u admin --authenticationDatabase admin

Enter password: ******
Using MongoDB:          7.0.43
Using Mongosh:          2.12.0

Enterprise rs0 [primary] test>

Enterprise rs0 [primary] test> rs.status().members.map(m => m.name + " " + m.stateStr + " health=" + m.health)
[
'mongo01:27017 PRIMARY health=1',  <----------
'mongo02:27017 SECONDARY health=1',
'mongo03:27017 SECONDARY health=1'
]

Step 2 – Check replication lag. Each secondary should be at or close to 0 seconds behind the primary.

Enterprise rs0 [direct: primary] test> rs.printSecondaryReplicationInfo()
source: mongo02:27017
{
syncedTo: 'Sun Nov 08 2026 01:38:41 GMT-0500 (Eastern Standard Time)',
replLag: '0 secs (0 hrs) behind the primary '   <---------------
}
---
source: mongo03:27017
{
syncedTo: 'Sun Nov 08 2026 01:38:41 GMT-0500 (Eastern Standard Time)',
replLag: '0 secs (0 hrs) behind the primary '  <------------------
}

Step 3 – Check the oplog window. This is how long a stopped member can stay down and still catch up without a full resync.

Enterprise rs0 [direct: primary] test> rs.printReplicationInfo()
actual oplog size
'10240 MB'
---
configured oplog size
'10240 MB'
---
log length start to end
'144034 secs (40.01 hrs)'
---
oplog first event time
'Fri Nov 06 2026 09:39:37 GMT-0500 (Eastern Standard Time)'
---
oplog last event time
'Sun Nov 08 2026 01:40:11 GMT-0500 (Eastern Standard Time)'
---
now
'Sun Nov 08 2026 01:40:12 GMT-0500 (Eastern Standard Time)'

• Oplog size: 10 GB. The configured and actual sizes match, so this is fine.
• Log length (40 hours): This is the replication window, meaning how long a secondary can be offline and still catch up by replaying the oplog.
• Oplog first event: Fri Nov 06, 09:39. Any secondary whose last synced operation is older than this cannot catch up and needs a full initial sync.
• Oplog last event: Sun Nov 08, 01:40. This is essentially “now”, which shows the primary is actively writing.

Note: the window is not fixed. A heavier write load shortens it, so 40 hours today does not guarantee 40 hours tomorrow.

Step 4 – (Optional) Check long-running operations that a shutdown would interrupt:

Enterprise rs0 [primary] test> db.currentOp({ active: true, secs_running: { $gt: 5 } }).inprog.length
4
Enterprise rs0 [primary] test>

-- 4 operations have been actively running for more than 5 seconds, run below command to find out and then inform to respective team.

db.currentOp({ active: true, secs_running: { $gt: 5 } }).inprog.map(op => ({
opid: op.opid,
op: op.op,
ns: op.ns,
secs: op.secs_running,
client: op.client,
desc: op.desc,
msg: op.msg
}))

⚠️ Before maintenance: take a backup if the work is risky (OS patching, upgrades, storage changes). Tell the application team about the window, because a primary change is a short interruption for clients that do not use retryable writes.

3. Stop Only the Primary

If you stop the primary directly, the secondaries need some seconds to notice and elect a new one. A planned rs.stepDown() is faster and cleaner, because the primary hands over its role on purpose.

Step 1 – Check the secondaries are healthy and caught up (Section 2 Pre-Checks Before Any Shutdown ).

Step 2 – Step down the primary. Connect to mongo01 and run:

# rs.stepDown(120) tells the current primary to give up its role and not try to become primary again for 120 seconds. Enterprise rs0 [direct: primary] test> rs.stepDown(120) { ok: 1, '$clusterTime': { clusterTime: Timestamp({ t: 1794125254, i: 1 }), signature: { hash: Binary.createFromBase64('LTbUnMEZJ5KZLejumqSPvu87T7E=', 0), keyId: Long('7705068198269288454') } }, operationTime: Timestamp({ t: 1794125254, i: 1 }) } Enterprise rs0 [direct: secondary] test> 

The number is the time in seconds during which mongo01 will not try to become primary again. The command waits up to 10 seconds for an electable secondary to catch up, then steps down. The shell connection drops and reconnects, which is normal.

Step 3 – Confirm a new primary.

Enterprise rs0 [direct: secondary] test> rs.status().members.map(m => m.name + " " + m.stateStr + " health=" + m.health)
[
'mongo01:27017 SECONDARY health=1',
'mongo02:27017 PRIMARY health=1',   <-------------- mongo02 has became primary now.
'mongo03:27017 SECONDARY health=1'
]
Enterprise rs0 [direct: secondary] test>

Step 4 – Stop mongo01 and do the maintenance.

[root@mongo01 ~]# systemctl stop mongod
[root@mongo01 ~]# pgrep -a mongod
[root@mongo01 ~]#
# ... maintenance ...
[root@mongo01 ~]# systemctl start mongod

Step 5 – Check after the start. mongo01 rejoins as a secondary, catches up, and returns as primary after its priority election. If you want it to wait or stay secondary, use a longer stepDown time.

[mongop@mongo01 ~]$ /app/mongodb/7.0.43/bin/mongosh --host localhost -u admin --authenticationDatabase admin
Enter password: ******
Current Mongosh Log ID: 6af02e97d012d14ec5333050
Connecting to:          mongodb://@localhost:27017/?directConnection=true&serverSelectionTimeoutMS=2000&authSource=admin&appName=mongosh+2.12.0
Using MongoDB:          7.0.43
Using Mongosh:          2.12.0
mongosh 2.14.1 is available for download: https://www.mongodb.com/try/download/shell

For mongosh info see: https://www.mongodb.com/docs/mongodb-shell/

Enterprise rs0 [direct: primary] test> rs.status().members.map(m => m.name + " " + m.stateStr + " health=" + m.health)
[
  'mongo01:27017 PRIMARY health=1',   <------------- mongo01 has became primary becuase it has priority 2 'mongo02:27017 SECONDARY health=1', 'mongo03:27017 SECONDARY health=1' ] Enterprise rs0 [direct: primary] test> Enterprise rs0 [direct: primary] test> rs.conf().members.map(m => ({ name: m.host, priority: m.priority, votes: m.votes }))
[
{ name: 'mongo01:27017', priority: 2, votes: 1 },
{ name: 'mongo02:27017', priority: 1, votes: 1 },
{ name: 'mongo03:27017', priority: 1, votes: 1 }
]
Enterprise rs0 [direct: primary] test> 

⚠️ Warning: while mongo01 is down, only two members remain. Do not stop or restart another node during this time, or you lose the majority and the application loses write access.

✅ The old primary is back and the replica set has one primary and two secondaries.

4. Quick Reference

💡 Tip: keep a shell open on a node that stays up during the work, and re-run the member status command after each step. It is the quickest way to see where the set is.

5. Common Mistakes and Troubleshooting

• Stopping the primary without rs.stepDown(). Clients see a longer interruption while the election runs.
• Stopping the old primary before a new primary is confirmed.
• Touching another node while the old primary is down. You lose the majority.
• Using kill -9 or powering off instead of a clean stop. The node has to run crash recovery at the next start.
• Leaving the node down longer than the oplog window. It cannot catch up and needs a full resync. Check rs.printReplicationInfo() before long maintenance.