Category Archives: ASM

Configure UDEV Rules for Oracle ASM

Configure UDEV Rules for Oracle ASM

Table of Contents
___________________________________________________________________________________________________

Pre-requisites:

a. Create OS user and groups for Oracle ASM disk owner
b. Add RAW disks to server (Check with sysadmin)

Configure UDEV Rules for Oracle ASM:

1. List the disks
2. Create partitions for the disks
3. Load updated block device partition tables
4. Find SCSI ID
5. Create udev rules
6. Reload the udev rules
7. List oracleasm disks

___________________________________________________________________________________________________


Pre-requisites


a. Create OS user and groups for Oracle ASM disk owner

http://www.br8dba.com/create-users-groups-and-paths-for-oracle-rac/


b. Add RAW disks to server

Operating SystemRed Hat Enterprise Linux release 8.7
Storage
/dev/sda100G for Linux and others
/dev/sdb100G for /u01
/dev/sdc100G for /orabackup
/dev/sdd100G for ASM DISK
/dev/sde100G for ASM DISK


Configure UDEV Rules for Oracle ASM

If ASMLIB kernel drivers are not available then we have to use udev rules to create the disks for Oracle ASM.

Setting up Oracle ASM udev rules is not so complicated. All you need is the udevadm command and editing one file.


1. List the disks

[root@testbox ~]# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    0  100G  0 disk
├─sda1        8:1    0    1G  0 part /boot
└─sda2        8:2    0   99G  0 part
  ├─ol-root 252:0    0 61.2G  0 lvm  /
  ├─ol-swap 252:1    0  7.9G  0 lvm  [SWAP]
  └─ol-home 252:2    0 29.9G  0 lvm  /home
sdb           8:16   0  100G  0 disk
└─sdb1        8:17   0  100G  0 part /u01
sdc           8:32   0  100G  0 disk
└─sdc1        8:33   0  100G  0 part /orabackup
sdd           8:48   0  100G  0 disk
sde           8:64   0  100G  0 disk
sr0          11:0    1 47.3M  0 rom
[root@testbox ~]#


2. Create partitions for the disks

fdisk /dev/sdd
fdisk /dev/sde

[root@testbox ~]# fdisk /dev/sdd

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x8c782d71.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-209715199, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199):

Created a new partition 1 of type 'Linux' and of size 100 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

[root@testbox ~]#
[root@testbox ~]# fdisk /dev/sde

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x36462c72.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-209715199, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199):

Created a new partition 1 of type 'Linux' and of size 100 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

[root@testbox ~]#


3. Load updated block device partition tables

# For Linux 5,6 and 7

# /sbin/partprobe /dev/sdd1
# /sbin/partprobe /dev/sde1

# For Linux8

[root@testbox ~]# /sbin/partx -u /dev/sdd1
[root@testbox ~]#
[root@testbox ~]# /sbin/partx -u /dev/sde1
[root@testbox ~]#


4. Find SCSI ID

[root@testbox ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdd
1ATA_VBOX_HARDDISK_VB0adc00d9-c5938e95
[root@testbox ~]#

[root@testbox ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sde
1ATA_VBOX_HARDDISK_VBdaa5e829-52e4b9b1
[root@testbox ~]#


5. Create udev rules

[root@testbox ~]# ls -ltr /etc/udev/rules.d
total 12
-rw-r--r--. 1 root root  67 Oct  2 18:03 69-vdo-start-by-dev.rules
-rw-r--r--. 1 root root 148 Nov  9 06:11 99-vmware-scsi-timeout.rules
-rw-r--r--. 1 root root 134 Apr  1 07:52 60-vboxadd.rules
[root@testbox ~]#

vi /etc/udev/rules.d/99-oracle-asmdevices.rules
and below lines and then save it.

KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB0adc00d9-c5938e95", SYMLINK+="oracleasm/disks/DISK01", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VBdaa5e829-52e4b9b1", SYMLINK+="oracleasm/disks/DISK02", OWNER="grid", GROUP="asmadmin", MODE="0660"


[root@testbox ~]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB0adc00d9-c5938e95", SYMLINK+="oracleasm/disks/DISK01", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VBdaa5e829-52e4b9b1", SYMLINK+="oracleasm/disks/DISK02", OWNER="grid", GROUP="asmadmin", MODE="0660"
[root@testbox ~]#

[root@testbox ~]# ls -ltr /etc/udev/rules.d/99-oracle-asmdevices.rules
-rw-r--r--. 1 root root 428 Apr  2 02:18 /etc/udev/rules.d/99-oracle-asmdevices.rules
[root@testbox ~]#


6. Reload the udev rules

The below commands will reload the complete udev configuration and will trigger all the udev rules. 
On a busy production system this could disrupt ongoing operations, applications running on the server. Please use the below command during a scheduled maintenance window only.

[root@testbox ~]# /sbin/udevadm control --reload-rules
[root@testbox ~]#
[root@testbox ~]# ls -ld /dev/sd*1
brw-rw----. 1 root disk 8,  1 Apr  2 02:23 /dev/sda1
brw-rw----. 1 root disk 8, 17 Apr  2 02:23 /dev/sdb1
brw-rw----. 1 root disk 8, 33 Apr  2 02:23 /dev/sdc1
brw-rw----. 1 root disk 8, 49 Apr  2 02:23 /dev/sdd1
brw-rw----. 1 root disk 8, 65 Apr  2 02:23 /dev/sde1
[root@testbox ~]#
[root@testbox ~]# /sbin/udevadm trigger
[root@testbox ~]#
[root@testbox ~]# ls -ld /dev/sd*1
brw-rw----. 1 root disk     8,  1 Apr  2 02:34 /dev/sda1
brw-rw----. 1 root disk     8, 17 Apr  2 02:34 /dev/sdb1
brw-rw----. 1 root disk     8, 33 Apr  2 02:34 /dev/sdc1
brw-rw----. 1 grid asmadmin 8, 49 Apr  2 02:34 /dev/sdd1
brw-rw----. 1 grid asmadmin 8, 65 Apr  2 02:34 /dev/sde1
[root@testbox ~]#


7. List the oracleasm disks

[root@testbox ~]# ls -ltra /dev/oracleasm/disks/*
lrwxrwxrwx. 1 root root 10 Apr  2 02:34 /dev/oracleasm/disks/DISK01 -> ../../sdd1
lrwxrwxrwx. 1 root root 10 Apr  2 02:34 /dev/oracleasm/disks/DISK02 -> ../../sde1
[root@testbox ~]#

[root@testbox ~]# ls -ld /dev/sd*1
brw-rw----. 1 root disk     8,  1 Apr  2 02:34 /dev/sda1
brw-rw----. 1 root disk     8, 17 Apr  2 02:34 /dev/sdb1
brw-rw----. 1 root disk     8, 33 Apr  2 02:34 /dev/sdc1
brw-rw----. 1 grid asmadmin 8, 49 Apr  2 02:34 /dev/sdd1
brw-rw----. 1 grid asmadmin 8, 65 Apr  2 02:34 /dev/sde1
[root@testbox ~]#

Note: symboliclinks are owned by root, but devices will be owned by grid:asmadmin

 

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

CSSD won’t start automatically

Cause

After server reboot, CSSD and ASM are not came up automatically.

Data Collection

[oracle@Linux4 ~]$ ps -ef | grep d.bin
oracle    4233     1  0 09:22 ?        00:00:03 /oracle11g/app/11.2.0/oracle/grid/bin/ohasd.bin reboot
oracle    4794     1  0 09:23 ?        00:00:01 /oracle11g/app/11.2.0/oracle/grid/bin/oraagent.bin
oracle    4810     1  0 09:23 ?        00:00:00 /oracle11g/app/11.2.0/oracle/grid/bin/evmd.bin
oracle    4812     1  0 09:23 ?        00:00:00 /oracle11g/app/11.2.0/oracle/grid/bin/tnslsnr LISTENER -inherit
oracle    4855  4810  0 09:23 ?        00:00:00 /oracle11g/app/11.2.0/oracle/grid/bin/evmlogger.bin -o 

/oracle11g/app/11.2.0/oracle/grid/evm/log/evmlogger.info -l /oracle11g/app/11.2.0/oracle/grid/evm/log/evmlogger.log
oracle    4942  4748  0 09:38 pts/1    00:00:00 grep d.bin
[oracle@Linux4 ~]$

[oracle@Linux4 ~]$ crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               OFFLINE OFFLINE      linux4
ora.LISTENER.lsnr
               ONLINE  ONLINE       linux4
ora.asm
               OFFLINE OFFLINE      linux4                   Instance Shutdown <----
ora.ons
               OFFLINE OFFLINE      linux4
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  OFFLINE <-----
ora.diskmon
      1        OFFLINE OFFLINE
ora.evmd
      1        ONLINE  ONLINE       linux4
[oracle@Linux4 ~]$

[oracle@Linux4 ~]$ crsctl stat res ora.cssd -p
NAME=ora.cssd
TYPE=ora.cssd.type
ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=
ACTIVE_PLACEMENT=0
AGENT_FILENAME=%CRS_HOME%/bin/cssdagent%CRS_EXE_SUFFIX%
AGENT_HB_INTERVAL=0
AGENT_HB_MISCOUNT=10
AUTO_START=never <----
CARDINALITY=1
CHECK_ARGS=
CHECK_COMMAND=
CHECK_INTERVAL=30
CLEAN_ARGS=abort
CLEAN_COMMAND=
CSSD_MODE=
CSSD_PATH=%CRS_HOME%/bin/ocssd%CRS_EXE_SUFFIX%
CSS_USER=oracle
DAEMON_LOGGING_LEVELS=CSSD=2,GIPCNM=2,GIPCGM=2,GIPCCM=2,CLSF=0,SKGFD=0,GPNP=1,OLR=0
DAEMON_TRACING_LEVELS=CSSD=0,GIPCNM=0,GIPCGM=0,GIPCCM=0,CLSF=0,SKGFD=0,GPNP=0,OLR=0
DEFAULT_TEMPLATE=
DEGREE=1
DESCRIPTION="Resource type for CSSD"
DETACHED=true
ENABLED=1
ENV_OPTS=
FAILOVER_DELAY=0
FAILURE_INTERVAL=3
FAILURE_THRESHOLD=5
HOSTING_MEMBERS=
INTERVAL_FS=3
LOAD=1
LOGGING_LEVEL=1
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
OMON_INITRATE=1000
OMON_POLLRATE=500
ORA_OPROCD_MODE=
ORA_VERSION=11.2.0.4.0
PID_FILE=
PLACEMENT=balanced
PROCD_TIMEOUT=1000
PROCESS_TO_MONITOR=
PROFILE_CHANGE_TEMPLATE=
REBOOT_OPTS=
RESTART_ATTEMPTS=5
SCRIPT_TIMEOUT=600
SERVER_POOLS=
START_ARGS=
START_COMMAND=
START_DEPENDENCIES=weak(concurrent:ora.diskmon)
START_TIMEOUT=600
STATE_CHANGE_TEMPLATE=
STOP_ARGS=
STOP_COMMAND=
STOP_DEPENDENCIES=hard(shutdown:ora.diskmon)
STOP_TIMEOUT=900
TIMEOUT_FS=0
UNRESPONSIVE_TIMEOUT=180
UPTIME_THRESHOLD=1m
USR_ORA_ENV=
VMON_INITLIMIT=16
VMON_INITRATE=500
VMON_POLLRATE=500

[oracle@Linux4 ~]$

[oracle@Linux4 ~]$ crsctl stat res ora.asm -p
NAME=ora.asm
TYPE=ora.asm.type
ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=
AGENT_FILENAME=%CRS_HOME%/bin/oraagent%CRS_EXE_SUFFIX%
ALIAS_NAME=
ASM_DISKSTRING=/dev/oracleasm/disks/*
AUTO_START=restore <----
CHECK_INTERVAL=1
CHECK_TIMEOUT=30
DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=asm) ELEMENT(INSTANCE_NAME= %GEN_USR_ORA_INST_NAME%)
DEGREE=1
DESCRIPTION=Oracle ASM resource
ENABLED=1
GEN_USR_ORA_INST_NAME=+ASM
LOAD=1
LOGGING_LEVEL=1
NLS_LANG=
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
PROFILE_CHANGE_TEMPLATE=
RESTART_ATTEMPTS=5
SCRIPT_TIMEOUT=60
SPFILE=+DATA/asm/asmparameterfile/registry.253.894413045
START_DEPENDENCIES=hard(ora.cssd) weak(ora.LISTENER.lsnr)
START_TIMEOUT=900
STATE_CHANGE_TEMPLATE=
STOP_DEPENDENCIES=hard(ora.cssd)
STOP_TIMEOUT=600
TYPE_VERSION=1.2
UPTIME_THRESHOLD=1d
USR_ORA_ENV=
USR_ORA_INST_NAME=+ASM
USR_ORA_OPEN_MODE=mount
USR_ORA_OPI=false
USR_ORA_STOP_MODE=immediate
VERSION=11.2.0.4.0

[oracle@Linux4 ~]$

Action Plan

Start the resources manually

[oracle@Linux4 ~]$ crsctl start res ora.cssd -init
CRS-2672: Attempting to start 'ora.cssd' on 'linux4'
CRS-2672: Attempting to start 'ora.diskmon' on 'linux4'
CRS-2676: Start of 'ora.diskmon' on 'linux4' succeeded
CRS-2676: Start of 'ora.cssd' on 'linux4' succeeded
[oracle@Linux4 ~]$
[oracle@Linux4 ~]$ ps -ef | grep cssd.bin
oracle    5184     1  0 10:19 ?        00:00:00 /oracle11g/app/11.2.0/oracle/grid/bin/ocssd.bin
oracle    5207  4748  0 10:20 pts/1    00:00:00 grep cssd.bin
[oracle@Linux4 ~]$
[oracle@Linux4 ~]$ srvctl start asm
[oracle@Linux4 ~]$
[oracle@Linux4 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       linux4
ora.LISTENER.lsnr
               ONLINE  ONLINE       linux4
ora.asm
               ONLINE  ONLINE       linux4                   Started
ora.ons
               OFFLINE OFFLINE      linux4
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       linux4
ora.diskmon
      1        OFFLINE OFFLINE
ora.evmd
      1        ONLINE  ONLINE       linux4
[oracle@Linux4 ~]$

We need to change the auto_start attribute of a resource to value ‘always’ or ‘1’ to start resource automatically after server reboot.

Run below commands as GI home owner.

[oracle@Linux4 ~]$ crsctl modify resource "ora.cssd" -attr "AUTO_START=always"
[oracle@Linux4 ~]$ crsctl modify resource "ora.asm" -attr "AUTO_START=always"
[oracle@Linux4 ~]$

[oracle@Linux4 ~]$ crsctl stat res ora.asm -p | grep "AUTO_START"
AUTO_START=always
[oracle@Linux4 ~]$ crsctl stat res ora.cssd -p | grep "AUTO_START"
AUTO_START=always
[oracle@Linux4 ~]$

Additional Information:

always (1)
 Causes the resource to restart when the node restarts regardless of the resource’s state when the node stopped.

restore (0)
 Does not start the resource at restart time if it was in an offline state, such as STATE=OFFLINE, TARGET=OFFLINE, when the node stopped. The 

resource is restored to its state when the node went down. The resource is started only if it was online before and not otherwise.

never (2)
 Oracle Clusterware never restarts the resource regardless of the resource’s state when the node is stopped.

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Instantiating disk: failed

Instantiating disk: failed while creating ASM disk.

Issue

[root@Linux4 ~]# oracleasm createdisk disk1 /dev/sdb1
Writing disk header: done
Instantiating disk: failed <----
Clearing disk header: done
[root@Linux4 ~]# 

Action Plan

[root@Linux4 ~]# /etc/init.d/oracleasm disable
Writing Oracle ASM library driver configuration: done
Dropping Oracle ASMLib disks:                              [  OK  ]
Shutting down the Oracle ASMLib driver:                    [  OK  ]
[root@Linux4 ~]#
[root@Linux4 ~]# /etc/init.d/oracleasm enable
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]
[root@Linux4 ~]#
[root@Linux4 ~]# oracleasm createdisk disk1 /dev/sdb1
Writing disk header: done
Instantiating disk: done <---

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Move ASM Spfile

How to move ASM SPFILE from +DATA to +VOTE diskgroup

Contents
___________________________________________________________________________________________________________________________________

1. Verify Available DiskGroups
2. Verify Current ASM SP File location
3. Create PFILE from SPFILE
4. Create SPFILE from PFILE
5. Stop CRS All Nodes
6. Start CRS All Nodes
7. Verify ASM SPFILE Location
___________________________________________________________________________________________________________________________________


1. Verify Available DiskGroups

ASMCMD> lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  NORMAL  N         512   4096  1048576      2940     2110              980             565              0             N  DATA/
MOUNTED  EXTERN  N         512   4096  1048576      7295       13                0              13              0             N  DATA1/
MOUNTED  EXTERN  N         512   4096  1048576      1019      892                0             892              0             Y  VOTE/
ASMCMD>


2. Verify Current ASM SP File location

[oracle@rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Fri Aug 14 03:47:52 2015

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> show parameter pfile

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +DATA/rac-scan/asmparameterfil
                                                 e/registry.253.833600843
SQL>


3. Create PFILE from SPFILE

SQL> create pfile='/home/oracle/initASM.ora' from spfile;

File created.

SQL>
SQL> !ls -ltr /home/oracle/initASM.ora
-rw-r--r-- 1 oracle oinstall 300 Aug 14 03:49 /home/oracle/initASM.ora

SQL>


4. Create SPFILE from PFILE

SQL> create spfile='+VOTE' from pfile='/home/oracle/initASM.ora';

File created.

SQL>

ASMCMD> pwd
+vote/rac-scan/ASMPARAMETERFILE
ASMCMD>
ASMCMD> ls -l
Type              Redund  Striped  Time             Sys  Name
ASMPARAMETERFILE  UNPROT  COARSE   AUG 14 03:00:00  Y    REGISTRY.253.887687509 <----
ASMCMD>


5. Stop CRS All Nodes

From Node 1

[root@rac1 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac1'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.VOTE.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.DATA1.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac1'
CRS-2677: Stop of 'ora.scan1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac2'
CRS-2677: Stop of 'ora.rac1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.rac1.vip' on 'rac2'
CRS-2677: Stop of 'ora.DATA1.dg' on 'rac1' succeeded
CRS-2676: Start of 'ora.rac1.vip' on 'rac2' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'rac1' succeeded
CRS-2677: Stop of 'ora.VOTE.dg' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac1'
CRS-2677: Stop of 'ora.ons' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac1'
CRS-2677: Stop of 'ora.net1.network' on 'rac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac1' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac1'
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'rac1'
CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac1 ~]#

From Node 2

[root@rac2 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac2'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.oc4j' on 'rac2'
CRS-2673: Attempting to stop 'ora.cvu' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.scan3.vip' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac2'
CRS-2677: Stop of 'ora.scan3.vip' on 'rac2' succeeded
CRS-2677: Stop of 'ora.scan2.vip' on 'rac2' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.rac2.vip' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.VOTE.dg' on 'rac2'
CRS-2673: Attempting to stop 'ora.DATA1.dg' on 'rac2'
CRS-2677: Stop of 'ora.rac2.vip' on 'rac2' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac2'
CRS-2677: Stop of 'ora.rac1.vip' on 'rac2' succeeded
CRS-2677: Stop of 'ora.DATA1.dg' on 'rac2' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'rac2' succeeded
CRS-2677: Stop of 'ora.cvu' on 'rac2' succeeded
CRS-2677: Stop of 'ora.VOTE.dg' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac2'
CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac2'
CRS-2677: Stop of 'ora.ons' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac2'
CRS-2677: Stop of 'ora.net1.network' on 'rac2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac2' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2'
CRS-2673: Attempting to stop 'ora.crf' on 'rac2'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac2'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac2'
CRS-2673: Attempting to stop 'ora.asm' on 'rac2'
CRS-2677: Stop of 'ora.mdnsd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.crf' on 'rac2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac2'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac2'
CRS-2677: Stop of 'ora.cssd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2'
CRS-2677: Stop of 'ora.gipcd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2'
CRS-2677: Stop of 'ora.gpnpd' on 'rac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
You have new mail in /var/spool/mail/root
[root@rac2 ~]#


6. Start CRS All Nodes

From Node 1

[root@rac1 ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac1 ~]#

From Node 2

[root@rac2 ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac2 ~]#
[root@rac1 ~]# crsctl check cluster -all
**************************************************************
rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
rac2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[root@rac1 ~]#


7. Verify ASM SPFILE Location

[oracle@rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Fri Aug 14 04:07:03 2015

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> show parameter pfile

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +VOTE/rac-scan/asmparameterfil
                                                 e/registry.253.887687509
SQL>

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

ASM Commands

1. ASMLIB commands
2. Start/Stop CSSD (Non-RAC)

1. ASMLIB commands


Most commonly used asmlib commands
 
/etc/init.d/oracleasm start
/etc/init.d/oracleasm stop
/etc/init.d/oracleasm restart
/etc/init.d/oracleasm configure
/etc/init.d/oracleasm status
/etc/init.d/oracleasm enable
/etc/init.d/oracleasm disable
/etc/init.d/oracleasm deletedisk
/etc/init.d/oracleasm createdisk DATA /dev/sdb1
/etc/init.d/oracleasm querydisk /dev/sdb1
/etc/init.d/oracleasm querydisk DATA
/etc/init.d/oracleasm renamedisk /dev/sdb1 VOL1
/etc/init.d/oracleasm force-renamedisk DATA NEW_DATA_DISK1
/etc/init.d/oracleasm scandisks
/etc/init.d/oracleasm listdisks

 
2. Start/Stop CSSD (Non-RAC)

Manually start CSSD (Non-RAC)
/etc/init.d/init.cssd start

Manually stop CSSD (Non-RAC)
/etc/init.d/init.cssd stop

still page under construction !!!