Category Archives: ASM

Configure ASMLib for Oracle ASM

Configure ASMLib for Oracle ASM

Table of Contents
___________________________________________________________________________________________________

Environment

Pre-requisites

a. Create OS user and groups for Oracle ASM disk owner
b. Add RAW disks to server (Check with osadmin)

Configure ASMLib for Oracle ASM (Only on Linux)

1. List the disks
2. Create partitions for the disks
3. Load updated block device partition tables
4. Install ASMLib packages
5. Configure Oracle ASMLib
6. Create ASM disks
7. List oracleasm disks

___________________________________________________________________________________________________


Environment

OSRed Hat Enterprise Linux release 8.7
Kernel Version4.18.0-477.27.1.el8_8.x86_64
ASM disk ownergrid user

Pre-requisites


a. Create OS user and groups for Oracle ASM disk owner


b. Add RAW disks to server (Please verify with your system administrator for ASM disk names)

Operating SystemRed Hat Enterprise Linux release 8.7
Storage
/dev/sdaFor Operating system and other
/dev/sdb100G for ASM DISK
/dev/sdc100G for ASM DISK
/dev/sdd100G for ASM DISK
/dev/sde100G for ASM DISK
/dev/sdf100G for ASM DISK


Configure ASMLib for Oracle ASM (Only on Linux)

What is ASMLib?
ASMLib, short for "Automatic Storage Management Library," is a support library provided by Oracle for managing Oracle ASM (Automatic Storage Management) on Linux systems. It simplifies the administration and management of Oracle ASM disk groups and volumes by providing a set of utilities and kernel modules. 
ASMLib is typically used in Oracle Database environments to interact with ASM disks and is specifically designed to work with Oracle ASM.
ASMlib driver is required on Linux operating systems, to enable a disk readable by ASM. Without ASMLIB, disks can’t be used at asm disks.
All ASMLib installations require the kmod-redhat-oracleasm, oracleasmlib and oracleasm-support packages appropriate for their machine.
ASMLib is not mandatory for the Automatic Storage Management (ASM) feature of Oracle Database on Linux and all features and functionality of ASM will work without ASMLib. 

oracleasm-support
kmod-redhat-oracleasm  <-- download from Redhat
oracleasmlib

You can also use udev rules instead of Oracle ASMLib.
https://www.br8dba.com/configure-udev-rules-for-oracle-asm/


1. List the disks

[root@db1 ~]# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    0  100G  0 disk
├─sda1        8:1    0    1G  0 part /boot
└─sda2        8:2    0   99G  0 part
  ├─ol-root 252:0    0 61.2G  0 lvm  /
  ├─ol-swap 252:1    0  7.9G  0 lvm  [SWAP]
  └─ol-home 252:2    0 29.9G  0 lvm  /home
sdb           8:16   0  100G  0 disk
sdc           8:32   0  100G  0 disk
sdd           8:48   0  100G  0 disk
sde           8:64   0  100G  0 disk
sdf           8:80   0  100G  0 disk
sr0          11:0    1 1024M  0 rom
[root@db1 ~]#


2. Create partitions for the disks

[root@db1 ~]# fdisk /dev/sdb

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xf1d4ac05.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-209715199, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199):

Created a new partition 1 of type 'Linux' and of size 100 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

[root@db1 ~]# fdisk /dev/sdc

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xfd2d6460.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-209715199, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199):

Created a new partition 1 of type 'Linux' and of size 100 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

[root@db1 ~]# fdisk /dev/sdd

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xa03481f1.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-209715199, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199):

Created a new partition 1 of type 'Linux' and of size 100 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

[root@db1 ~]# fdisk /dev/sde

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xbc1867fb.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-209715199, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199):

Created a new partition 1 of type 'Linux' and of size 100 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

[root@db1 ~]# fdisk /dev/sdf

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x4807d719.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-209715199, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199):

Created a new partition 1 of type 'Linux' and of size 100 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

[root@db1 ~]#
[root@db1 ~]# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    0  100G  0 disk
├─sda1        8:1    0    1G  0 part /boot
└─sda2        8:2    0   99G  0 part
  ├─ol-root 252:0    0 61.2G  0 lvm  /
  ├─ol-swap 252:1    0  7.9G  0 lvm  [SWAP]
  └─ol-home 252:2    0 29.9G  0 lvm  /home
sdb           8:16   0  100G  0 disk
└─sdb1        8:17   0  100G  0 part
sdc           8:32   0  100G  0 disk
└─sdc1        8:33   0  100G  0 part
sdd           8:48   0  100G  0 disk
└─sdd1        8:49   0  100G  0 part
sde           8:64   0  100G  0 disk
└─sde1        8:65   0  100G  0 part
sdf           8:80   0  100G  0 disk
└─sdf1        8:81   0  100G  0 part
sr0          11:0    1 1024M  0 rom
[root@db1 ~]#


3. Load updated block device partition tables

# For Linux 5,6 and 7

# /sbin/partprobe /dev/sdb1
# /sbin/partprobe /dev/sdc1
# /sbin/partprobe /dev/sdd1
# /sbin/partprobe /dev/sde1
# /sbin/partprobe /dev/sdf1

# For Linux8

[root@db1 ~]# /sbin/partx -u /dev/sdb1
[root@db1 ~]# /sbin/partx -u /dev/sdc1
[root@db1 ~]# /sbin/partx -u /dev/sdd1
[root@db1 ~]# /sbin/partx -u /dev/sde1
[root@db1 ~]# /sbin/partx -u /dev/sdf1


4. Install ASMLib packages

*** Download and install ASMLib packages that support your kernel. 
[root@db1 ~]# uname -rms
Linux 4.18.0-477.27.1.el8_8.x86_64 x86_64
[root@db1 ~]#
[root@db1 ~]# yum clean all ; yum repolist
24 files removed
repo id                                                                                   repo name
ol8_UEKR7                                                                                 Latest Unbreakable Enterprise Kernel Release 7 for Oracle Linux 8 (x86_64)
ol8_appstream                                                                             Oracle Linux 8 Application Stream (x86_64)
ol8_baseos_latest                                                                         Oracle Linux 8 BaseOS Latest (x86_64)
[root@db1 ~]#

[root@db1 ~]# yum install -y oracleasm
Oracle Linux 8 BaseOS Latest (x86_64)                                                                                                                                                                         31 MB/s |  64 MB     00:02
Oracle Linux 8 Application Stream (x86_64)                                                                                                                                                                    23 MB/s |  49 MB     00:02
Latest Unbreakable Enterprise Kernel Release 7 for Oracle Linux 8 (x86_64)                                                                                                                                    23 MB/s |  23 MB     00:01
Last metadata expiration check: 0:00:09 ago on Fri 13 Oct 2023 10:10:41 PM EDT.
Package kernel-uek-core-5.15.0-3.60.5.1.el8uek.x86_64 is already installed.
Dependencies resolved.
=============================================================================================================================================================================================================================================
 Package                                                       Architecture                                   Version                                                        Repository                                                 Size
=============================================================================================================================================================================================================================================
Installing:
 kmod-redhat-oracleasm                                         x86_64                                         8:2.0.8-17.0.2.el8                                             ol8_baseos_latest                                          45 k
Installing dependencies:
 kernel-core                                                   x86_64                                         4.18.0-477.27.1.el8_8                                          ol8_baseos_latest                                          42 M

Transaction Summary
=============================================================================================================================================================================================================================================
Install  2 Packages

Total download size: 42 M
Installed size: 70 M
Downloading Packages:
(1/2): kernel-core-4.18.0-477.27.1.el8_8.x86_64.rpm                                   34 MB/s |  42 MB     00:01
(2/2): kmod-redhat-oracleasm-2.0.8-17.0.2.el8.x86_64.rpm                              29 kB/s |  45 kB     00:01
---------------------------------------------------------------------------------------------------------------------
Total                                                                                 26 MB/s |  42 MB     00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                             1/1
  Installing       : kernel-core-4.18.0-477.27.1.el8_8.x86_64                                                    1/2
  Running scriptlet: kernel-core-4.18.0-477.27.1.el8_8.x86_64                                                    1/2
  Installing       : kmod-redhat-oracleasm-8:2.0.8-17.0.2.el8.x86_64                                             2/2
  Running scriptlet: kmod-redhat-oracleasm-8:2.0.8-17.0.2.el8.x86_64                                             2/2
  Running scriptlet: kernel-core-4.18.0-477.27.1.el8_8.x86_64                                                    2/2
  Running scriptlet: kmod-redhat-oracleasm-8:2.0.8-17.0.2.el8.x86_64                                             2/2
  Verifying        : kernel-core-4.18.0-477.27.1.el8_8.x86_64                                                    1/2
  Verifying        : kmod-redhat-oracleasm-8:2.0.8-17.0.2.el8.x86_64                                             2/2

Installed:
  kernel-core-4.18.0-477.27.1.el8_8.x86_64                                                                          kmod-redhat-oracleasm-8:2.0.8-17.0.2.el8.x86_64

Complete!
[root@db1 ~]#


[root@db1 ~]# rpm -qa | grep -i oracleasm
kmod-redhat-oracleasm-2.0.8-17.0.2.el8.x86_64
[root@db1 ~]#


[root@db1 ~]# yum install -y oracleasm-support
Last metadata expiration check: 0:10:32 ago on Fri 13 Oct 2023 10:10:41 PM EDT.
No match for argument: oracleasm-support
Error: Unable to find a match: oracleasm-support
[root@db1 ~]#

[root@db1 ~]# yum install oracleasmlib
Last metadata expiration check: 0:10:59 ago on Fri 13 Oct 2023 10:10:41 PM EDT.
No match for argument: oracleasmlib
Error: Unable to find a match: oracleasmlib
[root@db1 ~]#

[root@db1 ~]# wget https://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.17-1.el8.x86_64.rpm
--2023-10-13 22:02:58--  https://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.17-1.el8.x86_64.rpm
Resolving download.oracle.com (download.oracle.com)... 119.56.0.22
Connecting to download.oracle.com (download.oracle.com)|119.56.0.22|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 27092 (26K) [application/x-redhat-package-manager]
Saving to: ‘oracleasmlib-2.0.17-1.el8.x86_64.rpm’

oracleasmlib-2.0.17-1.el8.x86_64.rpm                        100%[========================================================================================================================================>]  26.46K  --.-KB/s    in 0.002s

2023-10-13 22:02:59 (13.1 MB/s) - ‘oracleasmlib-2.0.17-1.el8.x86_64.rpm’ saved [27092/27092]

[root@db1 ~]# wget https://public-yum.oracle.com/repo/OracleLinux/OL8/addons/x86_64/getPackage/oracleasm-support-2.1.12-1.el8.x86_64.rpm
--2023-10-13 22:04:10--  https://public-yum.oracle.com/repo/OracleLinux/OL8/addons/x86_64/getPackage/oracleasm-support-2.1.12-1.el8.x86_64.rpm
Resolving public-yum.oracle.com (public-yum.oracle.com)... 119.56.0.210, 2401:7400:8888:18d::2a7d, 2401:7400:8888:184::2a7d
Connecting to public-yum.oracle.com (public-yum.oracle.com)|119.56.0.210|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 99852 (98K) [application/x-rpm]
Saving to: ‘oracleasm-support-2.1.12-1.el8.x86_64.rpm’

oracleasm-support-2.1.12-1.el8.x86_64.rpm                   100%[========================================================================================================================================>]  97.51K  --.-KB/s    in 0.007s

2023-10-13 22:04:11 (13.7 MB/s) - ‘oracleasm-support-2.1.12-1.el8.x86_64.rpm’ saved [99852/99852]

[root@db1 ~]#
[root@db1 ~]# ls -ltr *.rpm
-rw-r--r--. 1 root root 99852 Feb 28  2020 oracleasm-support-2.1.12-1.el8.x86_64.rpm
-rw-r--r--. 1 root root 27092 Jun  1  2020 oracleasmlib-2.0.17-1.el8.x86_64.rpm
[root@db1 ~]#
[root@db1 ~]#

[root@db1 ~]# yum localinstall ./oracleasm-support-2.1.12-1.el8.x86_64.rpm ./oracleasmlib-2.0.17-1.el8.x86_64.rpm
Last metadata expiration check: 0:13:19 ago on Fri 13 Oct 2023 10:10:41 PM EDT.
Dependencies resolved.
=============================================================================================================================================================================================================================================
 Package                                                       Architecture                                       Version                                                     Repository                                                Size
=============================================================================================================================================================================================================================================
Installing:
 oracleasm-support                                             x86_64                                             2.1.12-1.el8                                                @commandline                                              98 k
 oracleasmlib                                                  x86_64                                             2.0.17-1.el8                                                @commandline                                              26 k

Transaction Summary
=============================================================================================================================================================================================================================================
Install  2 Packages

Total size: 124 k
Installed size: 400 k
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                     1/1
  Installing       : oracleasmlib-2.0.17-1.el8.x86_64                                                                                                                                                                                    1/2
  Installing       : oracleasm-support-2.1.12-1.el8.x86_64                                                                                                                                                                               2/2
  Running scriptlet: oracleasm-support-2.1.12-1.el8.x86_64                                                                                                                                                                               2/2
Note: Forwarding request to 'systemctl enable oracleasm.service'.
Synchronizing state of oracleasm.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable oracleasm
Created symlink /etc/systemd/system/multi-user.target.wants/oracleasm.service → /usr/lib/systemd/system/oracleasm.service.

  Verifying        : oracleasm-support-2.1.12-1.el8.x86_64                                                                                                                                                                               1/2
  Verifying        : oracleasmlib-2.0.17-1.el8.x86_64                                                                                                                                                                                    2/2

Installed:
  oracleasm-support-2.1.12-1.el8.x86_64                                                                                   oracleasmlib-2.0.17-1.el8.x86_64

Complete!
[root@db1 ~]#

[root@db1 ~]# rpm -qa | grep -i oracleasm
oracleasmlib-2.0.17-1.el8.x86_64
kmod-redhat-oracleasm-2.0.8-17.0.2.el8.x86_64
oracleasm-support-2.1.12-1.el8.x86_64
[root@db1 ~]#


5. Configure Oracle ASMLib

[root@db1 ~]# /usr/sbin/oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting  without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
[root@db1 ~]#

[root@db1 ~]# /usr/sbin/oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": failed
Unable to load module "oracleasm"
Mounting ASMlib driver filesystem: failed
Unable to mount ASMlib driver filesystem
[root@db1 ~]#

*** Please consult with the OS Administrator. The ASM library does not load with the current kernel version. I simply want to modify the default kernel boot order. Before making kernel changes, please consult with Oracle and Redhat support.

[root@db1 ~]# ls -l /boot/vmlinuz-*
-rwxr-xr-x. 1 root root 13287760 Feb 24  2023 /boot/vmlinuz-0-rescue-93eed19701864c2cac4193714f746de8
-rwxr-xr-x. 1 root root 10744352 Nov  8  2022 /boot/vmlinuz-4.18.0-425.3.1.el8.x86_64
-rwxr-xr-x. 1 root root 10851840 Sep 27 14:23 /boot/vmlinuz-4.18.0-477.27.1.el8_8.x86_64
-rwxr-xr-x. 1 root root 13287760 Oct 19  2022 /boot/vmlinuz-5.15.0-3.60.5.1.el8uek.x86_64
[root@db1 ~]#
[root@db1 ~]# grubby --set-default /boot/vmlinuz-4.18.0-477.27.1.el8_8.x86_64
The default is /boot/loader/entries/93eed19701864c2cac4193714f746de8-4.18.0-477.27.1.el8_8.x86_64.conf with index 1 and kernel /boot/vmlinuz-4.18.0-477.27.1.el8_8.x86_64
[root@db1 ~]# 

[root@db1 ~]# shutdown -Fr now


[root@db1 ~]# /usr/sbin/oracleasm init

[root@db1 ~]# /usr/sbin/oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes
[root@db1 ~]#


6. Create ASM disks

[root@db1 ~]# /usr/sbin/oracleasm createdisk DISK1 /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@db1 ~]# /usr/sbin/oracleasm createdisk DISK2 /dev/sdc1
Writing disk header: done
Instantiating disk: done
[root@db1 ~]# /usr/sbin/oracleasm createdisk DISK3 /dev/sdd1
Writing disk header: done
Instantiating disk: done
[root@db1 ~]# /usr/sbin/oracleasm createdisk DISK4 /dev/sde1
Writing disk header: done
Instantiating disk: done
[root@db1 ~]# /usr/sbin/oracleasm createdisk DISK5 /dev/sdf1
Writing disk header: done
Instantiating disk: done
[root@db1 ~]#

[root@db1 ~]# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
[root@db1 ~]#
[root@db1 ~]# /usr/sbin/oracleasm listdisks
DISK1
DISK2
DISK3
DISK4
DISK5
[root@db1 ~]#


7. List oracleasm disks

[root@db1 ~]# ls -ld /dev/sd*1
brw-rw----. 1 root disk 8,  1 Oct 14 06:32 /dev/sda1
brw-rw----. 1 root disk 8, 17 Oct 14 07:13 /dev/sdb1
brw-rw----. 1 root disk 8, 33 Oct 14 07:13 /dev/sdc1
brw-rw----. 1 root disk 8, 49 Oct 14 07:13 /dev/sdd1
brw-rw----. 1 root disk 8, 65 Oct 14 07:13 /dev/sde1
brw-rw----. 1 root disk 8, 81 Oct 14 07:13 /dev/sdf1
[root@db1 ~]#

[root@db1 ~]# ls -ltra /dev/oracleasm/disks/*
brw-rw----. 1 grid asmadmin 8, 17 Oct 14 07:11 /dev/oracleasm/disks/DISK1
brw-rw----. 1 grid asmadmin 8, 33 Oct 14 07:12 /dev/oracleasm/disks/DISK2
brw-rw----. 1 grid asmadmin 8, 49 Oct 14 07:12 /dev/oracleasm/disks/DISK3
brw-rw----. 1 grid asmadmin 8, 65 Oct 14 07:12 /dev/oracleasm/disks/DISK4
brw-rw----. 1 grid asmadmin 8, 81 Oct 14 07:12 /dev/oracleasm/disks/DISK5
[root@db1 ~]#

[root@db1 ~]# systemctl status oracleasm
● oracleasm.service - Load oracleasm Modules
   Loaded: loaded (/usr/lib/systemd/system/oracleasm.service; enabled; vendor preset: disabled)
   Active: active (exited) since Sat 2023-10-14 06:32:29 EDT; 2h 55min ago
  Process: 852 ExecStart=/usr/sbin/oracleasm.init start_sysctl (code=exited, status=0/SUCCESS)
 Main PID: 852 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 17392)
   Memory: 0B
   CGroup: /system.slice/oracleasm.service

Oct 14 06:32:27 db1.rajasekhar.com systemd[1]: Starting Load oracleasm Modules...
Oct 14 06:32:28 db1.rajasekhar.com oracleasm.init[852]: Initializing the Oracle ASMLib driver: OK
Oct 14 06:32:29 db1.rajasekhar.com oracleasm.init[852]: Scanning the system for Oracle ASMLib disks: OK
Oct 14 06:32:29 db1.rajasekhar.com systemd[1]: Started Load oracleasm Modules.
[root@db1 ~]#

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

Reference:

Oracle Linux 8: How To Install ASMLib (Doc ID 2720215.1)
Oracle ASMLib Software Update and Support Policy (Doc ID 1089399.1)
https://docs.oracle.com/en/database/oracle/oracle-database/12.2/ladbi/installing-and-configuring-oracle-asmlib-software.html#GUID-79F9D58F-E5BB-45BD-A664-260C0502D876
https://www.oracle.com/linux/downloads/linux-asmlib-v8-downloads.html

Configure UDEV Rules for Oracle ASM

Configure UDEV Rules for Oracle ASM

Table of Contents
___________________________________________________________________________________________________

Pre-requisites:

a. Create OS user and groups for Oracle ASM disk owner
b. Add RAW disks to server (Check with sysadmin)

Configure UDEV Rules for Oracle ASM:

1. List the disks
2. Create partitions for the disks
3. Load updated block device partition tables
4. Find SCSI ID
5. Create udev rules
6. Reload the udev rules
7. List oracleasm disks

___________________________________________________________________________________________________


Pre-requisites


a. Create OS user and groups for Oracle ASM disk owner

https://www.br8dba.com/create-users-groups-and-paths-for-oracle-rac/


b. Add RAW disks to server

Operating SystemRed Hat Enterprise Linux release 8.7
Storage
/dev/sda100G for Linux and others
/dev/sdb100G for /u01
/dev/sdc100G for /orabackup
/dev/sdd100G for ASM DISK
/dev/sde100G for ASM DISK


Configure UDEV Rules for Oracle ASM

If ASMLIB kernel drivers are not available then we have to use udev rules to create the disks for Oracle ASM.

Setting up Oracle ASM udev rules is not so complicated. All you need is the udevadm command and editing one file.


1. List the disks

[root@testbox ~]# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    0  100G  0 disk
├─sda1        8:1    0    1G  0 part /boot
└─sda2        8:2    0   99G  0 part
  ├─ol-root 252:0    0 61.2G  0 lvm  /
  ├─ol-swap 252:1    0  7.9G  0 lvm  [SWAP]
  └─ol-home 252:2    0 29.9G  0 lvm  /home
sdb           8:16   0  100G  0 disk
└─sdb1        8:17   0  100G  0 part /u01
sdc           8:32   0  100G  0 disk
└─sdc1        8:33   0  100G  0 part /orabackup
sdd           8:48   0  100G  0 disk
sde           8:64   0  100G  0 disk
sr0          11:0    1 47.3M  0 rom
[root@testbox ~]#


2. Create partitions for the disks

fdisk /dev/sdd
fdisk /dev/sde

[root@testbox ~]# fdisk /dev/sdd

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x8c782d71.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-209715199, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199):

Created a new partition 1 of type 'Linux' and of size 100 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

[root@testbox ~]#
[root@testbox ~]# fdisk /dev/sde

Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x36462c72.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-209715199, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199):

Created a new partition 1 of type 'Linux' and of size 100 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

[root@testbox ~]#


3. Load updated block device partition tables

# For Linux 5,6 and 7

# /sbin/partprobe /dev/sdd1
# /sbin/partprobe /dev/sde1

# For Linux8

[root@testbox ~]# /sbin/partx -u /dev/sdd1
[root@testbox ~]#
[root@testbox ~]# /sbin/partx -u /dev/sde1
[root@testbox ~]#


4. Find SCSI ID

[root@testbox ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdd
1ATA_VBOX_HARDDISK_VB0adc00d9-c5938e95
[root@testbox ~]#

[root@testbox ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sde
1ATA_VBOX_HARDDISK_VBdaa5e829-52e4b9b1
[root@testbox ~]#


5. Create udev rules

[root@testbox ~]# ls -ltr /etc/udev/rules.d
total 12
-rw-r--r--. 1 root root  67 Oct  2 18:03 69-vdo-start-by-dev.rules
-rw-r--r--. 1 root root 148 Nov  9 06:11 99-vmware-scsi-timeout.rules
-rw-r--r--. 1 root root 134 Apr  1 07:52 60-vboxadd.rules
[root@testbox ~]#

vi /etc/udev/rules.d/99-oracle-asmdevices.rules
and below lines and then save it.

KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB0adc00d9-c5938e95", SYMLINK+="oracleasm/disks/DISK01", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VBdaa5e829-52e4b9b1", SYMLINK+="oracleasm/disks/DISK02", OWNER="grid", GROUP="asmadmin", MODE="0660"


[root@testbox ~]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB0adc00d9-c5938e95", SYMLINK+="oracleasm/disks/DISK01", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VBdaa5e829-52e4b9b1", SYMLINK+="oracleasm/disks/DISK02", OWNER="grid", GROUP="asmadmin", MODE="0660"
[root@testbox ~]#

[root@testbox ~]# ls -ltr /etc/udev/rules.d/99-oracle-asmdevices.rules
-rw-r--r--. 1 root root 428 Apr  2 02:18 /etc/udev/rules.d/99-oracle-asmdevices.rules
[root@testbox ~]#


6. Reload the udev rules

The below commands will reload the complete udev configuration and will trigger all the udev rules. 
On a busy production system this could disrupt ongoing operations, applications running on the server. Please use the below command during a scheduled maintenance window only.

[root@testbox ~]# /sbin/udevadm control --reload-rules
[root@testbox ~]#
[root@testbox ~]# ls -ld /dev/sd*1
brw-rw----. 1 root disk 8,  1 Apr  2 02:23 /dev/sda1
brw-rw----. 1 root disk 8, 17 Apr  2 02:23 /dev/sdb1
brw-rw----. 1 root disk 8, 33 Apr  2 02:23 /dev/sdc1
brw-rw----. 1 root disk 8, 49 Apr  2 02:23 /dev/sdd1
brw-rw----. 1 root disk 8, 65 Apr  2 02:23 /dev/sde1
[root@testbox ~]#
[root@testbox ~]# /sbin/udevadm trigger
[root@testbox ~]#
[root@testbox ~]# ls -ld /dev/sd*1
brw-rw----. 1 root disk     8,  1 Apr  2 02:34 /dev/sda1
brw-rw----. 1 root disk     8, 17 Apr  2 02:34 /dev/sdb1
brw-rw----. 1 root disk     8, 33 Apr  2 02:34 /dev/sdc1
brw-rw----. 1 grid asmadmin 8, 49 Apr  2 02:34 /dev/sdd1
brw-rw----. 1 grid asmadmin 8, 65 Apr  2 02:34 /dev/sde1
[root@testbox ~]#


7. List the oracleasm disks

[root@testbox ~]# ls -ltra /dev/oracleasm/disks/*
lrwxrwxrwx. 1 root root 10 Apr  2 02:34 /dev/oracleasm/disks/DISK01 -> ../../sdd1
lrwxrwxrwx. 1 root root 10 Apr  2 02:34 /dev/oracleasm/disks/DISK02 -> ../../sde1
[root@testbox ~]#

[root@testbox ~]# ls -ld /dev/sd*1
brw-rw----. 1 root disk     8,  1 Apr  2 02:34 /dev/sda1
brw-rw----. 1 root disk     8, 17 Apr  2 02:34 /dev/sdb1
brw-rw----. 1 root disk     8, 33 Apr  2 02:34 /dev/sdc1
brw-rw----. 1 grid asmadmin 8, 49 Apr  2 02:34 /dev/sdd1
brw-rw----. 1 grid asmadmin 8, 65 Apr  2 02:34 /dev/sde1
[root@testbox ~]#

Note: symboliclinks are owned by root, but devices will be owned by grid:asmadmin

 

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Thank you,
Rajasekhar Amudala
Email: br8dba@gmail.com
Linkedin: https://www.linkedin.com/in/rajasekhar-amudala/

CSSD won’t start automatically

Cause

After server reboot, CSSD and ASM are not came up automatically.

Data Collection

[oracle@Linux4 ~]$ ps -ef | grep d.bin
oracle    4233     1  0 09:22 ?        00:00:03 /oracle11g/app/11.2.0/oracle/grid/bin/ohasd.bin reboot
oracle    4794     1  0 09:23 ?        00:00:01 /oracle11g/app/11.2.0/oracle/grid/bin/oraagent.bin
oracle    4810     1  0 09:23 ?        00:00:00 /oracle11g/app/11.2.0/oracle/grid/bin/evmd.bin
oracle    4812     1  0 09:23 ?        00:00:00 /oracle11g/app/11.2.0/oracle/grid/bin/tnslsnr LISTENER -inherit
oracle    4855  4810  0 09:23 ?        00:00:00 /oracle11g/app/11.2.0/oracle/grid/bin/evmlogger.bin -o 

/oracle11g/app/11.2.0/oracle/grid/evm/log/evmlogger.info -l /oracle11g/app/11.2.0/oracle/grid/evm/log/evmlogger.log
oracle    4942  4748  0 09:38 pts/1    00:00:00 grep d.bin
[oracle@Linux4 ~]$

[oracle@Linux4 ~]$ crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               OFFLINE OFFLINE      linux4
ora.LISTENER.lsnr
               ONLINE  ONLINE       linux4
ora.asm
               OFFLINE OFFLINE      linux4                   Instance Shutdown <----
ora.ons
               OFFLINE OFFLINE      linux4
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  OFFLINE <-----
ora.diskmon
      1        OFFLINE OFFLINE
ora.evmd
      1        ONLINE  ONLINE       linux4
[oracle@Linux4 ~]$

[oracle@Linux4 ~]$ crsctl stat res ora.cssd -p
NAME=ora.cssd
TYPE=ora.cssd.type
ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=
ACTIVE_PLACEMENT=0
AGENT_FILENAME=%CRS_HOME%/bin/cssdagent%CRS_EXE_SUFFIX%
AGENT_HB_INTERVAL=0
AGENT_HB_MISCOUNT=10
AUTO_START=never <----
CARDINALITY=1
CHECK_ARGS=
CHECK_COMMAND=
CHECK_INTERVAL=30
CLEAN_ARGS=abort
CLEAN_COMMAND=
CSSD_MODE=
CSSD_PATH=%CRS_HOME%/bin/ocssd%CRS_EXE_SUFFIX%
CSS_USER=oracle
DAEMON_LOGGING_LEVELS=CSSD=2,GIPCNM=2,GIPCGM=2,GIPCCM=2,CLSF=0,SKGFD=0,GPNP=1,OLR=0
DAEMON_TRACING_LEVELS=CSSD=0,GIPCNM=0,GIPCGM=0,GIPCCM=0,CLSF=0,SKGFD=0,GPNP=0,OLR=0
DEFAULT_TEMPLATE=
DEGREE=1
DESCRIPTION="Resource type for CSSD"
DETACHED=true
ENABLED=1
ENV_OPTS=
FAILOVER_DELAY=0
FAILURE_INTERVAL=3
FAILURE_THRESHOLD=5
HOSTING_MEMBERS=
INTERVAL_FS=3
LOAD=1
LOGGING_LEVEL=1
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
OMON_INITRATE=1000
OMON_POLLRATE=500
ORA_OPROCD_MODE=
ORA_VERSION=11.2.0.4.0
PID_FILE=
PLACEMENT=balanced
PROCD_TIMEOUT=1000
PROCESS_TO_MONITOR=
PROFILE_CHANGE_TEMPLATE=
REBOOT_OPTS=
RESTART_ATTEMPTS=5
SCRIPT_TIMEOUT=600
SERVER_POOLS=
START_ARGS=
START_COMMAND=
START_DEPENDENCIES=weak(concurrent:ora.diskmon)
START_TIMEOUT=600
STATE_CHANGE_TEMPLATE=
STOP_ARGS=
STOP_COMMAND=
STOP_DEPENDENCIES=hard(shutdown:ora.diskmon)
STOP_TIMEOUT=900
TIMEOUT_FS=0
UNRESPONSIVE_TIMEOUT=180
UPTIME_THRESHOLD=1m
USR_ORA_ENV=
VMON_INITLIMIT=16
VMON_INITRATE=500
VMON_POLLRATE=500

[oracle@Linux4 ~]$

[oracle@Linux4 ~]$ crsctl stat res ora.asm -p
NAME=ora.asm
TYPE=ora.asm.type
ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=
AGENT_FILENAME=%CRS_HOME%/bin/oraagent%CRS_EXE_SUFFIX%
ALIAS_NAME=
ASM_DISKSTRING=/dev/oracleasm/disks/*
AUTO_START=restore <----
CHECK_INTERVAL=1
CHECK_TIMEOUT=30
DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=asm) ELEMENT(INSTANCE_NAME= %GEN_USR_ORA_INST_NAME%)
DEGREE=1
DESCRIPTION=Oracle ASM resource
ENABLED=1
GEN_USR_ORA_INST_NAME=+ASM
LOAD=1
LOGGING_LEVEL=1
NLS_LANG=
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
PROFILE_CHANGE_TEMPLATE=
RESTART_ATTEMPTS=5
SCRIPT_TIMEOUT=60
SPFILE=+DATA/asm/asmparameterfile/registry.253.894413045
START_DEPENDENCIES=hard(ora.cssd) weak(ora.LISTENER.lsnr)
START_TIMEOUT=900
STATE_CHANGE_TEMPLATE=
STOP_DEPENDENCIES=hard(ora.cssd)
STOP_TIMEOUT=600
TYPE_VERSION=1.2
UPTIME_THRESHOLD=1d
USR_ORA_ENV=
USR_ORA_INST_NAME=+ASM
USR_ORA_OPEN_MODE=mount
USR_ORA_OPI=false
USR_ORA_STOP_MODE=immediate
VERSION=11.2.0.4.0

[oracle@Linux4 ~]$

Action Plan

Start the resources manually

[oracle@Linux4 ~]$ crsctl start res ora.cssd -init
CRS-2672: Attempting to start 'ora.cssd' on 'linux4'
CRS-2672: Attempting to start 'ora.diskmon' on 'linux4'
CRS-2676: Start of 'ora.diskmon' on 'linux4' succeeded
CRS-2676: Start of 'ora.cssd' on 'linux4' succeeded
[oracle@Linux4 ~]$
[oracle@Linux4 ~]$ ps -ef | grep cssd.bin
oracle    5184     1  0 10:19 ?        00:00:00 /oracle11g/app/11.2.0/oracle/grid/bin/ocssd.bin
oracle    5207  4748  0 10:20 pts/1    00:00:00 grep cssd.bin
[oracle@Linux4 ~]$
[oracle@Linux4 ~]$ srvctl start asm
[oracle@Linux4 ~]$
[oracle@Linux4 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       linux4
ora.LISTENER.lsnr
               ONLINE  ONLINE       linux4
ora.asm
               ONLINE  ONLINE       linux4                   Started
ora.ons
               OFFLINE OFFLINE      linux4
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       linux4
ora.diskmon
      1        OFFLINE OFFLINE
ora.evmd
      1        ONLINE  ONLINE       linux4
[oracle@Linux4 ~]$

We need to change the auto_start attribute of a resource to value ‘always’ or ‘1’ to start resource automatically after server reboot.

Run below commands as GI home owner.

[oracle@Linux4 ~]$ crsctl modify resource "ora.cssd" -attr "AUTO_START=always"
[oracle@Linux4 ~]$ crsctl modify resource "ora.asm" -attr "AUTO_START=always"
[oracle@Linux4 ~]$

[oracle@Linux4 ~]$ crsctl stat res ora.asm -p | grep "AUTO_START"
AUTO_START=always
[oracle@Linux4 ~]$ crsctl stat res ora.cssd -p | grep "AUTO_START"
AUTO_START=always
[oracle@Linux4 ~]$

Additional Information:

always (1)
 Causes the resource to restart when the node restarts regardless of the resource’s state when the node stopped.

restore (0)
 Does not start the resource at restart time if it was in an offline state, such as STATE=OFFLINE, TARGET=OFFLINE, when the node stopped. The 

resource is restored to its state when the node went down. The resource is started only if it was online before and not otherwise.

never (2)
 Oracle Clusterware never restarts the resource regardless of the resource’s state when the node is stopped.

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Instantiating disk: failed

Instantiating disk: failed while creating ASM disk.

Issue

[root@Linux4 ~]# oracleasm createdisk disk1 /dev/sdb1
Writing disk header: done
Instantiating disk: failed <----
Clearing disk header: done
[root@Linux4 ~]# 

Action Plan

[root@Linux4 ~]# /etc/init.d/oracleasm disable
Writing Oracle ASM library driver configuration: done
Dropping Oracle ASMLib disks:                              [  OK  ]
Shutting down the Oracle ASMLib driver:                    [  OK  ]
[root@Linux4 ~]#
[root@Linux4 ~]# /etc/init.d/oracleasm enable
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]
[root@Linux4 ~]#
[root@Linux4 ~]# oracleasm createdisk disk1 /dev/sdb1
Writing disk header: done
Instantiating disk: done <---

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

Move ASM Spfile

How to move ASM SPFILE from +DATA to +VOTE diskgroup

Contents
___________________________________________________________________________________________________________________________________

1. Verify Available DiskGroups
2. Verify Current ASM SP File location
3. Create PFILE from SPFILE
4. Create SPFILE from PFILE
5. Stop CRS All Nodes
6. Start CRS All Nodes
7. Verify ASM SPFILE Location
___________________________________________________________________________________________________________________________________


1. Verify Available DiskGroups

ASMCMD> lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  NORMAL  N         512   4096  1048576      2940     2110              980             565              0             N  DATA/
MOUNTED  EXTERN  N         512   4096  1048576      7295       13                0              13              0             N  DATA1/
MOUNTED  EXTERN  N         512   4096  1048576      1019      892                0             892              0             Y  VOTE/
ASMCMD>


2. Verify Current ASM SP File location

[oracle@rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Fri Aug 14 03:47:52 2015

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> show parameter pfile

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +DATA/rac-scan/asmparameterfil
                                                 e/registry.253.833600843
SQL>


3. Create PFILE from SPFILE

SQL> create pfile='/home/oracle/initASM.ora' from spfile;

File created.

SQL>
SQL> !ls -ltr /home/oracle/initASM.ora
-rw-r--r-- 1 oracle oinstall 300 Aug 14 03:49 /home/oracle/initASM.ora

SQL>


4. Create SPFILE from PFILE

SQL> create spfile='+VOTE' from pfile='/home/oracle/initASM.ora';

File created.

SQL>

ASMCMD> pwd
+vote/rac-scan/ASMPARAMETERFILE
ASMCMD>
ASMCMD> ls -l
Type              Redund  Striped  Time             Sys  Name
ASMPARAMETERFILE  UNPROT  COARSE   AUG 14 03:00:00  Y    REGISTRY.253.887687509 <----
ASMCMD>


5. Stop CRS All Nodes

From Node 1

[root@rac1 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac1'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.VOTE.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.DATA1.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac1'
CRS-2677: Stop of 'ora.scan1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac2'
CRS-2677: Stop of 'ora.rac1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.rac1.vip' on 'rac2'
CRS-2677: Stop of 'ora.DATA1.dg' on 'rac1' succeeded
CRS-2676: Start of 'ora.rac1.vip' on 'rac2' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'rac1' succeeded
CRS-2677: Stop of 'ora.VOTE.dg' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac1'
CRS-2677: Stop of 'ora.ons' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac1'
CRS-2677: Stop of 'ora.net1.network' on 'rac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac1' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac1'
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'rac1'
CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac1 ~]#

From Node 2

[root@rac2 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac2'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.oc4j' on 'rac2'
CRS-2673: Attempting to stop 'ora.cvu' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.scan3.vip' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac2'
CRS-2677: Stop of 'ora.scan3.vip' on 'rac2' succeeded
CRS-2677: Stop of 'ora.scan2.vip' on 'rac2' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.rac2.vip' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.VOTE.dg' on 'rac2'
CRS-2673: Attempting to stop 'ora.DATA1.dg' on 'rac2'
CRS-2677: Stop of 'ora.rac2.vip' on 'rac2' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac2'
CRS-2677: Stop of 'ora.rac1.vip' on 'rac2' succeeded
CRS-2677: Stop of 'ora.DATA1.dg' on 'rac2' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'rac2' succeeded
CRS-2677: Stop of 'ora.cvu' on 'rac2' succeeded
CRS-2677: Stop of 'ora.VOTE.dg' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac2'
CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac2'
CRS-2677: Stop of 'ora.ons' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac2'
CRS-2677: Stop of 'ora.net1.network' on 'rac2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac2' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2'
CRS-2673: Attempting to stop 'ora.crf' on 'rac2'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac2'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac2'
CRS-2673: Attempting to stop 'ora.asm' on 'rac2'
CRS-2677: Stop of 'ora.mdnsd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.crf' on 'rac2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac2'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac2'
CRS-2677: Stop of 'ora.cssd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2'
CRS-2677: Stop of 'ora.gipcd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2'
CRS-2677: Stop of 'ora.gpnpd' on 'rac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
You have new mail in /var/spool/mail/root
[root@rac2 ~]#


6. Start CRS All Nodes

From Node 1

[root@rac1 ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac1 ~]#

From Node 2

[root@rac2 ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac2 ~]#
[root@rac1 ~]# crsctl check cluster -all
**************************************************************
rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
rac2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[root@rac1 ~]#


7. Verify ASM SPFILE Location

[oracle@rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Fri Aug 14 04:07:03 2015

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> show parameter pfile

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +VOTE/rac-scan/asmparameterfil
                                                 e/registry.253.887687509
SQL>

Caution: Your use of any information or materials on this website is entirely at your own risk. It is provided for educational purposes only. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

ASM Commands

1. ASMLIB commands
2. Start/Stop CSSD (Non-RAC)

1. ASMLIB commands


Most commonly used asmlib commands
 
/etc/init.d/oracleasm start
/etc/init.d/oracleasm stop
/etc/init.d/oracleasm restart
/etc/init.d/oracleasm configure
/etc/init.d/oracleasm status
/etc/init.d/oracleasm enable
/etc/init.d/oracleasm disable
/etc/init.d/oracleasm deletedisk
/etc/init.d/oracleasm createdisk DATA /dev/sdb1
/etc/init.d/oracleasm querydisk /dev/sdb1
/etc/init.d/oracleasm querydisk DATA
/etc/init.d/oracleasm renamedisk /dev/sdb1 VOL1
/etc/init.d/oracleasm force-renamedisk DATA NEW_DATA_DISK1
/etc/init.d/oracleasm scandisks
/etc/init.d/oracleasm listdisks

 
2. Start/Stop CSSD (Non-RAC)

Manually start CSSD (Non-RAC)
/etc/init.d/init.cssd start

Manually stop CSSD (Non-RAC)
/etc/init.d/init.cssd stop

still page under construction !!!