UDEV SCSI Rules Configuration for ASM in Oracle Linux 5 and 6，udevasm
UDEV SCSI Rules Configuration for ASM in Oracle Linux 5 and 6
For Oracle Automatic Storage Manager (ASM) to use disks, it needs to be able to identify the devices consistently and for them to have the correct ownership and permissions. In Linux you can use ASMLib to manage these tasks, but it is seen as an additional layer of complexity and has never really gained any popularity. Instead, many people use the Linux device manager "udev" to perform these tasks. This article presents a brief overview of setting up udev rules with respect to disks for use with ASM in Oracle 11g. The examples are all done using Oracle Linux 5 and 6, so they will be consistent with RHEL and CentOS 5 and 6.
- Background
- Identify the Disks (/sbin/scsi_id)
- Make SCSI Devices Trusted
- Create UDEV Rules File
- Load Updated Block Device Partitions (/sbin/partprobe)
- Test Rules (udevtest)
- Restart UDEV Service
- Check Ownership and Permissions
Background
Essentially, what udev does is apply rules defined in files in the "/etc/udev/rules.d" directory to the device nodes listed in the "/dev" directory. The rules can be defined in a variety of ways, but what we need to do is identify the device and say what we want udev to do with it.
In this case I know all my disk devices are named "/dev/sd?1", where the "?" represents a letter from a-d, so I can identify the devices of interest using the following rule parameters.
KERNEL=="sd?1", BUS=="scsi"
I want to tie each specific device to an alias, so it is always identified the same way, regardless of the device name Linux assigns it. So I need to be able to test each device that matches the previous pattern to see if it is the disk I am interested in. Each disk has a unique SCSI ID, so I can place a test into the rule, telling it how to perform the test, and the result it should return for a positive match. The following rule parameters explain how to test the device and what result constitutes a match in Oracle Linux 5.
PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_"
The scsi_id command works a little differently in Oracle Linux 6, so for that the following test works better.
PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_"
Once we have identified the specific device of interest, we need to indicate what actions should be performed on it. The following parameters specify an alias, the ownership and the permissions for the device.
NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
So the whole rule for each disk will look something like this in Oracle Linux 5.
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
Or this in Oracle Linux 6.
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
This means that the device pointing to the partition "sd*1" on the disk with the SCSI ID of "SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_" will always be called "/dev/asm-disk1", regardless of the letter "?" Linux assigns when the device is discovered. In addition, the device will have the correct ownership and permissions for ASM.
There are a number of wildcards and matching patterns that can be used if you don't want to write device-specific rules.
Now we know roughly what we are trying to achieve, we will look at each step necessary for setting up the disks for ASM to use.
Identify the Disks (/sbin/scsi_id)
We are going to write device-specific rules, so we need to be able to identify each device consistently, irrespective of the order in which Linux discovers it. To do this we are going to use the SCSI ID for each disk (not the partition), which we get using
the scsi_id command. The "-s" option makes the paths relative to the "/sys" directory. For Oracle Linux 5, use the following command.
The "-s" is not available in Oracle Linux 6, so you must use the following syntax.# /sbin/scsi_id -g -u -s /block/sdb SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_ # /sbin/scsi_id -g -u -s /block/sdc SATA_VBOX_HARDDISK_VB46dec7e0-192e8000_ # /sbin/scsi_id -g -u -s /block/sdd SATA_VBOX_HARDDISK_VBce8c63bb-ac67a172_ # /sbin/scsi_id -g -u -s /block/sde SATA_VBOX_HARDDISK_VB7437a3b7-95b199cd_ #
# /sbin/scsi_id -g -u -d /dev/sdb SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_ # /sbin/scsi_id -g -u -d /dev/sdc SATA_VBOX_HARDDISK_VB46dec7e0-192e8000_ # /sbin/scsi_id -g -u -d /dev/sdd SATA_VBOX_HARDDISK_VBce8c63bb-ac67a172_ # /sbin/scsi_id -g -u -d /dev/sde SATA_VBOX_HARDDISK_VB7437a3b7-95b199cd_ #
Make SCSI Devices Trusted
Add the following to the "/etc/scsi_id.config" file to configure SCSI devices as trusted. Create the file if it doesn't already exist.
options=-g
Create UDEV Rules File
Create the "/etc/udev/rules.d/99-oracle-asmdevices.rules" file.
# vi /etc/udev/rules.d/99-oracle-asmdevices.rules
The file should contain the following lines for Oracle Linux 5. The PROGRAM parameter must match the command you used to retrieve the SCSI ID, and the
RESULT parameter must match the value returned from your disks.
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660" KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VB46dec7e0-192e8000_", NAME="asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660" KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VBce8c63bb-ac67a172_", NAME="asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660" KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VB7437a3b7-95b199cd_", NAME="asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"
The equivalent for Oracle Linux 6 is shown below.
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660" KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VB46dec7e0-192e8000_", NAME="asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660" KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VBce8c63bb-ac67a172_", NAME="asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660" KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VB7437a3b7-95b199cd_", NAME="asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"
Load Updated Block Device Partitions (/sbin/partprobe)
Load updated block device partition tables.
# /sbin/partprobe /dev/sdb1 # /sbin/partprobe /dev/sdc1 # /sbin/partprobe /dev/sdd1 # /sbin/partprobe /dev/sde1
Test Rules (udevtest)
Test the rules are working as expected.
# #OL5 # udevtest /block/sdb/sdb1 # udevtest /block/sdc/sdc1 # udevtest /block/sdd/sdd1 # udevtest /block/sde/sde1 # #OL6 # udevadm test /block/sdb/sdb1 # udevadm test /block/sdc/sdc1 # udevadm test /block/sdd/sdd1 # udevadm test /block/sde/sde1
The output from the first disk should look something like this.
# udevtest /block/sdb/sdb1 main: looking at device '/block/sdb/sdb1' from subsystem 'block' udev_rules_get_name: add symlink 'disk/by-id/scsi-SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3-part1' udev_rules_get_name: add symlink 'disk/by-path/pci-0000:00:0d.0-scsi-1:0:0:0-part1' run_program: '/lib/udev/vol_id --export /dev/.tmp-8-17' run_program: '/lib/udev/vol_id' returned with status 4 run_program: '/sbin/scsi_id -g -u -s /block/sdb/sdb1' run_program: '/sbin/scsi_id' (stdout) 'SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_' run_program: '/sbin/scsi_id' returned with status 0 udev_rules_get_name: rule applied, 'sdb1' becomes 'asm-disk1' udev_device_event: device '/block/sdb/sdb1' already in database, validate currently present symlinks udev_node_add: creating device node '/dev/asm-disk1', major = '8', minor = '17', mode = '0660', uid = '1100', gid = '1200' udev_node_add: creating symlink '/dev/disk/by-id/scsi-SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3-part1' to '../../asm-disk1' udev_node_add: creating symlink '/dev/disk/by-path/pci-0000:00:0d.0-scsi-1:0:0:0-part1' to '../../asm-disk1' main: run: 'socket:/org/kernel/dm/multipath_event' main: run: 'socket:/org/kernel/udev/monitor' main: run: '/lib/udev/udev_run_devd' main: run: 'socket:/org/freedesktop/hal/udev_event' main: run: '/sbin/pam_console_apply /dev/asm-disk1 /dev/disk/by-id/scsi-SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3-part1 /dev/disk/by-path/pci-0000:00:0d.0-scsi-1:0:0:0-part1' #
Restart UDEV Service
Restart the UDEV service.
# #OL5 # /sbin/udevcontrol reload_rules # #OL6 # udevadm control --reload-rules # #OL5 and OL6 # /sbin/start_udev
Check Ownership and Permissions
Check the disks are now available with the "asm-disk*" alias and the correct ownership and permissions.
# cd /dev # ls -al asm-disk* brw-rw---- 1 oracle dba 8, 17 Apr 8 22:47 asm-disk1 brw-rw---- 1 oracle dba 8, 33 Apr 8 22:47 asm-disk2 brw-rw---- 1 oracle dba 8, 49 Apr 8 22:47 asm-disk3 brw-rw---- 1 oracle dba 8, 65 Apr 8 22:47 asm-disk4 #
So the ASM_DISKSTRING initialization parameter in the ASM instance can be set to '/dev/asm-disk*' to identify the ASM disks.
利用UDEV服务解决RAC ASM存储设备名
在<Why ASMLIB and why not?>我们介绍了使用ASMLIB作为一种专门为Oracle Automatic Storage Management特性设计的内核支持库(kernel support library)的优缺点，同时建议使用成熟的UDEV方案来替代ASMLIB。这里我们就给出配置UDEV的具体步骤，还是比较简单的:1.确认在所有RAC节点上已经安装了必要的UDEV包[root@rh2 ~]# rpm -qa|grep udevudev-095-14.21.el52.通过scsi_id获取设备的块设备的唯一标识名，假设系统上已有LUN sdc-sdpfor i in c d e f g h i j k l m n o p ;doecho "sd$i" "`scsi_id -g -u -s /block/sd$i` ";donesdc 1IET_00010001sdd 1IET_00010002sde 1IET_00010003sdf 1IET_00010004sdg 1IET_00010005sdh 1IET_00010006sdi 1IET_00010007sdj 1IET_00010008sdk 1IET_00010009sdl 1IET_0001000asdm 1IET_0001000bsdn 1IET_0001000csdo 1IET_0001000dsdp 1IET_0001000e 以上列出于块设备名对应的唯一标识名3.创建必要的UDEV配置文件，首先切换到配置文件目录[root@rh2 ~]# cd /etc/udev/rules.d定义必要的规则配置文件[root@rh2 rules.d]# touch 99-oracle-asmdevices.rules [root@rh2 rules.d]# cat 99-oracle-asmdevices.rulesKERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010001", NAME="ocr1", OWNER="grid", GROUP="asmadmin", MODE="0660"KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010002", NAME="ocr2", OWNER="grid", GROUP="asmadmin", MODE="0660"KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010003", NAME=&quo......余下全文>>
正好刚整理了一个这方面的资料,先贴给你吧.
准备ASM环境
ASM使用一个名叫“+ASM”的数据库实例来管理ASM磁盘，因此在配置ASM磁盘之前，需要先启动ASM实例。另外还需要注意，ASM 实例必须要先于数据库实例启动，和数据库实例同步运行，迟于数据库实例关闭。ASM 实例的创建和删除可以用DBCA 工具来操作。在DBCA的第一个界面选择配置自动存储管理就可以进入ASM配置的界面。
根据提示运行脚本就可以配置和启动CSS（Cluster Synchronization Service）了，注意要以root的身份运行这个脚本，运行情况如下：
# /u01/app/oracle/product/10.2.0/db_1/bin/localconfig add
/etc/oracle does not exist. Creating it now.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configuration for local CSS has been initialized
Adding to inittab
Startup will be queued to init within 90 seconds.
Checking the status of new Oracle init process...
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
s1
CSS is active on all nodes.
Oracle CSS service is installed and running under init(1M)
ASM可以使用裸设备或者ASMLib方式， 因为裸设备的维护更罗嗦一些，本文只讨论ASMLib方式。为了在Linux系统中使用ASMLib方式准备ASM磁盘，需要安装相关的软件，下载链接如下：
www.oracle.com/...x.html
下载时注意选择自己的操作系统和内核的版本，我下载到的是以下三个软件：
oracleasm-2.6.18-164.el5-2.0.5-1.el5.i686.rpm
oracleasmlib-2.0.4-1.el5.i386.rpm
oracleasm-support-2.1.4-1.el5.i386.rpm
这里特别需要注意第一个软件要和你的Linux内核的版本一致。（其实我的内核版本是2.6.18-155.el5，但是在官方网站找不到完全对应的版本，只好使用这个2.6.18-164.el5的版本了，后面会讲怎么解决这个问题。）接下来进行软件的安装，只需要使用rpm命令即可。
# rpm -ivh oracleasm*
现在安装oracleasm模块可能会报错，......余下全文>>