Full-stack Philosophies

James Morle's Blog

RSS Feed

Udev Rules OK

Posted on 12:00 pm April 20, 2012 by James Morle

Dear Linux Hackers,

Have we not finished changing the 'best' way to have persistent naming and security attributes for disk devices? Seriously, just when we thought it was great to put the "uid", "gid" and "mode" specifications for disk devices into /etc/multipath.conf, then it gets once again deprecated in favour of udev. This push and pull game has to stop!

Yours Faithfully

All Oracle Users


OK, that's  enough ranting and raving, let's actually have some technical content in this posting. So, it would appear that the multipath.conf niceties afforded to us in the latter versions of Red Hat 5 no longer function in Red Hat 6. Instead we have to work out how to do this nicely again in Udev rules. By the way, if you don't understand the title of this post, check here.

I figured that I would find a decent way to segregate LUNs that are destined to become ASM disks and to set the right ownership and permission.  We still can't use ASMLib on Red Hat 6 to do this, so Udev is our only hope now that multipath.conf is ignoring those convenient options. I came up with an answer which works really nicely, using a combination of the multipath alias names and a new Udev rule file.

So, first of all, create aliases for all the WWIDs that you want to use in ASM, as you normally would/should:

multipath {
     wwid  360a98000646767724c4a6a4338506c71
     alias asmdisk1

Now create a new Udev rule and call it something with a low index number, like /etc/udev/rules.d/12-dm-asmdevices.rules so that it gets executed early on in the process. The content of the file should look a little like this:

ENV{DM_NAME}=="asmdisk?*", NAME:="asmdisks/%E{DM_NAME}", OWNER:="oracle", GROUP:="dba", MODE:="660"

Let's explain that a little. The first portion, ENV{DM_NAME}=="asmdisk?*" is a pattern matching rule that looks for the alias we gave our devices in multipath.conf. The next part of the line, NAME:=, instructs udev to create a new special file in /dev within a subdirectory named "asmdisks". The rest of the line specifies the owner, group and permissions for this new device special file.
The execution of this rule against our example alias in multipath.conf above will produce a new special file named '/dev/asmdisks/asmdisk1', with all the right security attributes. If there were tens or hundreds of aliased LUNs in multipath.conf, they would all end up nicely created in the /dev/asmdisks directory. This is a really nice way to organise your ASM disks, allowing for a single discovery string of "/dev/asmdisks/*".
One thing to note here is that you can have any number of different alias patterns, as long as there is a pattern that matches that alias in the udev rule file. So, one might elect to have different tiers of storage with different aliases, for example, and create multiple udev rules for each, as follows:

multipath {
     wwid  1234
     alias fastdisk01
multipath {
     wwid  1235
     alias fastdisk02
multipath {
     wwid  1236
     alias ssd01
multipath {
     wwid  1237
     alias slowasanything01

Udev rule:

ENV{DM_NAME}=="fastdisk?*", NAME:="asmdisks/%E{DM_NAME}", OWNER:="oracle", GROUP:="dba", MODE:="660"
ENV{DM_NAME}=="ssd?*", NAME:="asmdisks/%E{DM_NAME}", OWNER:="oracle", GROUP:="dba", MODE:="660"
ENV{DM_NAME}=="slowasanything?*", NAME:="asmdisks/%E{DM_NAME}", OWNER:="oracle", GROUP:="dba", MODE:="660"

Well that's about it - I think it's a nice way to organise your ASM disks in /dev and to keep the permissions and names persistent. Until they decide to change it again.

4 comments on “Udev Rules OK

  1. Thanks James. Looks workable to me and it would be great if everyone in the Oracle community follows your lead so we don't have to search for how it has been done at each client site.

    I have one *slight quibble* with your posting in the exact usage of "deprecate." Now if they had decided to merely "deprecate" doing it the multipath.conf way, it would have been a questionable change (because what does it hurt?), but they apparently broke existing configurations. Sigh. I completely agree with your sentiment that I take to be: There is no huge advantage to be gained. The cost of making these fundamental changes to things that are not broken far exceeds any potential benefit.

  2. Hi,

    Your udev rules seems much simpler then what I am currently using (on OL 5.x), but shouldn't you also include the last_rule option to prevent the block device from being picked up by other rules?

    My current rules:

    SUBSYSTEM!="block", GOTO="end_oracle"
    KERNEL!="dm-[0-9]*", GOTO="end_oracle"
    PROGRAM!="/sbin/mpath_wait %M %m", GOTO="end_oracle"
    ACTION=="add", RUN+="/sbin/dmsetup ls --target multipath --exec '/sbin/kpartx -a -p p' -j %M -m %m"
    PROGRAM=="/sbin/dmsetup ls --target multipath --exec /bin/basename -j %M -m %m", RESULT=="*_Ora*_asm[0-9]*", NAME="oracle/%c", OWNER="grid", GROUP="dba", MODE="0660", OPTIONS="last_rule"
    PROGRAM!="/bin/bash -c '/sbin/dmsetup info -c --noheadings -j %M -m %m | /bin/grep -q .*:.*:.*:.*:.*:.*:.*:part[0-9]*-mpath-'", GOTO="end_oracle"
    PROGRAM=="/sbin/dmsetup ls --target linear --exec /bin/basename -j %M -m %m", RESULT=="*_Ora*_asm[0-9]*p*", NAME="oracle/%c", OWNER="grid", GROUP="dba", MODE="0660", OPTIONS="last_rule"

  3. Pingback: Log Buffer #269, A Carnival of the Vanities for DBAs | The Pythian Blog

Leave a Reply