Full-stack Philosophies

James Morle's Blog

RSS Feed

Diagnosing ASMlib

Posted on 7:01 am February 7, 2013 by James Morle

To use ASMlib or not to use ASMlib? That is the question. Or at least, that is a question that frequently crops up when I'm on consulting engagements with customers. I have personally changed my mind a couple of times on this exact question, and I'm currently sat with one leg on either side of the fence, which doesn't do anyone any good. There are good reasons to go with ASMlib (if you are running the Oracle Database on a Linux platform) and good reasons not to consider going with it, as follows. Pros of using ASMlib:

  1. It's a convenient way for storage/sysadmin's to hand off LUNs to DBAs
  2. It provides solid device name persistence
  3. It provides access permission persistence
  4. A single file is opened by each process, rather than one per database file accessed
  5. I/O to all files can be completely suspended through control of the single file descriptor
  6. Other things that are not yet supported/externalised

Cons of using ASMlib:

  1. Requires a kernel module to be loaded and kept in sync with kernel patching
  2. Requires additional software packages
  3. Diagnostics are more difficult
  4. Seems to interact with the Block layer in the Linux kernel differently to Kernel Asynchronous IO (KAIO)
  5. UDEV rules can adequately provide Pros 3-5, to a greater or lesser extent, without any additional modules or packages

So really, the reasons for using ASMlib are mostly predicated on Pros 4-6, and this is where the indecision creeps in: Is it really worth the aggravation of maintaining an extra kernel module and software packages for advantages that cannot really be felt or seen by the end user? I always felt that it was worthwhile, particularly if it is possible to run Oracle Linux. Oracle Linux removes the management burden of keeping the kernel module in sync by building it directly into the UEK kernel distribution. In this case, there really is no burden and using ASMlib does indeed give a nice interface to the DBA without them having to rummage around in /dev. The items that have made me a little uncertain have been Cons 3-4, both of which have been tripping me up of late out in the field. This blog post is mostly about Con 3, the difficulty of getting good diagnostics, which is where I have been doing some research recently.

Future Directions

Before jumping into the diagnostics, and the internals of ASMlib in general, it's worth touching on Pro 6 - things that are not currently supported and/or externalised. As part of this research it became clear that ASMlib has an enormous amount of potential for extending the amount of control that the database has over the I/O subsystem. It is very possible, for example, to provide timing information for each and every Async I/O. This is something that is not possible with the KAIO implementation in Linux. It is also possible to do such things as prioritise/resource manage I/O as directed by Oracle, which is another thing that is increasingly important as we move into an era of Cloud Computing and Pluggable Databases. Standards such as T10 Protection Information (T10 PI), which is already incorporated in the Linux kernel, will be implemented all the way up into the Oracle Database data layer where checksum computation already takes place. Existing I/O interfaces do not provide a way to push this information into the kernel, but ASMlib provides that capability by simply changing the definition of a single data structure in the interface. Oracle already announced this functionality, which pushes the T10 PI protection right up into the database layer, with all metadata passed through to the storage tier through the ASMlib interface. This functionality starts in the 2.6.39-300 branch of the Oracle Linux UEK2 kernel. So, the future of ASMlib looks very interesting to me, and could become the de-facto standard for users of ASM. And, unless you are running NAS storage with Direct NFS, you really should not be using anything except ASM to present storage to Oracle. Regardless of whether it is a sensible thing to do right now or not, the fact is that many of my customers use ASMlib. This presented me with a problem when trying to diagnose performance problems with their storage tier, because the interface to the kernel via ASMlib is very different to the interface for either synchronous I/O or Kernel asynchronous I/O and so began this strand of research.

System Calls

One of the most useful tools for examining the behaviour of Oracle processes on Linux systems is the strace(1) tool. It's available on all UNIX-alikes (of which Linux is just another variety, don't let anyone tell you otherwise) under various guises: truss on Solaris and AIX, tusc on HP-SUX. It provides a view of which system calls the process is issuing, along with other handy functionality such as high resolution timestamps, call timing, and buffer decoding. If you are not 100% clear about what a system call is, I strongly recommend reading up on it, perhaps starting with the Wikipedia link I included in the last sentence. In very brief summary though, a system call is the interface used by processes to request service from the kernel. The kernel, of course, has pretty much total control over access to hardware and so calls must be made via the kernel for important things such as performing any I/O. You can probably see why strace is a useful tool when viewed in this context. When Oracle requires I/O, it makes system calls. These system calls can be viewed using strace, and a good deal of information can be gleaned by observing exactly which I/Os Oracle makes to which devices and in which order. I've collected a few examples here for various scenarios to demonstrate the kind of output that we see when using strace. I have selected the simple case of the Oracle LGWR process first of all, because it is relatively easy to understand what is going on. Specifically, all these examples show a LGWR process writing a single log flush to a pair of multiplexed (two members) log members. First up, good old-fashioned (and it is old fashioned - if you see this on your system, you have a problem) synchronous I/O:

gettimeofday({1355779106, 636655}, NULL) = 0
pwrite(260, "\1\"\0\0\267\0\0\0i\1\0\0\20\200\277Xl\0\0\0\5\0\0\0\351\314+\0\1\0\0\0"..., 512, 93696) = 512
gettimeofday({1355779106, 637183}, NULL) = 0
gettimeofday({1355779106, 637206}, NULL) = 0
pwrite(261, "\1\"\0\0\267\0\0\0i\1\0\0\20\200\277Xl\0\0\0\5\0\0\0\351\314+\0\1\0\0\0"..., 512, 93696) = 512
gettimeofday({1355779106, 637453}, NULL) = 0

The highlighted lines show the actual writes to the two log members. They are opened using file descriptors 260 and 261, and we can validate that by using another tool called lsof(8), a fragment of which is shown here:

oracle 9465 oracle 260u REG 253,3 52429312 704517 /opt/oracle/oradata/SIMORA/onlinelog/o1_mf_2_8592wtqq_.log
oracle 9465 oracle 261u REG 253,5 52429312 430802 /opt/oracle/fast_recovery_area/SIMORA/onlinelog/o1_mf_2_8592wvbh_.log

We can determine from the strace output that the writes are taking place using synchronous I/O because they use the pwrite(2) system call, which is just a variant of the even older write(2) system call that allows the specification of a seek offset into the file without a preceding lseek(2) call. We can see that each of the writes is 'wrapped' by a pair of calls to gettimeofday(2) which allows Oracle to measure the I/O latency for each write and record it in the standard wait interface within the Oracle database kernel. It should be apparent from this output that having two log members will cause log writes to take twice as long when using synchronous I/O, as each I/O is issued serially. Now let's have a look at what happens with Kernel asynchronous I/O (KAIO) is used:

gettimeofday({1355780023, 168516}, NULL) = 0
io_submit(139793758486528, 2, {{0x7f2443aa2f58, 0, 1, 0, 258}, {0x7f2443aa31b8, 0, 1, 0, 259}}) = 2
io_getevents(139793758486528, 2, 128, {{0x7f2443aa2f58, 0x7f2443aa2f58, 512, 0}, {0x7f2443aa31b8, 0x7f2443aa31b8, 512, 0}}, {600, 0}) = 2
gettimeofday({1355780023, 169330}, NULL) = 0

With KAIO, both the write requests are issued within the scope of a single system call to io_submit(2). These I/Os occur more or less in parallel because Oracle is able to request both writes concurrently, without waiting for one to finish before sending the next. This is one example of why Asynchronous I/O is vitally important to database performance. Again, the write operations are 'wrapped' in gettimeofday(2) calls to capture the timing. However, the actual timing logic here will only ever record the longest I/O (because the io_getevents(2) call waits for both writes to complete before returning) rather than each individual I/O, because the timing information for each I/O is not available in the Linux KAIO interface. There's still lots of information available, however, because strace decodes the structures in the system calls. It shows type of I/O for the io_submit(2), which in this case is 1 which is a write, and also the file descriptor (258 and 259 in this case. The returned buffer of io_getevents(2) shows which I/O requests have been completed - the first address is the address of the IO control block specified in the corresponding io_submit, and the third argument is the I/O size. The last two examples have shown the simple case where the two log members were plain old files residing in a normal filesystem. This makes it easy to understand. If the logs existed in an ASM diskgroup, we might expect more than two I/Os to occur, depending upon whether or not the write crossed a stripe boundary in the diskgroup. Apart from that, it would look mostly the same. So that's a KAIO strace. Now let's look at a write from LGWR when the log files are in an ASM diskgroup and the disks in the diskgroup are provisioned using ASMlib:

gettimeofday({1358370099, 154571}, NULL) = 0
read(32, "MSA\0\2\0\10\0P\0\0\0\0\0\0\0\220t\361\v\0\0\0\0\300/\334\247\225\177\0\0"..., 80) = 80
gettimeofday({1358370099, 155085}, NULL) = 0
read(32, "MSA\0\2\0\10\0P\0\0\0\0\0\0\0\220t\361\v\0\0\0\0\300/\334\247\225\177\0\0"..., 80) = 80
gettimeofday({1358370099, 155475}, NULL) = 0

Weird, huh? I assure you that this output is taken from the log writer process, so why do we just see reads? Finally, after much deliberation, we get to the point of this post - Diagnosing ASM.

Digging Into ASM

In order to understand how the interface to ASMlib worked, I decided to download the source for the kernel module, oracleasm, and to have a read through it. It's not often that the source is so readily available, so this made the job much easier than it might have been. I started with the easily available 2.0.5 tarballs available on Oracle's OSS pages, but found that it was too old for certain analyses and moved onto the UEK git repository (here). Thanks to Martin Petersen at Oracle for helping me locate the latest code (and for other assistance in the preparation of this blog). The oracleasm block driver module is the component of ASMlib that exists in the kernel. It provides the system call interface that the database then uses to issue I/O requests via ASMlib. It is implemented as a virtual filesystem, which is a common way to interface to the kernel in UNIX-alike systems given its defining model of everything is a file. It is a virtual filesystem, which means that the oracleasm module itself is not actually providing any traditional filesystem functionality such as its own storage layout, buffering, file permissions, etc. The presented filesystem consists solely of device special files, in the same way that the kernel presents access to other devices via /dev. In fact, the oracleasm filesystem is mounted under /dev as /dev/oracleasm. Below /dev/oracleasm are just two directories: disks and iid. The disks directory contains simple device special aliases for the underlying devices that have been labelled for use within ASMlib. It's actually a handy way to map labelled disks back to their underlying OS devices. Here's an example, using simple sd devices - in a multipath environment the disks would map back to the parent multipath device in /dev/mpath:

[root@asmlib01 oracleasm]# oracleasm listdisks
DATA01
DATA02
[root@asmlib01 oracleasm]# pwd
/dev/oracleasm
[root@asmlib01 oracleasm]# ls -l disks
total 0
brw-rw---- 1 oracle dba 8, 33 Jan  9 10:49 DATA01
brw-rw---- 1 oracle dba 8, 34 Jan  9 12:40 DATA02
[root@asmlib01 oracleasm]# ls -l /dev/sdc1 /dev/sdc2
brw-rw---- 1 root disk 8, 33 Jan  9 10:49 /dev/sdc1
brw-rw---- 1 root disk 8, 34 Jan  9 12:40 /dev/sdc2

It's the iid directory where all the magic starts to happen. This is the directory that contains all the instance interface descriptors that allow user processes to perform I/O via the oracleasm driver. Here's an example of the contents:

[root@asmlib01 oracleasm]# ls -l iid
total 0
-rwxr-x--- 1 oracle oinstall 0 Jan  9 10:48 0000000000000001
-rwxr-x--- 1 oracle oinstall 0 Jan  9 10:50 0000000000000002
-rwxr-x--- 1 oracle oinstall 0 Jan  9 10:50 0000000000000003
-rwxr-x--- 1 oracle oinstall 0 Jan  9 10:50 0000000000000004
-rwxr-x--- 1 oracle oinstall 0 Jan  9 10:50 0000000000000005
-rwxr-x--- 1 oracle oinstall 0 Jan  9 12:43 0000000000000006
-rwxr-x--- 1 oracle oinstall 0 Jan  9 12:44 0000000000000007
-rwxr-x--- 1 oracle oinstall 0 Jan  9 12:44 0000000000000008
-rwxr-x--- 1 oracle oinstall 0 Jan  9 12:46 0000000000000009

There is one special file in iid for every open instance on the system, plus a few others which don't seem to be open:

[root@asmlib01 ~]# fuser /dev/oracleasm/iid/*
/dev/oracleasm/iid/0000000000000005:  3222  3224  3230  3232  3238
/dev/oracleasm/iid/0000000000000009: 25548 25561 25567 25569 25571 25573 25575 25577 25581 25599 25654 25679 25701 25727
[root@asmlib01 ~]#

So, only the files ending "5" and "9" are opened by any process. File 5 is open by processes belonging to my ASM instance, and file 9 is open by processes belonging to my database instance. I have not yet fully explored the reasons that so many special files get created here, which can number in the hundreds. I suspect a bug of some kind, given that one of my customers (admittedly using an old version of oracleasm, as they are on RHEL5) collects one more entry in iid every hour, and I've never found any of them to be open by any process. So let's go back to that strace output that we got from LGWR using ASMlib. It was performing read(2) system calls to file descriptor 32, so let's use lsof to see what that file is:

[root@asmlib01 ~]# lsof -p 25569 | grep 32u
oracle  25569 oracle   32u   REG   (omitted) /dev/oracleasm/iid/0000000000000009

Well, no surprise there: LGWR is one of the processes that has the IID file that ends with "9" open, because that's the IID allocated for that instance. So LGWR is making read(2) system calls to IID file, and somehow achieving not only to do writes, but also to submit them using an asynchronous I/O interface as this is one of the features provided by ASMlib. How is it possible to do this solely through blocking read calls? It's actually all quite simple and rather ingenious. First of all, forget that it's a read(2) call - the name of the call simply does not matter. It just happens to be a convenient system call that allows the passing of (a) a file descriptor and (b) a pointer to an area of memory in user address space. It could just as well be a system call named wibble(2), as long as it provided those very two arguments and the oracleasm kernel driver could register a handler for that call. All the system call is doing it passing control to the kernel along with the address of the buffer in user address space. The user space buffer contains a bunch of data which conform to the ABI that  is defined in the header files for ASMlib. In the case of the Oracle binaries, this is the library that is included in the "oracleasmlib" RPM, which is used by the Oracle binary as an abstraction layer providing the implementation of the actual I/O. I strongly suspect that the oracleasmlib library conforms to the Oracle Disk Manager (ODM) specification, as this is a long established abstraction in the Oracle kernel. So, when the LGWR want to perform a write, the actual write request is loaded up into a memory structure and submitted to the oracleasm kernel driver by way of a read(2) system call.

So let's go back to that strace output we extracted from LGWR earlier in the blog and look at the contents in more detail:

read(32, "MSA\0\2\0\10\0P\0\0\0\0\0\0\0\220t\361\v\0\0\0\0\300/\334\247\225\177\0\0"..., 80) = 80
gettimeofday({1358370099, 155085}, NULL) = 0
read(32, "MSA\0\2\0\10\0P\0\0\0\0\0\0\0\220t\361\v\0\0\0\0\300/\334\247\225\177\0\0"..., 80) = 80

The first 16-bytes of this memory buffer are an oracleasm_abi_info structure that has the following format:

struct oracleasm_abi_info
{
/*00*/	__u32			ai_magic;	/* ASM_ABI_MAGIC */
		__u16				ai_version;	/* ABI version */
		__u16				ai_type;	/* Type of operation */
		__u32				ai_size;	/* Size of passed struct */
		__u32				ai_status;	/* Did it succeed */
/*10*/
};

The first four bytes are the oracleasm_abi_info.ai_magic member of the request. It's an unsigned 32-bit integer type, which means it is 4-bytes long. In reality this is just a null-terminated string which is "ASM" printed backwards. The next two bytes are an unsigned 16-bit (2-byte) version number (ai_version), which is "2" in little-endian format. All making sense so far, as this is version 2 of the oracleasm driver.

The next 16-bit word is the ai_type: The oracleasm driver checks the ai_type, to see what the actual request is and then uses the appropriate data structure to decode that request. These ai_types are also defined in the oracleasm header files (in this case abi.h):

enum asm_operation_types
{
ASMOP_NONE = 0,
ASMOP_QUERY_VERSION,
ASMOP_GET_IID,
ASMOP_CHECK_IID,
ASMOP_QUERY_DISK,
#define ASM_LAST_TRANSACTION_OP ASMOP_QUERY_DISK
ASMOP_OPEN_DISK,
ASMOP_CLOSE_DISK,
ASMOP_IO32,
ASMOP_IO64,
ASM_NUM_OPERATIONS  /* This must always be last */
};

In our example, we have an ai_type of "8", which enumerates to an ASMOP_IO64 type. Just to help you orient yourself, the text in the strace buffer that we are currently looking at says "\10\0", which is a 2-byte little-endian representation of "8" (\10 is octal representation of 8).

When oracleasm gets an ASMOP_IO64 request type,  it knows that the request buffer is actually now going to be of type 'oracleasm_io_v2', and can use that structure definition to decode more of the request. The structure looks like this:

struct oracleasm_io_v2
{
/*00*/	struct oracleasm_abi_info	io_abi;		/* ABI info */
/*10*/	__u64						io_handle;	/* asm_ctx */
		__u64						io_requests;	/* asm_ioc ** */
/*20*/	__u64						io_waitreqs;	/* asm_ioc ** */
		__u64						io_completions;	/* asm_ioc ** */
/*30*/	__u64						io_timeout;	/* struct timespec * */
		__u64						io_statusp;	/* __u32 * */
/*40*/	__u32						io_reqlen;
		__u32						io_waitlen;
		__u32						io_complen;
		__u32						io_pad1;	/* Pad to 64bit aligned size */
/*50*/
};

So we can see that the oracleasm_abi_info structure is always included at the beginning of the structure, so oracleasm can now use this oracleasm_io_v2 structure to access all of the request buffer. We can also see that it is 0x50 bytes long, which is reflected in the strace buffer as the "P\0\0\0" 32-bit/4-byte value for the ai_size field. 'P' is the ASCII representation of 0x50. The very final field in the oracleasm_abi_info is the ai_status field (again, 32-bits/4-bytes in size). This is completely unremarkable (and zero, or rather '\0\0\0\0', in this case) apart from the fact that this is a RETURN value, not a request parameter. There is nowhere in the oracleasm code that reads this value, only places that update the ai_status field, and so it is quite different to all the other values we have looked at so far - all of those were set by the user process, whereas the ai_status is populated by the kernel. Moving on...

We have now reached the end of our oracleasm_abi_info structure, and so the next data values pertain to the remainder of what we now have identified as an oracleasm_io_v2 structure. Only the first two of the fields of the oracleasm_io_v2 structure are shown in our strace output, each being 64-bit/8-byte pointers. The first one is the io_handle, which is the I/O context for this session, and the second is the first of a set of three pointer arrays:

  1. io_requests
  2. io_waitreqs
  3. io_completions

At this point we won't decode any more of the strace data because that gets a bit tedious. It was this stage (actually a bit before) where I started to write systemtap and Dtrace monitors for this stuff. That's going to be the subject of another blog post, though, because this one's getting a bit lengthy. The important thing to realise at this point, though, is that the contents of the buffer supplied to the read(2) system call are different on the way into the call to the way out of the call. This is a two way protocol implemented in a single system call. Using strace alone will only ever give you the output details (the return data) of the call to oracleasm, because strace is written to provide the most useful information for each system call. In the case of a read(2), that will normally be the contents of the buffer after the read call has completed. Not so in this case, so we need to use different tools anyway!

Switching back to the I/O request, we have a set of three different types of I/O request: a request, a wait and a completion. These are all implemented as arrays of pointers to a new structure of type asm_ioc, with the array size defined by the corresponding io_reqlen, io_waitlen and io_complen fields.  The asm_ioc structure looks like this:

struct _asm_ioc64 {
	__u32		ccount_asm_ioc;
	__s32		error_asm_ioc;
	__s32		warn_asm_ioc;
	__u32		elaptime_asm_ioc;
	__u16		status_asm_ioc;
	__u16		flags_asm_ioc;
	__u8		operation_asm_ioc;
	__u8		priority_asm_ioc;
	__u16		hint_asm_ioc;
	__u64   	disk_asm_ioc;
	__u64		first_asm_ioc;
	__u32		rcount_asm_ioc;
	__u16		xor_asm_ioc;
	__u16		abs_asm_ioc;
	__u32		abn_offset_asm_ioc;
	__u32		abn_asm_ioc;
	__u32		abn_mask_asm_ioc;
	__u32		spare1_asm_ioc;
	__u64		tag_asm_ioc;
	__u64		reserved_asm_ioc;
	__u64		buffer_asm_ioc;
	__u64		check_asm_ioc;
};

There is clearly a great deal of information in each of these structures, and I have not got to the bottom of all the fields as yet. One of the very interesting ones is the operation_asm_ioc field, which determines what the actual I/O operation is. This is taken from the following list:

/* I/O operations */
#define ASM_NOOP        0x00    /* no-op to key check or pass a hint */
#define ASM_READ        0x01    /* Read data from disk */
#define ASM_WRITE       0x02    /* write data to disk */
/* 0x03 is unused */
#define ASM_COPY        0x03    /* copy data from one location to another */
#define ASM_GETKEY      0x04    /* get value of one or more disk keys */
#define ASM_SETKEY      0x05    /* set value of one or more disk keys */

So now we are finally getting to the meat of the call, and we can start to see what is going on in the actual requests. Wouldn't it be useful to be able to decode all this from a diagnostic perspective? I thought so, so look out for part two of this journey, coming soon. One parting thought to observe: You may have noticed that the standard Linux KAIO interface does not provide timing information for each individual I/O. ASMlib has the technological flexibility to fix that, and you might just be interested to note the asm_ioc.elaptime_asm_ioc field...


9 comments on “Diagnosing ASMlib

  1. Hello James,

    I'm just wondering why could you have so many files under the /dev/oracleasm/iid/ directory.
    I tried running my VM for a few hours and I didn't see any new files generated there, it is just 3 files all along.
    But, I do notice that only 2 files are being accessed.
    Here is what I have in my VM:

    # ll /dev/oracleasm/disks/
    total 0
    brw-rw---- 1 grid asmadmin 8, 17 Jun 13 01:22 ASMDISK1
    brw-rw---- 1 grid asmadmin 8, 33 Jun 13 01:22 ASMDISK2

    # ll /dev/oracleasm/iid/
    total 0
    -rwxr-x--- 1 grid oinstall 0 Jun 13 09:24 0000000000000001
    -rwxr-x--- 1 grid oinstall 0 Jun 13 09:24 0000000000000002
    -rwxr-x--- 1 oracle asmadmin 0 Jun 13 09:24 0000000000000003

    # fuser /dev/oracleasm/iid/*
    /dev/oracleasm/iid/0000000000000002: 2951 2953 2959 2961
    /dev/oracleasm/iid/0000000000000003: 3110 3120 3126 3128 3130 3132 3134 3136 3140 3211 3218 3235 3241

    ps -f -p 2951 2953 2959 2961
    UID PID PPID C STIME TTY STAT TIME CMD
    grid 2951 1 0 09:24 ? Ss 0:00 asm_dbw0_+ASM
    grid 2953 1 0 09:24 ? Ss 0:00 asm_lgwr_+ASM
    grid 2959 1 0 09:24 ? Ss 0:00 asm_rbal_+ASM
    grid 2961 1 0 09:24 ? Ss 0:00 asm_gmon_+ASM

    I also tried running an strace on the lgwr process, but I didn't get the same thing as what you've shown.
    With this command: strace -p 2953 -e trace=all -r -o /tmp/lgwr.txt
    I'm only getting this kind of output:
    0.000027 semtimedop(98304, 0x7fffe5866fd0, 1, {3, 0}) = -1 EAGAIN (Resource temporarily unavailable)
    3.000517 getrusage(RUSAGE_SELF, {ru_utime={0, 148977}, ru_stime={0, 17997}, ...}) = 0
    0.000049 getrusage(RUSAGE_SELF, {ru_utime={0, 148977}, ru_stime={0, 17997}, ...}) = 0
    0.000042 times({tms_utime=14, tms_stime=1, tms_cutime=0, tms_cstime=0}) = 432016211
    0.000036 open("/proc/2935/stat", O_RDONLY) = 37
    0.000056 read(37, "2935 (oracle) S 1 2935 2935 0 -1"..., 999) = 227
    0.000042 close(37) = 0
    0.000042 times({tms_utime=14, tms_stime=1, tms_cutime=0, tms_cstime=0}) = 432016211
    0.000040 times({tms_utime=14, tms_stime=1, tms_cutime=0, tms_cstime=0}) = 432016211

    can you share how you did it?

    Thanks in advance for replying,
    Adhika

      • Hi James,

        I've waited for a while to get similar output as what I have seen on your result.
        Could you suggest what I should be doing during the strace capturing process?
        Do I need to execute lots of DML statements against the database so I could get the lines started with read(...)?

        If I don't do anything would I still see the read(...) lines?
        My assumption is, probably it would show, but I didn't have enough space for that, I was hoping that it would show very soon if I keep doing DML statements.

        Thanks,
        Adhika

        • Adhika,

          If you are strace-ing the LGWR process, you must do *something* to make LGWR do a write in order to see those read() calls. So, if you wit long enough the system will do some DML and you'll see it. It might be a while... Or you could make some kind of change yourself. I recommend just creating an empty table, because this will implicitly change data in the SYS schema and do a commit. That will produce some output. Or just update any row in any table and either wait 3s or issue a commit. All of those will make LGWR perform a write.
          Cheers

          James

  2. Pingback: Oak table world UK 2013 collateral #otwuk2013 | Oracle Observations

  3. Hi James,

    Excellent Write-up. Thanks for that.Do you plan to write part 2 in any near future ? .Eagerly waiting .

    With write calls showing as read.This adds weight to your cons point mo 3

    Diagnostics are more difficult..

    Re : A single file is opened by each process, rather than one per database file accessed

    This is an advantage as we will not be running out of open file descriptors ? . Is my understanding correct .

    Ta
    Jag

    • Hi Jag,

      Part 2 is coming, don't worry. The biggest delay in that is that it will include the source code for the tracing utility, and that has the ability to crash the kernel from executing in kernel mode. So I want to make sure that I have some reasonable degree of safety in the code before doing that!
      The main advantage of the single file descriptor is that ALL access to the files can be suspended directly by the ASMlib device driver. It is also a fact that there are significantly less open file descriptors, but this is really just a side benefit.
      Cheers

      James

      • Thanks James. I am hoping the ways to disable the ASM disk access using ASMLib will be part of Part 2 :-)

        Thanks again.

Leave a Reply