Jai Ho Rahman [Music Mastereo A. R. Rahman] by Sreekanth

25th February 2009

A 9 year old loses his father, drops out of school and begins working as a musician to feed his family. Few decades later, he becomes the modern day pied piper mesmerizing the world with his music and is anointed as the Emperor of music. This, my dear readers, is not a story line for an upcoming movie but the life summary a musical genius named Dileep Sekhar better known as AR Rahman.

A young man in 1992 began his career with a movie called ROJA. No one realized that his career would become the turning point for Indian film music and trumpet its potential to the entire world. In doing so, AR Rahman became the face of Indian music to the world.

AR Rahman's entry and subsequent success encouraged many to consider the music industry as a career. Before the Rahman era, (South) Indian film industry depended on a handful of playback singers. Songs of a film were sung by one male singer and one/two female singers. It was difficult to obtain entry or enjoy continued success as a singer. One reason might have been the possibility of benchmarking against the existing singers, who had attained legendary status. Music directors at that time were simply not willing to take the risk of entrusting their compositions to new singers fearing rejection from the audience who were accustomed to hearing a particular singer.

AR Rahman splintered this myth and scattered it to the winds. He was instrumental in breaking the "single-card" concept and brought many lesser known yet talented singers into main-stream. It is jocularly mentioned that if a well known singer was not available for a song, Rahman would look around his studio and give the song to a chorus or a track singer. This trend was followed by many contemporaries and future musicians, who introduced new talent into the industry. Rahman was one of the first musicians who insisted on including names of all his orchestra team on the inlay card.

AR Rahman brought revolution in music composition by mixing traditional instruments with latest electronics sounds and digital technology. His creative talent at composing tunes coupled with his tech savvy nature brought a new innovation in film music. His involvement with a project gave the producer an upper hand to dictate terms to the distributors. Music rights were bought for unheard amounts and audio sales touched mind boggling heights.

South Indian musicians never made it big in Bollywood. Even the nation's greatest music director Ilayaraja never became a force in Hindi despite being supremely blessed. AR Rahman first broke the imaginary barrier separating South Indian film industry and Bollywood by working for the biggest directors and stars of Bollywood. He later took Indian music beyond the confines of the country becoming a global ambassador of Indian music.

AR Rahman wrote music for Bombay Dreams and Lord of the rings musical. In the year 2003, he composed music for a Chinese film titled Warriors of Heaven and Earth, which won him applause for its rich background score. Some of his well known tunes were used for Hollywood movies as well. I was pleasantly surprised when I heard "chaiya chaiya" song as the title music for Denzel Washington's Inside Man.

An Oscar for an India was impending and for some strange reason, I always felt (and hoped) that it would come from music fraternity. I had my money on two Indians to bring home the Oscar namely Maestro Ilayaraja and AR Rahman. As an Indian, I feel proud and happy that Oscar is no longer a "sour grape" for us.

AR Rahman's life is inspirational to every individual and makes him an ideal candidate to be awarded the BHARATA RATNA. How of many of us can expect to fight back from the vulnerable position he found himself at his 9th year? His journey from such an abyss like situation to an Oscar win is true testament of an individual's will power and indomitable fighting spirit. Like many Indians, I too agree that Slumdog Millionaire is not AR Rahman's best work - personally, I don’t even think it would figure in his top 10 albums. Nevertheless, the applause and celebration is befitting for a man who has inspired a billion people with a remarkable accomplishment of achieving the once impossible.

Sreekanth Devarakonda

RAID Information [Basic]

RAID Information

Many people are requesting that RAID be part of a system, but don't thoroughly understand the implications of what they're asking for. The reliability and performance implications of the various RAID levels will be discussed in detail here.

This paper will try to explain the different RAID levels, along with their individual strengths and weaknesses. If goes fairly deeply into the theory and implementation, but diagrams are provided to help.

In this paper, there will be references similar to 'RAID-3' or 'RAID-5'. These are accepted abbreviations for 'RAID Level 3' and 'RAID Level 5', respectively. There are also references similar to 'RAID-2 4+2' or 'RAID-5 6+1+1'. The second part of the reference may be interpreted as '4 data drives and 2 parity drives' and '6 data drives, 1 parity drive and 1 hot standby drive', respectively.

It should be noted that no RAID level, regardless of the redundancy provided, is an adequate replacement for backups. This is because the redundant levels of RAID are designed to solve uptime problems, in that they increase overall system uptime by reducing the number of times that backup media are necessary to restore the system. RAID is not a backup solution, as it can be effected by the most common single cause of data loss, namely user error. Also, RAID is generally difficult to remove from the system, and store in a safe location, such as off-site, or in a fireproof safe.

RAID Information [Definitions]

Definitions

Before we dive into RAID Levels, we need to define a few terms that will be used throughout this paper. The purpose of this is so that when a term is used, you will know what it is intended to mean.

Concatenated array

This is an array where multiple disk drives or arrays are logically connected together, end-to-end. This was the earliest method to combine multiple disks. This type of array has no redundancy. It is not one of the RAID levels. Some manufacturer's incorrectly call this JBOD.

Data Drive

A data drive is a disk drive that is dedicated to storing data, as opposed to parity, Hamming code, or a hot standby. In RAID levels where the data and parity move around, the space equivalent of this many drives is available to store data.

The number of data drives in a RAID array is designated by the first number in the size description. In some cases, this is the only number.

Hamming Code

A Hamming Code is an algorithm that can be used to determine if an error exists in a data stream, and sometimes (dependent on the exact code used) correct that error. This is sometimes referred to as an Error Correction Code (ECC).

The number of Hamming Code drives in a RAID array is designated by the number after the first plus sign ('+'). If there is no first plus sign, then there are no Hamming Code drives. This is the same location that the parity number is found in. Except for RAID-1, if the value is one, then it is a parity drive; if it is greater then one, then they are Hamming Code drives.

Hot Standby

This is a disk drive that is reserved for use, in case a failure occurs in one of the active drives on the array. If such a failure occurs, the hot standby drive will start being rebuilt to replace the failed drive. This usually happens automatically, but some RAID implementations may require human intervention.

The number of hot standby drives in a RAID array is designated by the number after the second plus sign ('+'). If there is no second plus sign, then there are no hot standby drives.

JBOD

This is an acronym for 'Just a Bunch Of Disks'. Most disk controllers don't have any RAID functionality built into them. In these cases, the Operating System sees the disk drives connected to the controller as JBOD.

In addition, many RAID controllers default to a JBOD configuration before being configured for RAID.

It should be noted that some manufacturers use the term JBOD to refer to a Spanned or Concatenated array.

Logical Disk

This is what a RAID array is. Although the RAID array is multiple disks, it appears to the Operating System as a single disk.

Mirror Disk

These disks are used in RAID-1. A mirror disk contains an exact duplicate of the disk that it mirrors. More than one mirror disk is sometimes used.

Parity

Parity is used in RAID-3, RAID-4 and RAID-5 to validate the data written to the RAID array. Parity across the array is computed using the XOR (Exclusive OR) logical operation. This is a very fast operation, but it transfers a great deal of information to and from memory. It should be noted that this is a special case of a Hamming Code.

Except for RAID-1, the number of parity drives in a RAID array is designated by the number after the first plus sign ('+'). If there is no first plus sign, then there are no parity drives. This is the same location where the Hamming Code number is found. In general, if the value is one, then it is a parity drive; if it is greater then one, then they are Hamming Code drives.

Physical Disk

A physical disk is a disk. This term is sometimes used to distinguish it from a logical disk.

RAID

This is an acronym for 'Redundant Array of Independent Disks'. Originally, the I stood for Inexpensive, and is sometimes still seen written that way.

Segment size

This is the number of blocks (sometimes expressed in bytes) that are written to one disk drive, before moving on to the next disk drive in the array. It does not apply to RAID-1 or RAID-3.

This is sometimes called Stripe Size. The term Stripe Size is only valid for RAID-0 arrays. Other RAID levels do not have stripes. All RAID levels (except RAID-1 and RAID-3) have segments.

Spanned array

This is another name for a Concatenated array.

Spindle Sync

Spindle Sync is a feature that allows multiple disk drives to operate in sync with each other. When enabled, and properly cabled, a series of disk drives will spin at the same speed, and a given sector will pass under the heads of all the drives at the same time.

This feature will result in a performance improvement in virtually all RAID arrays. This is because it has an impact on the statistical rotational latency of a RAID array. As an example, in a RAID-3 array using 7200RPM disk drives, configured as 8+1, the statistical rotational latency will be 4.17ms if the drives are in sync, but 8.30ms if the drives are not in sync.

This feature is available on many SCSI and Fiber disk drives, but I have never seen it on an IDE disk drive.

Stripe size

This is similar to Segment size, except that it is only valid for RAID-0 arrays. Many manufacturers use this term when they mean Segment size.

Stripe width

This is the number of blocks that must be written to the array, so that every data drive has had a complete segment written.

XOR

The Exclusive OR (XOR) logical function is used to generate parity. It is also used in Hamming codes. The XOR function is similar to a binary addition, without any carry operations. The binary truth table for an XOR is as follows:

Basic RAID Levels

Basic RAID Levels

Basic RAID levels are the building blocks of RAID. Compound RAID levels are built using the concepts described here.

JBOD

JBOD is NOT RAID. JBOD stands for 'Just a Bunch Of Disks'. This accurately describes the underlying physical structure that all RAID structures rely upon. When a hardware RAID controller is used, it normally defaults to JBOD configuration for attached disks.

Some disk controller manufacturers incorrectly use the term JBOD to refer to a Concatenated array.

Concatenated array

A Concatenated array is NOT RAID, although it is an array. It is a group of disks connected together, end-to-end, for the purpose of creating a larger logical disk. Although it is not RAID, it is included here as it is the result of early attempts to combine multiple disks into a single logical device. There is no redundancy with a Concatenated array. Any performance improvement over a single disk is achieved because the file-system uses multiple disks. This type of array is usually slower than a RAID-0 array of the same number of disks.

The good point of a Concatenated array is that different sized disks can be used in their entirety. The RAID arrays below require that the disks that make up the RAID array be the same size, or that the size of the smallest disk be used for all the disks.

The individual disks in a Concatenated array are organized as follows:

RAID-0

In RAID Level 0 (also called striping), each segment is written to a different disk, until all drives in the array have been written to.

The I/O performance of a RAID-0 array is significantly better than a single disk. This is true on small I/O requests, as several can be processed simultaneously, and for large requests, as multiple disk drives can become involved in the operation. Spindle-sync will improve the performance for large I/O requests.

This level of RAID is the only one with no redundancy. If one disk in the array fails, data is lost.

The individual segments in a 4-wide RAID-0 array are organized as follows:

RAID-1

In RAID Level 1 (also called mirroring), each disk is an exact duplicate of all other disks in the array. When a write is performed, it is sent to all disks in the array. When a read is performed, it is only sent to one disk. This is the least space efficient of the RAID levels.

A RAID-1 array normally contains two disk drives. This will give adequate protection against drive failure. It is possible to use more drives in a RAID-1 array, but the overall reliability will not be significantly effected.

RAID-1 arrays with multiple mirrors are often used to improve performance in situations where the data on the disks is being read from multiple programs or threads at the same time. By being able to read from the multiple mirrors at the same time, the data throughput is increased, thus improving performance. The most common use of RAID-1 with multiple mirrors is to improve performance of databases.

Spindle-sync will improve the performance of writes. but have virtually no effect on reads. The read performance for RAID-1 will be no worse than the read performance for a single drive. If the RAID controller is intelligent enough to send read requests to alternate disk drives, RAID-1 can significantly improve read performance.

RAID-2

RAID Level 2 is an intellectual curiosity, and has never been widely used. It is more space efficient then RAID-1, but less space efficient then other RAID levels.

Instead of using a simple parity to validate the data (as in RAID-3, RAID-4 and RAID-5), it uses a much more complex algorithm, called a Hamming Code. A Hamming code is larger than a parity, so it takes up more disk space, but, with proper code design, is capable of recovering from multiple drives being lost. RAID-2 is the only simple RAID level that can retain data when multiple drives fail.

The primary problem with this RAID level is that the amount of CPU power required to generate the Hamming Code is much higher then is required to generate parity.

A RAID-2 array has all the penalties of a RAID-4 array, with an even larger write performance penalty. The reason for the larger write performance penalty is that it is not usually possible to update the Hamming Code. In general, all data blocks in the stripe modified by the write, must be read in, and used to generate new Hamming Code data. Also, on large writes, the CPU time to generate the Hamming Code is much higher that to generate Parity, thus possibly slowing down even large writes.

The individual segments in a 4+2 RAID-2 array are organized as follows:

RAID-3

RAID Level 3 is defined as bytewise (or bitwise) striping with parity. Every I/O to the array will access all drives in the array, regardless of the type of access (read/write) or the size of the I/O request.

During a write, RAID-3 stores a portion of each block on each data disk. It also computes the parity for the data, and writes it to the parity drive.

In some implementations, when the data is read back in, the parity is also read, and compared to a newly computed parity, to ensure that there were no errors.

RAID-3 provides a similar level of reliability to RAID-4 and RAID-5, but offers much greater I/O bandwidth on small requests. In addition, there is no performance impact when writing. Unfortunately, it is not possible to have multiple operations being performed on the array at the same time, due to the fact that all drives are involved in every operation.

As all drives are involved in every operation, the use of spindle-sync will significantly improve the performance of the array.

Because a logical block is broken up into several physical blocks, the block size on the disk drive would have to be smaller than the block size of the array. Usually, this causes the disk drive to need to be formatted with a block size smaller than 512 bytes, which decreases the storage capacity of the disk drive slightly, due to the larger number of block headers on the drive.

RAID-3 also has configuration limitations. The number of data drives in a RAID-3 configuration must be a power of two. The most common configurations have four or eight data drives.

Some disk controllers claim to implement RAID-3, but have a segment size. The concept of segment size is not compatible with RAID-3. If an implementation claims to be RAID-3, and has a segment size, then it is probably RAID-4.

RAID-4

RAID Level 4 is defined as blockwise striping with parity. The parity is always written to the same disk drive. This can create a great deal of contention for the parity drive during write operations.

For reads, and large writes, RAID-4 performance will be similar to a RAID-0 array containing an equal number of data disks.

For small writes, the performance will decrease considerably. To understand the cause for this, a one-block write will be used as an example.

1. A write request for one block is issued by a program.
2. The RAID software determines which disks contain the data, and parity, and which block they are in.
3. The disk controller reads the data block from disk.
4. The disk controller reads the corresponding parity block from disk.
5. The data block just read is XORed with the parity block just read.
6. The data block to be written is XORed with the parity block.
7. The data block and the updated parity block are both written to disk.

It can be seen from the above example that a one block write will result in two blocks being read from disk and two blocks being written to disk. If the data blocks to be read happen to be in a buffer in the RAID controller, the amount of data read from disk could drop to one, or even zero blocks, thus improving the write performance.

The individual segments in a 4+1 RAID-4 array are organized as follows:

RAID-5

RAID Level 5 is defined as blockwise striping with parity. It differs from RAID-4, in that the parity data is not always written to the same disk drive.

RAID-5 has all the performance issues and benefits that RAID-4 has, except as follows:

• Since there is no dedicated parity drive, there is no single point where contention will be created. This will speed up multiple small writes.

• Multiple small reads are slightly faster. This is because data resides on all drives in the array. It is possible to get all drives involved in the read operation.

The individual segments in a 4+1 RAID-5 array are organized as follows:

Compound RAID Levels

Compound RAID Levels

There are times when more then one type of RAID must be combined, in order to achieve the desired effect. In general, this would consist of RAID-0, combined with another RAID level (I've seen RAID-1, RAID-3 and RAID-5 used with RAID-0).

The primary reason for combining multiple RAID architectures would be to get either a very large, or a very fast, logical disk.

The list below contains a few examples. It is not the limit of what can be done.

RAID-1+0

RAID Level 1+0 (also called RAID-10) is the result of RAID-0 applied to multiple RAID-1 arrays. This will create a very fast, stable array. In this array, it is possible to have multiple disk failures, without loosing any data, and with a minimum performance impact.

To recover from a failed disk, it is necessary to replace the failed disk, and rebuild that disk from its mirror.

For two-drive failures, the probability of survival is 66% for a 4-disk array, and approaches 100% as the number of disks in the array increases.

RAID-0+1

RAID Level 0+1 is the result of RAID-1 applied to multiple RAID-0 arrays. This will create a very fast array. If the RAID-0 controllers (hardware or software) are capable of returning an error for data requests to failed drives, then this array has all the abilities of RAID-10. If an entire RAID-0 array is disabled when one drive fails, this becomes only slightly more reliable then RAID-0.

To recover from a failed disk, it is necessary to replace the failed disk, and rebuild the entire RAID-0 array from its mirror. This requires much more disk I/O than is required to recover from a disk failure in RAID-10. It should be noted that some enterprise-level RAID controllers are capable of tracking which drives in a RAID-0 array have failed, and only rebuilding that drive. These controllers are very expensive.

For two-drive failures, the probability of survival is 33% for a 4-disk array, and approaches 50% as the number of disks in the array increases.

This RAID level is significantly less reliable than RAID-1+0. This is because the structure is inherently less reliable in a multi-disk failure, combined with the longer time to reconstruct after a failure (due to a larger amount of data needing to be copied). The longer time increases the probability of a second disk failing before the first disk has been completely rebuilt.

RAID-3+0

RAID Level 3+0 is the result of RAID-0 applied to multiple RAID-3 arrays. This will improve the performance of a RAID-3 array, and allow multiple RAID-3 arrays to be dealt with as a single logical device. RAID-3+0 has a reliability similar to RAID-3, with improved performance.

This type of array is most commonly found when combining multiple hardware RAID devices into a single logical device.

RAID-5+0

RAID Level 5+0 (also called RAID-53 for some unknown reason) is the result of RAID-0 applied to multiple RAID-5 arrays. This will improve the performance of a RAID-5 array, and allow multiple RAID-5 arrays to be dealt with as a single logical device. The reliability of this type of array is similar to that of a RAID-1+0 array, but it has the performance impacts of RAID-5.

This type of array is most commonly found when combining multiple hardware RAID devices into a single logical device.

RAID-3+3+0

I saw this in 1992. This extreme measure was done to achieve a very high transfer rate on the swap partition of a Cray supercomputer. Using the RAID configuration below, the 4GB of memory could be transferred to disk in under 11 seconds. I include it here as an example of what can be done.

The individual disk drives (they were Seagate ST82368K) had a rudimentary RAID-3 controller built into them, where the individual data bits were routed to different heads, and the parity bit was routed to yet another head. Since the drive had eighteen physical heads, it was configured for two logical heads. The disk was capable of correcting for a bad head, but was not able to rebuild the data after the failed head was repaired (usually, a failure on a head amplifier card). This drive was capable of transferring 24MB/S sustained, which was excellent for the time (a fast SCSI drive could rarely sustain more than 3MB/S).

The individual drives were combined in a RAID-3 4+1 array, using a hardware controller. This was a full implementation of RAID-3, and could rebuild a drive, if necessary. This allowed a failed drive to be removed, and repaired, without damaging the array. This portion of the array was capable of transferring 96MB/S sustained.

The RAID-3 arrays were combined in a RAID-0 4-wide array, in software. This gave a sustained transfer rate of 384MB/S to the array.