Confusion in the Clouds

Where does the Cloud begin, and where does it end? You may say: “Yes, I have access to my data from various locations. I can choose the capacity of services. I work in the Cloud…”. Well, the semantics of it have become muddled.

No one knows when, but as it was co-mingled and mainstreamed into colloquial language, the Cloud concept has shared the fate of other similar phenomena. Its foundations have become, shall we say, “cloudy”. That’s why it is important to define it and all other associated concepts. Let’s start from the beginning.

The Definition

The official NIST definition states: “Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.” (source)

There are five key features: self-service on demand, resource pooling, network access, scalability on demand, and measured service. Almost everything seems to be clear… almost. Even with such a precise and detailed definition, there is still a lot of room for various understatements and doubts which may lead to misuse of the term “Cloud”- let’s take a closer look.

Facts and Myths

The first thing is the idea of hosting. The fact that you are keeping software in any case on a remote machine does not mean it is the “Cloud”. It doesn’t even meet the primary objectives of the NIST definition. Hosting can only be seen as a part of the idea of Cloud computing. Also the fact that one is using XEN, VMware or any virtualization solution does not mean that they are “in the Cloud”. It only means they are using a component of a cloud solution.

Comparing Cloud with Grid computing is also a mistake. Both are scalable, operate with multi-tenancy and multitask, but there are also differences. The first is a model of scalability. In the case of Grid, there is just a need to add a new server. Cloud requires the allocation of resources at once – at the request of the customer. This phenomenon was described by Rich Wolski – one of the authors of “Right Scale” – a popular application for managing services in the cloud: “If we have a cluster that consists of 1000 units, its users may allocate all of them or half or 200. Some of these allocations can be serviced on time, and some of them can be queued until resources are returned. As a result, we can point to advanced algorithms used for resource planning. That’s why we can say that Cloud computing is a set of small allocated resources.” (you can read more about it and about Amazon EC2 solutions on: RightScale Blog). What can be concluded from this? In short: Grid is something finite and allocated with the use of queues. Cloud is a dynamically scalable platform, which provides enlarged resources on demand, in real time.

Another issue concerns the “on demand” concept. In the case of Cloud it means not only that any action will be performed on user demand, but that it will also happen without human intervention – the developer, administrator, or anyone. For example, if someone says: “Yes, my storage is scalable – it’s Cloud and stuff. Just let me know and I’ll give you more space. It won’t take me more than three days.” – they’re wrong and have nothing in common with work in the Cloud.

In conclusion, we can say that one swallow doesn’t make a summer. If your infrastructure meets any of the assumptions of the NIST definition it doesn’t mean it works as the Cloud. It is important to emphasize that Cloud Computing is a complex structure that must meet a number of specific conditions to fulfill its role.

Fear of the New

As is easy to see, many have become self-proclaimed Cloud users – sometimes out of ignorance, sometimes through a false self-promotion (“I work in the Cloud” – one may brag). On the other hand, one can hear a lot of voiced concerns which indicate a fear of new technology.

Some of them are related to security – rightly and wrongly, which sounds contradictory. Why? “Cloud” does not mean “secure” (not directly). As often happens, human factors play a role. One can say: “Yes, Cloud is safe”, but its security will always depend on the service provider, proper internal control, a solid design and operational rigor – nothing more, nothing less. Another thing is that any solution requires consideration of the required level of safety. It means that in some cases, a regular cloud solution appears to be a good idea, and in others it is better to think about a hybrid cloud solution or virtual private cloud. The same applies to performance. Its level may vary depending on a provider and implemented solution.

Also, there is reliability. While we can be sure that Cloud solutions are designed to provide the highest possible availability and redundancy, nothing is perfect. To narrow the choices, one should base their consideration not on a comparison of capabilities, but value for money. Viewed in this light, cloud solutions may appear to be the best ones.

A Marketing Conspiracy?

All these doubts, coupled with a few misunderstandings, may be the reasons why “Cloud” is considered a marketing gimmick, an empty slogan or just a simple fad. And it has nothing to do with homegrown IT marketers.

Have you heard Richard Stallman’s statement about the Cloud? “It’s stupidity. It’s worse than stupidity: it’s a marketing hype campaign,” he told The Guardian. Or Larry Ellison from Oracle who said: “The computer industry is the only industry that is more fashion-driven than women’s fashion. Cloud Computing. I remember I was reading W and I read that orange is the new pink. And Cloud is the new SaaS. (Software as a Service) Or cloud is the new virtualization. It is the most nonsensical. I mean I read these articles… I have no idea what anybody is talking about. I mean it is really just complete gibberish” (source).

Richard Stallman is the enemy of cloud solutions because of the possibility of losing control. Well, yes, but is that the basic assumption of cloud? Is it truly a loss of control? Any product or service, when shifting the responsibility somewhere or to someone else is connected with a kind of risk. The only thing that can be said to the doubters is this: If you are afraid of the Cloud, if your data is confidential, valuable, whatever, then the only solution is secured offline storage. In other words, the advantages of the Cloud are not for you.

What about Larry Ellison? His charges relate mainly to the fact that Cloud solutions were ”nothing new”. He may be right. But it is also important to know that the popularity associated with the possibilities of Cloud computing have appeared recently and together with the development of the global network. What’s more, Cloud is not only “virtualization” or “SaaS”. It is a synergy of these and other elements, not just a rebranding of old names. It is a development – the evolution of old solutions.

If you disagree, look up the concept of “Orchestration“. It shows that Cloud is not only just a “new way of virtualization”. Yes, that’s a part of it, but the synergy also applies to such items as: network management, communications between servers, scheduling, queuing within the infrastructure etc. You can draw your own conclusions.

Overall, although the statements and doubts of experts may have a basis in reality, the phenomenon of Cloud computing is not only a marketing gimmick. So what’s it all about?

The Cloud – The Other Side of the Coin

Cloud storage, cloud music, cloud games, cloud services, cloud whatever… Yes, it sounds nice. It’s new, and trendy. It is also the convergence of multiple entities to a need. Everything becomes clearer if we leave the marketing bubble. Let’s try to do this not by the NIST definition, but from the perspective of a business user.

Cloud solutions were designed and created to facilitate work. We can point to various uses, but one is the most important – it is the outsourcing of IT processes outside of a company. Cloud allows businesses to focus on primary objectives and at the same time gets rid of costly IT processes. Data are stored on multiple servers in a network and available via website or other protocols from anywhere. Additionally, each service is implemented immediately and on demand. There is no need to wait, there’s no need for complicated and expensive technological facilities. This is the simplicity of the Cloud: just take a credit card and buy the appropriate service for a number of workers in a company.

Cloud has led to a situation in which access to IT technology has become widespread. This is inarguable. So, is Cloud just a new level of virtualization technology based on the existing IT infrastructure, or a brand new cutting-edge solution?

Cloud computing is rather, a set of values passed to an end user. Where does the Cloud begin, and where does it end? The second part of this question is hard to answer, but the first one is easy. Cloud begins at the same point where its true potential starts.

Yes, we all know why…it’s because of the PRICE! Unfortunately this is still the main reason why we all give up on using the SSD disks despite the benefits they bring –  the comparable capacity we can get in HDD for significantly less money. This price gap is still noticeable although the surging prices for hard drives caused by the massive flooding in Thailand which devastated much of the industrial area where about 40% of the world’s supply of hard drives are manufactured (according to market researcher IDC Corp.). This cost factor is the reason why this great technology has not yet conquered the market and…of course the server rooms, as it was expected at its premiere in 2007. But even the prices pale into insignificance if the fast processing of thousands of queries to large databases simultaneously becomes a crucial way for the effective functioning of a company.

Random Access vs. Sequential Access. What is your priority?

SSDs are at their most useful wherever random access is required more than a sequential access. Its memory modules can read and write data almost without any delay. Just think that in SSD access data occurs in about 0.1 milliseconds! While the traditional mechanical drive system must first set the read/write head, then go over to the place in which it has to read or write something. So, for a hard disk it can take from about 3 to even 15 milliseconds. It means that using SSDs can improve average access time from 30 up to 150 times!

Photo: Intel 

Furthermore, the latest SSD models are characterized by high endurance and can achieve much higher transfer speeds than hard drives. They can transmit up to 1 GB of data per second!
Their resilience to shocks and less sensitivity to impact is also worth the money. Because even slight shakes in the drive head in traditional magnetic disks when retrieving data may interrupt transmission, cause damage to magnetic disks, and thus even the permanent loss of data. So everything that we would prefer to avoid… But one of the most important and convincing arguments for investment in SSD is the fact that their memory modules consume far less energy than the electric motors that drive heads in HDDs what in the long run may bring real cost savings to your company. For portable computers it also means a much longer battery life. But most of all SSDs don’t heat up as much, so you will consume much less electricity to keep them cool. You and rest of the company team will also appreciate that thanks to the lack of mechanical parts SSD generate less operating noise – in fact we can even say that they work soundlessly. Can you imagine that? A quite server room? And finally, SSDs don’t need any maintenance. So yes, surely we can all agree that SSDs will fully meet its mission in server rooms… but only when the costs are a secondary issue. Most companies, however do not have such a comfortable situation, that is why the comparison of the capacity and TCO still effectively disqualifies the use of SSDs in most data centers, despite the obvious benefits.

So how we can reject the opportunity to gain higher reliability, faster data access and less power usage?

Well, fortunately there are some compromise solutions, maybe not as cheap as we’d like them to be, but they will not ruin your company’s budget. If you don’t want to give up the opportunity to increase the performance of your storage you may consider some affordable, mixed solutions. You can for example build a RAID both from SSDs disks and HDDs which will allow you to gain at the same time the benefits of these two worlds – the great speed of Solid State Drives supported by the capacity of Hard Drives. In this case you would simply create one volume group with two units – where the first unit with HDDs will be responsible for storing data and the second unit with the SDDs will be liable for the performance.

Another solution to reach satisfactory performance, responsiveness and reliability is to use a RAID card with SSD caching support. That kind of solution is delivered by Adaptec within the MaxIQ SSD Cache Performance or by LSI with MegaRAID CacheCade or with Intel RAID SSD Cache with FastPath* I/O.These solutions simply assist in maximizing both the read and write performances and will efficiently host more users at the same time while reducing the need for additional equipment. Thanks to the “hot spot” detection algorithm, all the files that require frequent access are automatically identified and assigned to the SSD cache. This sophisticated mechanism quickly recognizes the probability that a specific data block may soon be accessed and then just move it into the SSD cache area. The SSD cache automatically displaces “cold” data by the frequently accessed files. That means that the so-called hot data are quickly written and read which significantly reduces I/O latency. In this case you should simply create one unit with HDD and SSD and make them work together like a hybrid.

Producers claim that thanks to their SSD cashing solutions the I/O performance can be faster by three, or even up to eight times in comparison with RAID array based only on HDD. Pretty impressive, don’t you think? And also keep in mind that it will have the additional green and financial benefits from servers being used faster and more efficiently — servers consume less power in data processing and they don’t need to be cooled down so much. So maybe the compromise solutions are the optimal solutions? What is your opinion on this matter?

Trademarks: The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

Data security policy

Briefly and generally – data security policy is a set of precise rules and procedures created to organize the resources of a company including the issues of protection, management or sharing data. Such policies also include scenarios for i.e. data loss, unauthorized access etc. We can ask “what’s it for?” but don’t you think such a question is misplaced? In the modern information society, data management has became a matter of “to be or not to be” on the market. It is not only about the law and the security of personal data. Information is a pillar of modern enterprise.

So, if we talk about data security, what should be taken into account? We can point out three main areas. The first area is software, the second is hardware and the last is models of security, like: events tracking, permission management of levels, control over access, mechanisms and methods of identification and authentication control etc. In the cases of the first and third one, we can talk about avoiding any kind of leakage of data currently stored and used. Supervision over the issue of hardware is however more complex. It concerns the prevention and protection not only on the physical plane but also such problems as utilization.

What to do with useless hardware? How to do it in relation to company’s policy on data security? Let’s think about the problem in two way. First we should take into consideration the situation in which we have to deal with data that does not require protection. In the second case we should take into account critical data – important for the functioning of the company on the market.

“Soft solutions” with storage software

There are plenty of software solutions created and used to remove data from HDD – in this way it will not be probably recovered. Every kind of professional storage software should have such tools. I.e. if you use Open-E DSS V6, there is the possibility to “Remove LV (logical Volume)” with “Fill with zeros” option.

click on image to enlarge

But remember, with a willingness one can do anything – nothing is impossible in the mind. If you keep any kind of critical data on your HDD, it may be a better idea to simply destroy it. The costs of data theft can be many times larger than the gains from storing it.

“Soft solutions” with specialized software

If you don’t use DSS V6 (remember, also DSS V6 Lite version has the “Fill with zeros” option) but you want to be sure that your data will be completely removed. There are a few software solutions for you, we can point you towards freeware, Eraser or commercial: Active@Eeraser, BC Wipe 3, Blancco File Shredder or East-TecEraser. Effectiveness of software deletion is debatable, however let’s suppose it may be a sufficient solution for the ordinary man in the street.

For a more cautious approach we have some advice. If you have any critical data you want to remove (before you will throw your HDD into the trash), try TrueCrypt. Securing data and then deleting them (in a safe way) should give you a hundred percent certainty that your data will be not recovered by an unauthorized person. Why? Even if someone manages to recover a piece of data, the decryption of it will be virtually impossible. Of course, it only concerns the files that have not been removed yet.

“Soft solutions” with OS

With some degree of doubt, but still worth mentioning are also the inner OS solutions. MAC^® users may use the built-in functionality. It is called “Secured Empty Trash” and you can find it in the “Finder” menu. There is also “Erease Free Space” – the Disk Utility program (in “Utilities” folder). It scans the HDD for unused space and deletes it to military (7 passes) or Guttman (35 passes) standards.

Windows^® users can also use its built-in functionality – “Diskpart”. It does not operate through a network connection. Therefore, in cases using the NAS drive(Network Attached Storage) , there is a need to remove the drive and connect it to the hard disk controller in your computer or as an external drive via a USB adapter. Here we present a small step-by-step:

1. Log-in as admin. In Windows Vista you will probably need to use the user account management tool.
2. Push “Windows” + “R” and type: “diskpart”.
3. Type “list disk” and push “Enter”. You will see a list of detected data devices that should identify the discs. The first one will have the number “zero”. The second drive will be number “1″ and so on.
4. Choose the disk – i.e. “select disk 0″.
5. Type “clean all” – your data will be lost now.
6. When the process is finished, type “exit”.

In Linux (and also MAC^®) you can use “dd” command – like:

dd if=/dev/zero of=/dev/device bs=1M

… where: bs = block size and device = sda, sdb etc.

Or better, the “shred” command (only in Linux). Here you will find more information about it: URL

We are not sure about the MAC^® and Windows^® solutions and their effectiveness. However, we can say that Linux and its “shred” command gives a really interesting and probably the best effects of these three solutions.

“Brutal solutions”

As it have been written, “with a willingness one can do anything”. Everyone knows the “deletion” of data is not always equal to an inability to recover them. Even if we use specialized software, risk still remains. That is why, if we take the rules of data security policy seriously, the only way to get rid of old storage devices is to destroy them.

There are many companies equipped with industrial shredding machines. Take a look at the video below – well, at least it looks like good fun.

However, you can also smash it on your own. It is a little bit less esthetic but it doesn’t mean it is ineffective.

People with scientific zeal can also try acid or Thermite. Our advice: Don’t do it at home, even if it looks really cool and effective!

Where to find the best way?

To summarize, only the actual physical destruction of your data storage device will give you one hundred percent certainty that any information won’t be recovered. Therefore, the only reasonable solution for companies is to leave the matter in the hands of specialists who can deal with the physical destruction of storage devices. Of course, there is nothing stopping you from doing it at your own way. In the case of “home users”, if there is no such need, it would be good to consider any of “soft solutions” mentioned above. The appropriate software should effectively discourage amateurs off someone else’s data.

Freebies

• Open-E DSS V6 Lite
• Ereaser
• TrueCrypt

All trademarks mentioned belong to their owners, third party brands, product names, trade names, corporate names and company names mentioned may be trademarks of their respective owners or registered trademarks of other companies and are used for purposes of explanation and to the owner’s benefit, without implying a violation of copyright law.

The first is to use a Small Boot Volume that resides on the Hardware RAID controller. This is a very reliable and stabile boot media and there is no need to buy an extra boot media. The small 2GB logical volume on the RAID Array can be created after creating a RAID set. What you will have is a small logical volume then the data volume will need to be created. Overall what you will have is a RAID set with two RAID volumes, the first for boot and the second for user data.

The DSS software installer will place the DSS boot image onto the small volume during the installation. Once the software is installed and system is rebooted the WEB GUI will show the user data volume as available for format. In this case we have data and operating system separated on the RAID logical volumes. In case of complete RAID corruption we lose the data and we lose the DSS boot image. When event like this has occurred the DSS can be re-installed again new and the data can be restored from a backup. If the administrator has saved the DSS settings, the settings can be restored as well from the settings.cnf file.

Pros:

• Very reliable thanks to RAID
• Costs saving
• Less RMA cases

Cons:

• Logical Volume Manager restoration cannot be used in the event of a RAID crash.
• The User must download LOGs every time the Volume Group and or Logical Volumes configuration is changed.

The second option is a separate boot device like SATA DOM, ATA DOM, HDD, SSD or even extra RAID1 with small HDD or SSD.

The reliability depends on the media quality. The DSS OS image requires a minimum of 1GB of space and a maximum of 2GB. The recommended size is 2GB. The space above 2GB which is available on a particular boot device cannot be used for user data or to create volumes. This is why the ideal size of the boot device is 2GB. If the boot device is flash based (SATA DOM, ATA DOM, SSD) and has a larger size than 2GB, for example 8GB, it is recommended to assign 2GB for DSS and the rest 6GB will remain unused. This will increase flash lifetime as the SSD wear leveling technique will use those unused space for prolonging the service life.

Pros:

• In case of RAID crash the LVM restore can be run from console
• Easy restoration of totally crashed node of Failover
• Easy installation

Cons:

• Need to purchase an extra boot device
• Potential RMA in case of failure
• Only 2GB space is used and the rest cannot be used

Conclusion

For Systems using the Failover function the preferred method will be to have separate boot media because it will be easier to re-create the Failover member after a total RAID crash. This is important only if the cluster services cannot be stopped and in case of total RAID crash and the failed system will need to be recovered as failback ready.

In case the Failover node is totally destroyed the user will still need to have the LOGs in order to re-create the destroyed node. This is again is important if the cluster services cannot be stopped while adding the missing cluster node after recreation.

Single systems can use both boot methods. In every case it is a very good idea to download and save system LOGs after every change in Volume Group or Logical Volume configuration.

NOTE: We do NOT recommend any kind of USB based DOM. The USB based media today have very limited reliability. The lifetime of the USB based media is mostly in the range of 2-3 years and generates a high number of RMA cases.

The evolution of a revolution

Have you ever heard of artificial DNA? How about the idea of using it to store data or perform computational operations? Before we start building castles in the sky, let’s get down to the basics.

DNA or deoxyribonucleic acid – is a nucleic acid which we can call a “genetic manual” used to develop and manage the functioning of all living organisms (with the exception of RNA viruses… We are unsure about aliens). DNA is a kind of storage for genetic information – genes – which we can define (in laymen’s terms) as “organic data”. What if we were also able to store inorganic data with use of deoxyribonucleic acid – like on a HDD or SSD? Leonard Adleman predicted this in his treatise – “Molecular Computation of Solutions To Combinatorial Problems”

We’ve already heard about scientists from the University in Tojama (vide: “Journal of the American Chemical Society”). They built a DNA helix based on non-biological components. What is more, the helix continues to form a clockwise molecule. This begs the question: “So what?” Well, all this sounds mysterious and incomprehensible until we think about one very special feature of DNA – its amazing storage properties. As is stated in the aforementioned publication, DNA can offer really high stability and unprecedented possibilities for the creation of new biotech materials. Don’t you think it would be a revolution in the storage industry?

The storage possibilities of deoxyribonucleic acid are incredible. It is estimated that 0,5 kg of DNA can hold more data than all of the modern HDDs that are now in use. When we compare computers with silicon-based systems to DNA… There is no comparison.

Let us go further. Let us think of DNA as software and enzymes as hardware – amazing, isn’t it? Although it sounds like a dream, remember that dreams can come true. Some time ago we had the opportunity to read in “Nature” magazine about a nano-computer constructed by scientists from the Weizmann Institute. The most striking information was that the mentioned “machine” had an ability to enter input data and receive output data.

There were also rumors about “Maya-I” and “Maya-II” – biocomputers constructed by scientists from the University of New Mexico and NY Columbia University. Equally interesting was the report (“Science”) about the experiments with biocomputers based on RNA working in yeast cells. Their systems were used to create logic gates performing standard calculations based on logical functions – like conjunction, co-denial, “either-or” and “or”. Amazing!

As is easy to see, the idea of biocomputers and DNA computing is still developing. What’s more, this is a direction we’re heading with no turning back – mainly due to the huge range of applications.

Bio and nano magic

While discussing biocomputers, we are really talking about biologically-derived materials prepared to carry out computational functions. This is a very general statement. For better understanding, it is good to make a distinction between technologies: biochemical, bio-mechanical and bioelectronic computers – to show the wide range of possible ways of development.

In the first case we are dealing with biochemical reactions. They may take different forms and can be based on various factors, but it works! Last year we could read in “New Scientist” magazine about Japanese scientists. They taught bacteria to solve simple puzzles. It may sound trivial or funny – bacteria playing Sudoku – but do not judge a book by its cover. However, “programming” bacteria is nothing new and has its limitations (we can store in their genomes only a limited amount of information), think about large structures which would form more powerful and complex structures. Creepy? Yes. Incredible? Definitely!

Bio-mechanical computers are quite similar to the previous ones. In bio-mechanical computers [...] the mechanical shape of a specific molecule or set of molecules under a set of initial conditions serves as the output. Bio-mechanical computers rely on the nature of specific molecules to adopt certain physical configurations under certain chemical conditions. The mechanical, three-dimensional structure of the product of the bio-mechanical computer is detected and interpreted appropriately as a calculated output (source).

Biocomputers can also be based on electrical conductivity. They use biomolecules that conduct electricity in highly specific manners based upon the initial conditions that serve as the input of the bioelectronic system (source).

Quite apart from specific divisions, there is one thing we can be certain of; DNA computing is changing the way we think about computers and silicon-based systems. We are dealing here with a kind of matrix with multiple rows of tubes – but in this case DNA is replacing silicon chips. It is not just a milestone for the scientific world, it is a revolution.

Possible solutions

Apart from academic parlance, let’s focus on the pragmatic side of this issue. There are many possibilities for the implementation of biocomputers. We can point to medicine. Some scientists claim that soon it will be possible to diagnose the presence of viruses in blood or cancer cells. In fact, it is happening now.

Also, an economic benefit can be taken into account. Imagine storage systems with the ability to self-replicate and self-assemble. Theoretically, such a solution may prove to be highly efficient and inexpensive in relation to non-biological computers and components.

It is still hard to tell the direction of the development of biocomputers. We can only talk about its potential – which is huge – if we take into account the most amazing system ever created: the human brain.

Ethics

Whatever one thinks about biocomputers, there is still one thing that should always be considered – ethics. DNA computing raises many questions. Should we treat systems based on DNA as live beings? What about the use of artificial helixes? What is the difference between a biocomputer and a human brain? If the difference is determined by ownership of a self-consciousness, should it be considered an “artificial brain” – as a potential carrier of a conscious existence? Is it still an ordinary object, a machine; or is it a person? It is possible that these and other similar questions will become really important issues faced by the storage industry. It may sound a bit abstract or even funny, but what will you do in case of a fatal error in your live server room? “rm -rf /”?

RAID 5

RAID 5 is one of the most popular implementations. It works almost the same as RAID 4 but with one difference. The parity bits are not recorded on a specifically prepared disk – they are dispersed throughout the matrix structure. What does this mean in the case of failure? Well, it is much easier to recover data (in case of failure of one disk). Also, the read speed is much higher (in comparison to RAID 0).

However, as usually happens, there are also some drawbacks. The necessity of calculating checksums causes a lower write speed. RAID 5 is also expensive in the case of the array reconstruction – if one of the disks have to be replaced after failure. Also read and write operations will slowdown in such case due to the need of calculating checksums.

RAID 5 is the best solution when we talk about data safety. In the case of failure the system will automatically rebuild the lost data so that they can be read (however, the current performance of the matrix will be reduced).

RAID 5

For more information about RAID 5 mode of action take a look here: “How does RAID 5 work? The Shortest and Easiest explanation ever!” by Janusz Bąk.

RAID 6. The next level of excellence and complexity

RAID 6 is sometimes called RAID 5+1. It is because its mechanism is based on RAID 5. Generally it is RAID 5 but array enriched with one additional disk. Thus it contains two independent checksums. It is more reliable, but its implementation is more expensive.

RAID 6

RAID 6 is resistant so much so that its work will be impossible only after the failure of at least three disks. What does it mean? One important thing: It is two times more resistant than RAID 5. What is more, the speed of the whole system is higher than in the case of a single disk. It makes RAID 6 an almost ideal solution (if we do not take into account the costs of implementation).

Enhanced RAID

RAID 5E

RAID 5E (Enhanced) works similar to RAID 5. It stripes data and parity across all of the drives in array. The main reason is a built-in spare drive. Thanks to that, RAID 5E offers good data protection and better throughput. Also read and write operations are more efficient in the case of RAID 5E. It is because of this that such actions proceed faster in the case of four physical disks rather than three with an idle hot spare.

If we take into account the advantages of RAID 5E – good data protection, large storage capacity and high performance (higher than in the case of RAID 5) – we may ask: why it is less used in practice? The response should be sought in relation to its disadvantages.

The main drawback of such a solution is that the spare drive cannot be a part of any other array. So, if there is a need to implement its functionality to another array, preparing another spare drive is the only way. The second thing is RAID 5E supports only one logical drive in one array. And the most important – the thing which makes RAID 5E a useless solution in most cases – it is firmware-specific. Not every kind of controller can support this RAID level (in contrast to i.e. RAID 5).

RAID 5EE – Hot Space

RAID 5EE is an extended version of RAID 5E. It stripes data and parity across all of the drives in the array but it has a more efficient distributed spare. Its rebuild time is also faster.

RAID 5EE provides good data protection – like in the case of RAID 5. It also increases throughput – by using an additional hot spare disk in the array. Thanks to that, in the case of failure, when we have to replace the failed drive, the data from the “Hot Spare” emergency area are automatically transferred to it (to the new drive). The exempt space is than raised as an emergency space.

In this case, the difference between RAID 5E and 5EE is that the RAID 5EE spare is interleaved with the parity blocks (it does not use contiguous free space for the spare).

RAID 5EE requires a minimum of four drives. Just like RAID 5E it is also firmware-specific. It means we can expect problems with a controller. Also implementation costs, and any other – i.e. in case of failure, can be high. So we can say that this option is similar to RAID 5E.

Other possibilities

As it is easy to imagine, there are many solutions based on RAID. We have presented only the basic possible implementations. Although it does not mean the end. We can still point toward: RAID 1E, RAID x0, RAID 6E, 6EE and other. If you want to read about any of them, just let us know. We will do our best to give you the information you need.

RAID – The Series. Check out:

• RAID 0
• RAID 1, RAID 1+0 and RAID 0+1
• RAID 2, RAID 3, RAID 4
• RAID 5 – by Janusz Bak

Do you love your data as much as your MacBook^®? Well, this is the article (and software) for you. Apple Time Machine^® is backup software developed by Apple^® Inc. and was first implemented on Mac OS^® 10.5 (Leopard^®). It cares about your data – your funny photos, important documents or MP3s – no matter what. It helps to store them in a safe way, so even if your expensive, unique and perfect MacBook^® Pro will break down (yup, it’s possible), your data will still be secure. Take a look at our small how-to – it will guide you through the installation process.

The Step-by-Step

First you need the latest version of Open-E DSS V6, 6.0up85 (or higher).  It supports Apple’s Lion^® (Mac OS^® X 10.7) – as well as Time Machine^® support. When the 6.0up85 version is running, you’ll need to create a Volume Group (VG) and NAS Logical Volume. Next enable the AFP functionality under the WebGUI > CONFIGURATION > NAS settings > AppleTalk (AFP) settings. Take a look below:

Now it is time to create a share on the NAS volume that will be used to store Time Machine^® backup files and make it AFP enabled.

I’ve used a guest profile in this manual – without any password. But you can create user’s profile with a password if you wish.

The next step is the configuration of the Mac OS^® X server (don’t worry, it’s easy). By default it does not support any external NAS devices – i.e.: an external storage for Time Machine^® – but it can be easily changed with just one simple command in the terminal:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

Now your MacOS^® X system should be ready to configure the Time Machine^® service. You will need to connect to the AFP enabled share on the DSS V6.

After everything is done, you’ll be able to open the Time Machine^® configuration utility and select the share as backup disk.

That’s it. After a while you should see the first Time Machine^® backup running.

Tip (you’re doing this at your own risk though)

Here’s one good tip for you. Time Machine^® is a nice and easy to use piece of software… with some exceptions. Some of you use slower iBooks^®. What if some of them are too slow?

The default frequency of backup has been set up for one hour – 3600 seconds. It may cause some problems and nuisances on slower machines. Well, nothing is perfect, but there is a solution for such a problem – well, actually two solutions, but solution you choose there are two ways solve the issue.

If you are a kind of odd-job person, you can put some changes in to the code. All you need to do is to find the file in System/Library/LaunchDaemons and search for this line:

StartInterval
3600

… Just replace this with any other frequency (whatever you like) – i.e. 10800 (seconds / 3 hours) and that’s all.

You can also use the Time Machine Scheduler. It will allow you to set up the frequency of backups in intervals – from 1 to 12 hours.

Easy, isn’t it? MacGyver would be proud of you.

MacBook^®, Time Machine^®, Apple^®, iBook^®, Mac OS^®
are trademarks of Apple Inc., registered in the U.S. and other countries.

A dynamic increase of data volumes leads to greater the hardware requirements and greater power consumption – forces us to search for new solutions that can reduce the costs associated with this. So, every new chance to significantly reduce energy costs because of hardware power usage and cooling requirements is more than welcome in most organizations. The same goes for every additional possibility of enhancing the existing, or implementing new, disaster recovery solutions and decreasing the risk of possible data loss. One of the opportunities for an easier and more cost-effective data management is the solution of boot an operating systems from a remote iSCSI disks — and that is why we decided to test it for you.

The iSCSI protocol can send SCSI commands over IP networks between clients (initiators) and SCSI storage devices (targets) – and thanks to that it allow for eliminating the need for a local hard drive. This configuration along with the Automatic iSCSI Failover allow to enjoy high availability of resources and even better prevention against data loss – the worst admins nightmare. This automatically presents you with the opportunity to easily make reliable backups and to improve redundancy and dynamic resources allocation.

Learn how to boot an operating system from highly available iSCSI LUN with Open-E DSS V6 via SAN.

We have checked it in our Lab and now we want to share with you instructions on how to carry out a successful configuration of this process.

So what you will need?

1.Two Open-E DSS V6 licenses (or just one if you choose an option without an Automatic iSCSI Failover)

2.Host Bus Adapter (HBA) for every diskless PC or server as an iSCSI inicjator

3.SANsurfer tool (installed on the operating system).

Our task was to boot an operating system from a remote iSCSI disk. For this purpose we used Open-E Data Storage Software V6, which provides iSCSI target functionality, and an iSCSI initiator, Qlogic HBA QLA 4050C (hardware). Integration with existing Ethernet networks does not require any special, expensive equipment or cables. Because Open-E DSS V6 has an iSCSI failover clustering feature – our iSCSI disk (called LUN – Logical Unit Number) would be perfect for this job. But if you need a low-cost solution, a LUN can be provided without failover functionality. In such an option you will need only one Open-E DSS V6 system.

Configuration in a few easy steps:

STEP 1

The first thing to do is to configure the iSCSi failover. The whole configuration of this process is described in detail here.

STEP 2

After that the iSCSI hardware initiator will have to be configured. To access all firmware settings, not only those which are available directly from controller bios, an iSCSI Qlogic SANsurfer tool will be needed (you can download it from Qlogic website here).

STEP 3

Unfortunately firmware settings can not be modified under an active connection between initiator and target, therefore an auxiliary computer system with SANsurfer installed will be needed (with a Qlogic HBA controller installed. In a prepared environment we ran an SANsurfer and enabled two firmware options: gratuitous ARP and ARP redirect. But why is this so important? Both Open-E DSS V6 iSCSI failover nodes – primary active and secondary passive, have network interfaces with unique MAC addresses and virtual IP’s assigned.

For example:

Node 1 /primary /active/ eth0 MAC : 00:0A:E6:3E:FD:E1 virtual IP:10.10.10.1

Node 2 /secondary /passive/ eth0 MAC : 00:0B:D4:3A:3D:0B
virtual IP:10.10.10.1

Let’s say that an iSCSI hardware initiator is connected to target on Node 1 (primary active, virtual IP:10.10.10.1, MAC : 00:0A:E6:3E:FD:E1 ) and at some point the power fails. Node 1 becomes unavailable so failover occurs. That means that the connection to target was switched to Node 2 with the same virtual IP but with a different MAC address. So to reconnect an iSCSI hardware initiator we have to be able to refresh the MAC address. Therefore, the mentioned firmware option has to be enabled.

STEP 4

Now it is time to connect to the Open-E DSS V6 target. It can be made now on an auxiliary system using the SANsurfer tool or later directly from controller bios. We will describe how to configure the hardware initiator controller bios to boot from the target’s LUN.

The first thing to do is to set the host adapter network:

Menu: configuration settings -> host adapter settings

Secondly, enter target IP and SCSI name.

In this case the target IP is the Open-E DSS V6 virtual IP and the SCSI name is the DSS target name:

Menu: configuration settings -> host adapter settings -> iSCSI boot settings -> primary boot device settings.

If controller network settings are configured correctly and the target is not password protected: (chap user / chap secret) function (menu: scan iSCSI devices) displays all LUN information.

STEP 5

The last thing to do is to set the LUN as the boot device in motherboard bios.

And that’s all. It’s simple, isn’t it? We are curious to hear about your experiences with booting from Open-E DSS V6 iSCSI Failover, so we would like to invite you to take part in the interesting discussions on our storage community forum.

RAID 2 – the bit-level striping with dedicated Hamming-code parity

In the case of RAID 2 all data are stripped (to the bit levels – not block). Each bit is written on a different drive/stripe. Such a solution requires the use of Hamming code for error correction.

Hamming code is a linear error-correcting code named after its inventor, Richard Hamming. Hamming codes can detect up to d – 1 bit errors, and correct (d – 1) / 2 bit errors, where d is the minimum hamming distance between all pairs in the code words; thus, reliable communication is possible when the Hamming distance between the transmitted and received bit patterns is less than or equal to d. By contrast, the simple parity code cannot correct errors, and can detect only an odd number of errors.

source

The number of discs in RAID 2 used to store information is equal to the logarithm of the number of discs that are protecting the mentioned data. All disks in RAID 2 work as one disk which has a capacity equal to the common capacity of all disks used to store data.

RAID 2

While RAID 2 is being used it is important to synchronize all disks. Such a solution requires that the controller, which makes disks, will spin at the same angular orientation – in other way the index will not be reached at the same time. Disintegration will lead to total uselessness of drives in array.

Such a requirement is not the only drawback. Also the need for long Hamming code generation may prove to be problematic by slowing the whole system down.

The mode of RAID 2 action may be hard to understand. The need for using Hamming code, special controllers for disks – it makes RAID 2 a not very popular solution. But if we think about it in a less pragmatic way, it may prove to be very interesting – mainly due to its modus operandi. It introduces many more complex solutions than RAID 0 and RAID 1. While everything works well, RAID 2 proves to be quite a good solution in area of data security. In case of HDD failure – no matter if it was the disk with data or the Hamming code – any part of the array may be reconstructed by the other disks used.

While it is interesting and it has its advantages, we have not heard about any commercial implementations of RAID 2. Solutions based on it were used only in the initial phase of RAID systems usage – before disks were equipped with their own correction code. Modern HDDs use various correction and optymalising algorithms. That is why the Hamming system has started to be less interesting in the area of professional usage and it is no longer implemented in modern controllers.

RAID 3 – another rare one in practice

RAID 3 works as RAID 0 does – it uses byte-level stripping – but it also uses an additional disk in the array. It is used to store checksums and it supports a special processor in parity codes calculating – so we may call it “the parity disk”.

RAID 3

In RAID 3, configuration data are divided into individual bytes and then saved on a disk. Parity byte is determined for each row of data and saved on the mentioned “parity disk”. In case of failure it allows to recover data by an appropriate calculation of the remaining bytes and parity bytes that correspond with them.

Although RAID 3 is rarely used in practice, it is worth pointing out its advantages. First of all is its resistance to damage of one disk in the arrangement. Secondly, high read speed. Unfortunately, it also has a couple of drawbacks.

The read speed is more than satisfactory but write speed is on the contrary – the reason being the necessity of checksums calculating (even RAID hardware controllers cannot solve this problem). She second disadvantage is a matter of disk failure. When it happens, the whole system will work much slower. What is more, although RAID 3 is resistant to breakdown (in case of failure of one disk in the array), replacing a damaged disk is very costly. A third problem is the disk used for calculating checksums  – it is usually the bottleneck in the performance of the entire array.

As can be easily seen, RAID 3 is not a good, reliable or cheap solution. Therefore, as it was mentioned earlier, its use is rare in practice. Systems based on RAID 3 are mostly purposed for implementations where a small number of users refer to the very large files.

RAID 4 – smells like RAID 3 and 5

RAID 4 is very similar to RAID 3. The main difference is the way of sharing data. They are divided in to blocks (16, 32, 64 lub 128 kB) and written on disk s – similar to RAID 0. For each row of written data, any recorded block is written on a parity disk. In short this means that RAID 4 does not strip data at  block levels but it uses byte levels for striping (block-level striping with a dedicated parity disk).

There are also similarities in relation to RAID 5, but it confines all parity data to a single drive. RAID 4 does not use distributed parity.

RAID 4

RAID 4 requires at least three disks for complete implementation and configuration. What is more, it also needs hardware support for parity calculations. This makes it possible to recover data by the appropriate mathematical operations.

If we asked: what is RAID 4 for? we would point out one particular need. Such a solution will work very well in the case of really large files – when sequential read and write data process is used. Using RAID 4 for small portions of data would be not a good idea. The reason is the need to carry out modifications of parity blocks for each I/O session. The need for continuous repeating of such an operation would cause large losses of time and slow down a whole system.

RAID 3 and RAID 4 solutions were replaced by RAID 5. You can read about it in an article by Janusz Bąk: “How does RAID 5 work? The Shortest and Easiest explanation ever!”

RAID – The Series. Check out:

• RAID 0
• RAID 1, RAID 1+0 and RAID 0+1
• RAID 5 – by Janusz Bak
• RAID 5, RAID 6, RAID 5E, RAID 5EE

What is RAID 1

RAID 1 (stripping or mirroring) – the basic idea of such a solution is the replication (mirroring) of two or more discs. It means that while using RAID 1, an exact copy (or mirror) is created on two or more discs. Such a solution gives good read performance and reliability, but storage capacity must be compartmentalized. The reason for this is that the total storage must be equal to the capacity of the smallest disk. i.e.: in the case of an array composed of 3 discs – 250GM, 500GB and 1TB – the usable space will be equal to 250GB.

Why RAID 1

RAID 1 is the easy way to secure data and maintain a reasonable value of write, read and access speed. Adopting this strategy allows for full optimization of these processes. While it is sequential, the overall write speed is equal to all the write operations of each drive in the array. It’s a simultaneous process, the write operations take place parallel to all drives – which means that the recording duration is equal to the duration of the operation of the slowest disk.
It is also possible to set up this strategy for the read process. It can be sequential – so the read speed can be almost equal to RAID 0 or it can take place only from specified drives. The second solution is used when there are significant differences between the read speeds of each of drive in the array.

But returning to the pros – the possibility to increase read speed and reduce access time is the main advantage.  The second one is data safety.  Wikipedia gives us a good and simple example:

Consider a RAID 1 with two identical models of a disk drive with a 5% probability that the disk would fail within three years. Provided that the failures are statistically independent, then the probability of both disks failing during the three year lifetime is 0.25%. Thus, the probability of losing all data is 0.25% over a three year period if nothing is done to the array. If the first disk fails and is never replaced, then there is a 5% chance the data will be lost. If only one of the disks fails, no data would be lost. As long as a failed disk is replaced before the second disk fails, the data is safe.

However, since two identical disks are used and since their usage patterns are also identical, their failures cannot be assumed to be independent. Thus, the probability of losing all data, if the first failed disk is not replaced, may be considerably higher than 5%.

As a practical matter, in a well-managed system the above is irrelevant because the failed hard drive will not be ignored but will be replaced. The reliability of the overall system is determined by the probability the remaining drive will continue to operate through the repair period, that is the total time it takes to detect a failure, replace the failed hard drive, and for that drive to be rebuilt. If, for example, it takes one hour to replace the failed drive, the overall system reliability is defined by the probability the remaining drive will operate for one hour without failure.

source

As it’s plain to see, the possibility of data loss is really small. This is the biggest advantage of RAID 1 – allowing for a quick resolution to the rare possibility of losing data through a disk failure.

However, there are also some drawbacks. As was mentioned, the whole storage capacity of an array is equal to the storage capacity of the smallest disk connected to it. The second thing is something we may call the “immediacy of write and change process.”  RAID 1 can prevent a data loss due to disk failure, but in the case of viruses or human factors (i.e.: accidental deletion of data) RAID 1 is useless. Changes on the first (of two) disc in the array will take place also on the second – immediately or shortly.  One mistake, malware or anything else and data will be lost.

What is RAID 0+1 and RAID 1+0

RAID 0+1 means arrays implemented as RAID 1, whose elements are RAID 0 arrays. Such implementation has the benefits of RAID 0 speed and RAID 1 safety.  It is also much easier to implement than RAID 3, RAID 5 or RAID 6. The main drawback of such a solution is its cost.

RAID 1+0

RAID 0+1

A RAID 1+0 array is implemented as RAID 0, whose elements are RAID 1. It combines the same advantages of RAID 0 (speed) and RAID 1 (safety), but in different way.  RAID 1+0 creates a large stripe of small mirrors.  After failure of a disk, while it is being replaced, only the small fragment of a whole array is rebuilt. Unfortunately, it has the same drawback as that of RAID 0+1 – cost of implementation.

RAID – The Series. Check out:

• RAID 0
• RAID 2, RAID 3, RAID 4
• RAID 5 – by Janusz Bak
• RAID 5, RAID 6, RAID 5E, RAID 5EE