You probably have noticed by monitoring IT publications that cloud storage, especially hybrid cloud storage, finally got on businesses' radar screen

At Zettar we have been focusing on creating software solutions that make local and cloud storage indistinguishable for enterprise users since 2008. The current trend is a powerful confirmation of our forward-looking view indeed.

In my STRCUTURE 2010 LaunchPad live interview, I outlined a few major issues that a hybrid approach to storage clouds would address effectively for all businesses. In this blog, I would like to share two more:

1. Policy-based data placements and a true storage optimization

It's a simple fact that for a computing task to be effective, it should keep the data to be processed as close to the computing power as possible - data take time to move. To accomplish this, there has been a major effort in the storage industry to create effective ''policy-based data placement" solutions for sometime.

Entering the cloud age; if a business utilizing a combination of internal and multiple public storage clouds, how can you "exercise" an effective policy-based data placements?

So far, the typical approach only applies to a single vendor's systems, or you must buy connectors from them to have everything play nicely together (or with each other), with the attendant mess of "mix and match".

Furthermore, with the diversity comes the issue and opportunity of creating the "optimal" storage for the best TCO, aka "storage optimization". Traditional vendor solutions are not much of use in this regard either.

A hybrid cloud storage approach, coupled with a software based policy engine, readily and elegantly solves all. Since its inception in mid-2008, Zettar's main design principle has always been "hybrid by design". Thus we have the cleanest, the most complete, and extensible software solutions in this category.

2. A new meaning of redundancy in the cloud age

Data redundancy "seems" to be a well understood concept for anyone in business. But, as the the recent high profile megaupload shut-down event has shown, governmental actions should be factors that a business must take into account in redundancy considerations in the cloud age. If a legit business stores its data with a third party, e.g. a public cloud storage services, government actions can render unexpected business continuity problems for the business!

What is an easy way to get around? Waiting for the issue to be resolved in a court certainly is possible, but not "feasible". "Strength in numbers" is the answer. Enter hybrid storage clouds again. Once more, what we have been advocating at Zettar for more than three years is now confirmed by a dramatic real life example.

Please feel free to post your comments either here or in our forums.

We are pleased to release our latest batch of Zettar ZCloud AWS S3 local sandbox virtual appliances.

The focus of this release is to support two popular desktop class virtualization platforms:

1. VMWare Fusion for Apple's OS X
2. VirtualPC 2007 SP1 for Microsoft Windows (from XP Professional SP3 to Windows 7)

Nevertheless, ZCloud virtual appliances for the three datacenter class virtualization platforms:

1. Linux KVM
2. Xen
3. VMWare Server 2

have also received careful attention and signficant polish from us as well.

Highlights of this release:

• For the two desktop class virtualization platforms, we have implemented our own intelligent time-keeping mechanism to overcome the long standing deficiency of these two in this regard, without resorting to each's proprietary and restrictive virtual machine extension and upgrade software
• For Xen, we have decided this time to base our development and test effort on CentOS 5.5, which is widely used by large scale Xen based virtual machine farms, and is the basis of the commercial Citrix XenServer
• For VMWare Server 2.02, despite the fact that VMWare doesn't officially support recent releases of Ubuntu, our first hand experience has indicated that the virtualization platform actually runs much better (after patching) on Ubuntu 10.04LTS and 10.10
• We are more explicit in recommending Linux distributions and software tools (e.g. browsers) to be used with these virtualization platforms. All our recommendations are based on our first hand experience
• All online documentation have been carefully reviewed and updated for clarity and preciseness
• We have created a new batch of screencasts, replacing all former ones. These new screencasts are shorter, more consistent, and packed with more information
• The licensing mechanism should be even more transparent to users, and more robust too.

The broadened virtualization platform support should make Zettar ZCloud virtual appliance usable to more cloud application developers.

We have addressed all issues that have been identified since the last release. We are confident that this release will help you be more productive in your cloud application development. Enjoy!

Mr. Robin Harris, a highly respected storage industry analyst wrote a succinct blog about Zettar last Friday. His well-read blog immediately generated many comments, both on his blog, and elsewhere on other blogs.

In the past several days, I have observed a few common misunderstandings about Zettar Inc. and what it strives to accomplish. In this blog, I would like to clarify them:

Other AWS S3 server-side implementation

Some parties raised the point that there are other implementations that duplicates the server side of AWS S3, for example, a commenter in @robin’s blog. Nevertheless, as I pointed out in my follow-up, the sandbox is just a prelude of what are coming soon. We have actually duplicated more than just AWS S3.

Our focus has always been to resolve the three big issues that CIOs face/will be facing should they move to object storage: (1) lack of standards (2) vendor lock-in and (3) availability issues. If a software just offers AWS S3 compatibility (e.g. ParkPlace, Eucalyptus’ Walrus) or something new (e.g. OpenStack), how does a CIO deal with the above three issues?

Filesystems that frontend AWS S3

Zettar’s virtual file system, as explicitly pointed out by @robin, frontends multiple object storage clouds concurrently, not just AWS S3. This is a critical difference.

The design and implementation is motivated again by our desire to resolve the three big CIO concerns mentioned above. If a file system only frontends AWS S3, then IMHO it’s insufficient.

Solutions, technologies, or software?

We at Zettar strive to provide solutions, instead of just technologies or software that have to be assembled by end users. Even the modest-looking sandbox includes professional documentation, packaging, management features, polished interface, proven robustness, and after sales support that a business user needs.

EC2/S3 vs Zettar local sandbox

Some parties consider our published benchmark unfair. Note that our benchmark page made it very clear what our test hosts were: hardware, OS, number of CPUs, clock speed, RAM, network connection speed and more.

Furthermore, we stated explicitly that identical tests were performed both for the EC2/S3 combo and our own setup. As much as possible, we have tried to make the comparison valid.

One blogger considered that using a notebook for our efficiency benchmark to be pointless. In fact, that’s exactly an important point we wish to make: having a local sandbox gives developers the freedom to develop, test, and stage cloud applications even in the comfort and convenience of your own notebook – Internet connection is optional.

In the meantime, should you have any questions or comments, please feel free to post them either here or in our forums.

Introduction

I recently ran across a Zdnet blog, Private cloud discredited, part 1. As soon as I was done reading, I knew that I must write something to set the record straight.

The concept of data storage tiering

Due to tradeoffs in cost and speed, the data storage used in any computational activities is always available in a tiered manner; mentally, you can envision this tier as a pyramid. At the top, you have CPU registers, followed by L1, L2, and possibly L3 cache, and then regular RAM. But what are more interesting in this nascent cloud computing age are layers below them: the mass storage, the local network storage, and the remote network storage.

A rule of thumb

Any modern computer architecture textbook should make it clear that although the first 4 or 5 tiers of data storage differ significantly in their respective capacity, the "speed" difference is always less than two orders of magnitude.

But, before the appearance of solid state drives (SSDs), even the fastest mechanical hard drives still offer seek time in the few ms range. The difference in speed between the previous layer, RAM, and the mechanical HD layer, is more than 3 orders of magnitude. In a sense, there was a big "performance gap" for processed data to jump back and forth, limiting throughput as a result.

SSDs has nicely filled this gap. The attention that this type of devices has received and their desirability even to regular computer users actually is a natural consequence of the aforementioned "filling". Human beings like "smooth transitions" -- no one enjoys a jerky ride.

What's next?

Below the layer of mechanical hard drives, the next one is either direct attached storage (DAS) or local network storage.

Over the past decade, tremendous engineering advances have reduced the speed differential between such storage and the immediate layer above to less than two orders of magnitude - making approaches such as Virtualized Tier-less Storage or fully automated storage tiering feasible.

A simple ratio to look at

Now, lets take a look of two numbers:

1. Very few parties, businesses or consumers, can obtain a sustained two way data transfer rate of more than a couple of Mbps over the Internet.
2. On a LAN, today's minimum is 1Gbps. The local network speed in many enterprise clusters can easily go beyond 10Gbps, e.g. infiniband offers 25 Gbps routinely these days.

For many businesses, using a public storage cloud involves data transfers over the Internet. But 1. and 2. above have a more than 3 orders of magnitude speed differential! With such a big "performance gap", bridging them becomes highly technically challenging, if possible at all.

Conclusions

Skipping the private storage cloud layer in the entire data storage pyramid just doesn't make sense at all.

Opinions from large cloud storage service providers, IT vendors, and "thought leaders" may confuse people for a while, but eventually the facts based on laws of physics will prevail. Private storage clouds will become a fixture in the enterprise data storage scene as the cloud computing paradigm wins over more and more CIOs and businesses. A private storage cloud, in many ways, will play a similar role as SSDs have done and will be doing in the data storage pyramid.

Zettar Inc.'s storage cloud federation technologies and products are ready for such a big trend. Are you?

We announced our intention to release an AWS S3 sandbox at the end of this past April at the CloudPlay event held at PlugandPlayTechCenter in Sunnyvale, California. Our live demo was a highly popular highlight of the event, a delightful surprise to us.

The Sandbox has been very useful to us, and by extension, should be useful to many other developers engaged in cloud application development.

After several months of intense work, numerous tests, and plenty of polishing, we made the Zettar AWS S3 Virtual Appliance (ZVA for short) available to the public in the evening of November 3rd, 2010.

The ZVA per se has established a few world's firsts:

ZVA and you

The ZVA should make developing cloud applications targeting AWS not only more productive, but also more fun too. Stay tuned for more delightful surprises from us.

Enjoy!