THE HYPERSCALE DATA DISTRIBUTION SOFTWARE PLATFORM

A software engine
for moving data at scale and speed

About Zettar

The world faces exponential data growth. As a result, along with storage, computing, and networking, modern IT needs a new pillar of support: data movement at scale and speed. In response, Zettar Inc. delivers a GA-grade, scale-out, petascale-proven, all-in-one, multi-100+Gbps capable hyperscale data distribution software solution, Zettar zx, and aims to make it the foundation of this fourth pillar.

Unlike most other commercial data transfer software, Zettar zx has a modern, ground-breaking design capable of comfortably tackling the aforementioned exponential data growth. It establishes the brand new "hyperscale data distribution" category for modern digital businesses. zx supports both file and object storage. In addition, as a single application it efficiently tackles the following three "seeminly unrelated" data movement needs:

  1. Moving bulk/stream data over digital connections (multiple Mbps, Gbps, 100Gbps; LAN, Metro, WAN)
  2. Replicate data incrementally, hundreds of millions of small files are fine
  3. Accelerate the data loading/unloading for data shipping devices (AWS Snowball alike)

In March 2019, Zettar won the Supercomputing Asia 2019 (SCA19) Data Mover Challenge (DMC), a grueling 2-month long international competition at the highest level. The Zettar team beat out six other elite national teams from the U.S. and Japan (slide 17) by a wide margin.

The seven SCA19 DMC particpating teams

Zettar is a revenue-supported startup in Palo Alto, California. It has also been awarded grants by both the U.S. National Science Foundation (NSF) and the Department of Energy (DOE) Office of Science.

Zettar zx's performance is unaffected by distance, encryption, and checksumming. This is the goal that numerous parties, both academic and commercial, have struggled to reach for decades but failed. The tremendous value to the business efficiency of all distributed data-intensive enterprises should be evident.

Zettar has deployed its solution at a large global biopharmaceutical company, an European Oil & Gas major, a US DOE Exascale Computing Project (ECP) project, a hyperscale Web property, a well-known research university, and more. Many use cases involve the moving of multiple petabytes of data over distance.

May 2017 Zettar Moves Petabyte Datasets at Unprecedented Speed via ESnet

Each end only employs a modest two-node cluster consisting of two inexpensive 1U commodity servers with 4x10Gbps unbonded Ethernet ports (thus 2 x 4 x10Gbps = 80Gbps - the bandwidth cap)

September 2018 ESnet's Network, Software Help SLAC Researchers in Record-Setting Transfer of 1 Petabyte of Data

With the same data transfer nodes as the 2017 trial, and the 4 AIC SB122A-PH 1U 10-bay storage servers updated with Intel E5-2699v4 CPUs along with 16 Intel Optane P4800X 375GB SSDs, Zettar transferred, with encryption and checksumming, one petabyte of data in just 29 hours over the same connection. The average transfer rate is 75Gbps, or 94% utilization of the available bandwidth.

The May 2017 trial results visualized

Zettar zx generated 1/3 plus of ESnet traffic

The September 2018 trial results visualized

Zettar zx generated 1/3 plus of ESnet traffic again

News

  1. ICM University of Warsaw, Poland news release, Poland-Singapore data transfer over new CAE-1 100G trans-continental link, November 13, 2019
  2. InsideHPC article, Production Trial Shows Global Science Possible with CAE-1 100Gbps link, November 11, 2019
  3. Chin Fang, Is A Science DMZ The Key To Solving Poor Data Utilization?, Bio-IT World, Contributed commentary, Apr 16, 2019
  4. ESnet News & Publications, ESnet’s Networking Prowess on Display at Singapore Conference, April 8, 2019
  5. Supercomputing Asia 2019, Data Mover Challenger Winners Announced!, National Supercomputing Centre, Singapore, Mar 13, 2019
  6. Thomas Coughlin, Wicked Fast Data Transport, Frobes, Feb 11, 2019
  7. Press release, BeeGFS based burst buffer enables world record hyperscale data distribution, Oct 25, 2018
  8. Machine Design, DoE Tests Newest Information Superhighway, October 23, 2018
  9. ESnet News & Publications, ESnet's Network, Software Help SLAC Researchers in Record-Setting Transfer of 1 Petabyte of Data, OCTOBER 17, 2018
  10. InsideHPC article, Big Data over Big Distance: Zettar Moves a Petabyte over 5000 Miles in 29 Hours, October 5, 2018
  11. Press release, Zettar Transferred, with Encryption, One Petabyte of Data in Just 29 Hours Using AIC Servers, October 04, 2018
  12. Chin Fang, Dealing With Fast Growing Data With Hyperscale Data Distribution, Bio-IT World, Contributed commentary, August 17, 2018

Contact Us