COMPANY

About Zettar

company overview

Zettar Inc. (Zettar) builds and delivers simple, scalable, efficient, and unified data mover software for distributed data intensive engineering and science workloads such as for genomics, life sciences, Oil & Gas, AI, machine learning, transporting data for large scale IoT deployments, autonomous vehicle fleets, smart cities, EDA, Media & Entertainment Studio post-production, light sources (large lasers), accelerators, large telescopes.

The Zettar team has rich first hand experience in helping tier-1 customers in the biopharmaceutical, Oil & Gas, Media & Entertainment Studios, and supercomputing centers in different countries. As a result, even as a software company, the Zettar engineering team has deep and comprehensive understanding of the entire infrastructure stack, storage, computing, networking (including network security).

Furthermore, from the engagement supporting the highly ambitious data transfer requirements (>= 1Tbps point-to-point by 2024) of Linac Coherent Light Source II (LCLS-II), a premier U.S. DOE Exascale Initiative (EI) preparation project hosted at the SLAC National Accelerator Laboratory in Menlo Park, California, all members have gained extensive experience applying the U.S. DOE Exascale Initiative’s “co-design” principle – integrated consideration of storage, computing, networking, and concurrent software for optimal performance. Thus, working with Zettar will help your business to gain such valuable experience as well.

news

Zettar helps enterprises overcome data gravity by providing them the ability to move data at scale and speed, regardless it’s on prem, in the cloud, or any combination thereof, across any distance. We are trusted by the world’s leading companies, well-known national labs and supercomputing centers, and industry-leading partners. You are invited to review the latest Zettar news.

Zettar presented at 2020 Rice Oil & Gas High-Performance Computing Conference, HPC Systems & Facilities

Watch Video Presentation: Moving Massive Amounts of Data across Any Distance Efficiently

Speaker: Chin Fang, CEO, Zettar Inc.

Tuesday, March 3: 12:00pm – 12:20pm

Zettar Announced
Overall Winner of Data Mover Challenge

Zettar is the winner of Data Mover Challenge at SCAsia 2019 Supercomputing
publications

March 25, 2021 |HPCWire

DOE Technical Report: When to Use rsync?

The US Department of Energy (DOE) SLAC National Accelerator Laboratory has released a new Technical Report and associated open source testing tools. The report describes and illustrates a rigorous, comprehensive, and fully automated investigation about the highly popular rsync data copying tool. It answers a key question “When to use rsync?”  We believe this is the first study at this level and scope, carried out using two expertly designed flexible testbeds: Zettar Inc’s and the U.S. DOE ESnet’s 100G SDN testbed. 

First released in 1996, rsync remains the go-to data mover for many IT professionals.  Yet the world is facing exponential data growth. So,

  1. Is rsync still the proper tool to use for almost every data moving task?
  2. If it is still useful, what are the proper range of operations?
  3. How about the effectiveness of some rsync-based tools that run multiple rsync instances?
  4. Are there any alternatives?

 

The report is available at https://slac.stanford.edu/pubs/slactns/tn06/slac-tn-21-001.pdf.

The testing tools are available at https://github.com/fangchin/test_rsync.  They enable any interested parties to use the same methodology to obtain more results in their own environment.

March 25, 2021 |U.S. Department of Energy, Office of Scientific and Technical Information

When to use rsync

Using a series of data transfer results obtained from two testbeds, when to use the popular data copying tool rsync and related tools. Tests have been conducted in local area network (LAN) and wide area network (WAN) environments. We conclude that for files in a certain size range and network latency ≦ 10 ms round trip time (RTT), rsync is still useful for data moving tasks in the category 4 of the U.S. DOE Technical Report “Data Movement Categories” . For more demanding data movement requirements, tools of different classes are suggested. Sample histograms from two DOE user facilities are provided to further support our conclusions.

February 3, 2021 |HPCWire

Elbencho – A New Storage Benchmark for AI

elbencho and the storage sweep tools finally give storage users world-wide the ability to quickly understand their storage systems, rather than depending on published numbers that are meaningless for their actual workloads. In contrast to benchmark suites like IO500 or SPEC SFS, elbencho does not try to predefine a certain workload and instead enables users to test what actually matters to them – be it on a single host or coordinated across multiple storage clients.

February 3, 2021 |InsideHPC

Elbencho – A New Storage Benchmark for AI

elbencho and the storage sweep tools finally give storage users world-wide the ability to quickly understand their storage systems, rather than depending on published numbers that are meaningless for their actual workloads. In contrast to benchmark suites like IO500 or SPEC SFS, elbencho does not try to predefine a certain workload and instead enables users to test what actually matters to them – be it on a single host or coordinated across multiple storage clients.

January 27, 2021 |Jaymie Scotto & Associates (JSA)

Stories from Stanford, Cloud Storage and the Future of Moving Data at Scale & Speed

Chin Fang of Zettar Inc. – 𝗦𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗳𝗼𝗿 𝗺𝗼𝘃𝗶𝗻𝗴 𝗱𝗮𝘁𝗮 𝗮𝘁 𝘀𝗰𝗮𝗹𝗲 𝗮𝗻𝗱 𝘀𝗽𝗲𝗲𝗱 shares how his career evolved from his studies at Stanford to founding tech startups to launching Zettar Inc.

January 08, 2021 | Forbes

Data Movement Types Impact Storage Requirements

The explosion of machine generated data will require advanced automated and controlled data movement to make optimal use of that data. Data movement tools and digital storage required for making use of this data depends upon the size and type of data and the means and uses for that data.  This Forbes article, written by the respected Silicon Valley based storage analyst, Dr. Thomas Coughlin, is based on the SLAC Technical Note, SLAC-TN-20-004 that Dr. Les Cottrell and Dr. Chin Fang co-authored and published on December 30, 2020 on the SLAC SciDoc Website https://stanford.io/3oveo3d.

December 25, 2020 | github.com

Storage sweep tools for elbencho

A storage sweep is a simple and effective way to learn about the performance and characteristics of a storage service. Such a sweep should be carried out by any IT professional responsible for an organization’s storage, especially a new deployment.

The storage sweep tools for elbencho empower users to get results for small files, large files and everything in between with a simple command and even including the automatic creation of a nice graph in the end.

December 21, 2020 | U.S. Department of Energy, SLAC National Accelerator Laboratory (SLAC)

Data Movement Categories

We have endeavored to classify the commonly seen data movement needs, as observed in data-intensive institutions (both commercial and non-profit), into four categories. Knowing how to map a data movement task into one of the four categories helps select proper data mover tools. For each category, how the data storage is involved, high-level examples and the nature of typical solutions are described. Finally, some general remarks are provided to help further orient readers new to this field – the 4th IT pillar.

December 17, 2020 | U.S. Department of Energy, Energy Sciences Network (ESnet)

Zettar zx Evaluation for ESnet DTNs

ESnet is prototyping a Data Transfer Node as-a-Service (DTNaaS) capability that aims to provide optimized, on-demand data movement tools and endpoints to users of the network.  Zettar offers a high-performance data movement solution, zx, that integrates with a number of storage technologies and provides mechanisms for API automation.  An evaluation of the solution within the ESnet testbed environment was performed over the duration of approximately 2 months.  The performance of disk I/O and network interactions were explored in a containerized software environment.

June 12, 2020 | Forbes

Data Center Infrastructure And Transport

The respected Silicon Valley based storage analyst, Dr. Thomas Coughlin wrote a blog about data center infrastructure and the importance of data transport to the effective use of the infrastructure. Two Zettar zx software’s prized features: predictable site-to-site transfer performance and symmetry were pointed out.

March 4, 2020 | 2020 Rice Oil & Gas High Performance Computing Conference 

Moving Massive Amounts of Data across Any Distance Efficiently

In this talk, Zettar shared its intense experience in moving data at scale and speed. Key points: the importance of such endeavors in this age of hybrid and multi-cloud; the current state of the art; common misconceptions and myths to avoid; results from a multi-vendor joint project and a world-leading production trials are used for illustration.

Jan 8, 2020 | Bio-IT World, Contributed Commentary
How To Improve Biopharmaceutical Research Data Utilization?
In this Bio.IT World guest commentary, Zettar shares its comprehensive IT infrastructure expertise related to moving data. The article points out a few very common mistakes often made by IT teams working in the biopharmaceutical industry. They always reduce the research data mobility in this important industry, causing data under-utilization and slowing down important clinical discoveries.
Nov 13, 2019 | ICM University of Warsaw, Poland News Release
Poland-Singapore data transfer over new CAE-1 100G trans-continental link
Interdisciplinary Centre for Mathematical and Computational Modelling (ICM) – University of Warsaw (Poland), A*STAR Computational Resource Centre (A*CRC, Singapore), and Zettar Inc. (U.S.) embarked to jointly conduct a production trial of historical 1st importance.
Nov 11, 2019 | InsideHPC Article
Production Trial Shows Global Science Possible with CAE-1 100Gbps link
Several important engineering take-aways from above historical 1st Poland-Singapore data transfer production trial over CAE-1 100Gbps link are described by the well-known InsideHPC. For example, modern TCP is more effective than proprietary protocols built upon UDP.
Apr 16, 2019 | Bio-IT World, Contributed Commentary
Is A Science DMZ A Key To Solving Poor Data Utilization?
In this Bio.IT World guest commentary, Zettar shares its understanding of network security setup’s impact to the attainable data transfer performance. The improper use of firewalls and its negative impact to achieving high data rates (and thus business progresses) is pointed out. A well-established security methodology widely used in the U.S. DOE national labs community is proposed as an alternative.
Apr 8, 2019 | ESnet News & Publications
ESnet’s Networking Prowess on Display at Singapore Conference
The rapid progress and maturity of Zettar’s data mover products are helped by having the access to a unique 5000-mile 100Gbps loop going from SLAC, Menlo Park, California to Atlanta, Georgia, and back to SLAC. The extensive and rich experience conducting product pre-production trials over the loop at high speed enabled Zettar to win the 2-months long, grueling Supercomputing Asia 2019 (SCA19) Data Mover Challenge (DMC), the only international competition of this type for the past decade.
Mar 13, 2019 | National Supercomputing Centre, Singapore

Supercomputing Asia 2019, Data Mover Challenger, Winners Announced!

Zettar was announced as the overall winner by the host of SCA19 DMC, National Supercomputing Centre Singapore (NSCC). What Zettar accomplished “They achieve an amazing 68 Gbps transfer rate between Chicago and NSCC (Singapore).” still stands unbroken.
Feb 11, 2019 | Forbes
Wicked Fast Data Transport
The well-known Silicon Valley based IT analyst, Dr. Thomas Coughlin wrote a very readable article describing Zettar’s record setting pre-production trial result for the Forbes magazine. The outstanding 94% bandwidth utilization of the available 80Gbps was pointed out.
Oct 25, 2018 | ThinkParQ GmbH Press Release
BeeGFS based burst buffer enables world record hyperscale data distribution
Since 2016, Zettar has been using AIC’s storage servers and the BeeGFS parallel file system, layered on top of Intel’s NVMe SSDs, to establish three world records including the ‘Holy Grail’ world record run: a long distance data transfer of 1 PB in 29 hours, under an 80Gbps bandwidth cap, with full TLS encryption and checksumming. The flat transfer speed profile and the 94% bandwidth utilization are excellent for such a pre-production trial over a production 100Gbps WAN connection.
Oct 23, 2018 | Machine Design
DoE Tests Newest Information Superhighway
Zettar’s world-record setting pre-production trial results are described from a different angle. The U.S. DOE historically has been always having distributed data-intensive projects. Zettar high-performance data mover products are definitely valuable to such projects.
Oct 17, 2018 | ESnet News & Publications
ESnet’s Network, Software Help SLAC Researchers in Record-Setting Transfer of 1 Petabyte of Data
Zettar’s world-record setting pre-production trial results were delightful to many, including the U.S. DOE Energy Sciences Network (ESnet) team. This ESnet official news reported the accomplishment from ESnet’s point of view. Moving petabyte or more per day over distance has becoming more and more important to many importnat DOE funded projects.
Oct 5, 2018 | InsideHPC Article
Big Data over Big Distance: Zettar Moves a Petabyte over 5000 Miles in 29 Hours
Zettar’s world-record setting pre-production trial results achieved in September, 2018 is described by the well known InsideHPC. Also pointed out are Zettar’s ability to move multiple petabytes per week and its transparency in reporting its accomplishments.
Oct 4, 2018 | AIC Press Release
Zettar Transferred, with Encryption, One Petabyte of Data in Just 29 Hours Using AIC Servers
AIC Inc., a long-time technology partner of Zettar Inc., provided the storage backend for the world-record setting pre-production trial completed in September, 2018. This press release provides good background of the joint-efforts of not only AIC & Zettar, but also other technology partners, such as Intel and ThinkParQ.
Aug 17, 2018 | Bio-IT World, Contributed Commentary
Dealing With Fast Growing Data With Hyperscale Data Distribution
This Bio.IT World guest commentary introduces the important concept of hyperscale data to IT teams working in the biopharmaceutical industry. The term is first defined by Zettar and covers the exponential data growth that the world faces, now and in the future.
Nov 17, 2018 | Journal of Physics: Conference Series, Volume 898, Track 4: Data Handling, November 2017
High Performance Data Transfer
This peer-reviewed paper explains why Zettar data mover products are of ever increasing importance to big data, cloud computing, and the needs of data intensive science, High Performance Computing (HPC), defense, the oil and gas industry.

JOIN US

Interested in a career at Zettar?