Pure Storage Joins Ultra Ethernet Consortium

Pure Storage is excited to announce that we've joined the Ultra Ethernet Consortium (UEC)-a collaborative effort with other tech industry leaders to rethink an Ethernet networking stack performant enough to support artificial intelligence (AI) and high-performance computing (HPC) environments at scale. Pure Storage's involvement as a contributing partner will allow us to help design another storage networking option for high-throughput supercomputing workloads that are almost exclusively serviced by InfiniBand.

The UEC: A Collaborative Effort for Network Evolution

Ethernet has existed since 1973, when it was designed by Robert Metcalfe at the history-making Xerox PARC laboratory in California. Its intention: to improve on (and eventually compete with) Token Ring and other proprietary protocols in providing a lower cost and more ubiquitous way of networking systems together. Since then, it has become the de facto standard for everything from large data centers to home Wi-Fi systems. Its growth and adoption can be attributed to the reasons it was invented-it was less proprietary than other protocols at the time and less expensive to deploy.

Figure 1: Bob Metcalfe's original concept drawing for Ethernet (image courtesy of Palo Alto Research Center).

But Ethernet is a general-purpose protocol, which means it's designed to support the widest number of networking use cases possible. And, while its design has enabled its ubiquity, it has also narrowed avenues for improving it over time-with the exception of network capacity, the only variable to increase with Ethernet over the years. From the early days of 10Mbps to 1Gbps to 400Gbps today to support high-end data center interconnection needs, the evolution has been beneficial and enabled network capabilities to generally keep up with the ever-increasing demands expected with continual workload growth and propagation.

The UEC's mission is to add another performance improvement dimension to Ethernet that will provide tremendous improvement to the protocol's efficiency-without the need to change any of the physical pieces that comprise an existing network. These new efficiencies would initially benefit enterprise storage use cases that demand a higher throughput but run on existing or expanded Ethernet in the data center. High-end computing models of AI and HPC workloads are potential immediate benefitters of a more efficient Ethernet stack.

Understanding the Storage Leg of AI and HPC Workloads

AI and HPC workloads are incredibly demanding of the CPU, memory, and storage resources they are provisioned. Because of this, resource clustering via interconnections between resources is necessary to scale system performance to meet demand. Historically, InfiniBand has been the networking standard to enable this. It was created in 1999 by the InfiniBand Trade Association (IBTA) to provide a high-throughput, low-latency standard that was more efficient than Ethernet and could connect all elements of supercomputing-servers and storage:

Figure 2: InfiniBand networking logical diagram.

This doesn't mean the Ethernet stack has not been leveraged in the storage leg-RDMA over Converged Ethernet (RoCE) has been introduced in some designs on top of the protocol to allow for a more direct way for compute and storage to communicate. The efforts of the UEC will focus on improving the finer details of the Ethernet networking stack to provide a more evolved alternative to the storage leg of the above model.

Think of the UEC's mission as being able to provide an express lane beside another road that is regulated with stoplights. Both are roads; one just has fewer stops along the way.

Pure Storage's Belief in Ultra Ethernet as a Viable Alternative: Our Reasoning

Important note: Pure Storage joining the UEC does not imply we believe in a zero-sum outcome that expects InfiniBand to be eventually replaced wholesale. It's a viable technology and has continually evolved to remain the dominant networking standard for HPC and AI infrastructure models for the foreseeable future.

Our efforts with the UEC are to enable another option for the storage networking leg in enterprises that are considering implementing data center expansions for high-performance workloads but want to remain on a single networking standard when they do. InfiniBand requires its own NIC, cabling, and switching. Some enterprises may find a better ROI by leveraging existing Ethernet infrastructure for their storage.

Another important clarifying note: if this storage design path is chosen, compute interconnects between nodes will still remain InfiniBand.

Ethernet's strength as a ubiquitous data center networking protocol still remains today from back in the '70s. Pure Storage's contributions to the UEC aim to continue that legacy by evolving it to benefit a wider set of workloads that are beginning to also become more accessible to more businesses and IT operations.

Learn More about the Ultra Ethernet Consortium

Many of the major tech manufacturing companies are involved at different levels of the UEC. It's organized as part of the Linux Joint Development Foundation, which provides a necessary incubation space for collaborative standards development. Visit the UEC's website to learn more.

Pure Storage's commitment to continual innovation extends beyond our own platform-our commitment to the UEC is proof!

Public link

Please use the above public link if you want to share this noodl on another website.