Open-Source GenAI Adoption with the Dell AI Factory and AMD

"How do I get started?" and "How do I keep my data secure?"

These are the two questions I hear most often when talking with customers adopting generative AI (GenAI). In this rapidly evolving landscape, innovating and creating competitive differentiation requires access to cutting-edge skills, technology, and infrastructure. As our customers embark on this journey, answering these questions is key.

At Dell Technologies, our commitment is to simplify companies' adoption of GenAI and securely bring it to their data. This approach empowers customers to safely uncover valuable insights, strengthen their competitive edge and make faster, data-driven decisions.

The long-standing collaboration between Dell and AMD has been driving innovation and offering a choice of powerful, scalable and efficient solutions for years. In this spirit of collaboration, we're excited to announce updates to the Dell AI Factory, including new servers, enhanced solutions, improved implementation services and key additions to the Dell Enterprise Hub.

Latest Updates to the Dell AI Factory

Returning to the initial questions - "How do I get started?" and "How can I keep my data secure?"

The Dell AI Factory approach simplifies GenAI adoption with tailored, right-sized strategies and engineered architectures for diverse needs. By bringing AI to the data, it ensures data integrity and protection, accelerating a secure and adaptable transformation.

Simplify Open-Source Architecture Using Dell Generative AI Solutions with AMD

The Dell AI Factory streamlines adoption and amplifies success. Adding Dell Generative AI Solutions with AMD enables focused and repeatable outcomes while reducing time to value by up to 86%.¹ These solutions are now updated with the PowerEdge XE9680 server featuring AMD Instinct™ MI300X accelerators. Incorporating this with PowerSwitch networking and PowerScale storage provides a complete hardware solution designed for the high performance and bandwidth requirements of GenAI. However, this solution could not be complete without software integration. By harnessing open-source software like Dell Omnia, Enterprise SONiC, AMD ROCm, PyTorch and Jupyter, organizations can deliver GenAI outcomes on a complete and optimized stack.

We do more than just outline the solution and architecture. Dell's validation process involves comprehensive testing to optimize model configurations, saving customers time spent experimenting. Whether testing Time to First Token (TTFT) for interactive applications like chatbots and customer support assistants, or throughput and latency for inferencing, we've done the testing so our customers don't have to experiment. By thoroughly assessing these elements, we ensure robust and efficient solutions tailored to diverse use cases and help our customers:

• De-risk their buying decisions and choose the right configuration.
• Configure their systems optimally, extracting the best performance out of their investment.
• Focus on addressing their business needs as opposed to trying to figure out the best way to configure and tune their workloads.

Deploy LLMs More Easily with Dell Enterprise Hub on Hugging Face

Our strong relationships were highlighted again as Dell, AMD, and Hugging Face collaborated to support the PowerEdge XE9680 with Instinct MI300X accelerators, providing custom containers and scripts for easier deployment of Llama and Mistral models. These containerized models are uniquely optimized and tuned to the server and accelerator to achieve optimized deployment in just a few clicks leveraging the Hugging Face Text Generation Inference (TGI) backend and available on the Dell Enterprise Hub.

Jumpstart Success with Dell Implementation Services for Generative AI

Getting started with new technology requires more than just generic deployment. New Dell Implementation Services offer a customized operational platform, including Kubernetes cluster configuration, advanced GenAI framework deployment and essential knowledge transfer for customer teams. By bridging the skills gap, we help organizations drive complete business outcomes, covering strategy development, data preparation, operations, management, and scaling.

Elevated Compute for AI in the Data Center

The server infrastructure portfolio has expanded to include five new PowerEdge servers that use the advanced performance of the new AMD EPYC™ 5th Generation processors to drive AI and general workloads.

Next Generation Dell PowerEdge

• Dell PowerEdge XE7745. Built for enterprise AI workloads, this server supports up to eight double-width or 16 single-width internal PCIe GPUs and eight front-facing PCIe network adapters in a dense 4U air-cooled chassis. It's ideal for AI inferencing, model fine-tuning and high-performance computing (HPC).
• Dell PowerEdge R6725 and R7725. Dual-socket servers optimized for scalability and exceptional performance in data analytics, HPC and AI workloads. These platforms can support up to 50% more cores, with an up to 37% increased performance per core resulting in greater performance, efficiency, and improved TCO.² These gains consolidate up to seven five-year-old servers into one server today, resulting in up to 65% lower CPU power consumption.³
• Dell PowerEdge R6715 and R7715. Single-socket servers with AMD 5th Gen EPYC processors, offering exceptional performance and efficiency. These servers provide up to 37% increased drive capacity for greater storage density, perfect for small-scale, cost- effective AI models.⁴ The R6715 sees world record performance for AI and virtualization tasks.⁵

Additionally, updates to the Integrated Dell Remote Access Controller (iDRAC) enhance IT management by allowing remote system monitoring and updates with improved security and efficiency. Collaborating with AMD, Dell transforms datacenters using cutting-edge technology, including support for the latest Instinct accelerators on PowerEdge XE servers, enabling exceptional performance for demanding workloads.

A great example of our partnership in action is OSF Healthcare. "The collaboration between Dell Technologies and AMD has revolutionized our operational performance at OSF Healthcare, allowing us to deliver faster and more reliable services, ultimately enhancing the patient care experience. With the cost-effective and high-performance solutions provided by Dell and AMD, we are achieving unprecedented efficiency, enabling our clinicians to focus more on patient well-being rather than technology constraints," said Joe Morrow, Director of Technology Services.

Unlock the Future of AI with Dell and AMD

All these announcements bring us back full circle to "How do I get started?" and "How do I keep my data secure?" The latest updates to the Dell AI Factory continue to bring AI to the data, and streamline the adoption process, enabling businesses to confidently embark on this transformative journey. Our collaboration with AMD has resulted in the development of new solutions that enhance scalability and modernize data centers, ensuring they maintain a competitive edge. Together, we're committed to delivering advanced technology that not only addresses your concerns but also empowers your business to innovate and thrive in the rapidly changing GenAI landscape.

Ready to transform your AI capabilities? Learn more about how Dell and AMD can empower your business to stay ahead in the AI-driven future. Get started with the Dell AI Factory today!

^{1 Estimate based on Dell analysis in May 2024 comparing time to setup a 2-node Kubernetes cluster for a general-purpose LLM using automated scripts vs deploying a common design manually. Setup time includes base installation only. Actual setup time will vary depending on solution configuration.
2 Based on Dell analysis comparing the top SKU supported in a Dell PowerEdge R7725 of the AMD EPYC 5th gen CPU with 192 zen 5c cores to that supported in the Dell PowerEdge R7625 of the AMD EPYC 4th gen CPUs with 128 zen 4c cores to the top SKU. Data accurate as of 10/2/2024. Actual performance might vary.
Based on Dell analysis of the SPECFPRate scores of the AMD EPYC 5th gen 9755 CPU of 2270 in the R7725with that of the AMD EPYC 4th gen 9754 CPU of 1420 in an R7625. Data accurate as of 10/2/2024. Actual performance will vary.
3 Based on Dell analysis comparing the SPECint and SPECFP scores of the AMD EPYC 5th Gen 9965 in a Dell R7725 (2980 and 2350) with the same scores for an Intel Xeon 8280 in a Dell PowerEdge R740XD (375 and 296). The ratio of the scores shows that 7 of the R740xd servers would give a total score similar to that for the single R7725 as configured above. The CPUs in a single R7725 would have a total TDP of 1000W (2x500W). The CPUs in 7x R740XDs would have a total TDP of 2870W (2*205*7) where each Intel Xeon 8280 has a TDP of 205W. This represents a CPU power reduction of 65%. Data accurate as of 10/2/2024. Actual performance will vary.
4 Based on Dell analysis of specifications comparing the Dell PowerEdge R67x5 server with up to 22 E3.s drive slots with the Dell PowerEdge R66x5 servers with up to 16 E3.s drive slots. Data was collected as of 10/2/2024.
5 Based on Dell PowerEdge servers achieving world record scores for SPECVirt (1-S 4-node SAN score of 3.38 with the Dell PowerEdge R6715 is a world record score for SPECVirt at the 32 core per system level), and TPCx-AI (Scores of 720.386 @SF3, 864.593@SF10 with the Dell PowerEdge R6715 are world records for performance) as of 10/2/2024. Actual performance might vary.}

Public link

Please use the above public link if you want to share this noodl on another website.