Advantech Builds Sensible Generative AI Market Entry Strategy Supported by Partnerships with NVIDIA, Phison, and Nota.ai

By Reece Hayden | 3Q 2024 | IN-7481

Registered users can unlock up to five pieces of premium content each month.

Log in or register to unlock this Insight.

Although largely nascent, deploying Generative Artificial Intelligence (Gen AI) at the enterprise edge would bring significant value to a number of "uncarpeted verticals," including industrial, smart cities, healthcare, energy, transportation, and logistics/fleet management. These verticals face significant challenges around connectivity, data privacy, and growing data transmission costs, which local, edge deployment of Artificial Intelligence (AI) models would certainly ease.

With this in mind, Advantech, a leading provider of Industrial Internet of Things (IioT) solutions, has built on its success in the machine vision market to monetize Gen AI at the edge. It recently released "Edge AI Platform"; an edge AI server that integrates NVIDIA Graphics Processing Unit (GPU) cards, edge AI Software Development Kit (SDK), and NVIDIA AI Enterprise to provide off-the-shelf support for Gen AI inference and even training workloads. The solution includes 10+ optimized Gen AI models, including Llama3-70B. However, given the significant challenges of deploying Gen AI at the edge, Advantech has sensibly supported its product's market entry with a raft of hardware/software partnership announcements, which aim to solve technical hurdles that hinder scaled enterprise adoption:

• Advantech Partnered with Nota.ai,which provides hardware-aware optimization tools for edge AI through NetsPresso. This platform enables training, compression, conversion, and benchmarking to develop optimized AI models for specific applications across various hardware. This reduces resource usage, deployment cost, and time to deployment for Gen AI workloads. Nota.ai supports a wide range of edge devices, mainly targeting the computer vision space. This includes NVIDIA Jetson, Intel, Raspberry Pi, Arm, Renesas, and more.
• Alongside integration with Intel Geti and other software ecosystems, Advantech recently announced collaboration and integration with NVIDIA AI Enterprise. This integration enables enterprises to build on-premises solutions with customized Large Language Models (LLMs). It also enables enterprises to deploy highly-optimized models/applications through NVIDIA Inference Microservices (NIMs).
• Unlike most edge AI vendors, Advantech's solution supports inference, training, and fine-tuning. To support this, it has partnered with Phison to integrate aiDAPTIV+. This ai100 SSD, coupled with aiDAPTIV management software, facilitates Non-Volatile Memory Express (NVMe) offload to enhance performance, reduce training times, and enable large model training on resource/power-constrained devices.

Advantech's decision to partner with these vendors will certainly support its enterprise Go-to-Market (GTM) strategy by reducing technical and commercial barriers to deployments. However, even though Advantech is hitting the right notes, ABI Research expects that enterprise edge Gen AI deployment will remain constrained in the short run. ABI Research's recent forecasts suggest that edge Gen AI spending will only make up 20% of the total (across cloud, edge, and device) by 2030 (see ABI Research's Gen AI Software market data (MD-AISG-101)), with the majority of revenue still coming from the cloud/data center. This is because Gen AI edge deployment will continue to face ongoing technical and commercial challenges:

• Resource Underutilization: Deploying and running GPUs at the edge is expensive. Demand for AI processing at the edge fluctuates depending on the application contributing to various levels of utilization. This may sometimes contribute to excess processing capacity and cost inefficiency. Mitigating this requires resource transparency and automated resource sharing.
• Management and Visibility: Model deployment and management at the edge is challenging. Given that Gen AI models have shown significant accuracy and performance degradation after deployment, this creates significant risk for enterprise deployment.
• Reliability and Alignment for Mission-Critical Edge Applications: Many uncarpeted verticals running AI models at the edge are looking to support mission-critical use cases with accuracy guarantees, e.g., item classification or predictive maintenance. However, Gen AI models are not capable of providing this degree of accuracy, as most are, at best, offering approximately 90% accuracy. This will significantly inhibit any Gen AI adoption.
• Ensuring Processing Availability: Unlike the cloud, edge resources are not scalable and are inherently limited. As processes scale, resource utilization could hit 100%, meaning that certain applications cannot function on-demand. Especially for Gen AI models with significant resource usage, this could have a huge impact on application performance and increase application latency.
• Cost of Always-on Resources: Cloud resources can scale up and down based on workload processing requirements, while they can also use batch processing to achieve better usage-based economics. However, edge servers must be "always-on" and available for processing. This brings significant additional costs and challenges for enterprise deployment.

Advantech's edge Gen AI solution has answered many of the technical questions that this nascent market poses; however, it is still a big step away from its previous edge AI solutions targeting mature, traditional AI use cases. Subsequently, Advantech and other vendors looking to successfully crack this emerging market should look to employ some of the commercial recommendations below:

• Build Ready-Made Solutions with System Integrators (SIs): Edge deployment requires the convergence of Information Technology (IT) and Operational Technology (OT) and can bring significant deployment challenges. Solution providers should engage with SIs to smooth the deployment process and accelerate enterprise deployment.
• Develop Return on Investment (ROI) Model to Showcase Value Creation: To many enterprises, cost will remain the biggest hurdle to Gen AI adoption at the edge. Showcase Total Cost of Ownership (TCO), costs saved (i.e., networking, cloud resources), and value creation depending on use cases deployed.
• Deploy Proofs of Concept (PoCs) with Partners across Different Verticals: Use cases for Gen AI remain immature, especially when considering uncarpeted verticals with mission-critical operations. Building PoCs internally, or preferably with partners, can showcase the operational value proposition and limit friction in enterprise decision-making.
• Leverage Software Platform Partners to Offer Customized Models That Target Specific Use Cases: Generalized models are helpful for AI-competent enterprises to start building and playing with edge deployment; however, the long-tail of the market relies on completely turnkey solutions. Building models optimized for specific applications and targeting specific verticals will accelerate enterprise deployment. Machine vision's commercial success was similarly founded on providing highly optimized, application-specific models.
• Expanding and Updating Model Collection in Line with Market Innovation: Newmodel releases and updates in the Gen AI market are arriving a remarkable pace with new frontier and small AI models being released daily from cloud vendors, AI specialists, and others. Vendors developing turnkey enterprise solutions must provide access to AI models and regularly update them to ensure alignment with market expectations. This will reduce inherent deployment risk, limiting test and wait mindset.
• Completely "Free" Access to Partner Developer Environment: Software like NVIDIA AI Enterprise has a fairly sizable cost associated with it, at around US$5,000 per user per GPU. This discourages a test and evaluation mindset. Accelerating enterprise deployments requires much lower barriers to testing.