Application Delivery and Security for AI Applications: Navigating Modern AI Architecture

The really interesting thing, to me, is the new tier, because that's where new and existing application delivery and security capabilities are going to be needed.

In fact, the introduction of a new tier is causing a new data center architecture to emerge with infrastructure capable of delivering the compute, storage, and network resources required to run AI inferencing at scale. This tier is where things like AI network fabrics or AI factories or whatever we're going to call them are emerging. Irrespective of the name, this new infrastructure construct includes the ability to operate existing infrastructure constructs on new hardware systems. That's #4 in the nifty diagram provided.

But there's some new capabilities needed at #2, as well. While the bridge here is a fairly standard N-S data path with an obvious need to scale, secure, and route API traffic (yes, that API is for inferencing but it's still an API) we are likely to see new load balancing algorithms-or at least, new decision criteria incorporated into existing algorithms-at this juncture.

For example, token counts and context windows are particularly important to understanding performance and the load a given request puts on the receiving system, not to mention the impact of token counts on cost. Thus, it's no huge leap of logic to recognize that these variables may become part of any load balancing/traffic routing decision made at #2.

Point #4 is perhaps the most interesting because it returns us to the days of leveraging hardware to offload network tasks from servers. Yes, this is the return of the "let servers serve " approach to architecture. In the modern world, that means leveraging DPUs as a holistic system on which application delivery and security can be deployed, leaving the CPU on the inferencing server to, well, inference. It's a pattern we've seen before, and one that will successfully address any issues with scaling (and thus, performance) inferencing services.

The impact of AI architecture on application delivery and security is both mundane and monumental. It is mundane because the challenges are mostly the same. It is monumental because it introduces additional points in architecture where organizations can strategically address those challenges.

How the industry responds to both the mundane and the monumental will shape the future of application delivery and security.