Agentic AI: Reshaping Enterprise Architecture & Costs

Agentic AI is fundamentally changing enterprise technology. Dell Technologies CTO John Roese says IT leaders must now rebalance their CPU and GPU systems, integrate data layers, and redesign human workflows. The last year has completely rewritten the enterprise architecture playbook. Roese explained that agentic AI allows users to give an AI system objectives, rather than just single tasks like a chatbot. For example, Google's search engine now uses agents to accomplish objectives and build full pages. This superior user experience is causing enterprises to rebuild generative AI use cases into agentic workflows. Roese also addressed the myth that enterprises need thousands of GPUs. He stated that Dell's biggest internal AI workload supports 40,000 people on only 16 GPUs. This is because enterprise AI primarily focuses on inference, not training, and agents only require inference. However, the architecture for inference workloads is also changing. While chatbots had a light CPU load, AI agents use external tools and knowledge graphs, which require a more balanced architecture of CPUs and GPUs. Roese noted that for every two GPUs, a CPU is often needed. This shift means businesses need to rethink their infrastructure and operational costs to leverage the full power of agentic AI.