BN
TechAI Desk•3 views

Google Unveils Specialized TPUs for AI Training and Inference

Google is launching a major hardware update for its eighth-generation Tensor Processing Unit (TPU), splitting its functionality into two specialized processors: one for AI model training and another for inference. This move aims to boost efficiency for the growing field of AI agents. The new chips boast significant performance gains, with the training unit offering 2.8 times the performance of the previous generation at the same cost. Industry adoption is accelerating, with major entities like Citadel Securities and all 17 U.S. Energy Department national labs already utilizing the technology. This development solidifies Google's position as a key provider of custom AI silicon alternatives.

Ad slot
Google Unveils Specialized TPUs for AI Training and Inference

Google is restructuring its Tensor Processing Unit (TPU) for its eighth generation, separating AI model training and inference into two distinct, specialized processors. This strategic move aims to enhance efficiency and meet the growing demands of advanced AI agents, positioning Google as a major competitor in the AI hardware space.

TPU Architecture Split for AI Efficiency

Google announced that the eighth-generation TPU will feature two specialized chips: one dedicated to training AI models and another optimized for inference (running those models). These chips are slated for release later this year.

According to Amin Vahdat, a Google senior vice president and chief technologist for AI and infrastructure, the separation was determined to benefit the community by providing chips individually specialized for the needs of training and serving AI agents.

Industry Context and Competition

The push for custom AI silicon is a trend across the tech industry. Major players are developing specialized hardware to maximize efficiency for unique use cases:

  • Apple: Has integrated neural engine AI components into its in-house iPhone chips for several years.
  • Microsoft: Announced a second-generation AI chip in January.
  • Meta: Is reportedly working with Broadcom to develop multiple versions of AI processors.
Ad slot

While Google is a significant customer of Nvidia, it continues to offer its TPUs as a viable alternative for organizations utilizing Google Cloud services.

Performance Benchmarks and Technology

Google provided specific performance metrics for the new hardware:

  • Training Chip: Enables 2.8 times the performance of the seventh-generation Ironwood TPU, at the same price point.
  • Inference Processor: Shows an 80% improvement in performance.

Both new chips incorporate Static Random-Access Memory (SRAM), with each unit containing 384 MB of SRAM—triple the amount found in the Ironwood TPU. This architecture is designed to deliver the throughput and low latency necessary to run millions of agents cost-effectively.

Growing Adoption of Google AI Chips

Adoption of Google's specialized AI chips is reportedly increasing across key sectors. Notable users include:

  • Citadel Securities: Utilizes quantitative research software built on Google's TPUs.
  • U.S. Energy Department: All 17 national laboratories employ AI co-scientist software running on these chips.
  • Anthropic: Has committed to using multiple gigawatts worth of Google TPUs.
Ad slot