MediaTek 
Data Center Solutions

An XPU is a class of AI accelerator purpose-built to handle the massive parallel compute demands of training and serving large AI models. Unlike a CPU (optimized for general-purpose serial tasks) or a GPU (originally designed for graphics), an XPU is designed from the ground up for AI workloads — maximizing throughput for tensor operations, matrix math, and sparse computations. In a modern AI data center, XPUs sit at the heart of the AI compute system, working alongside CPUs and DPUs (data processing unit is a specialized processor designed to offload networking, storage, and security tasks from a computer's CPU) in a heterogeneous architecture where each processor handles the phase of the AI lifecycle it runs most efficiently. MediaTek designs custom XPUs that are co-optimized with advanced packaging, high-bandwidth memory (HBM), and interconnect as a single unified system.

A custom ASIC (Application-Specific Integrated Circuit) for AI is a chip designed from scratch to match a specific company’s model architecture, workload profile, and infrastructure constraints — rather than relying on off-the-shelf silicon. Hyperscalers and cloud service providers build custom ASICs to gain architectural control, optimize performance per watt and performance per TCO, and avoid the inefficiencies of general-purpose hardware. MediaTek partners with these customers to architect custom ASICs and XPUs end-to-end: from logic die design and memory interface architecture through advanced 2.5D/3.5D packaging and rack-level integration. The result is a workload-specific compute engine that outperforms commodity solutions on the metrics that matter — tokens per watt and tokens per dollar at scale.

An AI compute system is the full hardware stack that trains large AI models and runs high-volume AI inference at scale — it goes well beyond a single chip. At its core is the XPU (AI accelerator), but around it sits a complete fabric: heterogeneous compute (CPUs, DPUs, and XPUs tuned for different workload phases), scale-up interconnect (on-package die-to-die links that keep bandwidth high and energy per bit low), and scale-out interconnect (rack- and data-center-level fabrics built for bandwidth, reach, and efficiency). Memory — HBM, cHBM, LPDDR, SRAM — is co-packaged with compute to reduce data-movement energy. Advanced packaging (2.5D, 3.5D, wafer-level) brings these heterogeneous components together into one coherent engine. MediaTek designs AI compute systems where every element is co-optimized, turning the rack into a strategic asset rather than a collection of parts.

Co-packaged optics (CPO) is a technology that integrates optical engines directly into the chip package — typically on the same silicon interposer or substrate as the compute die and high-bandwidth memory — rather than placing optical transceivers on a separate line card. In AI data centers, where scale-out networking must move enormous volumes of data between racks and rows at low latency and low power, co-packaged optics dramatically reduce the energy per bit and signal loss compared to traditional pluggable optics. As AI clusters scale from hundreds to tens of thousands of XPUs, interconnect power and bandwidth density become first-order constraints, and CPO is emerging as the leading solution. MediaTek integrates co-packaged optics into its AI compute system designs using interposer-based and standard packaging approaches, depending on each customer’s I/O and system-level connectivity needs.

Advanced packaging — including 2.5D and 3.5D integration — enables AI chip designers to escape the physical limits of a single die by combining multiple chiplets (compute dies, memory interface dies, HBM stacks, SerDes, optical engines) into one package that behaves as a unified system. In 2.5D packaging, components are placed side-by-side on a silicon interposer, delivering extreme memory bandwidth inside the package. 3.5D integrations adds vertical stacking, tightening coupling between logic and memory to reduce latency and power. For AI workloads, this matters enormously: memory bandwidth is often the primary bottleneck, and advanced packaging can achieve bandwidth densities that monolithic dies cannot. MediaTek applies design-technology co-optimization (DTCO) and system-technology co-optimization (STCO) across its AI compute systems, with the ambition of achieving 100× improvements in system-level performance per watt and 40× effective increases in reticle-equivalent system size.

AI model training is the computationally intensive process of teaching a large language or multimodal model on massive datasets — it demands extreme memory bandwidth, high sustained throughput, and tolerance for long job durations. AI inference is deploying that trained model to serve real-time predictions, decisions, or generative outputs at scale — it prioritizes low latency, high concurrency, and energy efficiency. Modern AI data centers run both workloads, often on different hardware optimized for each phase. XPUs are tuned for the high-throughput demands of training; DPUs manage network and storage offload; CPUs handle orchestration and control-plane tasks. MediaTek designs heterogeneous AI compute systems where the memory hierarchy, interconnect, and packaging are co-optimized for both training and inference, including differentiated architectures for the prefill and decode stages of inference.

TOPS stands for Trillion Operations Per Second (or Tera Operations Per Second). It measures raw Theoretical throughput, but it ignores power consumption, memory bandwidth constraints, and real workload efficiency — making it a poor proxy for actual AI data center economics. Tokens per watt measures how much useful AI output (tokens generated, predictions served) a system delivers for each watt of power consumed, which directly maps to operating costs and facility capacity. As global data center power consumption climbs and hyperscalers run into grid limits, tokens per watt has become the defining benchmark for evaluating AI accelerators and custom ASICs. MediaTek treats energy as a first-class design constraint across its custom XPU and AI compute system designs — from low-voltage circuit design and compute-in-memory architectures to co-packaged optics and system-level power delivery — to maximize tokens per watt at the rack level.

High Bandwidth Memory (HBM) is stacked DRAM co-packaged with AI accelerators and custom ASICs to provide the extreme memory bandwidth required by large AI model training and inference. As model sizes have grown from billions to trillions of parameters, the memory wall — the gap between compute throughput and memory bandwidth — has become the primary performance constraint in AI systems. HBM addresses this by placing wide-interface memory dies directly on or beside the compute die via advanced 2.5D/3D packaging, achieving bandwidth of multiple terabytes per second per package. MediaTek designs dedicated memory interface dies that connect compute dies to HBM and other memory technologies (cHBM, LPDDR, SRAM), co-optimizing bandwidth, capacity, and latency for each workload class. Looking ahead, emerging technologies like HBF (High Bandwidth Flash) are being designed to enable large-scale inference with unprecedented memory capacity.

Off-the-shelf AI chips are general-purpose accelerators designed to serve many customers across diverse workloads — they are optimized for the median use case, not your specific model architecture, fleet size, or infrastructure constraints. MediaTek’s end-to-end approach means co-designing the custom ASIC or XPU, advanced packaging (2.5D/3.5D), memory hierarchy, high-speed interconnect, co-packaged optics, power delivery, and rack integration as one unified AI compute system — all tailored to a customer’s specific models, power budgets, and TCO targets. This vertical integration allows MediaTek customers to achieve superior performance per TCO and performance per watt versus commodity solutions, while owning their platform architecture for long-term competitive advantage. MediaTek partners with hyperscalers, cloud service providers, and AI platform companies who are making long-term bets on how they train and serve models at scale.

Custom silicon for AI infrastructure refers to chips and systems designed specifically for a company’s AI workloads, rather than purchased as standard commercial products. It spans custom ASICs (application-specific integrated circuits), custom XPUs (accelerator processors), memory interface dies, SerDes chiplets, and optical integration — all architected together as a full AI compute system. Hyperscalers (Google, Meta, Amazon, Microsoft) have led the trend, building custom silicon to optimize training and inference at the scale of millions of GPUs-equivalent compute. Enterprise cloud providers and AI-native companies are increasingly following. The driver is economics: custom silicon can deliver 2–5× better performance per watt and performance per TCO versus off-the-shelf alternatives when designed for a specific model class and deployment environment. MediaTek designs and delivers custom silicon solutions end-to-end — from logic die architecture through advanced packaging, interconnect, and rack integration — in partnership with leading fabs.