technologybriefs
9:51in productionCh. 1 · Why it started/ 9:51 · ceiling 15 min
Semiconductors

Hardware for artificial intelligence

AI hardware doesn’t make intelligence—it makes computation cheaper, faster, and more brittle.

AI hardware is a set of purpose-built chips—Lisp machines, GPUs, TPUs, NPUs—that accelerate AI workloads by optimising for parallelism, memory bandwidth, and low-precision arithmetic. It delivers real speed and efficiency gains, but at the cost of flexibility, portability, and architectural transparency. Its dominance reflects engineering necessity—not inevitability.

SemiconductorsArtificial intelligenceHardwareSemiconductorsArtificial intelligence across the network →
Chapters & takeaways4
  1. 1:03
    Why it started

    AI hardware exists because general-purpose chips were too slow and inefficient for symbolic AI—and later, neural networks.

  2. 2:58
    Who runs the stack

    GPUs won the training race; TPUs and NPUs carved out inference niches; by 2023, they dominate the market.

  3. 4:37
    The scaling engine

    Compute demand exploded—300,000× in five years—driving precision, interconnect, and memory innovation.

  4. 6:07
    Where it landed

    NPUs moved into laptops and phones by 2017, but remain isolated accelerators—not unified compute subsystems.

Worth your time?

Yes. Study the whole thing.

4/ 5
What works
  • cutting training time for large models
  • enabling real-time video inference on mobile devices
  • reducing energy per inference by orders of magnitude
What does not
  • make AI models more interpretable
  • reduce dependency on cloud infrastructure
  • support arbitrary programming models out of the box
Study it if
  • ML engineers deploying at scale
  • system architects designing edge inference pipelines
  • chip designers building domain-specific accelerators
Skip it if
  • front-end developers
  • policy analysts assessing AI governance
  • researchers working primarily in symbolic logic or probabilistic programming
The written brief1 min read

What it is and the problem it solves

AI hardware is specialised silicon designed to execute AI programs faster and with less energy than general-purpose CPUs. It solves the bottleneck of moving data during massive parallel computation—especially for deep learning training and inference.

How it works

AI hardware uses specialised architectures—dataflow engines, systolic arrays, processing-in-memory—to accelerate matrix operations and reduce data movement. It relies on high-bandwidth memory (e.g., HBM3), optical interconnects, and low-precision arithmetic (e.g., FP8) to improve throughput and energy efficiency.

What works

GPUs have dominated since the 2010s due to parallelism and memory bandwidth. TPUs (introduced 2016) and consumer NPUs (in SoCs since 2017) deliver efficient inference. By 2019, GPUs displaced CPUs for large-scale cloud AI training. From AlexNet (2012) to AlphaZero (2017), compute increased 300,000-fold—doubling every 3.4 months.

What does not

It does not eliminate the software-hardware co-design burden. Fixed-function chips like Taalas ASICs cannot adapt to new model architectures. Processing-in-memory sacrifices memory capacity for efficiency. Event cameras and memristors alone are excluded from the definition—neither qualifies as AI hardware without full system integration.

What it changes

It shifts AI development from algorithm-first to hardware-aware engineering. It concentrates infrastructure leverage in chip vendors and cloud providers. It makes deep learning training and inference orders of magnitude faster and cheaper—but only for workloads that map cleanly to the underlying architecture.

Is it worth your time

Yes—if you train or deploy models at scale, or design systems where latency, power, or inference cost matters. For general software development or small-scale prototyping, the complexity and lock-in often outweigh gains.

Same field · Semiconductors4 of 51
9:20
ChipletChiplets are modular ICs—functional fragments designed for assembly on an interposer. They enable heterogeneous integration, pre-test validation, and IP reuse. But they shift complexity from fabrication to integration, and require standards compliance to deliver on their promise.
9:27
ECC memoryECC memory adds redundancy and decoding logic to catch and fix single-bit memory errors before they propagate. It works via SECDED Hamming or Hsiao codes, using 8 extra bits per word—64+8 for DDR–DDR4, 32+8 per subchannel for DDR5. Mandatory on-die ECC in DDR5 and LPDDR6 addresses rising soft-error rates from shrinking process nodes. It delivers deterministic single-bit correction in servers and critical infrastructure—but offers no protection beyond that, and no benefit where silent corruption is tolerable.
9:33
EUV lithographyEUV lithography replaces transmissive deep-UV optics with reflective Mo/Si mirrors, tin-plasma light sources, and vacuum operation to pattern sub-7 nm semiconductor features. It works — but only after decades of co-development across continents, and only where photon budget, thermal drift and stochastic noise can be managed. It is necessary, not optional, for leading-edge logic — but it does not generalise, simplify or cheapen.
9:45
Multigate deviceThe multigate device is a MOSFET with more than one gate on a single transistor. It solves short-channel effects that arise when planar transistors shrink below ~32 nm. More gates wrap the channel — improving electrostatic control. This suppresses off-state leakage and boosts on-state drive current. N-channel FinFETs reached 17 nm in 1998. A 25 nm FinFET operating at 0.7 V was demonstrated in December 2002. Bulk FinFETs enabled mass production in 2004. It does not eliminate short-channel effects — only mitigates them at the cost of fabrication complexity.
Up next in Technology

Apache ActiveMQ

2006 · 9:36

ActiveMQ made JMS mainstream—but its Java roots and 2007-era architecture now hold back the very systems it was meant to unify.

Play nextBack to Semiconductors
9:36