technologybriefs
9:28in productionCh. 1 · What LoRA replaces/ 9:28 · ceiling 15 min
Models · Software

LoRA (machine learning)

LoRA doesn’t shrink models—it shrinks the cost of changing them.

LoRA makes fine-tuning large language models radically cheaper to train without slowing inference—by replacing full-weight updates with low-rank matrix additions. It works. It scales. It merges. But it assumes weight updates are low-rank—and that assumption fails where tasks demand dense, global parameter shifts.

ModelsSoftwareMachine learningAlgorithmsArtificial intelligenceMachine learning across the network →
Chapters & takeaways5
  1. 0:56
    What LoRA replaces

    LoRA is a 2021 Microsoft technique that cuts fine-tuning costs by freezing base models and adding tiny trainable matrices.

  2. 2:10
    How it computes change

    It works by approximating weight updates as low-rank matrices—linear algebra, not magic.

  3. 3:33
    The numbers that hold up

    On GPT-3, it cut trainable parameters by ~10,000× and training memory by 3×.

  4. 4:41
    No tax on inference

    Merged adapters deploy at native speed—no runtime penalty.

  5. 5:56
    Where it lives today

    It’s now embedded in real post-training pipelines—not just research code.

Worth your time?

Yes. Study the whole thing.

4.5/ 5
What works
  • cuts trainable parameters by ~10,000× on GPT-3
  • reduces training memory by 3× on GPT-3
  • adds zero inference latency after merging
  • enables adapter composition and DPO integration
What does not
  • eliminate need for task-specific data
  • guarantee performance parity with full fine-tuning
  • reduce inference cost before merging
Study it if
  • ML engineers fine-tuning LLMs on constrained hardware
  • teams integrating preference optimisation into deployment pipelines
Skip it if
  • researchers probing fundamental limits of weight plasticity
  • practitioners requiring full-parameter gradient updates
The written brief1 min read

What it is and the problem it solves

LoRA is a parameter-efficient fine-tuning method for large language models and deep neural networks. It solves the problem of prohibitive computational cost and memory demand when adapting massive pre-trained models to new tasks.

How it works

LoRA freezes pre-trained model weights and injects trainable low-rank matrices into each Transformer layer. It decomposes weight update matrices into lower-rank representations. It exploits the hypothesis that fine-tuning updates have low intrinsic rank.

What works

On GPT-3, LoRA reduced trainable parameters from 175 billion to roughly 18 million and GPU memory requirements during training from 1.2 terabytes to 350 gigabytes. After merging, it adds zero inference latency. It supports flexible adapter composition and has been widely adopted in preference optimisation workflows.

What does not

LoRA does not eliminate the need for task-specific data or domain expertise. It does not guarantee performance parity with full fine-tuning across all tasks or architectures. It does not reduce inference cost before adapter merging—only after.

What it changes

LoRA changes post-training workflows by enabling lightweight, composable, and mergeable adapters. It shifts the economics of LLM adaptation: training becomes feasible on consumer-grade hardware for many use cases. It enables integration with methods like DPO without inflating inference latency.

Is it worth your time

Yes—if you fine-tune large models and need to cut training cost, memory, or parameter count without sacrificing inference speed. No—if your task requires full-parameter adaptation or you lack tooling for adapter composition and merging.

Same field · Models4 of 5
9:18
Diffusion modelDiffusion models are a 2015 method for sampling from complex distributions using thermodynamic-inspired noise reversal. They work by learning to undo a diffusion process—turning noise into data, step by step. They succeed in computer vision tasks as of 2024. They do not generalise beyond vision. They require iterative sampling, trading speed for quality. Their mechanism is physical in inspiration, statistical in execution, and narrow in current utility.
8:14
Gaussian splattingGaussian splatting is a volume rendering technique introduced by Lee Westover in the early 1990s and revitalised in 2023 by Inria for real-time radiance field rendering. It uses differentiable, anisotropic, explicit 3D Gaussians integrated from sparse camera-calibrated points. It is rasterisation-based, GPU-optimised, and avoids surface or line primitives. It enables fast photorealistic reconstruction from few images—but offers no mesh output, no topological guarantees, and limited editability.
9:11
Generative adversarial network2014GANs are an adversarial framework for unsupervised generation. They work by pitting two neural networks against each other: a generator learns to produce data that fools a discriminator, which itself improves at spotting fakes. This indirect training yields samples statistically aligned with the training set—including photorealistic images that pass superficial human inspection. But GANs offer no guarantees on semantic coherence, stability, or interpretability. Their value lies in enabling distribution learning without labels—not in reliability, control, or fidelity.
9:29
Reinforcement learning from human feedbackRLHF replaces hand-coded rewards with learned ones—using human rankings as ground truth. It enables alignment at scale for language models, but only within the narrow bounds of the annotation task. It is widely used, not universally reliable.
Up next in Technology

Mac (computer)

1984 · 9:38

The Mac didn’t invent the GUI — it weaponised it for mass adoption.

Play nextBack to Models
9:38