Pair Programming after the Pandemic and Beyond
Published July 11, 2024
This video features Gerald Carlton and Tobias McNulty at DjangoCon US 2026 in Chicago, Illinois, USA.
Searching for human names in large datasets is harder than it looks. A user searches for "Smith, John," but the database contains "Smyth, Jon" or "Smithe, Johnny." Standard LIKE queries are too strict, full-text search doesn't handle name nuances well, and unindexed fuzzy matching won't scale.
This talk covers how we solved these problems for RecordsDB, a Django app that searches over 50 million North Carolina court records for legal aid organizations and public defenders.
Talk Structure
The session is structured as a technical walkthrough using a Jupyter notebook and a sample Django app. I’ll demonstrate the pitfalls of name searching and how to implement performant solutions. The notebook and source code will be available at the start of the talk so you can follow along.
We’ll cover how to use PostgreSQL phonetic algorithms (Soundex, Daitch-Mokotoff), fuzzy string matching (Levenshtein), and specific indexing strategies to keep searches fast across 50M+ records.
Problem Space
The Challenges: Why name search fails (spelling variations, typos, nicknames).
Scale: Why techniques that work for 10k records break at 1M+.
Case Study: Lessons from a 54-million-record production database.
PostgreSQL Techniques
Phonetic Matching: Using Daitch-Mokotoff to find similar-sounding names with different spellings.
Fuzzy Matching: Using Levenshtein distance for typo tolerance.
Indexing for Performance:
GIN indexes for phonetic matching.
B-tree with text_pattern_ops for prefix searches.
Performance gains from COLLATE "C".
Trade-offs between index size and query speed.
Django Implementation
ORM Integration: Writing custom QuerySet methods and wrapping Postgres functions with RawSQL or custom Func expressions.
Custom Indexes: Defining specific Postgres indexes in Django models.
Fast Type-ahead: Building search-as-you-type interfaces that stay responsive at scale.
Benchmarks: Comparing query times across different search strategies.
Lessons Learned
UX: When to use fuzzy matching versus when it creates too much noise.
Maintenance: Managing large indexes in production.
Accuracy: Balancing precision and recall in legal and public service contexts.
This talk was presented by Tobias McNulty and Gerald Carlton at DjangoCon US 2026 in Chicago, Illinois, USA
https://2026.djangocon.us/talks/search-as-you-type-for-54-million-names-postgresql-django-for-fuzzy-name-matching-at-scale/
Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026