LLM extraction for a 50,000-entry bibliography

*A 50,000-entry Pompeii bibliography that existed only in print, turned into a clean database by language models and checked by AI agents.*

**Summary:** I built an OCR and language-model pipeline for the Nova Bibliografia Pompeiana, a review tool, and AI review loops that check each flagged entry three times against the page. The backlog of roughly 10,000 to 13,000 flagged citations reached zero unresolved.

**Status:** Review run complete, September 2026. **Year:** 2023 to 2026. **Role:** Data engineer (contract), Department of Classics, UMass Amherst. **Tags:** OCR, LLM extraction, Agent workflows, Python

---

The problem The Nova Bibliografia Pompeiana is the standard reference list for research on Pompeii and Herculaneum. It lived only on paper, in dense columns mixing several languages, abbreviations and cross-references. People could read it. Software couldn’t.

How it works A text-recognition model that keeps the page layout reads each scan. Rules split the text into entries, and language models fill in author, title, journal and year. I chose the models by scoring candidates against a hand-labelled answer key.

```math {m^{\star} = \arg\min_{m}\ \mathrm{cost}(m)}\quad{\text{s.t.}\quad \mathrm{accuracy}(m) \ge \tau} ``` *The cheapest model combination that clears the accuracy bar on the answer key.*

```math \mathrm{route}(x) = \begin{cases} \text{accept} & \hat p(x) \ge \theta \\ \text{re-run, then review} & \hat p(x) < \theta \end{cases} ``` *Anything below a confidence threshold is re-run or flagged instead of guessed.*

Checking every flag I built a review tool that shows each flagged entry beside its page, then ran AI agents that checked every flag against the source three times. Roughly 10,000 to 13,000 flags went to zero unresolved. Each one got a decision; that doesn’t promise every entry is perfect.

• [Open the review studio](https://pbmpstudio.pggzup.chatgpt.site)