Two Models, Two Jobs: When Meeting Summaries Lose Action Items

# What this is

This piece walks through a small, practical experiment and code you can run in about 25 minutes that compares two approaches to getting action items out of meeting transcripts. The goal is not to sell a tool but to measure whether splitting the work across two models actually helps find commitments people make in passing.

# What they built

• A pipeline: one model produces a summary, a second model extracts owners and deadlines.
• A measurement harness: compares model outputs to a ground-truth list of action items and reports recall.
• A realistic transcript file used to test the pipeline, intentionally messy to mimic raw Zoom/Meet output.

Prerequisites listed in the write-up: Python 3.9+, pip, one API key. The tutorial assumes comfort running a Python script.

# The central finding

The measured result is counterintuitive to common practice. A single inexpensive model caught 98% of the action items in the test. The pipeline that people often build — summariser first, then extractor working off that summary — lost items. Specifically, feeding the second model the summary instead of the full transcript dropped recall to 71%, a 29-percentage-point loss. Missing items included a number said once, the explicitly low-priority item, and a conditional promise.

# Why this happens

# The transcript example (what was actually lost)

• A vendor contract renews on the thirtieth and nobody pulled usage numbers.
• The recommendation service was running on a founder's personal API key in production.
• A team member says, "I'll get you the numbers Thursday." That sentence appears in passing and was dropped by the summariser in the tested pipeline.

Those short, conversational statements are exactly the kind of commitments extraction must catch.

# Practical steps the article provides

• Create a working directory, set up a Python virtual environment, and install the required package.
• Export the KODEKEY_API_KEY environment variable (the tutorial uses KodeKey).
• Save a raw transcript file and run the two-model pipeline plus the scoring harness.

The tutorial includes runnable commands and the transcript text so you can reproduce the measurement on your own meeting.

# Recommendations for teams

• Measure before you over-engineer. Run a quick scoring harness on a few realistic transcripts to see whether a single model already meets your recall needs.
• If you run a multi-step pipeline, feed extraction models the raw transcript rather than a condensed summary.
• Use ground-truth test cases that include asides, conditional promises, and single-mention numbers — these expose the common failure modes.

# Bottom line

Summaries can be great prose but bad inputs for action-item extraction. Test on real, messy transcripts. The test harness and the transcript in the write-up make it straightforward to check whether you actually need two models.