AI Release Notes Generator: The Honest Comparison

An AI release notes generator turns a repository's git activity — commits, pull requests — into release notes your customers can actually read. Almost every tool in the category, from git-cliff to release-please to the newer AI layers, actually reads commit messages and PR titles: the quality of your notes depends entirely on your team's naming discipline. A newer approach derives the notes from the real diffs — the content of each change — and holds up even when half your commits are named "fix".

This is an honest comparison of the three tool families, the limit they share (documented by their own users), and what changes when notes are generated from diffs — for your human readers and for the AI agents that, increasingly, read your release notes too.

What a release notes generator actually is

Release notes generator: a tool that automatically produces, for each published version, the readable list of what changed — new features, fixes, breaking changes — from the project's git history.

The definition hides the question that separates the tools: from which part of the history, exactly? Git history has two layers. The metadata — commit messages, PR titles and descriptions, labels — written by humans, of variable quality. And the diffs — the line-by-line content of every change — exact by construction, because they're what the machine runs. Nearly every generator reads the first layer. That single choice explains both their popularity and their disappointing output.

The honest landscape: three families

Deterministic generators: git-cliff, release-please

git-cliff is an open-source, template-driven changelog generator: it parses conventional commits (feat:, fix:, feat!:…) and produces a clean changelog grouped by change type. release-please, maintained by Google, pushes automation further: from the same conventional commits it derives the semver version bump and opens a "release PR" which, once merged, publishes the release and its changelog. semantic-release, in the same family, automates versioning and publishing.

These tools are excellent within their contract — deterministic, auditable, free. The contract is explicit: well-named commits in, a faithful changelog out. If your team enforces conventional commits, they do the job. That's a real "if": the convention has to be imposed on everyone, survive Friday-evening hotfixes, and be followed by the AI agents that now commit to your repo too.

GitHub's built-in button

GitHub offers "Generate release notes" when you create a release: the list of merged pull requests since the previous version, grouped by label, with contributor credits. Zero setup, and honestly the right default for an open-source project with small, well-titled PRs. It is also, literally, a list of PRs — hold that phrase.

The AI layers

The recent wave adds a language model on top of the same sources: the AI rewrites commit and PR titles into customer-facing sentences, groups them by theme, applies a tone. The rewriting is real — the notes read better. But the raw material doesn't change: if the input is a list of titles, the output is a better-written list of titles. A language model cannot recover from "fix" the information that was never put there.

The shared limit, in their users' own words: "a list of PRs doesn't cut it"

This limit isn't our finding — it's a recurring criticism, on Hacker News in particular, of "commit-dump" generators: a list of PRs doesn't cut it. One HN thread with 241 points mocked changelogs that boil down to "bug fixes and performance improvements"; and customers of developer tools describe demanding real patch notes so they can judge whether an update is safe to apply.

The mechanism is structural, not anecdotal: git metadata is a declarative copy of the change, written by a human in a hurry. A generator that reads it inherits its silences. Three practical consequences:

• Badly named commits produce empty notes. "fix", "wip", "address review" — the generator has nothing to say, so it says nothing, or worse, fills the gap with boilerplate.
• The "why" is never there. A PR title says at best what was done, almost never why — and "why" is exactly what your customers ask when a behavior changes.
• Silent breaking changes slip through. A behavior change buried in a PR titled "refactor" triggers no "breaking changes" section, and no caution from the customer about to update.

Release notes derived from real diffs

The structural alternative: generate the notes from the exact layer of the history — the diffs. An illustrative example: a commit named "fix" actually changes the default separator of the CSV export. A metadata-based generator writes "Miscellaneous fixes". Reading the diff surfaces the changed constant, the touched file, the adjusted tests — and can write: "CSV export now uses semicolons by default; imports configured for commas need adjusting." Same commit, two unrelated notes — and only the second one prevents a support ticket.

That's DeployIt's approach: it reads the real code (re-synced on every push) and the git history — commits and diffs — and verifies in the code before asserting. Release notes become a generated view of that knowledge: derived from the content of the changes, they hold up even with badly named commits, they surface behavior changes, and they regenerate on every deploy instead of waiting for someone to write them.