KubeVirt release-1.9 AI Attribution Review

Following on from my release-1.8 AI attribution review, I’ve repeated the analysis for the release-1.9 development cycle. The results show a dramatic shift in AI tooling adoption across the project.

Background

KubeVirt’s AI Contribution Policy asks contributors to disclose AI-assisted work using git trailers such as Assisted-by:, Co-authored-by:, or Generated-by:. This review covers the commit range origin/release-1.8..origin/release-1.9 (the v1.9.0 development cycle).

Overview

The headline number: half of all non-merge commits now carry AI attribution. That’s a 7x increase in absolute count and a nearly 5x increase in share compared to release-1.8. The number of contributors using AI tooling has grown from 15 to 38. DCO compliance remains perfect at 100%.

AI Tools Used

Note: some commits reference multiple tools, so the per-tool totals exceed 536.

Claude remains dominant but the ecosystem has diversified significantly. In 1.8, only Claude and Cursor appeared. In 1.9, seven distinct AI tools or platforms are represented. Claude accounts for ~87% of commits using AI, Cursor for ~13%, with GPT/Codex, Gemini, Composer, sourcery-ai, and IBM Bob making up the tail.

An interesting new pattern is contributors specifying the underlying model when using Cursor, e.g. Assisted-by: Cursor (claude-4.6-opus-high-thinking) or Assisted-by: Cursor (gpt-5.3-codex-high). This gives finer-grained visibility into what models are being used through editor-integrated AI.

94.4% of AI-attributed commits come from Red Hat engineers (33 of 38 contributors), consistent with the 1.8 pattern. Nvidia, IBM, and community contributors make up the remainder.

PR Quality: AI vs Non-AI

I pulled metrics for 219 AI-attributed PRs and 284 non-AI PRs merged during the cycle via the GitHub API.

Time to Merge

Unlike 1.8, where AI PRs merged slightly faster, the pattern has shifted. AI PRs now take longer on median (13.0 vs 9.0 days), though the mean is virtually identical. When controlled for PR size, the picture is more nuanced:

Medium AI PRs merge notably faster (13.7 vs 18.7 days), consistent with the 1.8 finding. Small AI PRs take considerably longer than small non-AI PRs (7.5 vs 2.8 days), which may reflect that even “small” AI-assisted changes tend to be more substantive and require more review than typical small fixes. Large AI PRs take longer (23.9 vs 18.5 days), possibly reflecting the increased review burden of large AI-generated diffs that was flagged in the 1.8 review.

PR Size

AI PRs continue to skew larger, with 30% at size L or above (vs 25% for non-AI), though the gap is narrower than in 1.8 (where 65% of AI PRs were L+). 13 AI-attributed PRs reference VEPs, including work on Node Hooks (VEP-190), multi-architecture software emulation (VEP-172), and NAD hotplug.

Changes Requested

The change-request rate for AI PRs has ticked up slightly to 11% (from 9% in 1.8), while non-AI PRs remain at 9%. This is a marginal difference and not statistically significant at these sample sizes, but worth continuing to monitor.

Review Intensity

(Median comments per 100 lines changed)

This is a notable improvement from 1.8. In the previous cycle, large AI PRs received significantly fewer comments per line than non-AI PRs (2.5 vs 6.2), raising questions about whether reviewers were scrutinizing AI-generated diffs less carefully. In 1.9, the review intensity for large PRs is essentially identical (1.5 vs 1.7). Medium PRs are also nearly identical (4.9 vs 4.8). The review depth concern from 1.8 appears to have resolved itself.

Statistical Analysis

The median comparisons above are useful for intuition, but they don’t tell us whether the differences are statistically meaningful or just noise. To go deeper, I computed Cliff’s delta (a non-parametric effect size measure suitable for skewed data like time-to-merge), Mann-Whitney U tests for significance, and Benjamini-Hochberg corrections for multiple comparisons. I also computed three complexity metrics beyond lines changed: directory entropy, test code ratio, and distinct subsystems touched.

Effect Sizes

Cliff’s delta ranges from -1 to +1. Positive values mean AI PRs have higher values. Per Romano et al. (2006), |δ| < 0.147 is negligible, 0.147-0.33 is small, 0.33-0.474 is medium, and ≥ 0.474 is large.

The key takeaway: most differences between AI and non-AI PRs are negligible to small. The statistically significant differences (after BH correction at q=0.05) are:

• Total lines changed (δ=+0.168, small) — AI PRs are moderately larger
• Time to merge (δ=+0.147, small) — AI PRs take slightly longer, but this is right at the negligible/small boundary
• Test code ratio (δ=+0.238, small) — AI PRs contain proportionally more test code (median 65% vs 29%)
• Review comments (δ=+0.191, small) — AI PRs receive more review comments in absolute terms

The test code ratio finding is striking. AI-attributed PRs have a median test code ratio of 65% vs 29% for non-AI PRs. This suggests AI tooling is being used particularly for writing tests, or that contributors using AI are more diligent about test coverage.

Notably, changes requested (δ=+0.018), directory entropy (δ=+0.052), and subsystems touched (δ=+0.047) are all negligible — AI PRs are not generating more reviewer pushback, and their structural complexity (how spread out changes are across the codebase) is indistinguishable from non-AI PRs.

Complexity Metrics

Directory entropy (Shannon entropy over the distribution of changed lines across directories) measures how spread out changes are. Higher values indicate changes touching more parts of the codebase. The difference in medians looks large (0.23 vs 0.02) but the effect size is negligible (δ=+0.052), meaning many non-AI PRs also have higher entropy — the distributions overlap heavily.

Kaplan-Meier Survival Analysis

A more rigorous way to compare merge timelines is Kaplan-Meier survival analysis, which shows what fraction of PRs remain unmerged over time:

The curves diverge early — non-AI PRs merge faster in the first ~15 days — then converge. The log-rank test gives p=0.206, meaning the overall difference in merge time distributions is not statistically significant. While the median TTM differs (13.0 vs 9.0 days), the full distributions are not distinguishable at conventional significance levels.

When stratified by PR size, the picture becomes clearer:

• Small PRs: The visible gap (AI slower) has a small effect size (δ=+0.238, p=0.002). Small AI PRs genuinely take longer, likely because they are more substantive than typical small non-AI changes.
• Medium PRs: No significant difference (δ=-0.073, p=0.485). AI and non-AI medium PRs merge at essentially the same rate.
• Large PRs: No significant difference (δ=+0.055, p=0.579). Despite the 5-day median difference (23.5 vs 18.5), the distributions overlap too much for this to be meaningful.

Methodology Note

The analysis uses Cliff’s delta rather than Cohen’s d because time-to-merge and comment counts are heavily right-skewed (not normally distributed). Mann-Whitney U provides non-parametric significance tests, with Benjamini-Hochberg correction for the 10 comparisons. Kaplan-Meier analysis uses the lifelines library with log-rank tests. The full analysis script is available alongside the raw data in the report repository.

Trailer Format: Still the Main Issue

The format inconsistency flagged in the 1.8 review has gotten worse in absolute terms. There are now 59 distinct trailer value formats across 536 commits, up from 17 formats across 76 commits. Some examples:

Key inconsistencies that make machine parsing difficult:

• Case variation: Assisted-by vs Assisted-By vs Co-Authored-By vs Co-authored-by
• Model name inclusion: ~60% include the model name, ~40% don’t
• Email format: some use <noreply@anthropic.com>, some use <claude@anthropic.com>, some omit email entirely
• Context window annotation: some include (1M context) in the model name
• Cursor model specification: some note the underlying model (Cursor (claude-4.6-opus-high-thinking)), most don’t
• Tool chain notation: Claude Sonnet 4.6 via Cursor vs separate trailers for each tool

The Generated-by: trailer defined in the policy remains unused in 1.9, same as 1.8.

Comparison with 1.8

Recommendations

The 1.8 review made five recommendations. Here’s the status and updated guidance:

1. Standardize trailer formats - Not implemented. With 59 formats across 536 commits, this is now urgent. A git interpret-trailers alias or commit hook should be provided to enforce 1-2 canonical formats.

2. Clarify model name inclusion - Not resolved. The split has shifted but remains inconsistent. The policy should either require or discourage model name inclusion.

3. Add CI validation - Not implemented. A prow check for trailer format would catch the 59 format variants before merge.

4. Monitor review depth on large AI PRs - The gap has closed. Large AI PRs now receive essentially identical review intensity to non-AI PRs. This concern can be downgraded but should continue to be tracked.

5. Continue tracking metrics across releases - This review does exactly that, and the trend data is already proving valuable.

Additional recommendations for 1.10:

6. Address multi-tool attribution - Contributors using Claude through Cursor are handling this inconsistently (separate trailers, combined trailers, or only mentioning one tool). The policy should provide guidance.

7. Consider the review load - With half of all commits now AI-attributed and AI PRs skewing larger, the review burden has increased substantially. The project should discuss whether review processes need to adapt.

Conclusion

The 1.9 cycle represents an inflection point for AI tooling adoption in KubeVirt. Moving from 10% to 50% of non-merge commits carrying AI attribution in a single release cycle is a remarkable shift. The data shows this isn’t concentrated among a few contributors — 38 distinct developers are now using AI tools, up from 15.

The quality story is broadly positive: DCO compliance is perfect, review intensity has normalized, and statistical analysis confirms that most differences between AI and non-AI PRs are negligible to small in effect size. The most interesting finding is that AI PRs contain proportionally more test code (median 65% vs 29%), suggesting AI tooling is being used to improve test coverage. The ecosystem has diversified beyond Claude and Cursor to include GPT/Codex, Gemini, Composer, sourcery-ai, and IBM Bob, though Claude remains dominant at ~87% of commits.

The main action item remains trailer format standardization. With 59 distinct formats, the attribution data is becoming increasingly difficult to parse programmatically. Implementing the trailer standardization and CI validation recommended in the 1.8 review should be a priority before the 1.10 cycle.