Assessment Criteria¶

This page defines how VibeCode QA assesses its own standards and reference implementations. It exists so a separate reviewer, including an independent AI agent, can produce a report on VCQA without relying on the claims written inside the rubric or repo being assessed.

The assessor must judge evidence, not intent. A rubric or reference repo can be useful without being perfect, but the report must say exactly where it is authoritative, where it is only illustrative, and where it should not yet be treated as a gold standard.

Assessment Targets¶

Assess these targets separately:

• Rubric: a versioned standard under /standards/<id>/vN/.
• Reference repo: a public vibecodeqa/ref-* repository.
• Composition: a documented relationship between a rubric, stack items, scanner signals, and one or more reference repos.
• VCQA itself: the standards catalog, published rubrics, reference repos, scanner behavior, docs, CI, issue trail, and claims made on the website.

Do not combine the scores into one number until each target has its own evidence table.

Scoring Bands¶

Use a 100-point score only after the written findings are complete.

Scores must be conservative. Passing CI is evidence, not a score by itself.

Rubric Quality Scorecard¶

Assess each rubric across these dimensions.

Severity, exceptions, and evidence should be judged against the shared rule contract. A rubric can define stricter stack-specific evidence, but it should not weaken the common exception template.

Rubric Failure Modes¶

Flag these even if the page reads well:

• generic framework advice with no VCQA-specific judgment
• rules that cannot be checked from repository evidence
• missing distinction between recommendation and requirement
• no source citations for ecosystem claims
• no exception path for legacy, generated, or vendor-owned code
• no CI/evidence expectation
• no examples of failing patterns
• scoring language that encourages cosmetic compliance

Reference Repo Quality Scorecard¶

Assess each ref-* repository across these dimensions.

Reference Repo Failure Modes¶

Flag these even if CI passes:

• toy code that does not exercise the risky stack boundaries
• mocked behavior that hides the actual integration being demonstrated
• docs claiming a high score without evidence
• no negative tests for safety or security behavior
• generated output committed without a clear reason
• secrets, real project IDs, or production credentials in examples
• CI that only installs or builds without testing the important stack seam
• a reference repo that duplicates a vendor starter without VCQA-owned additions

Independent Agent Protocol¶

An independent AI agent assessing VCQA must follow this protocol.

1. Identify the target and assessment date.
2. Record the exact commit SHA, branch, and repository URL.
3. Read the relevant rubric, docs page, README.md, CI workflow, tests, and report file.
4. Run available local checks when practical.
5. Inspect recent GitHub Actions runs when the repo is public.
6. Compare claims to evidence file by file.
7. Produce findings before scores.
8. Score each dimension with a one-sentence justification.
9. List blocking gaps separately from ordinary improvements.
10. State whether the target is gold, good, partial, draft, or unreliable.

The agent must not accept self-reported scores at face value. Existing docs/vcqa-report.md files are inputs to review, not authoritative verdicts.

Required Report Format¶

Every independent assessment report should use this structure:

# VCQA Independent Assessment: <target>

Date: <YYYY-MM-DD>
Assessor: <agent/model or reviewer>
Target scope: <stack page|rubric|reference repo|composition|VCQA self-assessment>
Source target: <source file, repo, or rubric URL>
Live target: <published target URL or n/a>
Published assessment URL: <published report URL>
Commit: <sha or published edition>

## Verdict
<gold|good|partial|draft|unreliable> — <one paragraph>

## Findings
- <severity>: <file or URL> — <issue and impact>

## Scorecard
| Dimension | Score | Evidence |
|---|---:|---|

## Evidence Reviewed
- <files, workflows, docs, commands, CI runs>

## Claims Not Proven
- <claim>

## Required Fixes
- <fix>

## Useful Follow-ups
- <follow-up>

VCQA Self-Assessment Scope¶

When reporting on VCQA itself, include these targets:

• standards catalog and registry consistency
• every authored rubric
• every public ref-* repo
• the CLI scanner behavior and report output
• GitHub Actions status for docs, standards, CLI, and reference repos
• whether docs, standards, and marketing pages make claims the evidence supports

The result should be published as a dated report, not as an evergreen badge. A current report can link to older reports so regressions and improvements are visible.

Anti-Gaming Rules¶

The assessment must penalize:

• changing wording to evade scanner checks without improving behavior
• deleting tests, docs, or examples to reduce findings
• marking generated or vendor code as owned source without a clear policy
• adding fake smoke tests that do not execute the built artifact
• claiming coverage from a broader standard when the relevant slice is not actually represented
• using a reference repo as proof for a different stack shape

The point of self-assessment is to make VCQA more trustworthy, not to manufacture a high score.