Methodology
RepoPilot is a rule-based analyzer. The same commit always produces the same findings, and every number in a report can be traced to the rules below. No language model is involved in producing findings or scores.
How the score works
The full algorithm, with a worked example, is in docs/SCORING.md in the repository.
- Each check ends as pass (full credit), warn (half credit), fail (no credit), or not applicable / unavailable, which are left out of the calculation rather than counted against the repository.
- Each check has a weight within one category. Category score = 100 × Σ(weight × credit) ÷ Σ(weight) over the scored checks.
- The overall score is the weighted average of the categories that could be scored. If a category has nothing to score, such as Testing in a documentation-only repository, its weight is shared among the others.
- Each rule belongs to exactly one category and runs once, so one problem is never counted twice.
- Bands: A from 90, B from 75, C from 60, D from 40, F below 40. The “Fix next” plan re-runs this calculation with each failing check set to pass, so its gains use exactly the same arithmetic.
| Category | Weight | Scored rules |
|---|---|---|
| Maintainability | 30% | 7 |
| Testing | 25% | 4 |
| Security signals | 25% | 7 |
| Project hygiene | 20% | 11 |
The weights are product decisions, not empirical benchmarks. A high score means good signals were detected. It doesn't certify security, quality or production readiness.
Rules (30)
Observation rules state facts read directly from the repository. Heuristic rules infer from names or text patterns and can be wrong.
kit marks rules with a ready-to-commit fix.
Structure
- STR001
.gitignore presentkit
A .gitignore keeps build output, dependencies, editor files, and local secrets out of version control.
- ×1 hygieneobservation
- STR002
Project manifest recognizedkit
A manifest (package.json, pyproject.toml, go.mod, Cargo.toml…) declares dependencies and lets others build the project reproducibly.
- ×1 hygieneobservation
Documentation
- DOC001
README present
The README is the first thing users and contributors read; without it the project is hard to evaluate or use.
- ×3 hygieneobservation
- DOC002
README completeness signals
READMEs that explain installation and usage with examples reduce onboarding time and support requests.
- ×2 hygieneheuristic
- DOC003
License declaredkit
Without a license, others have no legal permission to use, modify, or distribute the code, which blocks adoption and contributions.
- ×2 hygieneobservation
- DOC004
Contribution guidekit
A contribution guide tells potential contributors how to set up, test, and submit changes.
- ×1 hygieneobservation
Testing
- TST001
Automated tests presentkit
Automated tests catch regressions before users do. RepoPilot detects test files by naming and directory conventions; it never runs them.
- ×3 testingheuristic
- TST002
Test framework configuredkit
A configured test framework makes tests runnable by anyone with a single, documented command.
- ×2 testingheuristic
- TST003
Test script in package.jsonkit
`npm test` is the conventional entry point that contributors and CI systems expect.
- ×1 testingobservation
- TST004
CI runs testskit
Tests only protect the main branch if they run automatically on every change.
- ×2 testingheuristic
Developer experience
- DX001
Linter configuredkit
Linters catch bugs and enforce consistent conventions automatically, which keeps reviews focused on logic.
- ×2 maintainabilityheuristic
- DX002
Formatter configuredkit
An automatic formatter removes style debates and keeps diffs focused on real changes.
- ×1 maintainabilityheuristic
- DX003
Continuous integration configuredkit
CI runs builds, linters and tests on every change so problems are caught before merging.
- ×2 maintainabilityobservation
- DX004
Standard npm scriptskit
Conventional scripts (build/dev/start, lint) give contributors a predictable way to work on the project.
- ×1 maintainabilityobservation
Dependencies
- DEP001
Lockfile committedkit
Lockfiles pin exact dependency versions so every install, CI run and deployment uses the same dependency tree.
- ×2 hygieneobservation
- DEP002
Single JavaScript package manager
Multiple lockfiles from different package managers drift apart and make installs non-deterministic.
- ×1 hygieneobservation
- DEP003
package.json files parse
An invalid package.json breaks every install and tool that reads it.
- ×2 hygieneobservation
- DEP004
Automated dependency updateskit
Dependabot or Renovate keep dependencies current and surface security fixes as pull requests.
- ×1 hygieneobservation
- DEP005
No unbounded dependency versions
Versions like "*" or "latest" accept any future release, including breaking or malicious ones.
- ×1 hygieneobservation
Security signals
- SEC001
No credential-like strings in scanned files
Credentials committed to a public repository can be harvested by automated scanners within minutes. Removing them from the latest commit is not enough; they remain in Git history.
- ×4 securityheuristic
- SEC002
No committed .env fileskit
.env files usually hold API keys, database URLs and other secrets that should never be committed.
- ×3 securityobservation
- SEC003
No committed private key files
Private keys and keystores grant access to servers, signing identities, or TLS endpoints.
- ×2 securityheuristic
- SEC004
Security policy publishedkit
A SECURITY.md tells researchers how to report vulnerabilities privately instead of opening public issues.
- ×1 securityobservation
- SEC005
Safe use of pull_request_target
pull_request_target workflows run with repository secrets and write permissions; checking out untrusted PR code in them lets any contributor execute code with those privileges.
- ×2 securityheuristic
- SEC006
No script injection in workflows
Interpolating attacker-controlled values (issue titles, PR bodies, branch names) directly into run: scripts allows shell command injection in CI.
- ×2 securityheuristic
- SEC007
Third-party actions pinned to commits
Tags and branches of third-party actions can be moved to malicious code; pinning to a full commit SHA makes the referenced code immutable.
- ×1 securityobservation
Maintainability
- MNT001
No oversized source files
Very large source files are harder to review, test and change safely. Size is measured in bytes from the Git tree; generated, vendored, minified and test files are excluded.
- ×2 maintainabilityheuristic
- MNT002
No committed dependency or cache directorieskit
Committed dependencies, virtualenvs and caches bloat the repository and cause platform-specific conflicts.
- ×2 maintainabilityobservation
- MNT003
No very large committed files
Large files slow every clone and fetch. GitHub warns above 50 MB and rejects files over 100 MB; Git LFS stores large assets efficiently.
- ×1 maintainabilityobservation
- REPO001
Repository actively maintained
Archived repositories are read-only and receive no fixes. This check is informational and not scored.
- not scoredobservation
Scan budgets
Limits that keep every scan fast, cheap and safe to run on untrusted repositories.
- Tree entries
- 100,000 (GitHub may truncate very large trees itself)
- Key files
- 40 manifests, README and CI files, 256.0 KB each
- Sampled files
- 80 text files for credential patterns, 100.0 KB each
- Content per scan
- 3.0 MB
- Time per scan
- 25 s
- Oversized source file
- 100.0 KB
- Large committed file
- 5.0 MB