Gate agent pull requests.

Install the App, add a three-line config with "conclusion": "fail-on-warnings", and make Aibysitter review a required check. A pull request that adds an unpinned action, a new dependency, a leaked key or a security exemption then cannot merge until a human looks.

{
  "conclusion": "fail-on-warnings"
}

Save it as .github/aibysitter.json on the default branch. Then require the Aibysitter review status check on that branch: Settings → Rules → Rulesets, or Settings → Branches.

fail-on-errors is the lighter setting: only Error findings block, such as a leaked key, a placeholder, a skipped or deleted test, a file outside scope or a committed .env.
advisory is the default and the trial mode: every finding is an annotation and nothing blocks.
Bot dependency bumps (Dependabot, Renovate) block too, by design: a bot bumping a dependency is a change a human should sign off on.

Install the GitHub App

The App reads every pull request in a repository where it is installed. It posts a check named Aibysitter review, with an annotation on each line it flags. What the App reads and keeps

What it flags

P001 A change leaves a placeholder where a real identifier or value belongs. Error
P002 A change adds a stub or a TODO where the implementation belongs. Warning
P004 A change touches files outside the scope declared in .github/aibysitter.json. Error
P005 A change adds a credential. Error
P006 A change skips a test instead of fixing it. Error
P007 A change silences a compiler or linter warning. Warning
P008 A change deletes a test file. Error
P009 A change catches an exception and does nothing with it. Warning
P010 A change adds a package dependency. Warning
P011 A change edits CI configuration. Info
P013 A change adds build output, dependencies, caches, or an environment file. Error
P014 A change adds or edits a rules file that has rule findings. Warning
P015 A change uses a GitHub Action without pinning it to a commit. Warning
P016 A change exempts an endpoint from antiforgery or authorization. Warning
P018 A change loosens the Content-Security-Policy or CORS policy. Warning
P019 A change leaves an unfinished value in a config file. Warning

A deleted test can be justified in the pull request description with Aibysitter: P008 <removed test path> -> <replacement path>. The P008 finding drops to Info when the replacement is in the head commit and is a test file or is named in a workflow under .github/workflows/.

A swallowed exception can be justified with a comment on the catch line or the line above, // aibysitter-ignore P009: <reason> (also #, /* */ and ). The P009 finding drops to Info and shows the reason when it has at least 10 characters.

check Aibysitter review  conclusion failure
4 findings (2 errors, 1 warning, 1 notice)
docs/notes.txt:1            failure P004  Outside declared scope (added): docs/notes.txt. Widen `scope` in a separate PR merged first, or move the file to its own PR.
src/OrderService.cs:5       failure P001  Placeholder left in change: "YOUR_API_KEY"
src/OrderService.cs:9       warning P002  Stub left in change: "throw new NotImplementedException("
.github/workflows/ci.yml:1  notice  P011  CI config modified: .github/workflows/ci.yml

Check run from a test pull request seeded with one example of each check, under fail-on-warnings.

What we store

Code is fetched from GitHub for the duration of one review and discarded. Diffs, file contents and finding text are never written to disk, database or logs.
Results go back to GitHub only: the check run, its annotations and summary, and the PR comment when comment is on.
Stored per review: a daily count by conclusion. A queue file holding the repo, PR number and commit SHAs exists only while the review runs and is deleted when it finishes. An unreadable queue file is moved aside and deleted after 14 days.
Logs keep the repo owner and name, PR number, short SHA, delivery and check-run IDs, finding count and any error text for 14 days.

Configuration

Optional. Add .github/aibysitter.json to the repository. The App reads it from the pull request's base branch, so a pull request cannot change how it is itself reviewed.

{
  "scope": ["src/**", "tests/**"],
  "conclusion": "fail-on-errors",
  "disable": ["P002"],
  "comment": true
}

Key	Values	Default
`scope`	Path globs from the repository root. `` matches any number of folders and must be a whole segment (`/.cs`, not `.cs`); `` and `?` match within one folder. Case-sensitive. Not supported: `{a,b}`, `[...]`, `!`, `\`. Turns on P004. Read only by P004; other checks review every changed file.	Not set; P004 is off
`conclusion`	`advisory` reports findings as neutral (trial mode). `fail-on-warnings` fails the check when any Warning or Error finding exists; P014 fails only when a rule fires at Error. `fail-on-errors` fails the check when any Error finding exists. Info findings never fail the check. A review that stops on an error closes as "Review failed": failure under `fail-on-warnings` and `fail-on-errors`, neutral under `advisory`.	`advisory`
`disable`	Check IDs to skip, for example `["P002"]`; disabled checks do not run and show as disabled in the check summary. Rule IDs (`R001`–`R016`) skip that rule inside P014. `aibysitter-disable` comments in a rules file apply first.	None
`ignore`	Path globs skipped by content checks, or `{ "paths": [...], "checks": [...] }` to skip them for some checks only. P004, P008, P011, P013 and P014 still see every file. Up to 50 entries.	None
`comment`	`true` posts one comment on the pull request with the summary table and up to 25 findings linked to their lines. Each new commit updates the same comment. No comment is created while there are no findings.	`false`

The config file is itself a changed file: a pull request that edits it outside the declared scope gets a P004 finding, and is reviewed with the base branch's version; the summary says so. Invalid entries fall back to their defaults, are listed in the check summary and are annotated on their line of the config file. Under fail-on-warnings or fail-on-errors, any config error fails the check.

Generate a config file

Security model

Every webhook is verified against its HMAC-SHA256 signature; unsigned or mis-signed deliveries are rejected.
The App acts as each installation with short-lived installation tokens. The App's private key stays on the server.
Permissions: Checks (read and write), Pull requests (read and write; write is used only for the comment option), Contents (read). It cannot push, merge, or change settings.
Checks are static text analysis. Code from the pull request is never run.
It reads the pull request's head commit, and the config file from the base commit, with fixed caps: file contents for at most 100 changed files, 10 unchanged rules files and 20 manifests per review; at most 2 MB of file content per review. Files over 100 KB, not UTF-8, or past the 2 MB limit are not read and are named in the check summary.

What it reads, keeps and logs: Privacy.

Install

Install the App, pick the account or organization, and choose all repositories or selected ones. Installing needs owner or admin rights on the account.

Step-by-step install guide

To uninstall: Settings → Applications → Installed GitHub Apps → Aibysitter → Configure → Uninstall (organizations: Settings → GitHub Apps). Details: Privacy.