CI checks

Bedrock verifies data with continuous integration (CI) checks. Every commit triggers background evaluation of the repo’s enabled CI checks against that commit’s state.

An extension can carry CI checks too, so that every repo that installs it runs them. See CI checks in an extension.

A CI check never blocks createRepoCommit. Instead, checks act as a merge gate on change requests: a change request can’t merge until every enabled required check has passed on the source branch’s head commit. A required check blocks the merge while its run is pending, when it reports findings, and when its run errors, because a check that can’t run might have failed. The merge fails with CIGateFailed, whose blocking lists each check that blocked it. Checks with advisory severity report their findings without blocking merges.

Check kinds and severity

A CI check definition (POST /repos/:repo_id/branches/:branch/ci-checks) is versioned in the repo’s DAG and specifies:

• name and description: What the check is called and what it verifies.
• kind: Either "stdlib" (a built-in rule that you configure) or "custom" (a TypeScript check function).
• severity: Either "required" (blocks change-request merges until it passes) or "advisory" (reports findings and blocks nothing).
• enabled: Whether the check runs. It defaults to true. To turn a check off, update it with "enabled": false.

Standard library checks

Standard library (stdlib) checks cover common tag and custom field invariants without custom code. Set kind: "stdlib", choose a stdlib_function_key, and pass its configuration object in stdlib_config:

bedrock ci-checks create-ci-check --repo "$REPO" --branch main --data '{
  "name": "Every line item has a cost center",
  "description": "Requires at least one Cost centers tag on every line_item record.",
  "kind": "stdlib",
  "stdlib_function_key": "required_tag",
  "stdlib_config": {
    "family_id": "tag_family_01j...",
    "models": ["line_item"]
  },
  "severity": "required"
}'

Custom checks with bedrock-ci-sdk

For rules that span multiple custom fields, compare related records, or check dates against the evaluation clock, create a "custom" CI check and provide TypeScript source code in custom_source_code.

A custom check runs synchronously inside an isolated Cloudflare Dynamic Worker sandbox with no network access, no timers, and a deterministic clock. It imports defineCheck and optional formula helpers from bedrock-ci-sdk and default-exports the check function:

import { defineCheck, formula } from "bedrock-ci-sdk";

export default defineCheck((ctx) => {
  const budgets = ctx.state.models.budget?.records ?? [];
  const lineItems = ctx.state.models.line_item?.records ?? [];

  for (const budget of budgets) {
    const allocated = formula.sum(
      lineItems.filter((item) => item.values.budget?.id === budget.id),
      (item) => item.values.amount,
    );
    const cap = budget.values.cap_cents;
    if (typeof cap === "number" && allocated > cap) {
      ctx.finding({
        key: `over-cap-${budget.id}`,
        message: `Allocated line items (${allocated}) exceed budget cap (${cap})`,
        target: { id: budget.id },
        field: "cap_cents",
      });
    }
  }
});

The CheckContext interface

The ctx argument passed to defineCheck((ctx) => ...) provides three members:

• ctx.state: The full repository snapshot at the evaluated commit:
  • ctx.state.tagFamilies: Array of tag families at the commit.
  • ctx.state.tags: Array of tags at the commit.
  • ctx.state.tagMemberships: Array of { id, tagId, targetRecordId } memberships, one for each tag on a record.
  • ctx.state.models.<api_name>: Each custom model with its customFields and records (Array<{ id, values, formulas }>). Just as in formulas, reference custom field values are linked in place to the target record object { id, values, formulas }, and user custom field values are objects of the shape { id }. A reference to a model in another repo is { id } too, since a check reads only its own repo’s records.
• ctx.now: A Date fixed to the CI run’s evaluation timestamp. Date.now() and new Date() inside the sandbox also read this fixed clock so that checks remain deterministic.
• ctx.finding({ key, message, target, field }): Reports a violation. key is a non-empty string identifying the finding, message explains the problem, target optionally names the offending record as { id: recordId }, and field optionally names the offending custom field’s apiName. A check that finishes without calling ctx.finding passes.

System One checks

A System One check judges what code can’t, such as whether a submittal’s description matches its spec section, by asking a System One model typed questions. See System One models with ctx.systemOne for what a model answers.

The check stays deterministic. It never calls the model itself: its ask reads the commit’s state and returns what it wants judged, Bedrock asks the model outside the sandbox, and its judge reports findings from the answers. Both run synchronously, in a Worker of their own.

import { defineSystemOneCheck, noul } from "bedrock-ci-sdk";

const submittals = (ctx) => ctx.state.models.submittal?.records ?? [];

export default defineSystemOneCheck({
  model: "jev-1.14.0",
  ask: (ctx) =>
    submittals(ctx).map((submittal) => ({
      key: submittal.id,
      state: { description: submittal.values.description, spec: submittal.values.spec_section },
      questions: { mismatch: noul("Does the description contradict the spec section?") },
    })),
  judge: (ctx, answers) => {
    for (const submittal of submittals(ctx)) {
      if (answers[submittal.id].mismatch.noul > 0.9) {
        ctx.finding({
          key: submittal.id,
          message: "The description contradicts its spec section",
          target: submittal,
          field: "description",
        });
      }
    }
  },
});

• model names a versioned model, such as jev-1.14.0. An alias such as jev-latest errors the run, because its answers could change under a commit. To move to a new model, edit the check, which runs it again as a new version.
• ask receives ctx.state and ctx.now, and returns up to 500 requests of { key, state, questions }. Make questions with noul, choice, and score from bedrock-ci-sdk. Each request asks up to 64 questions, and each key is unique.
• judge receives the same context as a custom check and answers, each request’s answers by its key, then each question’s answer by its name.

Bedrock keeps every answer by a hash of its request, so a later commit that asks the same about an unchanged record reads the answer back instead of asking again. An answer that no run reads for 30 days, such as an edited record’s, is dropped, and a run that asks it again asks the model. Ask about one record per request, so that a change to one record costs one request. A check’s requests that the model hasn’t answered count against the org’s limit of about 600 System One requests a minute, which its extensions share.

A probability isn’t a proof, so a required System One check can block a merge on a judgement that a person would make differently. Make a check advisory until its thresholds have earned your trust.

A run errors at once when the model refuses a request, is down or rate limited, or when the stage has no System One model. An errored required check blocks a merge, and a new commit runs it again.

CI run inspection

CI runs belong to the commit they evaluated rather than to versioned branch state. You can list the CI runs for a commit or for a branch’s head commit:

# List CI runs on a branch's head commit
bedrock ci-runs list-branch-ci-runs --repo "$REPO" --branch q4-budget

# List CI runs on a specific commit
bedrock ci-runs list-commit-ci-runs --repo "$REPO" --commit "$COMMIT_ID"

The response includes pending (whether background check execution is still in progress for the commit) and runs (each check’s status of passed, failed, or errored, along with its reported findings).