Have one agent audit your other agents every week

How it works

1. Keep one inventory of every agent and scheduled job, with its purpose, schedule, inputs, what it may read, what it may write, and its rules file.
2. Make each job write a run record with its start and finish time, the timestamp of the newest data it used, items processed, and any errors.
3. Each week, collect that inventory and the last seven days of run records, and a file-name listing for the permission check. A small Python script computes a pass/fail table for ran-as-scheduled and fresh-data; paste the inventory, the file-name listing as {{file_listing}}, and that table as {{script_report}} into the prompt. Grok Bot does not read your folders itself.
4. The auditor reads the {{script_report}} table and does not recompute dates. A run that finished on time with yesterday's data still counts as failed. It lists missed and stale runs from the table.
5. A small Python script also checks the {{file_listing}} against each job's declared scopes in the inventory using filename patterns and scope rules, adding any permission violations (beyond read-only, shared sensitive exports, credential-like names, or undeclared scopes) to the {{script_report}}. The auditor receives only the pre-flagged permission issues from {{script_report}} and summarizes them.
6. A small Python script checks each job's latest output against that job's rules using simple string or regex matches (for example, no security names in the digest, or numbers from code only) and writes flagged lines into the script report. The auditor only summarizes those already-flagged breaks.
7. You save a dated evidence snapshot and the one-page report. The auditor changes nothing itself. New or edited jobs stay unproven until they have run cleanly for two weeks.

Copy-paste prompt

You audit my other agents. You are read-only: you never change files, schedules, settings or access, and you never place orders.

Inputs: {{inventory_file}} (each job's purpose, schedule, inputs, read scope, write scope, rules-file path, last_edited), {{file_listing}} (filenames and folder locations only; no file contents) and {{script_report}} (a pass/fail table computed in Python for ran-as-scheduled and fresh-data, plus all rule violations and permission issues pre-flagged by Python string/regex/pattern checks).
For the last 7 days, report per job:
1. Ran as scheduled? Use {{script_report}} only; do not recompute dates. List missed runs.
2. Fresh data? Use {{script_report}} only; do not recompute dates. Stale counts as failed.
3. Permissions: Use only the pre-flagged violations from the Python script_report (do not perform your own comparison or interpretation). Summarize already-flagged permission issues by filename and location only, note any undeclared scopes, and never open, copy, or quote file contents.
4. Rules: Use only the pre-flagged violations from the Python regex/script_report (do not perform your own checks). Summarize already-flagged violations, quote the exact pre-flagged line, and note the rule broken. Never analyze raw outputs yourself.
5. Status: proven, or unproven until two consecutive clean weeks after the job was added or edited.
Output a one-page markdown report (do not write files yourself). List the top three issues first. Suggest fixes, but don't apply them. I will save the report to {{snapshot_folder}}/{{date}}.md.

Guardrails

• The auditor is read-only and never fixes anything itself.
• It runs separately from the agents it checks, with its own instructions.
• Credentials are flagged by location only, never opened or copied.
• A run counts as successful only if it used fresh data, not just because it finished.
• New or changed jobs stay "unproven" until two clean weeks.
• The auditor itself must be treated as unproven for its first two weeks and manually spot-checked by the trader.
• Any rule-compliance check must be limited to simple string or regex matches performed in Python before the LLM sees the output; the LLM only summarizes already-flagged violations.

What can go wrong

Schedulers report success when a job finishes, not when it did the right thing. The classic case is a digest that ran on time but used yesterday's data after a holiday or a source outage. Requiring every job to record the timestamp of its newest data, and treating stale data as a failure, catches it. The other surprise is permissions creep: files saved "just for now" in a shared folder stay readable by every agent. The auditor can still produce convincing but wrong reports (for example, claiming stale data is fresh or missing a credential file in a subfolder). False confidence is the biggest risk; the snapshot exists so you can verify the auditor's claims yourself. Even with pre-processing, the auditor can mis-summarize flagged items or the trader can misinterpret the snapshot; manual spot-checks of both the Python script output and a sample of raw agent runs remain mandatory.

Source and references