socraticblock.
← All work

SYSTEMS / AGENTIC ENGINEERING

Hermes engineering
fleet.

Designing an environment where AI agents can investigate, propose, implement and challenge work—while scope, testing and consequential decisions remain visible.

CUSTOM ORCHESTRATIONITERATIVE / ONGOINGAI-NATIVE DEVELOPMENT

Conceptual workflow illustration, not a live system trace.

01 / THE PROBLEM

Make engineering work more structured.

I needed a repeatable way to coordinate AI-assisted work across different products. A raw chat or a stream of code changes isn't enough: tasks need context, explicit scope, review, checks, documentation and responsibility for release.

02 / MY CONTRIBUTION

The system around the agents.

I designed, configured and continually improve custom role definitions, task workflows, reusable procedures, automations, review boundaries and operational checks around the third-party Hermes Agent foundation.

I rely heavily on AI to read, debug and implement code. My personal contribution includes identifying problems, shaping workflows, directing the agents, commissioning adversarial audits, examining their results and testing observable behavior.

Important credit boundary: the underlying Hermes Agent framework is third-party; this case describes my own orchestration and operating environment built around it.

03 / ARCHITECTURE & PROVENANCE

The design lives around the runtime.

The Hermes Agent runtime comes from Nous Research. My work centers on the operating layer around it: role design, skills, task coordination, model choices, review practices and product decisions. The diagram distinguishes those responsibilities and their limits.

ARCHITECTURE / RESPONSIBILITY

An operating layer, not a black box.

Separate what is supplied by Hermes Agent, what I customize and direct, and what can actually be verified.

  1. 01 / FRAMEProblem & constraints
  2. 02 / DIRECTAgent-led work
  3. 03 / INSPECTArtifacts & checks
  4. 04 / DECIDEHuman acceptance

Four boundaries. Tap any layer for its explanation.

01 / Human directionAccountable decisions

I frame the problem, set scope, direct the work, challenge results and test observable behavior myself.

Owner responsibility
02 / Custom operating layerOrchestration around agents

I shape specialized profiles, reusable skills, scripts, model-selection conventions and review practices.

Custom configuration
03 / Hermes Agent runtimeA third-party foundation

Nous Research provides the agent loop, tools, profile and skill loading, scheduling, gateway and MCP-client capabilities.

Upstream framework
04 / Evidence and reviewClaims need supporting records

Plans, code changes, test outputs, review findings and observed behavior inform decisions where records exist.

Task-specific evidence
Conceptual responsibilities, not an enforced software pipeline, a live architecture trace or a replay of a specific task.
Next: explore the six-stage workflow

OBSERVED / 09 OCT 2026

Behind the workflow.

A point-in-time internal inventory. These numbers describe what was present, not how reliably it operated.

agent profiles
13

Configured roles, not a measurement of independently active agents

live skill documents
119

Observed alongside quarantined and archived materials

automation-directory files
43

Files present, not 43 operationally validated automations

What's observed, described or still unproven?

Observed: profile definitions, skill documents, automation files, model-provider session records and project decision materials.

Working method: I direct AI-led investigation and implementation, commission model-based challenges, and perform hands-on QA. Not every review or approval is proven to be a technically enforced gate for every task.

Not established: a fleet-wide success rate, universally reliable autonomous operation, validated behavior for every script, or a complete Hermes-managed task trace suitable for public reproduction.

04 / INSIDE THE FLEET

One workflow. Several responsibilities.

Select a stage to examine human judgment, agent contribution, evidence and limits. This is a high-level operating model, not a reconstruction of a verified individual task.

THE WORK / SIX STAGES

Six stages, one continuous journey. The evidence and its limits live inside each card whenever you want the detail.

  1. 01 / 06Human-led direction

    FrameMake the problem explicit.

    THE QUESTION

    What would count as solving the right problem?

    Human direction

    I identify the friction, desired outcome and release constraints.

    Agent contribution

    Agents can surface ambiguity and missing context.

    Evidence & limitations

    What to inspect

    A stated goal and scoped task—where documented.

    What it doesn't prove

    A task description cannot prove the solution is correct.

  2. 02 / 06AI-led investigation

    InvestigateChallenge the initial picture.

    THE QUESTION

    Which assumptions remain untested?

    Human direction

    I challenge the investigation and ask for alternatives or missing cases.

    Agent contribution

    Agents inspect relevant code, behavior and available records.

    Evidence & limitations

    What to inspect

    Investigation notes and concrete findings, when retained.

    What it doesn't prove

    An agent explanation can still be inaccurate.

  3. 03 / 06Human-guided decision

    PlanMake consequential choices visible.

    THE QUESTION

    Why this approach rather than the simpler alternative?

    Human direction

    I choose the direction, scope and approval boundaries.

    Agent contribution

    Agents propose implementation paths and tradeoffs.

    Evidence & limitations

    What to inspect

    A proposed plan or decision record—not a claim every task used the same form.

    What it doesn't prove

    An approved plan is not proof that its implementation will work.

  4. 04 / 06AI-directed implementation

    BuildTurn the decision into a change.

    THE QUESTION

    What changed, and what else might that affect?

    Human direction

    I steer the work and validate requirements and resulting behavior.

    Agent contribution

    Agents write, read, debug and revise the implementation code.

    Evidence & limitations

    What to inspect

    Changed artifacts, tests and relevant explanations.

    What it doesn't prove

    I do not claim unaided coding or a personal line-by-line source audit.

  5. 05 / 06Model-led challenges

    ReviewTry to disprove the result.

    THE QUESTION

    Could the reviewers share the same blind spot?

    Human direction

    I request adversarial reviews using different models, assess their findings and seek further checks.

    Agent contribution

    Separate agents challenge behavior, design and potential failure modes.

    Evidence & limitations

    What to inspect

    Review findings and targeted automated checks where available.

    What it doesn't prove

    Model agreement and green tests are not guarantees of safety.

  6. 06 / 06Hands-on human QA

    VerifyTest, decide, learn.

    THE QUESTION

    What has actually been tested—and what remains unknown?

    Human direction

    I personally attempt to break user-facing behavior and decide when available evidence warrants release.

    Agent contribution

    Automated checks and assistants support investigation and release verification.

    Evidence & limitations

    What to inspect

    Observed behavior and release checks at a specific revision.

    What it doesn't prove

    Exploratory QA cannot certify every code path or deployment risk.

Illustrative working method—not a live trace. All six stages, including their evidence, remain readable without animation.

FOLLOW THE DECISION

Six stages, one at a time. Read the essentials first; open the evidence whenever you want to go deeper.

STAGE 01 / Human-led direction

Make the problem explicit.

Frame
THE QUESTION

What would count as solving the right problem?

Human direction

I identify the friction, desired outcome and release constraints.

Agent contribution

Agents can surface ambiguity and missing context.

Inspect evidence & limits

What to inspect

A stated goal and scoped task—where documented.

What it doesn't prove

A task description cannot prove the solution is correct.

STAGE 02 / AI-led investigation

Challenge the initial picture.

Investigate
THE QUESTION

Which assumptions remain untested?

Human direction

I challenge the investigation and ask for alternatives or missing cases.

Agent contribution

Agents inspect relevant code, behavior and available records.

Inspect evidence & limits

What to inspect

Investigation notes and concrete findings, when retained.

What it doesn't prove

An agent explanation can still be inaccurate.

STAGE 03 / Human-guided decision

Make consequential choices visible.

Plan
THE QUESTION

Why this approach rather than the simpler alternative?

Human direction

I choose the direction, scope and approval boundaries.

Agent contribution

Agents propose implementation paths and tradeoffs.

Inspect evidence & limits

What to inspect

A proposed plan or decision record—not a claim every task used the same form.

What it doesn't prove

An approved plan is not proof that its implementation will work.

STAGE 04 / AI-directed implementation

Turn the decision into a change.

Build
THE QUESTION

What changed, and what else might that affect?

Human direction

I steer the work and validate requirements and resulting behavior.

Agent contribution

Agents write, read, debug and revise the implementation code.

Inspect evidence & limits

What to inspect

Changed artifacts, tests and relevant explanations.

What it doesn't prove

I do not claim unaided coding or a personal line-by-line source audit.

STAGE 05 / Model-led challenges

Try to disprove the result.

Review
THE QUESTION

Could the reviewers share the same blind spot?

Human direction

I request adversarial reviews using different models, assess their findings and seek further checks.

Agent contribution

Separate agents challenge behavior, design and potential failure modes.

Inspect evidence & limits

What to inspect

Review findings and targeted automated checks where available.

What it doesn't prove

Model agreement and green tests are not guarantees of safety.

STAGE 06 / Hands-on human QA

Test, decide, learn.

Verify
THE QUESTION

What has actually been tested—and what remains unknown?

Human direction

I personally attempt to break user-facing behavior and decide when available evidence warrants release.

Agent contribution

Automated checks and assistants support investigation and release verification.

Inspect evidence & limits

What to inspect

Observed behavior and release checks at a specific revision.

What it doesn't prove

Exploratory QA cannot certify every code path or deployment risk.

Illustrative operating model, not a verified replay or live system trace. Third-party Hermes Agent foundation; custom orchestration, direction and QA by Jeff.

Continue: how the results are verified

05 / VERIFICATION

Different checks establish different things.

AI-led review

Agents investigate code and challenge changes. Their conclusions remain fallible.

Executable checks

Automated tests and deployment probes validate defined assertions at a particular revision.

My manual QA

I try real behavior and edge cases myself. That helps discover observable problems, but doesn't establish code-level safety.

06 / LIMITS & NEXT

Evidence is not certainty.

More model reviewers do not guarantee independent correctness, and a passing test suite cannot cover everything. The diagrams describe responsibilities and methods, not a replay of private task records.

Ask me about this work