CertaDNS
Skip to lesson

Automation and Judgement · lesson 2 of 2

Designing the handoff

After this lesson you can

Define where automation stops and what it must hand a person to work with.

Assumes you have read What automates well.

The handoff is the interface between the automated and the human halves of the programme. Designed well it saves most of the work; designed badly it is why the queue stops being read.

What a handed-off finding should contain

ItemWhy it matters at the handoff
The name, and why it matchedWhich watchlist entry or rule fired. Without it the analyst reverse-engineers the trigger.
What escalated itResolving, certificate, MX — named individually rather than as a score.
The full DNS and RDAP snapshotAlready gathered. This is most of the enrichment.
The evidence package, with its manifestCaptured at detection, so it exists whatever happens next.
Asset comparison resultsWhich hashes matched yours, and which did not.
Any related findingsShared nameservers, a shared certificate, a registration window.
A rank, with the contributing signals shownA number alone cannot be argued with; the signals can.

Note what is absent: a verdict, a recommended action, and a confidence percentage. Those are the analyst’s output, and supplying them causes the analyst to check the machine rather than the finding.

Two ways the handoff fails

Too little:
   "suspicious domain found: northwlnd.example"
   -> the analyst now does twenty minutes of gathering
      that the pipeline could have done in one second.
      The queue backs up. People stop opening it.

Too much:
   "VERDICT: PHISHING (confidence 94%). Recommended
    action: file abuse report."
   -> the analyst rubber-stamps it, because disagreeing
      with a confident machine requires effort.
      The judgement layer has quietly been removed.

The one-way door

  • Sending an abuse report is not reversible. The effect on a legitimate business happens before anyone can retract it.
  • So a person sends it. Always, on every report, regardless of how confident the pipeline is.
  • Everything before it is reversible — detection, enrichment, capture, ranking — and should be automatic.
  • That is the line. Reversibility, not confidence.

A confidence score is not a safeguard

A threshold of 95% still fires on the one in twenty it is wrong about, and it fires repeatedly, because the rule that matched a legitimate business once matches it every cycle. The safeguard is a person reading the page before the report leaves — which takes two minutes and is the only thing between the pipeline and a real company.

Knowledge check

Where should the automated half of the programme stop?

Last reviewed