Handle incomplete evidence
Distinguish an optional evidence gap from a required-input failure.
No evidence is not evidence of absence
A report with zero findings can mean that nothing matched. It can also mean that a collector failed. Those outcomes need different explanations. Define which sources are required for the tool to operate and which are optional additions.
The primary authentication export is required here. A VPN export is optional: when missing, the report can still describe the primary evidence but must record a limitation. A present, header-only VPN file means the exporter supplied zero records; it does not create the same missing-source limitation.
Keep diagnostics out of data streams
Use nonzero exit status and stderr for a required-input failure. For a successful partial investigation, write limitations into a separate report file rather than polluting TSV findings with free-form warnings. Another tool can then parse both outputs reliably.
Check required files before replacing reports. Keep source names and event IDs in every selected row, and retain input order within each source. This provides a clear, repeatable account of what was examined. Never convert a missing required file into an empty one just to make the script appear successful: that would hide the very collection problem an analyst needs to know about.
if [ -f vpn.tsv ]; then printf 'VPN evidence supplied\n'; else printf 'VPN evidence unavailable\n'; fi
Put it into practice.
primary.tsv is required; vpn.tsv is optional. Both have validated id<TAB>user<TAB>result rows. If primary.tsv is missing, exit nonzero with Missing required evidence on stderr before replacing reports. Otherwise write findings.tsv (source<TAB>id<TAB>user), selecting exact FAIL rows from primary then VPN in input order. Write limitations.txt as vpn.tsv: unavailable followed by a newline only when VPN is missing; otherwise it is empty. Missing optional evidence still exits zero. Replace both reports on reruns.
See an explained solution
[ -f primary.tsv ] || { printf 'Missing required evidence\n' >&2; exit 1; }
printf 'source\tid\tuser\n' > findings.tsv
: > limitations.txt
for source in primary.tsv vpn.tsv; do
if [ ! -f "$source" ]; then printf '%s: unavailable\n' "$source" >> limitations.txt; continue; fi
awk -F '\t' -v source="$source" 'NR>1 && $3=="FAIL" {printf "%s\t%s\t%s\n",source,$1,$2}' "$source" >> findings.tsv || exit 1
done
The required-file guard prevents a failed investigation from masquerading as an empty successful report. Optional gaps remain visible in limitations.txt while valid evidence continues to produce traceable findings.