Capstone: assess fleet exposure
Combine recorded TCP probes and fleet facts into a tested, reproducible assessment.
Build an evidence-based fleet assessment
The capstone brings the scanner and administration work together. Your tool receives a declared fleet, recorded hardware facts, an endpoint policy and recorded TCP attempts. It produces a repeatable report about that supplied evidence. It does not discover machines, contact services, stop real processes or execute remote commands.
Separate startup failures from rejected rows. Missing required files, wrong headers and malformed policy prevent a meaningful assessment and must preserve previous reports. Individual malformed records instead become errors with a filename and physical line number, allowing the remaining valid evidence to be assessed. The first valid identity wins; a malformed row must not reserve an ID.
Preserve the limits of every conclusion
An unexpected open service needs review, while a timeout remains uncertain. Missing system facts are unknown rather than zero. Disabled hosts stay in the accepted inventory but are excluded from assessment. Report every applicable capacity and exposure finding, not just one convenient headline per host.
Retry attempts are selected by their numeric attempt number, independent of input order. Bound their count and cumulative recorded duration. Stop on open or refused, and keep the last inconclusive state when the next numbered observation is absent. A later record after a gap cannot silently stand in for that missing attempt.
Stage the reports, sort their data deterministically and replace previous outputs on successful repeated runs. The original evidence must remain byte-for-byte unchanged. This produces an auditable local tool whose planning, observation and reporting boundaries can later support a real authorised-network adapter. The optional downloadable scanner is a separate starting point for actual TCP connections; its less detailed failure vocabulary must be mapped honestly.
for name in assets.tsv facts.tsv endpoints.tsv attempts.tsv;do
printf '%s: ' "$name"
awk 'END {printf "%d data rows\n",NR-1}' "case/$name"
done
Put it into practice.
Implement bash main.sh [input_directory] [output_directory], defaulting to case and reports. Inputs are UTF-8 TSV with the exact supplied headers, no embedded tabs/newlines; IDs match ^[A-Za-z0-9_-]+$. Required files: assets.tsv, facts.tsv, endpoints.tsv, attempts.tsv, policy.conf. Missing input exits 2 with “Missing input”; wrong TSV header with “Invalid header”; bad policy with “Invalid policy”. These failures preserve existing reports. Paths may contain spaces/Unicode; supplied input/output directories are distinct, non-alias workspace paths.
Policy has exactly four unique key=value lines, canonical unsigned decimals: min_cores 1–1024, min_memory_mib 0–1048576, max_attempts 1–10, budget_ms 0–60000. Treat it as data, never source it.
Validate files in the order above, rejecting a row at its first applicable reason. assets: fields (3), host (ID), address (canonical IPv4), enabled (yes/no), duplicate (accepted host). facts: fields (4), host (ID), os (nonempty), cores (0–1024), memory_mib (0–1048576), unknown_host, duplicate (accepted host). endpoints: fields (4), endpoint (ID), host (ID), port (canonical 1–65535), expected_open (yes/no), unknown_host, duplicate (accepted endpoint). attempts: fields (4), endpoint (ID), attempt (canonical 1–10), state (open/refused/timed_out/error), elapsed_ms (canonical 0–60000), unknown_endpoint, duplicate (accepted endpoint/attempt). Reserve identities only after validation. errors.tsv contains source<TAB>line<TAB>reason in file-processing/physical-line order.
Assess enabled=yes hosts only. fleet.tsv: host<TAB>os<TAB>cores<TAB>memory_mib<TAB>capacity, sorted by host. Missing facts use three dashes and unknown; known facts are review when either value is below policy, otherwise ok. Emit one finding per low value: low_cores or low_memory (facts.tsv line). Missing facts: facts_missing (assets.tsv line). Fleet findings use endpoint=-.
Assess every accepted endpoint of an enabled host. Start state=missing, attempts=0, elapsed_ms=0. Look for attempt 1, then 2, etc. Stop if the next numbered record is absent, retaining the current state. Before consuming an existing record, if count is already max_attempts or its duration would exceed budget_ms, stop with budget_exhausted without counting it. Otherwise consume it and stop on open/refused. Equality fits the budget. open maps to expected_open (policy yes) or unexpected_open (no); refused maps to unavailable (yes) or expected_refusal (no); all other states map to unknown. scan.tsv: endpoint<TAB>host<TAB>address<TAB>port<TAB>state<TAB>attempts<TAB>elapsed_ms<TAB>finding, sorted by endpoint.
findings.tsv: host<TAB>endpoint<TAB>code<TAB>source<TAB>line, sorted by complete row under LC_ALL=C. Include fleet findings plus endpoint findings only for unexpected_open, unavailable and unknown. Endpoint findings cite the last consumed attempts.tsv line, or the endpoints.tsv line if no attempt was consumed. summary.txt has, in order: accepted_hosts=N, active_hosts=N, assessed_endpoints=N, findings=N, rejected_records=N, scope=recorded evidence; no live reachability guarantee. All four TSV reports always have headers. Replace all five outputs on success; repeated runs must not accumulate rows or modify input bytes.
See an explained solution
export LC_ALL=C
input=${1:-case};output=${2:-reports}
for name in assets.tsv facts.tsv endpoints.tsv attempts.tsv policy.conf;do
[ -f "$input/$name" ] || { printf 'Missing input: %s\n' "$name" >&2;exit 2; }
done
for name in assets.tsv facts.tsv endpoints.tsv attempts.tsv;do
case "$name" in
assets.tsv) expected=$'host\taddress\tenabled';;
facts.tsv) expected=$'host\tos\tcores\tmemory_mib';;
endpoints.tsv) expected=$'endpoint\thost\tport\texpected_open';;
attempts.tsv) expected=$'endpoint\tattempt\tstate\telapsed_ms';;
esac
IFS= read -r actual < "$input/$name"
[ "$actual" = "$expected" ] || { printf 'Invalid header: %s\n' "$name" >&2;exit 2; }
done
work=$(mktemp -d .assessment.XXXXXX) || exit 2
trap 'rm -rf "$work"' EXIT
if ! awk -F '\t' -v out="$work" -f lib/network.awk -f - "$input/policy.conf" "$input/assets.tsv" "$input/facts.tsv" "$input/endpoints.tsv" "$input/attempts.tsv" <<'AWK'
function id(s){return s~/^[A-Za-z0-9_-]+$/}
function uint(s,lo,hi){return s~/^(0|[1-9][0-9]*)$/ && length(s)<=7 && s>=lo && s<=hi}
function reject(source,reason){if(reason!=""){printf "%s\t%d\t%s\n",source,FNR,reason > (out "/errors");errors++;return 1}return 0}
function finding(h,e,c,s,l){printf "%s\t%s\t%s\t%s\t%d\n",h,e,c,s,l > (out "/findings");findings++}
BEGIN{printf "" > (out "/errors");printf "" > (out "/findings");printf "" > (out "/fleet");printf "" > (out "/scan")}
FILENAME==ARGV[1]{
n=split($0,p,"=");key=p[1];v=p[2];lo=0;hi=0
if(key=="min_cores"){lo=1;hi=1024}
else if(key=="min_memory_mib")hi=1048576
else if(key=="max_attempts"){lo=1;hi=10}
else if(key=="budget_ms")hi=60000
else badpolicy=1
if(n!=2 || key in policy || !uint(v,lo,hi))badpolicy=1
policy[key]=v+0;policycount++;next
}
FNR==1{next}
FILENAME==ARGV[2]{
reason=""
if(NF!=3)reason="fields"
else if(!id($1))reason="host"
else if(ipv4($2)<0)reason="address"
else if($3!="yes" && $3!="no")reason="enabled"
else if($1 in address)reason="duplicate"
if(!reject("assets.tsv",reason)){address[$1]=$2;enabled[$1]=$3;assetline[$1]=FNR;hosts++}
next
}
FILENAME==ARGV[3]{
reason=""
if(NF!=4)reason="fields"
else if(!id($1))reason="host"
else if($2=="")reason="os"
else if(!uint($3,0,1024))reason="cores"
else if(!uint($4,0,1048576))reason="memory_mib"
else if(!($1 in address))reason="unknown_host"
else if($1 in os)reason="duplicate"
if(!reject("facts.tsv",reason)){os[$1]=$2;cores[$1]=$3;memory[$1]=$4;factline[$1]=FNR}
next
}
FILENAME==ARGV[4]{
reason=""
if(NF!=4)reason="fields"
else if(!id($1))reason="endpoint"
else if(!id($2))reason="host"
else if(!port($3))reason="port"
else if($4!="yes" && $4!="no")reason="expected_open"
else if(!($2 in address))reason="unknown_host"
else if($1 in host)reason="duplicate"
if(!reject("endpoints.tsv",reason)){host[$1]=$2;ports[$1]=$3;expected[$1]=$4;endpointline[$1]=FNR}
next
}
{
reason="";key=$1 SUBSEP $2
if(NF!=4)reason="fields"
else if(!id($1))reason="endpoint"
else if(!uint($2,1,10))reason="attempt"
else if($3!="open" && $3!="refused" && $3!="timed_out" && $3!="error")reason="state"
else if(!uint($4,0,60000))reason="elapsed_ms"
else if(!($1 in host))reason="unknown_endpoint"
else if(key in state)reason="duplicate"
if(!reject("attempts.tsv",reason)){state[key]=$3;duration[key]=$4;attemptline[key]=FNR}
}
END{
if(badpolicy || policycount!=4)exit 2
for(h in address){
if(enabled[h]!="yes")continue
active++;capacity="unknown";o="-";c="-";m="-"
if(h in os){
o=os[h];c=cores[h];m=memory[h];capacity="ok"
if(c<policy["min_cores"]){capacity="review";finding(h,"-","low_cores","facts.tsv",factline[h])}
if(m<policy["min_memory_mib"]){capacity="review";finding(h,"-","low_memory","facts.tsv",factline[h])}
}else finding(h,"-","facts_missing","assets.tsv",assetline[h])
printf "%s\t%s\t%s\t%s\t%s\n",h,o,c,m,capacity > (out "/fleet")
}
for(e in host){
h=host[e];if(enabled[h]!="yes")continue
assessed++;result="missing";count=0;elapsed=0;source="endpoints.tsv";line=endpointline[e]
for(i=1;i<=10;i++){
key=e SUBSEP i;if(!(key in state))break
if(count>=policy["max_attempts"] || elapsed+duration[key]>policy["budget_ms"]){result="budget_exhausted";break}
count++;elapsed+=duration[key];result=state[key];source="attempts.tsv";line=attemptline[key]
if(result=="open" || result=="refused")break
}
assessment="unknown"
if(result=="open")assessment=(expected[e]=="yes" ? "expected_open" : "unexpected_open")
else if(result=="refused")assessment=(expected[e]=="yes" ? "unavailable" : "expected_refusal")
if(assessment=="unexpected_open" || assessment=="unavailable" || assessment=="unknown")finding(h,e,assessment,source,line)
printf "%s\t%s\t%s\t%s\t%s\t%d\t%d\t%s\n",e,h,address[h],ports[e],result,count,elapsed,assessment > (out "/scan")
}
printf "accepted_hosts=%d\nactive_hosts=%d\nassessed_endpoints=%d\nfindings=%d\nrejected_records=%d\nscope=recorded evidence; no live reachability guarantee\n",hosts,active,assessed,findings,errors > (out "/summary")
}
AWK
then printf 'Invalid policy\n' >&2;exit 2;fi
printf 'host\tos\tcores\tmemory_mib\tcapacity\n' > "$work/fleet.tsv"
sort "$work/fleet" >> "$work/fleet.tsv"
printf 'endpoint\thost\taddress\tport\tstate\tattempts\telapsed_ms\tfinding\n' > "$work/scan.tsv"
sort "$work/scan" >> "$work/scan.tsv"
printf 'host\tendpoint\tcode\tsource\tline\n' > "$work/findings.tsv"
sort "$work/findings" >> "$work/findings.tsv"
printf 'source\tline\treason\n' > "$work/errors.tsv"
cat "$work/errors" >> "$work/errors.tsv"
mv "$work/summary" "$work/summary.txt"
mkdir -p "$output" || exit 2
for name in fleet.tsv scan.tsv findings.tsv errors.tsv summary.txt;do mv "$work/$name" "$output/$name" || exit 2;done
The assessment separates declared scope, recorded observations and policy. Ordered validation keeps rejected evidence traceable; bounded attempt selection preserves uncertainty and provenance. Independent expected-output fixtures check complete reports, unchanged inputs and repeat-run replacement without granting the script any live network access.