YOUR CAPSTONE PROJECT · LESSON 17

Capstone: review a network snapshot

Build a repeatable network assessment with configuration findings, evidence limits and rejected records.

55 min · learn at your pace

Bring configuration and evidence together

A useful network review combines desired state with observations while keeping their limits visible. Inventory tells this project which devices should be assessed. Interface exports describe observed addressing and configuration. DNS and probe exports contribute different evidence; neither should silently replace the other.

Start by validating the required files, exact headers and policy. These are startup failures: the tool cannot interpret the case reliably, so it must leave previous reports intact and fail clearly. Malformed individual records are different. Keep a rejection record with its source and physical line number, then continue with usable records.

Make selection and missing evidence explicit

Only the first valid occurrence reserves a device, observation or probe ID. Normalise DNS names before detecting duplicate name/address pairs. Select the latest accepted probe by timestamp, then lexical probe ID, before interpreting its outcome. A later refusal must not be hidden by an older successful attempt.

A device can have several findings. Missing interface or DNS coverage, stale probes and timeouts make its overall state unknown even if another observation shows a definite mismatch. Otherwise a finding means review; no findings means ok within this supplied evidence. Disabled devices remain available for joins but are excluded from the active report.

Publish a reproducible handover

Write deterministic reports with source references and counts derived from their final contents. Successful repeated runs replace the previous outputs without changing input bytes. The summary states that this is a snapshot assessment, not a live reachability guarantee. The supplied addresses are from documentation ranges; the script performs no network access or device changes.

Try an example.sh
for name in devices.tsv interfaces.tsv dns.tsv probes.tsv;do
 printf '%s: ' "$name"
 awk 'END{printf "%d data rows\n",NR-1}' "network/$name"
done
YOUR TURN

Put it into practice.

Build bash main.sh [input_directory] [output_directory], defaulting to network and reports. Use trusted relative workspace directory paths without . or .. components, backslashes, tabs or newlines; the output directory is separate from the input. Read devices.tsv, interfaces.tsv, dns.tsv, probes.tsv and policy.conf from input. Headers and fields are shown in the supplied files. All files use UTF-8 and LF; fields contain no tabs/newlines or ASCII control characters. Successful runs replace network_report.tsv, findings.tsv, errors.tsv and summary.txt in output, preserving all input bytes. Print no stdout. Missing required inputs fail nonzero with Missing input on stderr; incorrect table headers fail with Invalid header. Invalid policy fails with Invalid policy. Startup failures must not replace existing reports.

Policy has exactly one each of now_second (canonical integer 0–1000000000), max_probe_age (1–86400) and max_latency_ms (1–60000). Ignore blank lines and lines beginning #. Reject unknown/duplicate keys, missing keys, spaces around =, leading zeros except 0, or invalid ranges. Never execute policy text.

Validate each record in the order below, emitting only its first rejection reason. Canonical integers have no signs, spaces or leading zeros except 0. Device/probe IDs match ^[A-Za-z][A-Za-z0-9_-]*$. Hostnames are ASCII dot-separated labels: at least two labels, each 1–63 alphanumeric/hyphen characters, starting and ending alphanumeric, total length at most 253 after removing one optional trailing dot. Normalise hostnames to lowercase without that dot. IPv4 uses four canonical decimal octets 0–255. VLAN is 1–4094; MTU is 576–9216. A rejected row never reserves its identity.

Validation order: devices — fields (six), device_id, hostname, address, vlan, mtu, enabled (yes/no), duplicate (accepted device ID). interfaces — fields (five), device_id, address, vlan, mtu, state (up/down), unknown_device, duplicate (accepted device ID). dns — fields (two), name, address, duplicate (normalised name/address pair). probes — fields (five), probe_id, device_id, second (0 through now_second), result (ok/timeout/refused/unreachable), latency (0–60000 for ok, exactly - otherwise), unknown_device, duplicate (accepted probe ID). Process files in that order; errors.tsv has source<TAB>line<TAB>reason in source/physical-line order. Source names are the bare input filenames.

Assess accepted enabled=yes devices only. For a missing interface emit interface_missing (unknown) from devices.tsv and that device line, detail No interface observation. Otherwise emit every applicable finding from the interface row: address_mismatch (Address differs from inventory), vlan_mismatch (VLAN differs from inventory), mtu_mismatch (MTU differs from inventory), and interface_down (Observed interface is down). For DNS, an accepted pair matching the device hostname/address satisfies policy even if other answers exist. If the hostname has no accepted DNS rows, emit dns_missing (unknown), from devices.tsv/device line, detail No DNS observation. If it has rows but none match, emit dns_mismatch from dns.tsv and the first accepted line for that name, detail Expected A record is absent.

Choose the latest accepted probe for the device by greatest second, then lexical greatest probe_id in LC_ALL=C. If absent, emit probe_missing (unknown), from devices.tsv/device line, detail No probe observation. Otherwise use probes.tsv/chosen line and exactly one rule, in this order: age > max_probe_age gives probe_stale (unknown), detail Latest probe is stale; timeout gives probe_timeout (unknown), detail Timeout is inconclusive; refused gives probe_rejected, detail Probe was refused; unreachable gives probe_route_error, detail Probe reported unreachable; ok with latency > max_latency_ms gives probe_slow, detail Latency exceeds policy. Equality at either threshold is allowed. Other successful probes add no finding.

findings.tsv has device_id<TAB>code<TAB>source<TAB>line<TAB>detail, sorted by device ID then code. network_report.tsv has device_id<TAB>hostname<TAB>observed_address<TAB>state<TAB>findings, sorted by device ID; observed_address is - if missing, hostname is normalised, findings is that device's count. State is unknown if any unknown finding exists, otherwise review if any finding exists, otherwise ok. summary.txt contains these key=value lines in order: accepted_devices, active_devices, disabled_devices, ok, review, unknown, rejected_records, findings, followed by scope=supplied snapshots; no live reachability guarantee. Counts come from accepted inventory, final active states, rejected rows and findings. Supplied lib/ipv4.awk may be reused.

See an explained solution
One way to solve it.sh
export LC_ALL=C
[ "$#" -le 2 ] || { printf 'Usage: bash main.sh [input_directory] [output_directory]\n' >&2;exit 2; }
input=${1:-network};output=${2:-reports}
for path in "$input" "$output";do
 case "$path" in ''|/*|.|..|./*|../*|*/./*|*/../*|*/.|*/..|*\\*)printf 'Invalid directory\n' >&2;exit 2;;esac
done
[ "$input" != "$output" ] || { printf 'Separate output directory required\n' >&2;exit 2; }
input=./$input;output=./$output
for name in devices.tsv interfaces.tsv dns.tsv probes.tsv policy.conf;do
 [ -f "$input/$name" ] || { printf 'Missing input: %s\n' "$name" >&2;exit 2; }
done
for name in devices.tsv interfaces.tsv dns.tsv probes.tsv;do
 IFS= read -r header < "$input/$name"
 case "$name" in
 devices.tsv)expected=$'device_id\thostname\taddress\tvlan\tmtu\tenabled';;
 interfaces.tsv)expected=$'device_id\taddress\tvlan\tmtu\tstate';;
 dns.tsv)expected=$'name\taddress';;
 probes.tsv)expected=$'probe_id\tdevice_id\tsecond\tresult\tlatency_ms';;
 esac
 [ "$header" = "$expected" ] || { printf 'Invalid header: %s\n' "$name" >&2;exit 2; }
done
policy=$(awk -F '=' '
 /^#/ || $0==""{next}
 {if(NF!=2 || ($1!="now_second"&&$1!="max_probe_age"&&$1!="max_latency_ms") || ($1 in v) || $2!~/^(0|[1-9][0-9]*)$/ || length($2)>10){bad=1;next}v[$1]=$2+0}
 END{if(bad || !("now_second" in v) || !("max_probe_age" in v) || !("max_latency_ms" in v) || v["now_second"]>1000000000 || v["max_probe_age"]<1 || v["max_probe_age"]>86400 || v["max_latency_ms"]<1 || v["max_latency_ms"]>60000)exit 2;printf "%.0f\t%.0f\t%.0f\n",v["now_second"],v["max_probe_age"],v["max_latency_ms"]}' "$input/policy.conf") || { printf 'Invalid policy\n' >&2;exit 2; }
IFS=$'\t' read -r now max_age max_latency <<< "$policy"
mkdir -p "$output" || exit 1
stage=$(mktemp -d "$output/.network.XXXXXX") || exit 1
trap 'rm -rf "$stage"' EXIT
: > "$stage/errors.data";: > "$stage/findings.data";: > "$stage/network.data"
awk -F '\t' -v now="$now" -v max_age="$max_age" -v max_latency="$max_latency" -v stage="$stage" -f lib/ipv4.awk -f - "$input/devices.tsv" "$input/interfaces.tsv" "$input/dns.tsv" "$input/probes.tsv" <<'AWK' || exit 1
function uint(s,lo,hi){return s~/^(0|[1-9][0-9]*)$/ && length(s)<=10 && s+0>=lo && s+0<=hi}
function ident(s){return s~/^[A-Za-z][A-Za-z0-9_-]*$/}
function normal(s){s=tolower(s);sub(/\.$/,"",s);return s}
function hostname(s,a,n,i){s=normal(s);if(length(s)>253)return 0;n=split(s,a,".");if(n<2)return 0;for(i=1;i<=n;i++)if(length(a[i])>63 || a[i]!~/^[a-z0-9]([a-z0-9-]*[a-z0-9])?$/)return 0;return 1}
function reject(source,reason){printf "%s\t%d\t%s\n",source,FNR,reason >> (stage "/errors.data");rejected++}
function finding(id,code,source,line,detail,uncertain){printf "%s\t%s\t%s\t%d\t%s\n",id,code,source,line,detail >> (stage "/findings.data");count[id]++;total_findings++;if(uncertain)unknown[id]=1}
FNR==1{next}
FILENAME==ARGV[1]{
 reason=""
 if(NF!=6)reason="fields";else if(!ident($1))reason="device_id";else if(!hostname($2))reason="hostname";else if(ipv4($3)<0)reason="address";else if(!uint($4,1,4094))reason="vlan";else if(!uint($5,576,9216))reason="mtu";else if($6!="yes"&&$6!="no")reason="enabled";else if($1 in names)reason="duplicate"
 if(reason!=""){reject("devices.tsv",reason);next}
 names[$1]=normal($2);address[$1]=$3;vlan[$1]=$4;mtu[$1]=$5;enabled[$1]=$6;device_line[$1]=FNR;accepted++;next
}
FILENAME==ARGV[2]{
 reason=""
 if(NF!=5)reason="fields";else if(!ident($1))reason="device_id";else if(ipv4($2)<0)reason="address";else if(!uint($3,1,4094))reason="vlan";else if(!uint($4,576,9216))reason="mtu";else if($5!="up"&&$5!="down")reason="state";else if(!($1 in names))reason="unknown_device";else if($1 in observed)reason="duplicate"
 if(reason!=""){reject("interfaces.tsv",reason);next}
 observed[$1]=$2;observed_vlan[$1]=$3;observed_mtu[$1]=$4;observed_state[$1]=$5;interface_line[$1]=FNR;next
}
FILENAME==ARGV[3]{
 reason="";name=normal($1);key=name SUBSEP $2
 if(NF!=2)reason="fields";else if(!hostname($1))reason="name";else if(ipv4($2)<0)reason="address";else if(key in dns)reason="duplicate"
 if(reason!=""){reject("dns.tsv",reason);next}
 dns[key]=1;if(!(name in dns_line))dns_line[name]=FNR;next
}
FILENAME==ARGV[4]{
 reason=""
 if(NF!=5)reason="fields";else if(!ident($1))reason="probe_id";else if(!ident($2))reason="device_id";else if(!uint($3,0,now))reason="second";else if($4!="ok"&&$4!="timeout"&&$4!="refused"&&$4!="unreachable")reason="result";else if(($4=="ok"&&!uint($5,0,60000))||($4!="ok"&&$5!="-"))reason="latency";else if(!($2 in names))reason="unknown_device";else if($1 in probes)reason="duplicate"
 if(reason!=""){reject("probes.tsv",reason);next}
 probes[$1]=1
 if(!($2 in probe_id)||$3>probe_second[$2]||($3==probe_second[$2]&&("x" $1)>("x" probe_id[$2]))){probe_id[$2]=$1;probe_second[$2]=$3;probe_result[$2]=$4;probe_latency[$2]=$5;probe_line[$2]=FNR}
 next
}
END{
 for(id in names){
  if(enabled[id]=="no"){disabled++;continue}active++
  if(!(id in observed))finding(id,"interface_missing","devices.tsv",device_line[id],"No interface observation",1)
  else{
   if(observed[id]!=address[id])finding(id,"address_mismatch","interfaces.tsv",interface_line[id],"Address differs from inventory",0)
   if(observed_vlan[id]!=vlan[id])finding(id,"vlan_mismatch","interfaces.tsv",interface_line[id],"VLAN differs from inventory",0)
   if(observed_mtu[id]!=mtu[id])finding(id,"mtu_mismatch","interfaces.tsv",interface_line[id],"MTU differs from inventory",0)
   if(observed_state[id]=="down")finding(id,"interface_down","interfaces.tsv",interface_line[id],"Observed interface is down",0)
  }
  if(!((names[id] SUBSEP address[id]) in dns)){
   if(!(names[id] in dns_line))finding(id,"dns_missing","devices.tsv",device_line[id],"No DNS observation",1)
   else finding(id,"dns_mismatch","dns.tsv",dns_line[names[id]],"Expected A record is absent",0)
  }
  if(!(id in probe_id))finding(id,"probe_missing","devices.tsv",device_line[id],"No probe observation",1)
  else if(now-probe_second[id]>max_age)finding(id,"probe_stale","probes.tsv",probe_line[id],"Latest probe is stale",1)
  else if(probe_result[id]=="timeout")finding(id,"probe_timeout","probes.tsv",probe_line[id],"Timeout is inconclusive",1)
  else if(probe_result[id]=="refused")finding(id,"probe_rejected","probes.tsv",probe_line[id],"Probe was refused",0)
  else if(probe_result[id]=="unreachable")finding(id,"probe_route_error","probes.tsv",probe_line[id],"Probe reported unreachable",0)
  else if(probe_latency[id]>max_latency)finding(id,"probe_slow","probes.tsv",probe_line[id],"Latency exceeds policy",0)
  state=unknown[id]?"unknown":count[id]>0?"review":"ok";states[state]++
  printf "%s\t%s\t%s\t%s\t%d\n",id,names[id],(id in observed)?observed[id]:"-",state,count[id] >> (stage "/network.data")
 }
 printf "accepted_devices=%d\nactive_devices=%d\ndisabled_devices=%d\nok=%d\nreview=%d\nunknown=%d\nrejected_records=%d\nfindings=%d\nscope=supplied snapshots; no live reachability guarantee\n",accepted,active,disabled,states["ok"],states["review"],states["unknown"],rejected,total_findings > (stage "/summary.txt")
}
AWK
{ printf 'device_id\thostname\tobserved_address\tstate\tfindings\n';sort "$stage/network.data"; } > "$stage/network_report.tsv" || exit 1
{ printf 'device_id\tcode\tsource\tline\tdetail\n';sort "$stage/findings.data"; } > "$stage/findings.tsv" || exit 1
{ printf 'source\tline\treason\n';cat "$stage/errors.data"; } > "$stage/errors.tsv" || exit 1
for name in network_report.tsv findings.tsv errors.tsv summary.txt;do mv "$stage/$name" "$output/$name" || exit 1;done

The reference separates startup failures from row rejection, joins only accepted evidence and retains missing coverage. Probe selection is deterministic, threshold equality is explicit, and every finding points to a source line. Staged reports and an independent expected-output oracle make repeated network reviews reproducible without claiming live reachability.