Validate device inventory
Keep usable device records and explain rejected export rows.
Build on records you can trust
Network exports often combine equipment from several collectors. Before joining a VLAN or interface observation, decide what identifies a device and which records are valid. Otherwise an empty device ID or a duplicated row can silently turn into a misleading count.
This course uses fictional TSV exports and documentation addresses. Each tab separates one field; values contain no embedded tabs or newlines. Awk can preserve an empty field between adjacent tabs. A shell loop with whitespace splitting may not, so parse the format deliberately before using its values.
Accept identity only after validation
Use one ordered chain of rejection rules. Check field count first, then identifiers and required labels, and finally duplicates. A malformed row must not reserve its device ID: a later corrected row should still be accepted. Once a valid ID is stored, reject another occurrence rather than guessing which observation was intended.
Keep the physical line number with each error. The next person can find and repair the source record without reverse-engineering your script. The lab is a separate Linux workspace with networking disabled; these files describe a fictional network, not devices discovered on your own computer. Bash Basics is recommended, and all lessons remain freely accessible.
awk -F '\t' 'NR>1 {printf "source line %d: %d fields\n",NR,NF}' devices.tsv
Put it into practice.
Validate devices.tsv with header device_id<TAB>hostname<TAB>role<TAB>site. Replace valid.tsv with that header and accepted rows in input order. Replace errors.tsv with line<TAB>reason. Reject in this order: fields (not four); device_id (not ^[A-Za-z0-9_-]+$); hostname (empty); role (not router, switch or firewall); site (empty); duplicate (ID already accepted). Reserve an ID only after every rule passes. Values are UTF-8 without embedded tabs/newlines.
See an explained solution
printf 'device_id\thostname\trole\tsite\n' > valid.tsv
printf 'line\treason\n' > errors.tsv
awk -F '\t' 'NR>1 {
reason=""
if(NF!=4)reason="fields"
else if($1!~/^[A-Za-z0-9_-]+$/)reason="device_id"
else if($2=="")reason="hostname"
else if($3!="router" && $3!="switch" && $3!="firewall")reason="role"
else if($4=="")reason="site"
else if($1 in seen)reason="duplicate"
if(reason!="")printf "%d\t%s\n",NR,reason >> "errors.tsv"
else {seen[$1]=1;print $0 >> "valid.tsv"}
}' devices.tsv
One ordered rejection rule gives each invalid record a reproducible explanation. Reserving only accepted IDs keeps a corrected later observation available while preventing duplicate devices in downstream joins.