03 / WORKING WITH TEXT · LESSON 11

Sorting, counting and duplicates

Count repeated labels and publish a deterministic summary.

13 min · learn at your pace

Group equal lines first

uniq compares adjacent lines. It does not remember every line it has ever seen. Sort before uniq -c when repeated values can occur anywhere in a file. sort -u is useful when you only need distinct values, but it discards their frequencies.

wc -l counts newline characters. It is not automatically a record validator: a final line without a newline and a blank line have different effects on a format’s record count. This task supplies newline-terminated, nonempty service labels so the counting contract is simple.

Reproducibility is a feature

Set LC_ALL=C to obtain bytewise ordering independent of the machine’s locale. Counts from uniq -c are padded for display; normalize them before writing a report. Here each label has no spaces, so a whitespace field parser can read the count and label safely.

State whether a report orders by label or by count. Both can be useful, but readers and automated checks should not have to guess. Use a tab between output fields so the result can become input to another program without parsing decorative formatting.

Try an example.sh
LC_ALL=C sort services.txt | uniq -c
YOUR TURN

Put it into practice.

Count each service label in `services.txt`. Write `service<TAB>count` rows to `counts.tsv`, ordered by service with `LC_ALL=C`, with no header. Labels contain only lowercase ASCII letters, digits and hyphens. Ignore nothing: every supplied nonempty line is one occurrence. Empty input produces an empty report.

See an explained solution
One way to solve it.sh
export LC_ALL=C
set -o pipefail
sort services.txt | uniq -c | awk '{printf "%s\t%d\n", $2, $1}' > counts.tsv

Sorting groups all occurrences. Awk swaps the display-oriented count/label order into the task’s tabular format.