03 / WORKING WITH TEXT · LESSON 10

Extracting and transforming text

Build a portable text pipeline that extracts, normalizes and trims fields.

13 min · learn at your pace

Give each program a small job

cut selects fields or columns, tr translates or deletes characters, and sed applies line-oriented editing rules. Connecting small transformations can be clearer than one large command. The order matters: extract the intended field before changing its text.

For a simple pipe-delimited file, cut -d '|' -f 2 selects field two. This is not a CSV parser: quoted separators and embedded newlines need a format-aware tool. Our supplied format explicitly forbids the delimiter inside a field.

Prefer a portable subset

tr '[:upper:]' '[:lower:]' converts ASCII labels predictably with LC_ALL=C. Sed’s s/old/new/ substitutes text; ^ and $ anchor its beginning and end. [[:space:]] is a portable character class. Use separate -e expressions when multiple edits make the intent easier to follow.

Avoid in-place sed -i: GNU and macOS disagree about its arguments. Write a new output file instead. Do not use newer Bash case-conversion expansions such as ${value,,} when targeting the built-in macOS Bash 3.2.

Try an example.sh
printf 'one|  WEB-01  |active\n' | cut -d '|' -f 2 | tr '[:upper:]' '[:lower:]' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'
YOUR TURN

Put it into practice.

For every line in `assets.txt`, extract its second `|`-delimited field, trim leading/trailing spaces and lowercase ASCII letters. Write one resulting name per line to `names.txt`, preserving order. Inputs always have three fields and no embedded `|`. Empty input produces an empty output.

See an explained solution
One way to solve it.sh
export LC_ALL=C
set -o pipefail
cut -d '|' -f 2 assets.txt | tr '[:upper:]' '[:lower:]' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' > names.txt

Each stage has one responsibility. Output goes to a separate file, and pipefail preserves upstream errors.