01 / SECURITY LOGS AND DETECTIONS · LESSON 01

Read and validate security events

Read a JSONL export while keeping valid records and explaining rejected lines.

15 min · learn at your pace

One event per line

Security products often export JSON Lines: each nonblank line is a separate JSON object. That makes it possible to keep useful records even when one line is damaged. json.loads() parses a single line; json.load() expects a whole JSON document. The two formats are not interchangeable.

Read the supplied events.jsonl as UTF-8, enumerate physical lines starting at one, and keep the original line number with any rejection. Blank lines still count toward that number. A useful error says where to look without copying an entire sensitive event into another report.

Parsing is only the first check

Valid JSON can be a list, a number, or an object missing the fields your detector needs. Validate shape before reading fields. For this exercise, id, timestamp, user, and outcome must be nonempty strings; only failure and success are accepted outcomes. Preserve valid event values unchanged. Timestamp interpretation comes next.

Use a narrow exception around JSON decoding. A missing file should raise FileNotFoundError; it is different from a file containing no events. Returning an empty list for every problem can turn a failed collection into an apparently clean investigation.

A small, reusable reader

Return (events, errors) from your function. The caller can inspect good data and still see its limitations. The checks supply other exports too, so calculate from the file rather than writing the example answer into a list. Running the example prints a short summary; checking calls your function directly.

Try an example.py
import json
line = '{"id":"e1","timestamp":"2026-04-01T09:00:00Z","user":"alex","outcome":"failure"}'
event = json.loads(line)
print(event["user"], event["outcome"])
YOUR TURN

Put it into practice.

Define read_events(path), returning (events, errors). Ignore blank lines. Accept only JSON objects with nonempty string id, timestamp, user and outcome, where outcome is failure or success. Preserve accepted objects. For malformed JSON append {"line": physical_line_number, "reason": "invalid JSON"}; for other invalid records use reason "invalid event". Keep input order. Missing files must raise FileNotFoundError.

See an explained solution
One way to solve it.py
import json
from pathlib import Path

def read_events(path):
    events, errors = [], []
    for line_number, line in enumerate(Path(path).read_text(encoding="utf-8").splitlines(), 1):
        if not line.strip():
            continue
        try:
            event = json.loads(line)
        except json.JSONDecodeError:
            errors.append({"line": line_number, "reason": "invalid JSON"})
            continue
        if (not isinstance(event, dict)
            or not all(isinstance(event.get(k), str) and event[k].strip() for k in ("id", "timestamp", "user", "outcome"))
            or event["outcome"] not in ("failure", "success")):
            errors.append({"line": line_number, "reason": "invalid event"})
            continue
        events.append(event)
    return events, errors

if __name__ == "__main__":
    print(read_events("events.jsonl"))

The reader separates decoding from record validation, records physical line numbers, and lets file errors reach the caller. Valid events are preserved rather than silently repaired.