Changelog Insights

In May 2026 ANAC migrated the format of the owner/operator fields in the RAB from pipe-separated to JSON inline. Without handling, ~85% of the "changes" here would be false positives: noise from the format itself, not real sales.

5 layers of defense automatically filter out the noise before it is written to history:

• 1. Dual-format parser, understands the new JSON (from May 2026) and the legacy pipe format (through April 2026). A canary aborts the import if >5% of the rows in PROPRIETARIOS or OPERADORES come in an unexpected format.
• 2. Placeholder normalization, "Nome Indisponível" and "Indisponível" (the label changed between formats) become null. Suppresses ~57% of the false diffs.
• 3. 100% pre-upsert validation, scans every record before writing; aborts if raw JSON slipped through.
• 4. Diff classifier, when the set of owners (CNPJ root) is identical before/after, the "new operator" is just a reorder of the JSON array, not a real sale. Suppresses an extra ~27%.
• 5. Database safety net, a CHECK constraint in Postgres rejects writing raw JSON (with tolerance for NBSP/BOM). A last-resort guard for any future pipeline.

Auditable post-mortem via scraping_logs.metadata.diff_stats on every import. Each change that survives here is a genuine sale/transfer candidate.