← All drillsRead the Freight Paperwork
What you'll be able to do- ✓Build a reader that finds a field by where its label sits on the page, and measure why flat text fails on forms that print values under labels
- ✓Build a layout-aware token classifier and test it on shippers, and on a form, it has never seen
- ✓Measure whether a reader's stated confidence is true, and calibrate it field by field
- ✓Set an auto-post threshold for each field from what a wrong value costs, instead of one global cut-off
- ✓Report the straight-through rate at a required accuracy floor, and choose between the floor and the cheapest operating point
- ✓Reconcile a Bill of Lading, a Proof of Delivery and a booking, and tell a reading error from a real discrepancy
- ✓Route what is left to a review queue ordered by the expected cost of a mistake
Drill 3 of the Freight & Logistics track. You build the paperwork desk for a freight broker, turning scanned Bills of Lading and Proofs of Delivery into posted records, held discrepancies and a review queue. The number it is measured on is the share of documents it can post without a person looking, at an error rate the business can live with.
⌁ Vendors of freight document automation, such as bill of lading and proof of delivery parsers, sell on the share of documents a customer no longer has to key by hand. This drill is about measuring that share honestly, at a stated error rate, instead of reporting accuracy.
Start this internshipCreate an account to unlock the 6 sections, the workbench, and askThili.
BeginSections
1. Read the Freight Paperwork
🔒 locked2. 1 · Read the page
🔒 locked3. 2 · Learn the layout
🔒 locked4. 3 · Confidence that means something
🔒 locked5. 4 · The consequence table and straight-through posting
🔒 locked6. 5 · Reconcile, route, measure
🔒 lockedDig deeper
📄LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking (Huang et al., 2022, arXiv:2204.08387)
paper📄On Calibration of Modern Neural Networks (Guo et al., 2017, arXiv:1706.04599)
paper📄Selective Classification for Deep Neural Networks (Geifman and El-Yaniv, 2017, arXiv:1705.08500)
paper📄DocILE Benchmark for Document Information Localization and Extraction (Simsa et al., 2023, arXiv:2302.05658)
paper🔗thili-ai/thilisynth: the data this drill runs on (scans, desk rules and ground truth)
code