The EnterpriseTalk Conversion Collection

Why Legal AI Misreads Tables — and a Better Way to Chunk Them

Tabular documents — financial schedules, deal-point tables, spreadsheets, database exports — are everywhere in legal and corporate work, yet most retrieval-augmented AI systems handle them poorly. The reason is upstream of the model itself: before any AI can reason over a document, that document must be chunked into smaller segments for retrieval. Standard chunking methods were built for flowing prose, so they slice tables at arbitrary points, separating headers from values and splitting a single record across multiple chunks. The result is fragmented context and harder-to-retrieve information.

This paper introduces Structure-Aware Tabular Chunking (STC), a method that respects the structure of the data instead of fighting it. STC represents each row as a key-value block within a hierarchical "Row Tree," then splits only at field boundaries when a token budget requires it and merges adjacent rows greedily to produce dense, non-overlapping chunks. Evaluated on MAUD, an attorney-annotated benchmark of real merger-and-acquisition contracts, STC delivers measurable gains in both retrieval accuracy and processing efficiency over standard recursive-chunking baselines. The approach is domain-agnostic and extends naturally to spreadsheets, logs, and other structured sources.

Inside this paper, you'll find:

• The chunking problem few teams measure. Why segmentation choices made before the model runs quietly shape what your AI can and can't retrieve.
• The Row Tree method, end to end. How key-value encoding, token-constrained splitting, and greedy merging preserve field-level relationships within each chunk.
• Retrieval results on real M&A contracts. STC improves recall from 0.347 to 0.539 in a hybrid retrieval setting, with even larger gains (0.366 → 0.754) in a lexical-only test that isolates the effect of chunk structure.
• Efficiency gains. A ~40% reduction in chunk count versus the recursive baseline, higher token utilization, and faster processing — for a leaner index.
• Where it goes next. Honest limitations and the path toward end-to-end RAG evaluation.