ToMMeR - Efficient Entity Mention Detection from Large Language Models
This work provides evidence that structured entity representations exist in early transformer layers and can be efficiently recovered with minimal parameters, and reveals that diverse architectures converge on similar mention boundaries, confirming that mention detection emerges naturally from language modeling
Abstract
Identifying which text spans refer to entities - mention detection- is both foundational for information extraction and a known performance bottleneck. We introduce ToMMeR, a lightweight model (\ensuremath75%), confirming that mention detection emerges naturally from language modeling. When extended with span classification heads, ToMMeR achieves competitive NER performance (80-87% F1 on standard benchmarks). Our work provides evidence that structured entity representations exist in early transformer layers and can be efficiently recovered with minimal parameters.