A Paradigm Shift: Reasoning at Enteprise Scale

When performing retrieval at scale on large sets of enteprise documents, it becomes very clear that current Retrieval Augmented Generation (RAG)-like approaches are not well suited (irrespective to the context windows becoming very large). The "RAG is dead" meme that comes out every so often, willfully ignores that 

• most interesting sets of documents are always beyond the latest largest context window that the cool kids talk about
• the reason we want a satisfying RAG is that we do not want to choose the documents that will come into the context window
• the current story is about text, get ready for images, voice and videos
• large context windows do not assure a level of recall quality

If company documents are the context needed to have a purposeful discussion with LLMs inside a company or if new services or products are built on internal documents, then we need to have new algorithms for an enriched experience with all the company knowledge.

At LightOn, we believe the future of AI retrieval lies in reasoning, not just pattern matching. As Antoine Chaffin explained in his Maven podcast appearance, single-vector embeddings collapse nuance into one dimension, limiting systems to shallow similarity. (Before you read the rest of the blog post, do not hesitate to get in touch if you want to help in building this new stack)

Late-interaction models take a different approach:

• Every token is preserved as its own vector.
• Matching happens late, at the interaction stage.
• The result: deeper semantic understanding and genuine reasoning.

This simple but powerful insight has sparked an open-source ecosystem that’s now shaping both academic research and production-scale AI systems.

PyLate: From Experimental Code to Peer-Reviewed Paper

PyLate began as an internal experiment to simplify multi-vector training. Today, it’s a full-fledged library with 527 GitHub stars and growing adoption.

• Academic recognition: PyLate’s paper was accepted at CIKM 2025 (see below), becoming the first peer-reviewed library dedicated to training ColBERT-style models.
• Practical impact: Researchers can train a state-of-art retrieval model on MS MARCO in under 2 hours with just ~80 lines of code.
• Real-world benefit: Out-of-domain search, reasoning-heavy tasks, and long-context retrieval become accessible to any team.

if you want to learn more about the library: PyLate documentation

ModernBERT: Re-Imagining the Encoder

In partnership with Answer.AI, LightOn co-developed ModernBERT, a model that fundamentally rethinks encoder architecture.

• 8192-token context with Flash Attention, running efficiently on consumer GPUs.
• 1,500 GitHub stars and 27M+ downloads on HuggingFace.
• Poster presentation at ACL 2025 (Vienna): validation from one of NLP’s most competitive venues.

ModernBERT has already been cited 305+ times, with variants like BioClinical ModernBERT emerging for healthcare applications.

👉 Explore: ModernBERT LightOn blog post

FastPlaid: Performance That Scales

Building great models is only half the challenge, making them work in production is the other. That’s where FastPlaid comes in.

• A Rust + CUDA engine for multi-vector search.
• Delivers +554% throughput improvements for multi-vector search compared to Stanford’s PLAID baseline.
• Designed for scalability: powering recommendation engines, retrieval-augmented generation (RAG), and real-time search.

As Raphael Sourty explains, static indexes solve many use cases, but mutable indexes (new in v1.10.0) unlock real-world applications where data evolves continuously.

👉 Read more: FastPlaid LightOn blogpost

PyLate-rs: Retrieval in the Browser

Finally, to push accessibility even further, PyLate-rs compiles late-interaction inference to WebAssembly (WASM).

That means:

• Run a state-of-the-art retriever directly in the browser.
• Achieve 97% faster cold-start performance on CPU.
• Remove server dependencies entirely.

This lowers the barrier for demos, education, and lightweight deployments, proving late-interaction isn’t just powerful, it’s portable.

From Theory to Production: A Movement

Taken together, these projects form a technical symphony:

• ModernBERT provides the backbone.
• PyLate enables fast and easy training of SOTA models.
• FastPlaid ensures scalable search performance.
• PyLate-rs brings inference to any environment.

The ecosystem has grown from an academic curiosity into a reasoning-first retrieval stack. With recognition at CIKM and ACL, adoption across GitHub and HuggingFace, and practical tools for real-world workflows, LightOn is helping shape the next era of AI search.

📖 Explore LightOn’s open-source ecosystem:

• PyLate
• ModernBERT
• FastPlaid
• PyLate has already enabled the development of state-of-the-art models, such as: 
• GTE-ModernColBERT and 
• Reason-ModernColBERT
• Other models:
• Ettin Suite: SoTA Paired Encoders and Decoders
• BioClinical ModernBERT: a new SOTA encoder model for Medical NLP
• MonoQwen-Vision, the first visual document reranker

Dataset

• FC-AMF-OCR Dataset : a 9.3 million images OCR dataset to improve real world document parsing

Pre-training libraries

• Passing the Torch: Training a Mamba Model for Smooth Handover
• Training Mamba Models on AMD MI250/MI250X GPUs with Custom Kernels

PyLate: Flexible Training and Retrieval for Late Interaction Models by Antoine Chaffin, Raphaël Sourty

Neural ranking has become a cornerstone of modern information retrieval. While single vector search remains the dominant paradigm, it suffers from the shortcoming of compressing all the information into a single vector. This compression leads to notable performance degradation in out-of-domain, long-context, and reasoning-intensive retrieval tasks. Multi-vector approaches pioneered by ColBERT aim to address these limitations by preserving individual token embeddings and computing similarity via the MaxSim operator. This architecture has demonstrated superior empirical advantages, including enhanced out-of-domain generalization, long-context handling, and performance in complex retrieval scenarios. Despite these compelling empirical results and clear theoretical advantages, the practical adoption and public availability of late interaction models remain low compared to their single-vector counterparts, primarily due to a lack of accessible and modular tools for training and experimenting with such models. To bridge this gap, we introduce PyLate, a streamlined library built on top of Sentence Transformers to support multi-vector architectures natively, inheriting its efficient training, advanced logging, and automated model card generation while requiring minimal code changes to code templates users are already familiar with. By offering multi-vector-specific features such as efficient indexes, PyLate aims to accelerate research and real-world application of late interaction models, thereby unlocking their full potential in modern IR systems. Finally, PyLate has already enabled the development of state-of-the-art models, including GTE-ModernColBERT and Reason-ModernColBERT, demonstrating its practical utility for both research and production environments.

🌐 Learn more about lighton.ai

** Nuit Blanche is now on Twitter: @NuitBlog **
Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:
Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog
About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv