Conference Corner

In our Conference Corner, readers will find Imminent’s take on the most important conferences in language research. Each edition highlights the most interesting talks, notable papers, and emerging trends presented at these events. Whether you’re exploring advances in linguistics, NLP, or broader language sciences, our curated summaries provide a clear and engaging snapshot of the ideas and innovations shaping the field.

NeurIPS 2025

This flagship conference represents the leading event for cutting-edge empirical and theoretical advances in Machine Learning, AI scaling, Foundation Models, Reinforcement Learning, and Multimodal Systems. 

Attending NeurIPS 2025 provided resources, motivation, and strategic context to refine quality estimation models, integrate advanced architectures into data pipelines, and advance multimodal models.

Emerging Trends

At NeurIPS 2025, several emerging areas gained prominence, particularly LLM efficiency, alignment for evaluation, ethics and reliability, and reasoning capabilities. These topics reflect the field’s shift toward large-scale real-world deployment, where models must overcome issues such as unreliable reasoning, privacy risks, and high computational costs. Research focused on making LLMs more practical through efficiency techniques like deeper architectures and quantization, improving evaluation through better alignment and scrutiny of LLM-as-judge systems, and addressing trust concerns including data memorization, privacy, and the risk of output homogenization across models.

Highlights from Talks and Papers

A standout oral, “Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?”, challenged assumptions about RLVR, showing that reinforcement learning can improve performance but does not fundamentally extend reasoning beyond the base model’s inherent capabilities.

Scaling models remains a powerful driver of new capabilities. The Best Paper, “1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities”, demonstrated that extreme network depth (up to 1024 layers) can dramatically boost goal-conditioned self-supervised robotic performance. Meanwhile, “Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)” revealed convergence patterns in LLM outputs, underscoring the need for diversity checks in MT and QA tasks.

Efficiency and trust were also key focuses. Talks on scaling data quality and privacy/legal risks in generative AI offered frameworks for high-volume data validation and strategies to mitigate privacy exposure, addressing compliance and production challenges. Posters like “LittleBit: Ultra Low-Bit Quantization via Latent Factorization” and “Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness” introduced ultra-low-bit quantization and rigorous unlearning evaluations, enabling efficient models while safeguarding sensitive information.

Overall, the conference emphasized the dual challenge of pushing LLM capabilities while ensuring reliability, ethical safeguards, and efficient deployment, highlighting that advances in reasoning, scale, and data stewardship are tightly intertwined.

Conference Main Themes

The conference highlighted the gap between benchmark performance and real-world reliability, stressing the need for large models that are efficient, practical, and broadly usable.

Trust, safety, and responsible deployment were key concerns, alongside smarter architectures and evaluation strategies. Human-AI collaboration emerged as a focus, combining human oversight with AI scalability.

New Resources

NeurIPS 2025 highlighted key open-source resources, including the Infinity-Chat dataset for diversity-aware generation, the aformentioned LittleBit ultra-low-bit quantization toolkit for efficient edge models, and the Toloka hybrid human-AI annotation platform for scalable data curation. These tools and datasets represent some of the most impactful contributions for practical NLP and AI research.

EURIPS 2025

EurIPS 2025, the first European satellite event of flagship event NeurIPS held in Copenhagen, provided a unique strategic advantage. By concentrating around 2000 members of the European AI talent pool and featuring a selective program (38 Orals, 241 posters), it offered high-density networking and access to scientific advancement in AI within an EU context.

Emerging Trends

At EurIPS 2025, Sustainable AI was a central theme, focusing on efficiency, adaptability, and environmental responsibility. Emtiyaz Khan’s talk, “Adaptive Bayesian Intelligence and the Road to Sustainable AI” introduced continual learning methods that reduce the need to retrain models from scratch, while Sepp Hochreiter’s “Sustainable, Low-Energy, and Fast AI Made in Europe” presented xLSTM, an efficient alternative to Transformers that lowers energy use, speeds inference, and handles longer contexts for practical, low-energy AI deployment.

Highlights from Talks and Papers

Some papers and talks The Art of (Artificial) Reasoning, by Yejin Choi who argues that it is possible to democratize generative AI transcending current Scaling Laws, which implies the only path is extreme scaling of resources, by innovating with unconventional data, algorithms and collaboration. She collaborated on her position presenting ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models demonstrating it is possible to push reasoning capabilities of small models with careful RL. For data she presents Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning demonstrating is possible to scale performance scaling synthetic data generation, and as example of extreme collaboration she present OpenThoughts: Data Recipes for Reasoning Models, a dataset created coordinating many universities, companies and startups.

Some interesting talk and posters, focusing on techniques directly applicable in language processes: 

Inference-Time Hyper-Scaling with KV Cache Compression – boosts reasoning accuracy by compressing Transformer’s KV caches to allow more token generation within the same memory footprint. The proposed method, named Dynamic Memory Sparsification (DMS), sparsifies KV caches achieving up to 8x compression by teaching pre-trained models which tokens can be scheduled for future eviction.

Beyond Oracle: Verifier-Supervision for Instruction Hierarchy in Reasoning and Instruction-Tuned LLMs (paper) – introduces a unified framework that improves instruction hierarchy in LLMs (e.g: System vs User) by utilizing programmatically verifiable signals instead of costly oracle labels. The method introduces a synthesis pipeline to create conflicting instruction pairs and executable verifiers (i.e: python functions), ensuring dataset quality through an automated unit testing and repair. Using RL with verifiable rewards with this dataset significantly enhances adherence to complex directives and safety robustness.

Hogwild! Inference: Parallel LLM Generation via Concurrent Attention (paper) – The research introduces a parallel generation method that allows multiple LLM instances to collaborate via a shared, concurrently-updated KV cache. The system enables workers to decide a shared strategy and synchronize seeing each other’s memory (KV entries) without requiring additional fine-tuning or recomputation. This new promising approach enables effective and efficient collaboration between multiple LLMs boosting accuracy on reasoning tasks.

Conference Main Themes

Beyond the central theme of Sustainable AI, the conference maintained a strong balance across a broad range of topics. The program featured diverse research areas including Model Optimization and Representation Learning, as well as specialized fields such as Computer Vision, Diffusion Models, Graph Neural Networks, Causal Inference, and Reinforcement Learning.

New Resources

EurIPS 2025 highlighted key open-source resources, including TabArena LivingBenchmark – a living benchmark for ML on Tabular Data, advocating for evolving evaluation benchmarks over static ones. Although applied to tabular data, the specific challenges addressed in the paper are likely applicable to other domains as well.

EMNLP 2025

EMNLP is one of the world’s leading conferences on empirical natural language processing, with a strong focus on practical results, large-scale experimentation, and emerging applications of language technologies. Its relevance is immediate: the conference spans machine translation, multimodality, model evaluation, dataset creation, ethical considerations, and the fast-evolving ecosystem of large language models. This year it was held in Suzhou, China.

EMNLP also hosts the Workshop on Machine Translation (WMT), the premier annual workshop on machine translation. Tracking WMT findings has long informed our research directions in quality estimation. Attending EMNLP 2025 therefore offered not only inspiration but also direct insight into the future landscape of applied NLP.

One of the clearest cross-cutting themes this year was LLM efficiency. As models grow ever larger, researchers are increasingly focused on making them faster, lighter, and more accessible. Topics such as advanced KV-caching, extreme quantization, and improved distillation methods stood out. These approaches aim to reduce computation, cut memory needs, and make high-performance models usable on modest hardware — an essential development for many applied research teams.

Another emerging space is the use of LLMs in every stage of the research pipeline, from synthetic dataset creation to automated evaluation. This shift is reshaping traditional workflows and prompting new questions about how the field measures quality, novelty, and reliability.

Highlights from Talks and Papers

Among the most memorable keynotes was Heng Ji’s “No more processing. Time to discover.” Speaking from the vantage point of drug discovery and the broader “AI for Science” movement, she challenged the community to re-center research around genuine breakthroughs rather than incremental improvements. Her call for models and methods that support true scientific discovery resonated strongly throughout the conference.

From an MT perspective, Lonyue Wang (Alibaba International) delivered a timely talk on the evolution of multilingual translation in the LLM era. He described the shift from fine-tuned LLMs to reasoning-driven models and now to LLM-based agents — while also highlighting the persistent gap between academic MT benchmarks and the needs of industrial-scale translation systems.

On the technical front, the paper “S1: Simple Test-time Scaling” drew considerable attention. It presents an open-source method for boosting model performance by increasing compute only at inference time — an ability previously demonstrated by proprietary systems such as OpenAI’s o1. With a modest finetuning set of just 1,000 questions on Qwen2.5-32B, the authors achieved competitive gains, opening the door to new experimentation strategies for many labs.

Here is a list of other research that stood out during the conference:

• Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index
• Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?
• Efficient Model Development through Fine-tuning Transfer
• Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models
• Evaluating Language Translation Models by Playing Telephone
• Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement

Conference Main Themes

Beyond efficiency and LLM-centric pipelines, several research threads recurred across workshops and oral sessions.
From the works that stood out from the sea of incremental research pieces, the common themes involved explorations inside LLM building blocks to hypothesize newer approaches in the architecture (e.g. a mixture of heterogenously-sized experts rather than homogenous ones in practice), and interpretations from LLM’s internal representations for their implications on surface-level output.

New Resources

For quality estimation work, several new models merit improved capabalities, including NVIDIA’s Qwen-MQM, Google’s MetricX-25 and GemSpanEval, and an RL-based MT evaluation model presented by researchers from the University of Amsterdam. Together, these resources represent some of the most relevant and forward-looking contributions to EMNLP and WMT 2025.

EACL 2026

EACL is one of the flagship conferences of the European Chapter of the Association for Computational Linguistics (ACL), bringing together researchers and practitioners working on all aspects of natural language processing. The conference is known for combining rigorous scientific contributions with a broad international perspective on language technologies, including machine translation, multilingual NLP, language resources, evaluation, ethics, and recent advances in large language models. Its relevance is especially strong in fostering collaboration across Europe and beyond while highlighting research with real-world impact. In 2026, EACL was held in Rabat, Morocco.

Emerging Trends

EACL 2026 featured a broad spectrum of topics, with several established fields continuing to have significant attention, such as Information Retrieval and Tokenization, alongside areas that appeared to be gaining notable momentum. Among these, for example:

• Agents & Interaction: highlighting a shift from viewing LLMs as static generators toward autonomous systems capable of planning, acting, and executing tasks within complex environments. Current research focuses on embedding LLMs into agentic pipelines and tool-use frameworks to enable multi-step reasoning and multi-agent cooperation, a critical direction as the industry moves from content generation to functional task automation.
• Safety & Alignment: addressing how to make models more robust against misuse, adversarial inputs, and misaligned behaviors. Given the continued deployment of LLMs in real-world applications, this line of research sits at the intersection of technical reliability and responsible AI.

Additionally, amplified by EACL’s first edition in Africa, multilingualism and low-resource support were at the forefront of the conference. The agenda highlighted this commitment through the inclusion of various affinity and language groups such as Arabic NLP, SomosNLP, and Muslims in ML. Two of the four keynote addresses were central to this narrative:

• In Arabic and Technology: A 40-Year Perspective traced Arabic NLP, Nizar Habash revisited the development of Arabic NLP from early foundational efforts to today’s large-scale generative systems, highlighting the language’s unique challenges: rich morphology, ambiguous orthography, and its extensive dialectal variation. He concluded with a forward-looking vision for a cohesive, sustainable ecosystem, advancing Arabic in AI through stronger collaboration and innovation for the next generation of researchers.
• In Omnilinguality: Scaling AI to Any Language (paper), Marta R. Costa-Juss presented a vision for extending AI capabilities across the world’s languages, framing “omnilinguality” not as a task-specific challenge, but as a broader arena for advancing general-purpose LLM techniques. While MT has historically driven multilingual progress, scaling to thousands of languages and producing key open resources like FLORES and BOUQuET, she argued that with the rise of LLMs, multilingual coverage should be addressed by design from the very beginning, particularly in early infrastructure choices, rather than treated as a downstream specialization.

This convergence of signals around multilingualism aligns with our long-term vision for Lara models.

Highlights from Talks and Papers

In addition to the previous two keynotes, here are some of the most impactful papers:

• Humans and Transformer LMs: Abstraction drives language learning:
  Winner of the Best Paper Award, this research investigates whether transformer models acquire linguistic categories through abstract feature-based learning or concrete exemplar-based memorization. By tracking a GPT-2 model during training, the study reveals that abstract, class-level behaviors emerge before item-specific ones, demonstrating that language models possess an inductive bias toward forming abstractions.

• DivMerge: A divergence-based model merging method for multi-tasking
  Winner of the Outstanding Paper Award, this research introduces DivMerge, a reference-free method that combines multiple specialized models into a single multi-task one. The method automatically balances task importance and mitigates interference without requiring explicit target distributions or labeled data, achieving this by minimizing the divergence between the outputs of the original fine-tuned models and the newly merged model. For our team, this presents a clear practical interest, as we could experiment to efficiently merge different translation models or combine models trained on entirely distinct tasks into one unified system, all while avoiding the high computational costs associated with standard multi-task retraining.

• Detecting Hallucination in Vision Language Models without generating a single token
  This paper tackles hallucination in VLMs from an interesting angle: rather than detecting errors after generation, it asks whether a model’s internal states, captured in a single forward pass before any token is produced, can already predict whether a hallucination is likely. By training lightweight probes on these representations, the authors show that late query-token states tend to be the most predictive, though the optimal layer varies by architecture. For our work, the prospect of anticipating hallucinations before they occur opens up some genuinely useful directions.

• Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning
  This paper addresses a genuinely interesting challenge: how to extend reinforcement learning techniques, typically effective in structured, verifiable domains like mathematics, to broader reasoning tasks. The proposed pipeline covers the full process, from data curation and template design to control answer-space complexity, rule-based reward modeling, and data blending strategies. The core finding is that integrating multi-domain data meaningfully improves both reasoning accuracy and token efficiency across diverse benchmarks. The methodology may help improve Lara translation accuracy and contextual understanding.

• Specialization through Collaboration: Understanding Expert Interaction in Mixture-of-Expert Large Language Models
  In this paper, the authors reframe MoE specialization from individual “subnetworks” to coordinated groups. Using Hierarchical Sparse Dictionary Learning (HSDL), they reveal that models specialize through cross-layer collaborations that map to distinct tasks. This enables a pruning algorithm, which leverages these patterns to prune up to 50% of experts with minimal performance loss, proving that maintaining collaborative structures is more effective than simply preserving high-frequency experts.

• ReflectiveRAG: Rethinking Adaptivity in Retrieval-Augmented Generation
  The authors address a practical RAG failure mode: performance degradation under noisy retrieval. Rather than scaling model size, they propose two lightweight modules,  a Self-Reflective Retrieval (SRR) and a Contrastive Noise Removal (NR), to first iteratively evaluate evidence sufficiency and perform query reformulation, then refine retrieved content via embedding-based contrastive filtering. This approach offers a fresh perspective on the retrieval problem, injecting reasoning into the process rather than relying on scale. The adaptive query reformulation mechanism could be particularly relevant for improving Lara’s adaptation capabilities.

• A Family of LLMs Liberated from Static Vocabularies(presented in the BoF Tokenization & Beyond)
  This paper from Aleph Alpha proposes replacing traditional tokenizers with a hierarchical byte-level architecture (HAT), where a small encoder aggregates bytes into standard word embeddings and a decoder converts outputs back to bytes. This approach makes the model inherently more robust to spelling variations and more adaptable to new languages and domains through continued training. Interestingly, the proposed techniques are compatible with pretrained models (hence “liberated”) and are able to match or outperform the original Llama 3.1 performance on most benchmarks..

Conference Main Themes

Beyond the prominent focus on multilinguality and under-resourced languages, the conference maintained a strong balance across the broader NLP landscape. Research contributions covered a wide range of topics, including inference optimization through techniques such as KV-cache compression and speculative decoding, bias mitigation and fairness, multimodal models, and data curation. This breadth highlighted the community’s continued focus on both advancing model capabilities and addressing practical challenges related to efficiency, reliability, and responsible deployment.

ICML 2026

The International Conference on Machine Learning (ICML) is one of the leading conferences in machine learning, bringing together researchers working across the field, from generative modelling and large language models to reinforcement learning, optimization, and evaluation. Its broad scope makes it an important opportunity to assess the state of the art, exchange practical insights with researchers facing similar training and data challenges, and identify directions that may shape future research. For Translated, ICML also provides valuable visibility within the research community, supporting future collaborations and initiatives such as the Imminent Research Grants. In 2026, the conference offered a particularly clear view of how the field is moving beyond scaling model capabilities toward improving efficiency, reliability, and real-world performance.

Emerging Trends

ICML 2026 featured several research directions gaining momentum, including developments that challenge established assumptions about language generation and model training.

• Diffusion Language Models: Both Outstanding Paper Awards went to research on diffusion, signalling a notable shift from the dominance of autoregressive transformers. The discussion has progressed beyond whether diffusion can support language generation toward identifying the training methods, sampling strategies, and theoretical foundations needed to make it competitive. Diffusion is also increasingly being explored as a way to accelerate inference in existing models, rather than solely as an alternative architecture.
• LLM-as-a-Judge and Evaluation Reliability: Model-based evaluation has become a standard approach as the number of models and systems outpaces the capacity for human annotation. However, research increasingly focuses on its limitations, including preferences for longer answers, biases toward particular model families, and inconsistent agreement with human evaluators on difficult tasks. Developing more reliable evaluation methods is essential to producing trustworthy benchmarks and assessing improvements meaningfully.
• Self-Improvement Agents: Agentic AI has become a central research direction, with growing interest in systems that generate, evaluate, and learn from their own outputs. These approaches address a fundamental constraint in model development: the availability of high-quality human-generated training data. The key question is how far models can generate useful training signals themselves without reinforcing errors or degrading performance.
• More Principled and Efficient Pretraining: Researchers are increasingly replacing established training heuristics with analytical approaches to optimization, learning-rate schedules, batch sizes, and data selection. Results presented at the conference suggest that better-designed training procedures can match or outperform heavily tuned baselines while reducing computational costs.
• Embodied AI and Robotics: A growing share of research is concerned with systems interacting with physical environments. Compared with text-only applications, these settings introduce additional challenges, including real-time constraints, sample efficiency, and transferring capabilities from simulation to the real world.

Together, these trends point to a broader shift in machine learning: progress depends not only on building more capable models, but also on making them more efficient to train, more reliable to evaluate, and more useful in practical settings.

Highlights from Talks and Papers

Several talks and papers offered particularly relevant insights into the future of model development, evaluation, and multilingual AI.

• What Will Be Left for Us to Work On? — Arvind Narayanan
  In one of the conference’s most thought-provoking invited talks, Arvind Narayanan examined what happens to human work as AI capabilities continue to improve. His central argument was that technical capability alone does not translate immediately into widespread economic or organizational change. Between a system becoming capable and its transforming a profession lie several slower stages: building a usable product, achieving adoption, and reorganizing work around it. Drawing an analogy with electrification, Narayanan argued that simply replacing existing tools with new technology does not unlock its full potential; meaningful gains often require redesigning the surrounding processes. He also highlighted a widening gap between model accuracy and reliability, distinguishing consistency, robustness, calibration, and operational safety as separate dimensions. These distinctions matter when designing agents for different purposes, since systems intended to automate tasks may require different characteristics from those designed to collaborate with people.The implications for translation are particularly relevant. Narayanan used machine translation as a counterexample to the assumption that automation necessarily eliminates human work. Despite substantial improvements in translation quality, he argued that employment has remained broadly stable, partly because demand for translation is not fixed: there is always more content to translate and more languages into which it can be translated.
  He concluded that as AI takes on more verifiable tasks, evaluation will become increasingly important.
  
• How Far Can Quadratics Take Us? Lessons for LLM Pretraining — Sham Kakade
  This invited talk explored how a relatively simple quadratic model can explain much of the optimization behaviour observed during large-scale language-model pretraining. The proposed framework offers a way to replace some widely used training heuristics with more principled methods.Two examples were presented: SeeSaw, which trades learning-rate decay for increasing batch sizes to achieve comparable loss with lower serial runtime, and Horizon-Free Pretraining, which aims to match tuned cosine-decay schedules without requiring the training horizon to be specified in advance. The talk also examined theoretical limits on the interaction between momentum and batch size. The broader takeaway is that a better understanding of optimization dynamics could make large-scale training more predictable and computationally efficient.
  
• mmBERT: A Modern Multilingual Encoder with Annealed Language Learning
  This encoder-only model was trained on three trillion tokens across more than 1,800 languages. Its distinctive approach combines an inverse mask-ratio schedule with inverse-temperature sampling, introducing more than 1,700 low-resource languages into the training mixture only during the decay phase.The reported results show that the timing of language exposure can significantly affect performance, suggesting that multilingual training is not simply a matter of adding more data. For Translated, this is a particularly relevant finding: the scheduling of low-resource language data may be an important design variable when training multilingual models, alongside dataset size and sampling proportions.
  
• Olmix: A Framework for Data Mixing Throughout LM Development
  Developed during the pretraining of Olmo 3, Olmix treats data mixing as an ongoing process rather than a decision made once at the beginning of training. The work systematically investigates design choices in data-mixing methods, including how to handle domains with limited available data.Its most practical contribution is mixture reuse: when a dataset changes, the framework recalculates mixing ratios only for the affected domains instead of recomputing the entire mixture. Across five dataset updates, this approach matched full recomputation while using 74% less compute. The framework, with its released code and data, offers a potentially useful reference for teams that regularly update training corpora or manage multilingual datasets.
  
• Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates
  This Spotlight paper introduces a training-free approach to adapting agents at inference time. Rather than updating model parameters through gradient-based reinforcement learning, the method maintains a non-parametric memory of previous trajectories, retrieves relevant examples, estimates advantages, and uses them to adjust output logits.The authors show that the additive update corresponds to a closed-form solution to a KL-constrained policy optimization objective, giving a principled foundation to the approach. The method achieves state-of-the-art results among training-free approaches on WebArena and Jericho, at a fraction of the cost of reinforcement-learning fine-tuning. It is an interesting direction for improving agent behaviour through experience without repeatedly retraining the underlying model.
  
• Quantifying Biases in LLM-as-a-Judge Evaluations
  This paper proposes a statistical framework based on Bayesian generalized linear models to identify, quantify, and correct biases in model-based evaluation. These include preferences for longer responses and for outputs generated by a judge’s own model family. The method supports both absolute scoring and pairwise comparisons, while providing uncertainty estimates in addition to conventional inter-rater agreement measures.Its relevance extends beyond benchmarking: if model-based evaluation is used to compare translation systems, assess generated content, or guide model development, systematic evaluator biases can distort the conclusions. Methods that explicitly measure these biases could help establish more reliable evaluation pipelines.

Conference Main Themes

Beyond these individual contributions, several themes recurred throughout ICML 2026.

Evaluation is becoming a central research problem. As models and agents improve, accuracy alone is insufficient to establish whether a system is dependable. Bias, calibration, robustness, sycophancy, and agent-specific benchmarks are receiving increasing attention. Narayanan’s invited talk reinforced the point that evaluation deserves a more prominent role as AI systems move into practical applications.

Training signals matter as much as training algorithms. Reinforcement learning remains widespread in post-training, but researchers are increasingly investigating how rewards are designed and delivered. Natural-language critiques, rubric-based feedback, and alternative optimization objectives offer ways to move beyond reliance on a single scalar reward.

Efficiency is becoming a core design constraint. Research on quantization-aware training, optimizer memory, token compression, and early-exit methods reflects a growing emphasis on reducing computational costs. Similarly, diffusion-inspired sampling techniques are beginning to offer ways of accelerating inference, in some cases without retraining existing models.

Multilingual performance depends on data strategy, not just scale. Work such as mmBERT and Olmix highlights the importance of when languages enter training, how data is sampled, and how mixtures are updated as datasets evolve. These questions are particularly important for translation systems that must balance high-resource languages with languages for which data remains scarce.

Agents are moving toward adaptation and self-improvement. Research on test-time learning, self-critique, and synthetic training signals points toward systems that can improve their behaviour beyond conventional static inference. The central challenge is ensuring that these mechanisms improve performance without amplifying errors or introducing new reliability problems.

Taken together, ICML 2026 suggests that the next phase of machine learning will be shaped by the interaction between capability, efficiency, and trustworthiness. For Translated, the most actionable directions include experimenting with multilingual data-scheduling strategies, improving evaluation methodologies, investigating more efficient training and inference techniques, and exploring approaches to agent adaptation that do not require expensive model updates.

ACL 2026: Emerging Trends in Natural Language Processing

The Association for Computational Linguistics (ACL) is one of the world’s leading conferences in Natural Language Processing (NLP) and Computational Linguistics. The 2026 edition brought together approximately 3,500 participants and featured 4,462 accepted papers, offering a broad overview of current research in language technologies.

For Translated, ACL was particularly relevant to Lara’s development, with a strong focus on machine translation evaluation, multilingual language models, and techniques for improving training. While much of the research presented incremental rather than groundbreaking advances, several contributions offered practical ideas worth exploring, particularly in automated evaluation, language fairness, and cultural understanding.

Emerging Trends

LLM-as-a-Judge and Evaluation Reliability

LLM-based evaluation was one of the most prominent themes at ACL 2026. Researchers explored how language models can assess system outputs against structured criteria, addressing the limitations of traditional metrics when evaluating high-quality translations. Sample-level rubrics, automatically generated evaluation criteria, judge calibration, and bias mitigation were recurring areas of investigation.

These methods are particularly relevant to Lara, as they could help identify translation errors and quality regressions that conventional automated metrics struggle to detect, while making human review more targeted and efficient.

From Multilinguality to Multiculturality

The conference highlighted the distinction between supporting many languages and understanding the cultural contexts associated with them. Multilingual training does not automatically equip models with knowledge of local conventions, idioms, preferences, and communication norms. Research suggested that cultural signals can diminish during successive stages of model training and alignment.

For machine translation, this distinction is especially important: a translation can be linguistically correct while failing to preserve the cultural meaning of the original text.

Fairness in Multilingual Tokenization

Researchers also investigated how tokenization can create disparities between languages. Conventional Byte-Pair Encoding optimizes compression across a training corpus, potentially favouring high-resource languages. Parity-Aware BPE proposes an alternative that improves the balance between languages while maintaining the performance of standard tokenization.

This work suggests that multilingual fairness can be improved through foundational design choices, rather than relying exclusively on changes to training data or model architecture.

Reinforcement Learning for Post-Training

Reinforcement learning remained a major research direction, with many papers investigating modifications to GRPO, a widely used algorithm for post-training language models. Several approaches reported incremental improvements in training stability or benchmark performance.

However, much of this research focused on mathematical and logical reasoning. Whether the same improvements transfer to translation and other language tasks remains an open question, making task-specific validation important for Translated.

More Efficient Training and Text Generation

Other contributions examined the organization of training data and the strategies used to generate text. Research suggested that sample ordering, gradual transitions between data distributions, periodic data reuse, and batch diversity can influence downstream performance. New sampling methods also explored how to control generation dynamically and reduce degenerate outputs at high temperatures.

Together, these studies illustrate how targeted changes to training and inference can yield practical improvements without requiring entirely new model architectures.

3. Highlights from Talks and Papers

Philip Resnick — A New Balancing Act: Reflections on the Relationship between Computational Linguistics and AI

Resnick’s keynote examined how the success of large language models has reshaped Computational Linguistics. He identified three potential imbalances: the decline of theory-driven explanations in favour of black-box experimentation, the concentration of research around a narrow set of methods and architectures, and the tendency to treat language primarily as an input/output modality rather than as a subject of study in its own right.

The talk emphasized the need for the research community to address these imbalances. For Translated, it reinforces the importance of combining empirical improvements in language models with a deeper understanding of language and communication.

Julia Kreutzer — From Multilingual to Multicultural Models

Kreutzer’s keynote at the Workshop on Multilinguality in the Era of Large Language Models explored why multilinguality does not guarantee cultural understanding. The accompanying paper, The Culture Funnel: You Can’t Align What Isn’t in the Data, argues that explicit cultural signals can diminish through pre-training, instruction tuning, alignment, and reasoning-data stages.

The work suggests that culture should be considered explicitly during data curation. This is particularly relevant to translation, where idioms, cultural references, and local communication norms are essential to preserving meaning.

Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization

This paper addresses the uneven representation of languages in conventional tokenization. By modifying BPE to maximize the minimum compression rate across languages, the proposed method aims to improve cross-lingual fairness while maintaining the performance of standard BPE.

The approach offers a practical research direction for investigating how tokenization choices affect language parity in multilingual models.

Autorubric: A Unifying Framework for Rubric-Based LLM Evaluation on Non-Verifiable Tasks

Autorubric brings together structured rubrics, ensemble judging, few-shot calibration, and bias mitigation in an open-source evaluation framework. These techniques address weaknesses in LLM-based judging, including position bias, criterion conflation, and low agreement between evaluators.

For Lara, rubric-based evaluation could support the identification of translation regressions and recurring error patterns, helping prioritize samples for human assessment.

Demystifying Data Organization for Enhanced LLM Training

This paper investigates how training-data ordering affects model performance. It identifies several useful principles: progressively increasing sample difficulty or quality, periodically reintroducing earlier data, ensuring smooth transitions between data distributions, and maintaining diversity within mini-batches.

The findings highlight data scheduling as a potential lever for improving training outcomes without changing the underlying model architecture.

Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting

This work proposes a modification to GRPO intended to improve reinforcement-learning stability by reweighting large covariance values between token entropy and advantage estimates. The authors report small but consistent gains over a standard GRPO baseline, although their experiments are limited to mathematical tasks.

The paper illustrates the potential of targeted improvements to established RL algorithms, while also highlighting the need to verify whether such gains transfer to translation.

Min-k Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamics

This paper introduces a dynamic sampling method that adjusts token truncation at each generation step rather than relying solely on fixed thresholds. The authors report promising results at unusually high temperatures without the degenerate outputs typically associated with such settings.

The approach offers another example of how modifications to inference-time strategies can improve control over generated text.

Conference Main Themes

Several recurring ideas connected the research presented at ACL 2026:

• Evaluation beyond conventional metrics: Structured rubrics, judge calibration, and bias mitigation are becoming increasingly important for assessing language-model outputs, particularly in tasks where quality cannot be captured by a single score.
• Multilingual fairness: Tokenization and data representation can introduce disparities between languages, making fairness a concern throughout the model-development pipeline.
• Cultural understanding: Multilingual coverage alone is insufficient for tasks such as translation, where preserving cultural meaning is central to output quality.
• Targeted improvements to training: Research on GRPO, data organization, and sampling demonstrates the potential value of relatively small changes to existing methods.
• Reproducibility and practical implementation: The conference featured datasets, evaluation protocols, and open-source tools intended to make research easier to reproduce and integrate into existing workflows.

Overall, ACL 2026 reinforced the importance of moving beyond general model capabilities towards more reliable evaluation, fairer multilingual representation, and a better understanding of the cultural dimensions of language.