Manan Suri

2026

1. 

   arXiv

   Frames2LoRA: Parametric Video Internalization for Vision-Language Models

   Manan Suri*, Sarvesh Baskar*, and Dinesh Manocha

   Jun 2026

   * equal contribution. Formerly titled Video2LoRA (arXiv v1).

   Abs Bib HTML PDF Code Website Page Checkpoints Press

   Video is expensive for vision-language models: every frame costs hundreds of tokens, and every repeated query pays that cost again. Frames2LoRA internalizes a video parametrically. A perceiver hypernetwork reads the frozen VLM’s layer-wise states as it encodes a video and generates a LoRA adapter in a single forward pass, with no gradient updates. The adapter-augmented model then answers arbitrary queries with zero visual tokens in context: up to 1,500x lower visual-token load and 6-80x faster time-to-first-token, with outputs statistically non-inferior to full video-in-context inference.

   @misc{suri2026frames2lora,
     title = {Frames2LoRA: Parametric Video Internalization for Vision-Language Models},
     author = {Suri*, Manan and Baskar*, Sarvesh and Manocha, Dinesh},
     year = {2026},
     month = jun,
     eprint = {2606.04351},
     archiveprefix = {arXiv},
     primaryclass = {cs.CV},
     url = {https://arxiv.org/abs/2606.04351},
     note = {* equal contribution. Formerly titled Video2LoRA (arXiv v1).},
   }

2. 

   arXiv

   Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models

   Puneet Mathur, Manan Suri, and Dinesh Manocha

   Aug 2026

   Abs Bib HTML PDF Page

   Extends the Frames2LoRA idea from video to joint audio-visual context. A Perceiver hypernetwork encodes omnimodal context into a LoRA adapter, with GRPO-based rank allocation driven by a modality-ablated counterfactual reward that preserves cross-modal coherence. 8-12% average gains over the strongest baselines on four audio-visual QA benchmarks at a 30% rank budget, and up to 12x lower time-to-first-token than full-context inference.

   @misc{mathur2026omni2lora,
     title = {Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models},
     author = {Mathur, Puneet and Suri, Manan and Manocha, Dinesh},
     year = {2026},
     month = aug,
     eprint = {2608.09227},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/abs/2608.09227},
   }

3. 

   arXiv

   CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Training

   Amine Lbath*, Manan Suri*, Aurélien Delaitre, Vadim Okun, Massih-Reza Amini, Ram D. Sriram, and Dinesh Manocha

   Aug 2026

   * equal contribution

   Abs Bib HTML PDF Code Website Page Press

   Security agents need training data that doesn’t exist: real repositories with known, executable vulnerabilities. CyberForge injects vulnerabilities into real C/C++ projects with dynamic validation: unit tests must pass and the proof-of-vulnerability must trigger only on the injected build. 1,034 validated vulnerabilities across 80 projects and 63 weakness categories. Fine-tuning on it lifts SEC-bench patch repair by 3.3-14.7 points, generalizing to PatchEval and other languages. Developed during a NIST PREP fellowship; featured by UMD Computer Science in "When AI Goes on Defense".

   @misc{lbath2026cyberforge,
     title = {CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Training},
     author = {Lbath*, Amine and Suri*, Manan and Delaitre, Aur{\'e}lien and Okun, Vadim and Amini, Massih-Reza and Sriram, Ram D. and Manocha, Dinesh},
     year = {2026},
     month = aug,
     eprint = {2608.06471},
     note = {* equal contribution},
     archiveprefix = {arXiv},
     primaryclass = {cs.CR},
     url = {https://arxiv.org/abs/2608.06471},
   }

4. 

   ACL

   CodeScout: Contextual Problem Statement Enhancement for Software Agents

   Manan Suri, Xiangci Li, Mehdi Shojaie, Songyang Han, Chao-Chun Hsu, Shweta Garg, Aniket Anand Deshmukh, and Varun Kumar

   In Findings of the Association for Computational Linguistics: ACL 2026, Jul 2026

   Abs Bib HTML PDF Page

   Current AI-powered code assistance tools struggle with poorly-defined problem statements lacking task context. CodeScout addresses this through contextual query refinement — converting underspecified requests into comprehensive problem statements via lightweight pre-exploration of target codebases. The method performs targeted context scoping, multi-perspective analysis examining potential fixes, and synthesizes insights into enhanced problem statements with reproduction steps and exploration hints. Evaluated on SWEBench-Verified, it demonstrated a 20% improvement in resolution rates with up to 27 additional issues resolved compared to baseline methods.

   @inproceedings{suri-etal-2026-codescout,
     title = {{C}ode{S}cout: Contextual Problem Statement Enhancement for Software Agents},
     author = {Suri, Manan and Li, Xiangci and Shojaie, Mehdi and Han, Songyang and Hsu, Chao-Chun and Garg, Shweta and Deshmukh, Aniket Anand and Kumar, Varun},
     booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
     month = jul,
     year = {2026},
     address = {San Diego, California, United States},
     publisher = {Association for Computational Linguistics},
     url = {https://arxiv.org/abs/2603.05744},
   }

5. 

   ACL

   Structured Uncertainty Guided Clarification for LLM Agents

   Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, and Dinesh Manocha

   In Findings of the Association for Computational Linguistics: ACL 2026, Jul 2026

   Abs Bib HTML PDF Page

   LLM agents extend large language models with tool-calling capabilities, but ambiguous user instructions often lead to incorrect invocations and task failures. We introduce a principled formulation of structured uncertainty over tool-call parameters, modeling joint tool-argument clarification as a POMDP with Expected Value of Perfect Information (EVPI) objective for optimal question selection and aspect-based cost modeling to prevent redundancy. Our SAGE-Agent increases coverage on ambiguous tasks by 7–39% while reducing clarification questions by 1.5–2.7x compared to baselines. We also present ClarifyBench, the first multi-turn tool-augmented disambiguation benchmark with realistic LLM-based user simulation across diverse domains.

   @inproceedings{suri-etal-2026-clarification,
     title = {Structured Uncertainty Guided Clarification for {LLM} Agents},
     author = {Suri, Manan and Mathur, Puneet and Lipka, Nedim and Dernoncourt, Franck and Rossi, Ryan A. and Manocha, Dinesh},
     booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
     month = jul,
     year = {2026},
     address = {San Diego, California, United States},
     publisher = {Association for Computational Linguistics},
     url = {https://arxiv.org/abs/2511.08798},
   }

2025

1. 

   NAACL

   VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation

   Manan Suri, Puneet Mathur, Franck Dernoncourt, Kanika Goswami, Ryan A. Rossi, and Dinesh Manocha

   In Proceedings of the 2025 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Apr 2025

   Bib HTML PDF Page

   @inproceedings{suri2025visdommultidocumentqavisually,
     title = {VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation},
     author = {Suri, Manan and Mathur, Puneet and Dernoncourt, Franck and Goswami, Kanika and Rossi, Ryan A. and Manocha, Dinesh},
     booktitle = {Proceedings of the 2025 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)},
     year = {2025},
     month = apr,
     eprint = {2412.10704},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/abs/2412.10704},
   }

2. 

   ACL

   ChartLens: Fine-grained Visual Attribution in Charts

   Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, and Dinesh Manocha

   In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2025

   Abs Bib HTML PDF Page

   The growing capabilities of multimodal large language models (MLLMs) have advanced tasks like chart understanding. However, these models often suffer from hallucinations, where generated text sequences conflict with the provided visual data. To address this, we introduce Post-Hoc Visual Attribution for Charts, which identifies fine-grained chart elements that validate a given chart-associated response. We propose ChartLens, a novel chart attribution algorithm that uses segmentation-based techniques to identify chart objects and employs set-of-marks prompting with MLLMs for fine-grained visual attribution. Additionally, we present ChartVA-Eval, a benchmark with synthetic and real-world charts from diverse domains like finance, policy, and economics, featuring fine-grained attribution annotations. Our evaluations show that ChartLens improves fine-grained attributions by 26-66%.

   @inproceedings{suri-etal-2025-chartlens,
     title = {{C}hart{L}ens: Fine-grained Visual Attribution in Charts},
     author = {Suri, Manan and Mathur, Puneet and Lipka, Nedim and Dernoncourt, Franck and Rossi, Ryan A. and Manocha, Dinesh},
     editor = {Che, Wanxiang and Nabende, Joyce and Shutova, Ekaterina and Pilehvar, Mohammad Taher},
     booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
     month = jul,
     year = {2025},
     address = {Vienna, Austria},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2025.acl-long.1094/},
     pages = {22447--22462},
     isbn = {979-8-89176-251-0},
   }

3. 

   EMNLP

   Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic Agents

   Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, Vivek Gupta, and Dinesh Manocha

   In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Nov 2025

   Abs Bib HTML PDF Page

   Flowcharts are a critical tool for visualizing decision-making processes. However, their non-linear structure and complex visual-textual relationships make it challenging to interpret them using LLMs, as vision-language models frequently hallucinate nonexistent connections and decision paths when analyzing these diagrams. This leads to compromised reliability for automated flowchart processing in critical domains such as logistics, health, and engineering. We introduce the task of Fine-grained Flowchart Attribution, which traces specific components grounding a flowchart referring LLM response. Flowchart Attribution ensures the verifiability of LLM predictions and improves explainability by linking generated responses to the flowchart’s structure. We propose FlowPathAgent, a neurosymbolic agent that performs fine-grained post hoc attribution through graph-based reasoning. It first segments the flowchart, then converts it into a structured symbolic graph, and then employs an agentic approach to dynamically interact with the graph, to generate attribution paths. Additionally, we present FlowExplainBench, a novel benchmark for evaluating flowchart attributions across diverse styles, domains, and question types. Experimental results show that FlowPathAgent mitigates visual hallucinations in LLM answers over flowchart QA, outperforming strong baselines by 10–14% on our proposed FlowExplainBench dataset.

   @inproceedings{suri-etal-2025-follow,
     title = {Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic Agents},
     author = {Suri, Manan and Mathur, Puneet and Lipka, Nedim and Dernoncourt, Franck and Rossi, Ryan A. and Gupta, Vivek and Manocha, Dinesh},
     editor = {Christodoulopoulos, Christos and Chakraborty, Tanmoy and Rose, Carolyn and Peng, Violet},
     booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
     month = nov,
     year = {2025},
     address = {Suzhou, China},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2025.emnlp-main.1144/},
     pages = {22496--22519},
     isbn = {979-8-89176-332-6},
   }

2024

1. 

   EMNLP

   DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding

   Manan Suri, Puneet Mathur, Franck Dernoncourt, Rajiv Jain, Vlad I Morariu, Ramit Sawhney, Preslav Nakov, and Dinesh Manocha

   In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Nov 2024

   Abs Bib HTML PDF Page

   Document structure editing involves manipulating localized textual, visual, and layout components in document images based on the user’s requests. Past works have shown that multimodal grounding of user requests in the document image and identifying the accurate structural components and their associated attributes remain key challenges for this task. To address these, we introduce the DocEditAgent, a novel framework that performs end-to-end document editing by leveraging Large Multimodal Models (LMMs). It consists of three novel components – (1) Doc2Command to simultaneously localize edit regions of interest (RoI) and disambiguate user edit requests into edit commands. (2) LLM-based Command Reformulation prompting to tailor edit commands originally intended for specialized software into edit instructions suitable for generalist LMMs. (3) Moreover, DocEditAgent processes these outputs via Large Multimodal Models like GPT-4V and Gemini, to parse the document layout, execute edits on grounded Region of Interest (RoI), and generate the edited document image. Extensive experiments on the DocEdit dataset show that DocEditAgent significantly outperforms strong baselines on edit command generation (2-33%), RoI bounding box detection (12-31%), and overall document editing (1-12%) tasks.

   @inproceedings{suri-etal-2024-docedit,
     title = {{D}oc{E}dit-v2: Document Structure Editing Via Multimodal {LLM} Grounding},
     author = {Suri, Manan and Mathur, Puneet and Dernoncourt, Franck and Jain, Rajiv and Morariu, Vlad I and Sawhney, Ramit and Nakov, Preslav and Manocha, Dinesh},
     editor = {Al-Onaizan, Yaser and Bansal, Mohit and Chen, Yun-Nung},
     booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
     month = nov,
     year = {2024},
     address = {Miami, Florida, USA},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2024.emnlp-main.867},
     pages = {15485--15505},
   }

2023

1. 

   ACL

   ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER

   Sreyan Ghosh, Utkarsh Tyagi, Manan Suri, Sonal Kumar, Ramaneswaran S, and Dinesh Manocha

   In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2023

   Abs Bib HTML PDF Page

   Complex Named Entity Recognition (NER) is the task of detecting linguistically complex named entities in low-context text. In this paper, we present ACLM Attention-map aware keyword selection for Conditional Language Model fine-tuning), a novel data augmentation approach based on conditional generation, to address the data scarcity problem in low-resource complex NER. ACLM alleviates the context-entity mismatch issue, a problem existing NER data augmentation techniques suffer from and often generates incoherent augmentations by placing complex named entities in the wrong context. ACLM builds on BART and is optimized on a novel text reconstruction or denoising task - we use selective masking (aided by attention maps) to retain the named entities and certain keywords in the input sentence that provide contextually relevant additional knowledge or hints about the named entities. Compared with other data augmentation strategies, ACLM can generate more diverse and coherent augmentations preserving the true word sense of complex entities in the sentence. We demonstrate the effectiveness of ACLM both qualitatively and quantitatively on monolingual, cross-lingual, and multilingual complex NER across various low-resource settings. ACLM outperforms all our neural baselines by a significant margin (1%-36%). In addition, we demonstrate the application of ACLM to other domains that suffer from data scarcity (e.g., biomedical). In practice, ACLM generates more effective and factual augmentations for these domains than prior methods.

   @inproceedings{ghosh-etal-2023-aclm,
     title = {{ACLM}: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex {NER}},
     author = {Ghosh, Sreyan and Tyagi, Utkarsh and Suri, Manan and Kumar, Sonal and S, Ramaneswaran and Manocha, Dinesh},
     booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
     month = jul,
     year = {2023},
     address = {Toronto, Canada},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2023.acl-long.8},
     doi = {10.18653/v1/2023.acl-long.8},
     pages = {104--125},
   }

2. 

   EMNLP

   CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network

   Sreyan Ghosh, Manan Suri, Purva Chiniya, Utkarsh Tyagi, Sonal Kumar, and Dinesh Manocha

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023

   Bib HTML PDF Page

   @inproceedings{ghosh2023cosyn,
     title = {CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network},
     author = {Ghosh, Sreyan and Suri, Manan and Chiniya, Purva and Tyagi, Utkarsh and Kumar, Sonal and Manocha, Dinesh},
     booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
     pages = {6159--6173},
     year = {2023},
     month = dec,
     url = {https://aclanthology.org/2023.emnlp-main.377/},
   }