Manan Suri

2025

1. VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation

   Manan Suri, Puneet Mathur, Franck Dernoncourt, Kanika Goswami, Ryan A. Rossi, and Dinesh Manocha

   In Proceedings of the 2025 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Apr 2025

2024

1. Doc2Command: Furthering Language Guided Document Editing

   Manan Suri, Puneet Mathur, Ramit Sawhney, Preslav Nakov, and Dinesh Manocha

   In The Second Tiny Papers Track at ICLR 2024, Apr 2024
2. DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding

   Manan Suri, Puneet Mathur, Franck Dernoncourt, Rajiv Jain, Vlad I Morariu, Ramit Sawhney, Preslav Nakov, and Dinesh Manocha

   In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Nov 2024

   Abs

   Document structure editing involves manipulating localized textual, visual, and layout components in document images based on the user’s requests. Past works have shown that multimodal grounding of user requests in the document image and identifying the accurate structural components and their associated attributes remain key challenges for this task. To address these, we introduce the DocEditAgent, a novel framework that performs end-to-end document editing by leveraging Large Multimodal Models (LMMs). It consists of three novel components – (1) Doc2Command to simultaneously localize edit regions of interest (RoI) and disambiguate user edit requests into edit commands. (2) LLM-based Command Reformulation prompting to tailor edit commands originally intended for specialized software into edit instructions suitable for generalist LMMs. (3) Moreover, DocEditAgent processes these outputs via Large Multimodal Models like GPT-4V and Gemini, to parse the document layout, execute edits on grounded Region of Interest (RoI), and generate the edited document image. Extensive experiments on the DocEdit dataset show that DocEditAgent significantly outperforms strong baselines on edit command generation (2-33%), RoI bounding box detection (12-31%), and overall document editing (1-12%) tasks.

2023

1. ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER

   Sreyan Ghosh, Utkarsh Tyagi, Manan Suri, Sonal Kumar, Ramaneswaran S, and Dinesh Manocha

   In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2023

   Abs Bib HTML PDF

   Complex Named Entity Recognition (NER) is the task of detecting linguistically complex named entities in low-context text. In this paper, we present ACLM Attention-map aware keyword selection for Conditional Language Model fine-tuning), a novel data augmentation approach based on conditional generation, to address the data scarcity problem in low-resource complex NER. ACLM alleviates the context-entity mismatch issue, a problem existing NER data augmentation techniques suffer from and often generates incoherent augmentations by placing complex named entities in the wrong context. ACLM builds on BART and is optimized on a novel text reconstruction or denoising task - we use selective masking (aided by attention maps) to retain the named entities and certain keywords in the input sentence that provide contextually relevant additional knowledge or hints about the named entities. Compared with other data augmentation strategies, ACLM can generate more diverse and coherent augmentations preserving the true word sense of complex entities in the sentence. We demonstrate the effectiveness of ACLM both qualitatively and quantitatively on monolingual, cross-lingual, and multilingual complex NER across various low-resource settings. ACLM outperforms all our neural baselines by a significant margin (1%-36%). In addition, we demonstrate the application of ACLM to other domains that suffer from data scarcity (e.g., biomedical). In practice, ACLM generates more effective and factual augmentations for these domains than prior methods.

   @inproceedings{ghosh-etal-2023-aclm,
     title = {{ACLM}: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex {NER}},
     author = {Ghosh, Sreyan and Tyagi, Utkarsh and Suri, Manan and Kumar, Sonal and S, Ramaneswaran and Manocha, Dinesh},
     booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
     month = jul,
     year = {2023},
     address = {Toronto, Canada},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2023.acl-long.8},
     doi = {10.18653/v1/2023.acl-long.8},
     pages = {104--125},
   }

2. CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network

   Sreyan Ghosh, Manan Suri, Purva Chiniya, Utkarsh Tyagi, Sonal Kumar, and Dinesh Manocha

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023
3. I Don’t Feel so Good! Detecting Depressive Tendencies Using Transformer-Based Multimodal Frameworks

   Manan Suri, Nalin Semwal, Divya Chaudhary, Ian Gorton, and Bijendra Kumar

   In Proceedings of the 2022 5th International Conference on Machine Learning and Natural Language Processing, Dec 2023

   Abs

   One of the most common mental illnesses that affects 5% of adults globally is depression. The advancement of social media has meant that more and more people have gained a platform to voice their thoughts and beliefs. People’s social media interactions and posted content can be used to infer critical characteristics such as depressive tendencies which will allow for timely intervention and help. This paper describes a novel supervised approach to detect depressive tendencies in Twitter users using multimodal frameworks which account for user interaction and online behaviour in addition to the tweet content processed using transformers like BERT. The performance of three multimodal frameworks is described with different methods for combining modalities. The best result is obtained a cross-modality based model which improves the baseline by 12% points.

2022

1. Boosting Pre-trained Language Models with Task Specific Metadata and Cost Sensitive Learning

   Manan Suri

   In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), Jul 2022

   Abs

   This paper describes our system for Task 4 of SemEval 2022: Patronizing and Condescending Language Detection. Patronizing and Condescending Language (PCL) refers to language used with respect to vulnerable communities that portrays them in a pitiful way and is reflective of a sense of superiority. Task 4 involved binary classification (Subtask 1) and multi-label classification (Subtask 2) of Patronizing and Condescending Language (PCL). For our system, we experimented with fine-tuning different transformer-based pre-trained models including BERT, DistilBERT, RoBERTa and ALBERT. Further, we have used token separated metadata in order to improve our model by helping it contextualize different communities with respect to PCL. We faced the challenge of class imbalance, which we solved by experimenting with different class weighting schemes. Our models were effective in both subtasks, with the best performance coming out of models with Effective Number of Samples (ENS) class weighting and token separated metadata in both subtasks. For subtask 1 and subtask 2, our best models were finetuned BERT and RoBERTa models respectively.