Publications

2025

1. 

   Multi-View Attention Fusion for Explainable Body Language Behavior Recognition

   Surbhi Madan, Rishabh Jain, Ramanathan Subramanian, and 1 more author

   In IEEE Transactions on Affective Computing, 2025

   Status: Accepted

   Abs Link

   Body language behavior, including gestures and fine-grained movements not only reflects human emotions, but also serves as a versatile cue for enhancing emotional intelligence and creating responsive technologies. In this work, we explore the efficacy of multiview-multimodal cues for explainable prediction of bodily behavior. This paper proposes an attention fusion method that combines features extracted from multiview RGB videos, their multiview Discrete Cosine Transform representations, and three stream skeleton features, via a transformer-based approach. We evaluate our approach on the diverse BBSI and Drive&Act datasets. Empirical results confirm that the RGB, DCT and Skeleton features enable discovery of multiple class-specific behaviors resulting in explainable predictions. Our key findings are that multimodal approaches outperform unimodal counterparts in categorizing bodily behavioral classes, efficient class predictions and plausible explanations are achieved with both unimodal and multimodal approaches, and that empirical results confirm the superiority of our approach compared to state-of-the-art methods on both datasets. The code is available at: https://github.com/surbhimadan92/MAGIC_TBR_Extended

2. 

   Periodic Spatio-Temporal Colored Hotspot Detection in Azure Traffic Data

   Rakesh Rajeev, Rishabh Jain, Venkata M. V. Gunturi, and 5 more authors

   In Asian Conference on Intelligent Information and Database Systems (ACIIDS), 2025

   Status: Accepted

   Abs Link

   The problem of periodic spatio-temporal colored hotspot detection (PST-Col-Hotspot) takes the following input: (a) a spatio-temporal color event framework E; (b) a collection of events A (over E), where each event is associated with a set of geo-spatial coordinates, a timestamp and a color. Colors are organized into a hierarchy, CH, with white being the root. Given the input, the goal of PST-Col-Hotspot problem is to determine spatial regions which show high “intensity” of events (of a particular color) at certain periodic intervals. The output of the PST-Col-Hotspot detection problem consists of the following: (a) a collection of spatial regions (which show high intensity of events), (b) their respective time intervals of high activity, periodicity values (e.g., daily, weekday-only, etc),and color according to the given color hierarchy CH. PST-Col-Hotspot detection poses significant challenge for designing a suitable interest measure. The aim here is to design a mathematical representation of a PST-Col-Hotspot such that it can differentiate interesting periodic patterns from trivial persistent patterns in the dataset. The current state of the art in the area of spatial and spatio-temporal hotspot detection focus on non-colored and non-periodic patterns. In contrast, our proposed approach is able to determine colored periodic hotspots. We experimentally evaluated our proposed algorithm using both real Azure traffic dataset(from Indian region) and synthetic dataset.

2023

1. 

   MAGIC-TBR: Multiview Attention Fusion for Transformer-Based Bodily Behavior Recognition in Group Settings

   Surbhi Madan, Rishabh Jain, Gulshan Sharma, and 2 more authors

   In Proceedings of the 31st ACM International Conference on Multimedia, 2023

   Status: Accepted

   Abs Link

   Bodily behavioral language is an important social cue, and its automated analysis helps in enhancing the understanding of artificial intelligence systems. Furthermore, behavioral language cues are essential for active engagement in social agent-based user interactions. Despite the progress made in computer vision for tasks like head and body pose estimation, there is still a need to explore the detection of finer behaviors such as gesturing, grooming, or fumbling. This paper proposes a multiview attention fusion method named MAGIC-TBR that combines features extracted from videos and their corresponding Discrete Cosine Transform coefficients via a transformer-based approach. The experiments are conducted on the BBSI dataset and the results demonstrate the effectiveness of the proposed feature fusion with multiview attention. The code is available at: https://github.com/surbhimadan92/MAGIC-TBR

2. 

   Spatio-Temporal Hotspot Detection in Microsoft Azure

   Rishabh Jain, Vishawam Datta, and Venkata M.V. Gunturi

   In B.Tech thesis, Indian Institute of Technology Ropar, India, 2023

   Type: Thesis document

   Abs PDF

   The objective of this project was to work on the different aspects of periodic spatio-temporal hotspots in Microsoft Azure Services. Without millions of users using Azure APIs, it becomes necessary to track usage spikes and perform neces- sary configuration changes in the servers to handle large number of requests. We thus implemented algorithms to process usage data from Azure services to return periodic usage spikes. The first part of this report involves the implementation and experiments based on the Mono-Chromatic Spatio-Temporal Hotspot algorithm, proposed by Gunturi et al. [2022]. We add several modules to the same, modify the framework, and run experiments to verify its performance. The second part involves design, implementation and experiments based on the novel Multi-Chromatic Spatio-Temporal Hotspot algorithm. We discuss the problem formulation, the design and implementation of the algorithm.

3. 

   Deep Federated Recommender Systems

   Rishabh Jain, and Shweta Jain

   In B.Tech Honors Thesis, Indian Institute of Technology Ropar, India, 2023

   Type: Thesis document

   Abs PDF

   With data being called the ’oil’ of the 21st century, it is becoming an invaluable commodity. Data is primarily being used on a large scale by industries worldwide to train extensive artificially intelligent systems. This data, however, often contains user sensitive information which causes privacy concerns. Furthemore, it is becoming increasingly ex- pensive to train large scale models due to ever increasing number of users. Federated Learning is a distributed machine learning paradigm which addresses the issue of data privacy, as well as offsets the cost of compute requirements to edge devices. Federated learning algorithms help train machine learning models without sharing the user data with the service providers. In this work, we focus on the application of user privacy in federated recommender systems.

4. 

   A General System for Dataset Generation from Smartwatch Sensors for Biomedical Research

   Rishabh Jain, Manav Mago, Vibham Kumar Dubey, and 3 more authors

   In Proceedings of International Conference on Frontiers in Computing and Systems, 2023

   Status: Accepted

   Abs Link

   This paper describes a novel Android Wear OS application, designed to enable researchers to conveniently collect sensor data from commonly available Wear OS smartwatches for dataset generation. The application was primarily developed to ensure the reliability of data collection during physical exercises. The application thus proposed implements a reliable dataset generation architecture, which researchers can extend and modify to adapt to their workflows. In the application, data is collected from sensors present on the smartwatch and then transferred to a paired smartphone. Thus, the application eliminates the need for a third device or service (such as cloud storage) for dataset generation. Data obtained from the smartwatch is stored in a predetermined folder/directory in the smartphone in CSV format making it convenient for performing data analysis. The application is tested by recording data from multiple subjects in the lab environment as well as outside the lab. The quality of data was confirmed by plotting it in normal working time, and during physical exercise and the changes are properly observed, which ensures that the data we are collecting is correct. Since the use of smartwatches has become prevalent for studying and diagnosing diseases, the application will be very useful for researchers across the globe for creating datasets for biomedical research.