Showing 1–50 of 79 results for author: Ganguly, S

AdaKD: Dynamic Knowledge Distillation of ASR models using Adaptive Loss Weighting

Authors: Shreyan Ganguly, Roshan Nayak, Rakshith Rao, Ujan Deb, Prathosh AP

Abstract: Knowledge distillation, a widely used model compression technique, works on the basis of transferring knowledge from a cumbersome teacher model to a lightweight student model. The technique involves jointly optimizing the task specific and knowledge distillation losses with a weight assigned to them. Despite these weights playing a crucial role in the performance of the distillation process, curre… ▽ More Knowledge distillation, a widely used model compression technique, works on the basis of transferring knowledge from a cumbersome teacher model to a lightweight student model. The technique involves jointly optimizing the task specific and knowledge distillation losses with a weight assigned to them. Despite these weights playing a crucial role in the performance of the distillation process, current methods provide equal weight to both losses, leading to suboptimal performance. In this paper, we propose Adaptive Knowledge Distillation, a novel technique inspired by curriculum learning to adaptively weigh the losses at instance level. This technique goes by the notion that sample difficulty increases with teacher loss. Our method follows a plug-and-play paradigm that can be applied on top of any task-specific and distillation objectives. Experiments show that our method performs better than conventional knowledge distillation method and existing instance-level loss functions. △ Less

Submitted 11 May, 2024; originally announced May 2024.

Connecting physics to systems with modular spin-circuits

Authors: Kemal Selcuk, Saleh Bunaiyan, Nihal Sanjay Singh, Shehrin Sayed, Samiran Ganguly, Giovanni Finocchio, Supriyo Datta, Kerem Y. Camsari

Abstract: An emerging paradigm in modern electronics is that of CMOS + $\sf X$ requiring the integration of standard CMOS technology with novel materials and technologies denoted by $\sf X$. In this context, a crucial challenge is to develop accurate circuit models for $\sf X$ that are compatible with standard models for CMOS-based circuits and systems. In this perspective we present physics-based, experime… ▽ More An emerging paradigm in modern electronics is that of CMOS + $\sf X$ requiring the integration of standard CMOS technology with novel materials and technologies denoted by $\sf X$. In this context, a crucial challenge is to develop accurate circuit models for $\sf X$ that are compatible with standard models for CMOS-based circuits and systems. In this perspective we present physics-based, experimentally benchmarked modular circuit models that can be used to evaluate a class of CMOS + $\sf X$ systems, where $\sf X$ denotes magnetic and spintronic materials and phenomena. This class of materials is particularly challenging because they go beyond conventional charge-based phenomena and involve the spin degree of freedom which involves non-trivial quantum effects. Starting from density matrices $-$ the central quantity in quantum transport $-$ using well-defined approximations, it is possible to obtain spin-circuits that generalize ordinary circuit theory to 4-component currents and voltages (1 for charge and 3 for spin). With step-by-step examples that progressively go higher in the computing stack, we illustrate how the spin-circuit approach can be used to start from the physics of magnetism and spintronics to enable accurate system-level evaluations. We believe the core approach can be extended to include other quantum degrees of freedom like valley and pseudospins starting from corresponding density matrices. △ Less

Submitted 30 April, 2024; originally announced April 2024.

Restricted Bayesian Neural Network

Authors: Sourav Ganguly, Saprativa Bhattacharjee

Abstract: Modern deep learning tools are remarkably effective in addressing intricate problems. However, their operation as black-box models introduces increased uncertainty in predictions. Additionally, they contend with various challenges, including the need for substantial storage space in large networks, issues of overfitting, underfitting, vanishing gradients, and more. This study explores the concept… ▽ More Modern deep learning tools are remarkably effective in addressing intricate problems. However, their operation as black-box models introduces increased uncertainty in predictions. Additionally, they contend with various challenges, including the need for substantial storage space in large networks, issues of overfitting, underfitting, vanishing gradients, and more. This study explores the concept of Bayesian Neural Networks, presenting a novel architecture designed to significantly alleviate the storage space complexity of a network. Furthermore, we introduce an algorithm adept at efficiently handling uncertainties, ensuring robust convergence values without becoming trapped in local optima, particularly when the objective function lacks perfect convexity. △ Less

Submitted 8 April, 2024; v1 submitted 6 March, 2024; originally announced March 2024.

Uniform $\mathcal{C}^k$ Approximation of $G$-Invariant and Antisymmetric Functions, Embedding Dimensions, and Polynomial Representations

Authors: Soumya Ganguly, Khoa Tran, Rahul Sarkar

Abstract: For any subgroup $G$ of the symmetric group $\mathcal{S}_n$ on $n$ symbols, we present results for the uniform $\mathcal{C}^k$ approximation of $G$-invariant functions by $G$-invariant polynomials. For the case of totally symmetric functions ($G = \mathcal{S}_n$), we show that this gives rise to the sum-decomposition Deep Sets ansatz of Zaheer et al. (2018), where both the inner and outer function… ▽ More For any subgroup $G$ of the symmetric group $\mathcal{S}_n$ on $n$ symbols, we present results for the uniform $\mathcal{C}^k$ approximation of $G$-invariant functions by $G$-invariant polynomials. For the case of totally symmetric functions ($G = \mathcal{S}_n$), we show that this gives rise to the sum-decomposition Deep Sets ansatz of Zaheer et al. (2018), where both the inner and outer functions can be chosen to be smooth, and moreover, the inner function can be chosen to be independent of the target function being approximated. In particular, we show that the embedding dimension required is independent of the regularity of the target function, the accuracy of the desired approximation, as well as $k$. Next, we show that a similar procedure allows us to obtain a uniform $\mathcal{C}^k$ approximation of antisymmetric functions as a sum of $K$ terms, where each term is a product of a smooth totally symmetric function and a smooth antisymmetric homogeneous polynomial of degree at most $\binom{n}{2}$. We also provide upper and lower bounds on $K$ and show that $K$ is independent of the regularity of the target function, the desired approximation accuracy, and $k$. △ Less

Submitted 2 March, 2024; originally announced March 2024.

Comments: 38 pages

MSC Class: 05E10 ACM Class: I.2.4; I.2.6; I.2.0

Reconfigurable Stochastic Neurons Based on Strain Engineered Low Barrier Nanomagnets

Authors: Rahnuma Rahman, Samiran Ganguly, Supriyo Bandyopadhyay

Abstract: Stochastic neurons are efficient hardware accelerators for solving a large variety of combinatorial optimization problems. "Binary" stochastic neurons (BSN) are those whose states fluctuate randomly between two levels +1 and -1, with the probability of being in either level determined by an external bias. "Analog" stochastic neurons (ASNs), in contrast, can assume any state between the two levels… ▽ More Stochastic neurons are efficient hardware accelerators for solving a large variety of combinatorial optimization problems. "Binary" stochastic neurons (BSN) are those whose states fluctuate randomly between two levels +1 and -1, with the probability of being in either level determined by an external bias. "Analog" stochastic neurons (ASNs), in contrast, can assume any state between the two levels randomly (hence "analog") and can perform analog signal processing. They may be leveraged for such tasks as temporal sequence learning, processing and prediction. Both BSNs and ASNs can be used to build efficient and scalable neural networks. Both can be implemented with low (potential energy) barrier nanomagnets (LBMs) whose random magnetization orientations encode the binary or analog state variables. The difference between them is that the potential energy barrier in a BSN LBM, albeit low, is much higher than that in an ASN LBM. As a result, a BSN LBM has a clear double well potential profile, which makes its magnetization orientation assume one of two orientations at any time, resulting in the binary behavior. ASN nanomagnets, on the other hand, hardly have any energy barrier at all and hence lack the double well feature. That makes their magnetizations fluctuate in an analog fashion. Hence, one can reconfigure an ASN to a BSN, and vice-versa, by simply raising and lowering the energy barrier. If the LBM is magnetostrictive, then this can be done with local (electrically generated) strain. Such a reconfiguration capability heralds a powerful field programmable architecture for a p-computer, and the energy cost for this type of reconfiguration is miniscule. △ Less

Submitted 1 April, 2024; v1 submitted 8 February, 2024; originally announced February 2024.

Comments: Some typos in the previous version have been corrected

AnthroNet: Conditional Generation of Humans via Anthropometrics

Authors: Francesco Picetti, Shrinath Deshpande, Jonathan Leban, Soroosh Shahtalebi, Jay Patel, Peifeng Jing, Chunpu Wang, Charles Metze III, Cameron Sun, Cera Laidlaw, James Warren, Kathy Huynh, River Page, Jonathan Hogins, Adam Crespi, Sujoy Ganguly, Salehe Erfanian Ebadi

Abstract: We present a novel human body model formulated by an extensive set of anthropocentric measurements, which is capable of generating a wide range of human body shapes and poses. The proposed model enables direct modeling of specific human identities through a deep generative architecture, which can produce humans in any arbitrary pose. It is the first of its kind to have been trained end-to-end usin… ▽ More We present a novel human body model formulated by an extensive set of anthropocentric measurements, which is capable of generating a wide range of human body shapes and poses. The proposed model enables direct modeling of specific human identities through a deep generative architecture, which can produce humans in any arbitrary pose. It is the first of its kind to have been trained end-to-end using only synthetically generated data, which not only provides highly accurate human mesh representations but also allows for precise anthropometry of the body. Moreover, using a highly diverse animation library, we articulated our synthetic humans' body and hands to maximize the diversity of the learnable priors for model training. Our model was trained on a dataset of $100k$ procedurally-generated posed human meshes and their corresponding anthropometric measurements. Our synthetic data generator can be used to generate millions of unique human identities and poses for non-commercial academic research purposes. △ Less

Submitted 7 September, 2023; originally announced September 2023.

Comments: AnthroNet's Unity data generator source code is available at: https://unity-technologies.github.io/AnthroNet/

Application of Quantum Pre-Processing Filter for Binary Image Classification with Small Samples

Authors: Farina Riaz, Shahab Abdulla, Hajime Suzuki, Srinjoy Ganguly, Ravinesh C. Deo, Susan Hopkins

Abstract: Over the past few years, there has been significant interest in Quantum Machine Learning (QML) among researchers, as it has the potential to transform the field of machine learning. Several models that exploit the properties of quantum mechanics have been developed for practical applications. In this study, we investigated the application of our previously proposed quantum pre-processing filter (Q… ▽ More Over the past few years, there has been significant interest in Quantum Machine Learning (QML) among researchers, as it has the potential to transform the field of machine learning. Several models that exploit the properties of quantum mechanics have been developed for practical applications. In this study, we investigated the application of our previously proposed quantum pre-processing filter (QPF) to binary image classification. We evaluated the QPF on four datasets: MNIST (handwritten digits), EMNIST (handwritten digits and alphabets), CIFAR-10 (photographic images) and GTSRB (real-life traffic sign images). Similar to our previous multi-class classification results, the application of QPF improved the binary image classification accuracy using neural network against MNIST, EMNIST, and CIFAR-10 from 98.9% to 99.2%, 97.8% to 98.3%, and 71.2% to 76.1%, respectively, but degraded it against GTSRB from 93.5% to 92.0%. We then applied QPF in cases using a smaller number of training and testing samples, i.e. 80 and 20 samples per class, respectively. In order to derive statistically stable results, we conducted the experiment with 100 trials choosing randomly different training and testing samples and averaging the results. The result showed that the application of QPF did not improve the image classification accuracy against MNIST and EMNIST but improved it against CIFAR-10 and GTSRB from 65.8% to 67.2% and 90.5% to 91.8%, respectively. Further research will be conducted as part of future work to investigate the potential of QPF to assess the scalability of the proposed approach to larger and complex datasets. △ Less

Submitted 28 August, 2023; originally announced August 2023.

Comments: 13 pages, 8 figures

Development of a Novel Quantum Pre-processing Filter to Improve Image Classification Accuracy of Neural Network Models

Authors: Farina Riaz, Shahab Abdulla, Hajime Suzuki, Srinjoy Ganguly, Ravinesh C. Deo, Susan Hopkins

Abstract: This paper proposes a novel quantum pre-processing filter (QPF) to improve the image classification accuracy of neural network (NN) models. A simple four qubit quantum circuit that uses Y rotation gates for encoding and two controlled NOT gates for creating correlation among the qubits is applied as a feature extraction filter prior to passing data into the fully connected NN architecture. By appl… ▽ More This paper proposes a novel quantum pre-processing filter (QPF) to improve the image classification accuracy of neural network (NN) models. A simple four qubit quantum circuit that uses Y rotation gates for encoding and two controlled NOT gates for creating correlation among the qubits is applied as a feature extraction filter prior to passing data into the fully connected NN architecture. By applying the QPF approach, the results show that the image classification accuracy based on the MNIST (handwritten 10 digits) and the EMNIST (handwritten 47 class digits and letters) datasets can be improved, from 92.5% to 95.4% and from 68.9% to 75.9%, respectively. These improvements were obtained without introducing extra model parameters or optimizations in the machine learning process. However, tests performed on the developed QPF approach against a relatively complex GTSRB dataset with 43 distinct class real-life traffic sign images showed a degradation in the classification accuracy. Considering this result, further research into the understanding and the design of a more suitable quantum circuit approach for image classification neural networks could be explored utilizing the baseline method proposed in this paper. △ Less

Submitted 21 August, 2023; originally announced August 2023.

Comments: 13 pages, 10 figures

Implementing Quantum Generative Adversarial Network (qGAN) and QCBM in Finance

Authors: Santanu Ganguly

Abstract: Quantum machine learning (QML) is a cross-disciplinary subject made up of two of the most exciting research areas: quantum computing and classical machine learning (ML), with ML and artificial intelligence (AI) being projected as the first fields that will be impacted by the rise of quantum machines. Quantum computers are being used today in drug discovery, material & molecular modelling and finan… ▽ More Quantum machine learning (QML) is a cross-disciplinary subject made up of two of the most exciting research areas: quantum computing and classical machine learning (ML), with ML and artificial intelligence (AI) being projected as the first fields that will be impacted by the rise of quantum machines. Quantum computers are being used today in drug discovery, material & molecular modelling and finance. In this work, we discuss some upcoming active new research areas in application of quantum machine learning (QML) in finance. We discuss certain QML models that has become areas of active interest in the financial world for various applications. We use real world financial dataset and compare models such as qGAN (quantum generative adversarial networks) and QCBM (quantum circuit Born machine) among others, using simulated environments. For the qGAN, we define quantum circuits for discriminators and generators and show promises of future quantum advantage via QML in finance. △ Less

Submitted 15 August, 2023; originally announced August 2023.

Comments: AI Summit, London

Quantum Circuit Optimization of Arithmetic circuits using ZX Calculus

Authors: Aravind Joshi, Akshara Kairali, Renju Raju, Adithya Athreya, Reena Monica P, Sanjay Vishwakarma, Srinjoy Ganguly

Abstract: Quantum computing is an emerging technology in which quantum mechanical properties are suitably utilized to perform certain compute-intensive operations faster than classical computers. Quantum algorithms are designed as a combination of quantum circuits that each require a large number of quantum gates, which is a challenge considering the limited number of qubit resources available in quantum co… ▽ More Quantum computing is an emerging technology in which quantum mechanical properties are suitably utilized to perform certain compute-intensive operations faster than classical computers. Quantum algorithms are designed as a combination of quantum circuits that each require a large number of quantum gates, which is a challenge considering the limited number of qubit resources available in quantum computing systems. Our work proposes a technique to optimize quantum arithmetic algorithms by reducing the hardware resources and the number of qubits based on ZX calculus. We have utilised ZX calculus rewrite rules for the optimization of fault-tolerant quantum multiplier circuits where we are able to achieve a significant reduction in the number of ancilla bits and T-gates as compared to the originally required numbers to achieve fault-tolerance. Our work is the first step in the series of arithmetic circuit optimization using graphical rewrite tools and it paves the way for advancing the optimization of various complex quantum circuits and establishing the potential for new applications of the same. △ Less

Submitted 4 June, 2023; originally announced June 2023.

Quantum Natural Language Processing based Sentiment Analysis using lambeq Toolkit

Authors: Srinjoy Ganguly, Sai Nandan Morapakula, Luis Miguel Pozo Coronado

Abstract: Sentiment classification is one the best use case of classical natural language processing (NLP) where we can witness its power in various daily life domains such as banking, business and marketing industry. We already know how classical AI and machine learning can change and improve technology. Quantum natural language processing (QNLP) is a young and gradually emerging technology which has the p… ▽ More Sentiment classification is one the best use case of classical natural language processing (NLP) where we can witness its power in various daily life domains such as banking, business and marketing industry. We already know how classical AI and machine learning can change and improve technology. Quantum natural language processing (QNLP) is a young and gradually emerging technology which has the potential to provide quantum advantage for NLP tasks. In this paper we show the first application of QNLP for sentiment analysis and achieve perfect test set accuracy for three different kinds of simulations and a decent accuracy for experiments ran on a noisy quantum device. We utilize the lambeq QNLP toolkit and $t|ket>$ by Cambridge Quantum (Quantinuum) to bring out the results. △ Less

Submitted 30 May, 2023; originally announced May 2023.

Comments: 6 pages, 9 figures

Journal ref: 10.1109/ICPC2T53885.2022.9776836

Optimal partition of feature using Bayesian classifier

Authors: Sanjay Vishwakarma, Srinjoy Ganguly

Abstract: The Naive Bayesian classifier is a popular classification method employing the Bayesian paradigm. The concept of having conditional dependence among input variables sounds good in theory but can lead to a majority vote style behaviour. Achieving conditional independence is often difficult, and they introduce decision biases in the estimates. In Naive Bayes, certain features are called independent… ▽ More The Naive Bayesian classifier is a popular classification method employing the Bayesian paradigm. The concept of having conditional dependence among input variables sounds good in theory but can lead to a majority vote style behaviour. Achieving conditional independence is often difficult, and they introduce decision biases in the estimates. In Naive Bayes, certain features are called independent features as they have no conditional correlation or dependency when predicting a classification. In this paper, we focus on the optimal partition of features by proposing a novel technique called the Comonotone-Independence Classifier (CIBer) which is able to overcome the challenges posed by the Naive Bayes method. For different datasets, we clearly demonstrate the efficacy of our technique, where we achieve lower error rates and higher or equivalent accuracy compared to models such as Random Forests and XGBoost. △ Less

Submitted 27 April, 2023; originally announced April 2023.

Comments: 7 pages, 13 figures

An Evaluation of Non-Contrastive Self-Supervised Learning for Federated Medical Image Analysis

Authors: Soumitri Chattopadhyay, Soham Ganguly, Sreejit Chaudhury, Sayan Nag, Samiran Chattopadhyay

Abstract: Privacy and annotation bottlenecks are two major issues that profoundly affect the practicality of machine learning-based medical image analysis. Although significant progress has been made in these areas, these issues are not yet fully resolved. In this paper, we seek to tackle these concerns head-on and systematically explore the applicability of non-contrastive self-supervised learning (SSL) al… ▽ More Privacy and annotation bottlenecks are two major issues that profoundly affect the practicality of machine learning-based medical image analysis. Although significant progress has been made in these areas, these issues are not yet fully resolved. In this paper, we seek to tackle these concerns head-on and systematically explore the applicability of non-contrastive self-supervised learning (SSL) algorithms under federated learning (FL) simulations for medical image analysis. We conduct thorough experimentation of recently proposed state-of-the-art non-contrastive frameworks under standard FL setups. With the SoTA Contrastive Learning algorithm, SimCLR as our comparative baseline, we benchmark the performances of our 4 chosen non-contrastive algorithms under non-i.i.d. data conditions and with a varying number of clients. We present a holistic evaluation of these techniques on 6 standardized medical imaging datasets. We further analyse different trends inferred from the findings of our research, with the aim to find directions for further research based on ours. To the best of our knowledge, ours is the first to perform such a thorough analysis of federated self-supervised learning for medical imaging. All of our source code will be made public upon acceptance of the paper. △ Less

Submitted 9 March, 2023; originally announced March 2023.

Training Machine Learning Models to Characterize Temporal Evolution of Disadvantaged Communities

Authors: Milan Jain, Narmadha Meenu Mohankumar, Heng Wan, Sumitrra Ganguly, Kyle D Wilson, David M Anderson

Abstract: Disadvantaged communities (DAC), as defined by the Justice40 initiative of the Department of Energy (DOE), USA, identifies census tracts across the USA to determine where benefits of climate and energy investments are or are not currently accruing. The DAC status not only helps in determining the eligibility for future Justice40-related investments but is also critical for exploring ways to achiev… ▽ More Disadvantaged communities (DAC), as defined by the Justice40 initiative of the Department of Energy (DOE), USA, identifies census tracts across the USA to determine where benefits of climate and energy investments are or are not currently accruing. The DAC status not only helps in determining the eligibility for future Justice40-related investments but is also critical for exploring ways to achieve equitable distribution of resources. However, designing inclusive and equitable strategies not just requires a good understanding of current demographics, but also a deeper analysis of the transformations that happened in those demographics over the years. In this paper, machine learning (ML) models are trained on publicly available census data from recent years to classify the DAC status at the census tracts level and then the trained model is used to classify DAC status for historical years. A detailed analysis of the feature and model selection along with the evolution of disadvantaged communities between 2013 and 2018 is presented in this study. △ Less

Submitted 7 March, 2023; originally announced March 2023.

Exploring Self-Supervised Representation Learning For Low-Resource Medical Image Analysis

Authors: Soumitri Chattopadhyay, Soham Ganguly, Sreejit Chaudhury, Sayan Nag, Samiran Chattopadhyay

Abstract: The success of self-supervised learning (SSL) has mostly been attributed to the availability of unlabeled yet large-scale datasets. However, in a specialized domain such as medical imaging which is a lot different from natural images, the assumption of data availability is unrealistic and impractical, as the data itself is scanty and found in small databases, collected for specific prognosis tasks… ▽ More The success of self-supervised learning (SSL) has mostly been attributed to the availability of unlabeled yet large-scale datasets. However, in a specialized domain such as medical imaging which is a lot different from natural images, the assumption of data availability is unrealistic and impractical, as the data itself is scanty and found in small databases, collected for specific prognosis tasks. To this end, we seek to investigate the applicability of self-supervised learning algorithms on small-scale medical imaging datasets. In particular, we evaluate $4$ state-of-the-art SSL methods on three publicly accessible \emph{small} medical imaging datasets. Our investigation reveals that in-domain low-resource SSL pre-training can yield competitive performance to transfer learning from large-scale datasets (such as ImageNet). Furthermore, we extensively analyse our empirical findings to provide valuable insights that can motivate for further research towards circumventing the need for pre-training on a large image corpus. To the best of our knowledge, this is the first attempt to holistically explore self-supervision on low-resource medical datasets. △ Less

Submitted 28 June, 2023; v1 submitted 3 March, 2023; originally announced March 2023.

Comments: Accepted at IEEE ICIP 2023

Configurable calorimeter simulation for AI applications

Authors: Francesco Armando Di Bello, Anton Charkin-Gorbulin, Kyle Cranmer, Etienne Dreyer, Sanmay Ganguly, Eilam Gross, Lukas Heinrich, Lorenzo Santi, Marumi Kado, Nilotpal Kakati, Patrick Rieck, Matteo Tusoni

Abstract: A configurable calorimeter simulation for AI (COCOA) applications is presented, based on the Geant4 toolkit and interfaced with the Pythia event generator. This open-source project is aimed to support the development of machine learning algorithms in high energy physics that rely on realistic particle shower descriptions, such as reconstruction, fast simulation, and low-level analysis. Specificati… ▽ More A configurable calorimeter simulation for AI (COCOA) applications is presented, based on the Geant4 toolkit and interfaced with the Pythia event generator. This open-source project is aimed to support the development of machine learning algorithms in high energy physics that rely on realistic particle shower descriptions, such as reconstruction, fast simulation, and low-level analysis. Specifications such as the granularity and material of its nearly hermetic geometry are user-configurable. The tool is supplemented with simple event processing including topological clustering, jet algorithms, and a nearest-neighbors graph construction. Formatting is also provided to visualise events using the Phoenix event display software. △ Less

Submitted 8 March, 2023; v1 submitted 3 March, 2023; originally announced March 2023.

Comments: 9 pages, 11 figures

A Deep Dive into the Computational Fidelity of High Variability Low Energy Barrier Magnet Technology for Accelerating Optimization and Bayesian Problems

Authors: Md Golam Morshed, Samiran Ganguly, Avik W. Ghosh

Abstract: Low energy barrier magnet (LBM) technology has recently been proposed as a candidate for accelerating algorithms based on energy minimization and probabilistic graphs because their physical characteristics have a one-to-one mapping onto the primitives of these algorithms. Many of these algorithms have a much higher tolerance for error compared to high-accuracy numerical computation. LBM, however,… ▽ More Low energy barrier magnet (LBM) technology has recently been proposed as a candidate for accelerating algorithms based on energy minimization and probabilistic graphs because their physical characteristics have a one-to-one mapping onto the primitives of these algorithms. Many of these algorithms have a much higher tolerance for error compared to high-accuracy numerical computation. LBM, however, is a nascent technology, and devices show high sample-to-sample variability. In this work, we take a deep dive into the overall fidelity afforded by this technology in providing computational primitives for these algorithms. We show that while the compute results show finite deviations from zero variability devices, the margin of error is almost always certifiable to a certain percentage. This suggests that LBM technology could be a viable candidate as an accelerator for popular emerging paradigms of computing. △ Less

Submitted 15 February, 2023; originally announced February 2023.

Comments: 5 pages, 8 figures

Choose your tools carefully: A Comparative Evaluation of Deterministic vs. Stochastic and Binary vs. Analog Neuron models for Implementing Emerging Computing Paradigms

Authors: Md Golam Morshed, Samiran Ganguly, Avik W. Ghosh

Abstract: Neuromorphic computing, commonly understood as a computing approach built upon neurons, synapses, and their dynamics, as opposed to Boolean gates, is gaining large mindshare due to its direct application in solving current and future computing technological problems, such as smart sensing, smart devices, self-hosted and self-contained devices, artificial intelligence (AI) applications, etc. In a l… ▽ More Neuromorphic computing, commonly understood as a computing approach built upon neurons, synapses, and their dynamics, as opposed to Boolean gates, is gaining large mindshare due to its direct application in solving current and future computing technological problems, such as smart sensing, smart devices, self-hosted and self-contained devices, artificial intelligence (AI) applications, etc. In a largely software-defined implementation of neuromorphic computing, it is possible to throw enormous computational power or optimize models and networks depending on the specific nature of the computational tasks. However, a hardware-based approach needs the identification of well-suited neuronal and synaptic models to obtain high functional and energy efficiency, which is a prime concern in size, weight, and power (SWaP) constrained environments. In this work, we perform a study on the characteristics of hardware neuron models (namely, inference errors, generalizability and robustness, practical implementability, and memory capacity) that have been proposed and demonstrated using a plethora of emerging nano-materials technology-based physical devices, to quantify the performance of such neurons on certain classes of problems that are of great importance in real-time signal processing like tasks in the context of reservoir computing. We find that the answer on which neuron to use for what applications depends on the particulars of the application requirements and constraints themselves, i.e., we need not only a hammer but all sorts of tools in our tool chest for high efficiency and quality neuromorphic computing. △ Less

Submitted 5 May, 2023; v1 submitted 10 February, 2023; originally announced February 2023.

Comments: 13 pages, 6 figures

Journal ref: Front. Nanotechnol. 5:1146852 (2023)

Roadmap for Unconventional Computing with Nanotechnology

Authors: Giovanni Finocchio, Jean Anne C. Incorvia, Joseph S. Friedman, Qu Yang, Anna Giordano, Julie Grollier, Hyunsoo Yang, Florin Ciubotaru, Andrii Chumak, Azad J. Naeemi, Sorin D. Cotofana, Riccardo Tomasello, Christos Panagopoulos, Mario Carpentieri, Peng Lin, Gang Pan, J. Joshua Yang, Aida Todri-Sanial, Gabriele Boschetto, Kremena Makasheva, Vinod K. Sangwan, Amit Ranjan Trivedi, Mark C. Hersam, Kerem Y. Camsari, Peter L. McMahon , et al. (26 additional authors not shown)

Abstract: In the "Beyond Moore's Law" era, with increasing edge intelligence, domain-specific computing embracing unconventional approaches will become increasingly prevalent. At the same time, adopting a variety of nanotechnologies will offer benefits in energy cost, computational speed, reduced footprint, cyber resilience, and processing power. The time is ripe for a roadmap for unconventional computing w… ▽ More In the "Beyond Moore's Law" era, with increasing edge intelligence, domain-specific computing embracing unconventional approaches will become increasingly prevalent. At the same time, adopting a variety of nanotechnologies will offer benefits in energy cost, computational speed, reduced footprint, cyber resilience, and processing power. The time is ripe for a roadmap for unconventional computing with nanotechnologies to guide future research, and this collection aims to fill that need. The authors provide a comprehensive roadmap for neuromorphic computing using electron spins, memristive devices, two-dimensional nanomaterials, nanomagnets, and various dynamical systems. They also address other paradigms such as Ising machines, Bayesian inference engines, probabilistic computing with p-bits, processing in memory, quantum memories and algorithms, computing with skyrmions and spin waves, and brain-inspired computing for incremental learning and problem-solving in severely resource-constrained environments. These approaches have advantages over traditional Boolean computing based on von Neumann architecture. As the computational requirements for artificial intelligence grow 50 times faster than Moore's Law for electronics, more unconventional approaches to computing and signal processing will appear on the horizon, and this roadmap will help identify future needs and challenges. In a very fertile field, experts in the field aim to present some of the dominant and most promising technologies for unconventional computing that will be around for some time to come. Within a holistic approach, the goal is to provide pathways for solidifying the field and guiding future impactful discoveries. △ Less

Submitted 27 February, 2024; v1 submitted 17 January, 2023; originally announced January 2023.

Comments: 80 pages accepted in Nano Futures

Journal ref: Nano Futures (2024)

Hybrid Quantum Generative Adversarial Networks for Molecular Simulation and Drug Discovery

Authors: Prateek Jain, Srinjoy Ganguly

Abstract: In molecular research, simulation \& design of molecules are key areas with significant implications for drug development, material science, and other fields. Current classical computational power falls inadequate to simulate any more than small molecules, let alone protein chains on hundreds of peptide. Therefore these experiment are done physically in wet-lab, but it takes a lot of time \& not p… ▽ More In molecular research, simulation \& design of molecules are key areas with significant implications for drug development, material science, and other fields. Current classical computational power falls inadequate to simulate any more than small molecules, let alone protein chains on hundreds of peptide. Therefore these experiment are done physically in wet-lab, but it takes a lot of time \& not possible to examine every molecule due to the size of the search area, tens of billions of dollars are spent every year in these research experiments. Molecule simulation \& design has lately advanced significantly by machine learning models, A fresh perspective on the issue of chemical synthesis is provided by deep generative models for graph-structured data. By optimising differentiable models that produce molecular graphs directly, it is feasible to avoid costly search techniques in the discrete and huge space of chemical structures. But these models also suffer from computational limitations when dimensions become huge and consume huge amount of resources. Quantum Generative machine learning in recent years have shown some empirical results promising significant advantages over classical counterparts. △ Less

Submitted 15 December, 2022; originally announced December 2022.

Variational Quantum Algorithms for Chemical Simulation and Drug Discovery

Authors: Hasan Mustafa, Sai Nandan Morapakula, Prateek Jain, Srinjoy Ganguly

Abstract: Quantum computing has gained a lot of attention recently, and scientists have seen potential applications in this field using quantum computing for Cryptography and Communication to Machine Learning and Healthcare. Protein folding has been one of the most interesting areas to study, and it is also one of the biggest problems of biochemistry. Each protein folds distinctively, and the difficulty of… ▽ More Quantum computing has gained a lot of attention recently, and scientists have seen potential applications in this field using quantum computing for Cryptography and Communication to Machine Learning and Healthcare. Protein folding has been one of the most interesting areas to study, and it is also one of the biggest problems of biochemistry. Each protein folds distinctively, and the difficulty of finding its stable shape rapidly increases with an increase in the number of amino acids in the chain. A moderate protein has about 100 amino acids, and the number of combinations one needs to verify to find the stable structure is enormous. At some point, the number of these combinations will be so vast that classical computers cannot even attempt to solve them. In this paper, we examine how this problem can be solved with the help of quantum computing using two different algorithms, Variational Quantum Eigensolver (VQE) and Quantum Approximate Optimization Algorithm (QAOA), using Qiskit Nature. We compare the results of different quantum hardware and simulators and check how error mitigation affects the performance. Further, we make comparisons with SoTA algorithms and evaluate the reliability of the method. △ Less

Submitted 14 November, 2022; originally announced November 2022.

A Lego-Brick Approach to Coding for Network Communication

Authors: Nadim Ghaddar, Shouvik Ganguly, Lele Wang, Young-Han Kim

Abstract: Coding schemes for several problems in network information theory are constructed starting from point-to-point channel codes that are designed for symmetric channels. Given that the point-to-point codes satisfy certain properties pertaining to the rate, the error probability, and the distribution of decoded sequences, bounds on the performance of the coding schemes are derived and shown to hold ir… ▽ More Coding schemes for several problems in network information theory are constructed starting from point-to-point channel codes that are designed for symmetric channels. Given that the point-to-point codes satisfy certain properties pertaining to the rate, the error probability, and the distribution of decoded sequences, bounds on the performance of the coding schemes are derived and shown to hold irrespective of other properties of the codes. In particular, we consider the problems of lossless and lossy source coding, Slepian-Wolf coding, Wyner-Ziv coding, Berger-Tung coding, multiple description coding, asymmetric channel coding, Gelfand-Pinsker coding, coding for multiple access channels, Marton coding for broadcast channels, and coding for cloud radio access networks (C-RAN's). We show that the coding schemes can achieve the best known inner bounds for these problems, provided that the constituent point-to-point channel codes are rate-optimal. This would allow one to leverage commercial off-the-shelf codes for point-to-point symmetric channels in the practical implementation of codes over networks. Simulation results demonstrate the gain of the proposed coding schemes compared to existing practical solutions to these problems. △ Less

Submitted 19 October, 2023; v1 submitted 14 November, 2022; originally announced November 2022.

PSP-HDRI$+$: A Synthetic Dataset Generator for Pre-Training of Human-Centric Computer Vision Models

Authors: Salehe Erfanian Ebadi, Saurav Dhakad, Sanjay Vishwakarma, Chunpu Wang, You-Cyuan Jhang, Maciek Chociej, Adam Crespi, Alex Thaman, Sujoy Ganguly

Abstract: We introduce a new synthetic data generator PSP-HDRI$+$ that proves to be a superior pre-training alternative to ImageNet and other large-scale synthetic data counterparts. We demonstrate that pre-training with our synthetic data will yield a more general model that performs better than alternatives even when tested on out-of-distribution (OOD) sets. Furthermore, using ablation studies guided by p… ▽ More We introduce a new synthetic data generator PSP-HDRI$+$ that proves to be a superior pre-training alternative to ImageNet and other large-scale synthetic data counterparts. We demonstrate that pre-training with our synthetic data will yield a more general model that performs better than alternatives even when tested on out-of-distribution (OOD) sets. Furthermore, using ablation studies guided by person keypoint estimation metrics with an off-the-shelf model architecture, we show how to manipulate our synthetic data generator to further improve model performance. △ Less

Submitted 11 July, 2022; originally announced July 2022.

Comments: PSP-HDRI$+$ template Unity environment, benchmark binaries, and source code will be made available at: https://github.com/Unity-Technologies/PeopleSansPeople

Classification of NEQR Processed Classical Images using Quantum Neural Networks (QNN)

Authors: Santanu Ganguly

Abstract: A quantum neural network (QNN) is interpreted today as any quantum circuit with trainable continuous parameters. This work builds on previous works by the authors and addresses QNN for image classification with Novel Enhanced Quantum Representation of (NEQR) processed classical data where Principal component analysis (PCA) and Projected Quantum Kernel features (PQK) were investigated previously by… ▽ More A quantum neural network (QNN) is interpreted today as any quantum circuit with trainable continuous parameters. This work builds on previous works by the authors and addresses QNN for image classification with Novel Enhanced Quantum Representation of (NEQR) processed classical data where Principal component analysis (PCA) and Projected Quantum Kernel features (PQK) were investigated previously by the authors as a path to quantum advantage for the same classical dataset. For each of these cases the Fashion-MNIST dataset was downscaled using PCA to convert into quantum data where the classical NN easily outperformed the QNN. However, we demonstrated quantum advantage by using PQK where quantum models achieved more than ~90% accuracy surpassing their classical counterpart on the same training dataset as in the first case. In this current work, we use the same dataset fed into a QNN and compare that with performance of a classical NN model. We built an NEQR model circuit to pre-process the same data and feed the images into the QNN. Our results showed marginal improvements (only about ~5.0%) where the QNN performance with NEQR exceeded the performance of QNN without NEQR. We conclude that given the computational cost and the massive circuit depth associated with running NEQR, the advantage offered by this specific Quantum Image Processing (QIMP) algorithm is questionable at least for classical image dataset. No actual quantum computing hardware platform exists today that can support the circuit depth needed to run NEQR even for the reduced image sizes of our toy classical dataset. △ Less

Submitted 29 March, 2022; originally announced April 2022.

Graph Neural Networks in Particle Physics: Implementations, Innovations, and Challenges

Authors: Savannah Thais, Paolo Calafiura, Grigorios Chachamis, Gage DeZoort, Javier Duarte, Sanmay Ganguly, Michael Kagan, Daniel Murnane, Mark S. Neubauer, Kazuhiro Terao

Abstract: Many physical systems can be best understood as sets of discrete data with associated relationships. Where previously these sets of data have been formulated as series or image data to match the available machine learning architectures, with the advent of graph neural networks (GNNs), these systems can be learned natively as graphs. This allows a wide variety of high- and low-level physical featur… ▽ More Many physical systems can be best understood as sets of discrete data with associated relationships. Where previously these sets of data have been formulated as series or image data to match the available machine learning architectures, with the advent of graph neural networks (GNNs), these systems can be learned natively as graphs. This allows a wide variety of high- and low-level physical features to be attached to measurements and, by the same token, a wide variety of HEP tasks to be accomplished by the same GNN architectures. GNNs have found powerful use-cases in reconstruction, tagging, generation and end-to-end analysis. With the wide-spread adoption of GNNs in industry, the HEP community is well-placed to benefit from rapid improvements in GNN latency and memory usage. However, industry use-cases are not perfectly aligned with HEP and much work needs to be done to best match unique GNN capabilities to unique HEP obstacles. We present here a range of these capabilities, predictions of which are currently being well-adopted in HEP communities, and which are still immature. We hope to capture the landscape of graph techniques in machine learning as well as point out the most significant gaps that are inhibiting potentially large leaps in research. △ Less

Submitted 25 March, 2022; v1 submitted 23 March, 2022; originally announced March 2022.

Comments: contribution to Snowmass 2021

Symmetry Group Equivariant Architectures for Physics

Authors: Alexander Bogatskiy, Sanmay Ganguly, Thomas Kipf, Risi Kondor, David W. Miller, Daniel Murnane, Jan T. Offermann, Mariel Pettee, Phiala Shanahan, Chase Shimmin, Savannah Thais

Abstract: Physical theories grounded in mathematical symmetries are an essential component of our understanding of a wide range of properties of the universe. Similarly, in the domain of machine learning, an awareness of symmetries such as rotation or permutation invariance has driven impressive performance breakthroughs in computer vision, natural language processing, and other important applications. In t… ▽ More Physical theories grounded in mathematical symmetries are an essential component of our understanding of a wide range of properties of the universe. Similarly, in the domain of machine learning, an awareness of symmetries such as rotation or permutation invariance has driven impressive performance breakthroughs in computer vision, natural language processing, and other important applications. In this report, we argue that both the physics community and the broader machine learning community have much to understand and potentially to gain from a deeper investment in research concerning symmetry group equivariant machine learning architectures. For some applications, the introduction of symmetries into the fundamental structural design can yield models that are more economical (i.e. contain fewer, but more expressive, learned parameters), interpretable (i.e. more explainable or directly mappable to physical quantities), and/or trainable (i.e. more efficient in both data and computational requirements). We discuss various figures of merit for evaluating these models as well as some potential benefits and limitations of these methods for a variety of physics applications. Research and investment into these approaches will lay the foundation for future architectures that are potentially more robust under new computational paradigms and will provide a richer description of the physical systems to which they are applied. △ Less

Submitted 11 March, 2022; originally announced March 2022.

Comments: Contribution to Snowmass 2021

Upper tail behavior of the number of triangles in random graphs with constant average degree

Authors: Shirshendu Ganguly, Ella Hiesmayr, Kyeongsik Nam

Abstract: Let $N$ be the number of triangles in an Erdős-Rényi graph $\mathcal{G}(n,p)$ on $n$ vertices with edge density $p=d/n,$ where $d>0$ is a fixed constant. It is well known that $N$ weakly converges to the Poisson distribution with mean ${d^3}/{6}$ as $n\rightarrow \infty$. We address the upper tail problem for $N,$ namely, we investigate how fast $k$ must grow, so that the probability of… ▽ More Let $N$ be the number of triangles in an Erdős-Rényi graph $\mathcal{G}(n,p)$ on $n$ vertices with edge density $p=d/n,$ where $d>0$ is a fixed constant. It is well known that $N$ weakly converges to the Poisson distribution with mean ${d^3}/{6}$ as $n\rightarrow \infty$. We address the upper tail problem for $N,$ namely, we investigate how fast $k$ must grow, so that the probability of $\{N\ge k\}$ is not well approximated anymore by the tail of the corresponding Poisson variable. Proving that the tail exhibits a sharp phase transition, we essentially show that the upper tail is governed by Poisson behavior only when $k^{1/3} \log k< (\frac{3}{\sqrt{2}})^{2/3} \log n$ (sub-critical regime) as well as pin down the tail behavior when $k^{1/3} \log k> (\frac{3}{\sqrt{2}})^{2/3} \log n$ (super-critical regime). We further prove a structure theorem, showing that the sub-critical upper tail behavior is dictated by the appearance of almost $k$ vertex-disjoint triangles whereas in the supercritical regime, the excess triangles arise from a clique like structure of size approximately $(6k)^{1/3}$. This settles the long-standing upper-tail problem in this case, answering a question of Aldous, complementing a long sequence of works, spanning multiple decades, culminating in (Harel, Moussat, Samotij,'19) which analyzed the problem only in the regime $p\gg \frac{1}{n}.$ The proofs rely on several novel graph theoretical results which could have other applications. △ Less

Submitted 14 February, 2022; originally announced February 2022.

Comments: 32 pages, 2 figures

SparseAlign: A Super-Resolution Algorithm for Automatic Marker Localization and Deformation Estimation in Cryo-Electron Tomography

Authors: Poulami Somanya Ganguly, Felix Lucka, Holger Kohr, Erik Franken, Hermen Jan Hupkes, K Joost Batenburg

Abstract: Tilt-series alignment is crucial to obtaining high-resolution reconstructions in cryo-electron tomography. Beam-induced local deformation of the sample is hard to estimate from the low-contrast sample alone, and often requires fiducial gold bead markers. The state-of-the-art approach for deformation estimation uses (semi-)manually labelled marker locations in projection data to fit the parameters… ▽ More Tilt-series alignment is crucial to obtaining high-resolution reconstructions in cryo-electron tomography. Beam-induced local deformation of the sample is hard to estimate from the low-contrast sample alone, and often requires fiducial gold bead markers. The state-of-the-art approach for deformation estimation uses (semi-)manually labelled marker locations in projection data to fit the parameters of a polynomial deformation model. Manually-labelled marker locations are difficult to obtain when data are noisy or markers overlap in projection data. We propose an alternative mathematical approach for simultaneous marker localization and deformation estimation by extending a grid-free super-resolution algorithm first proposed in the context of single-molecule localization microscopy. Our approach does not require labelled marker locations; instead, we use an image-based loss where we compare the forward projection of markers with the observed data. We equip this marker localization scheme with an additional deformation estimation component and solve for a reduced number of deformation parameters. Using extensive numerical studies on marker-only samples, we show that our approach automatically finds markers and reliably estimates sample deformation without labelled marker data. We further demonstrate the applicability of our approach for a broad range of model mismatch scenarios, including experimental electron tomography data of gold markers on ice. △ Less

Submitted 21 January, 2022; originally announced January 2022.

MSC Class: 65K10; 65M32

PeopleSansPeople: A Synthetic Data Generator for Human-Centric Computer Vision

Authors: Salehe Erfanian Ebadi, You-Cyuan Jhang, Alex Zook, Saurav Dhakad, Adam Crespi, Pete Parisi, Steven Borkman, Jonathan Hogins, Sujoy Ganguly

Abstract: In recent years, person detection and human pose estimation have made great strides, helped by large-scale labeled datasets. However, these datasets had no guarantees or analysis of human activities, poses, or context diversity. Additionally, privacy, legal, safety, and ethical concerns may limit the ability to collect more human data. An emerging alternative to real-world data that alleviates som… ▽ More In recent years, person detection and human pose estimation have made great strides, helped by large-scale labeled datasets. However, these datasets had no guarantees or analysis of human activities, poses, or context diversity. Additionally, privacy, legal, safety, and ethical concerns may limit the ability to collect more human data. An emerging alternative to real-world data that alleviates some of these issues is synthetic data. However, creation of synthetic data generators is incredibly challenging and prevents researchers from exploring their usefulness. Therefore, we release a human-centric synthetic data generator PeopleSansPeople which contains simulation-ready 3D human assets, a parameterized lighting and camera system, and generates 2D and 3D bounding box, instance and semantic segmentation, and COCO pose labels. Using PeopleSansPeople, we performed benchmark synthetic data training using a Detectron2 Keypoint R-CNN variant [1]. We found that pre-training a network using synthetic data and fine-tuning on various sizes of real-world data resulted in a keypoint AP increase of $+38.03$ ($44.43 \pm 0.17$ vs. $6.40$) for few-shot transfer (limited subsets of COCO-person train [2]), and an increase of $+1.47$ ($63.47 \pm 0.19$ vs. $62.00$) for abundant real data regimes, outperforming models trained with the same real data alone. We also found that our models outperformed those pre-trained with ImageNet with a keypoint AP increase of $+22.53$ ($44.43 \pm 0.17$ vs. $21.90$) for few-shot transfer and $+1.07$ ($63.47 \pm 0.19$ vs. $62.40$) for abundant real data regimes. This freely-available data generator should enable a wide range of research into the emerging field of simulation to real transfer learning in the critical area of human-centric computer vision. △ Less

Submitted 11 July, 2022; v1 submitted 16 December, 2021; originally announced December 2021.

Comments: PeopleSansPeople template Unity environment, benchmark binaries, and source code is available at: https://github.com/Unity-Technologies/PeopleSansPeople

On the Use and Misuse of Absorbing States in Multi-agent Reinforcement Learning

Authors: Andrew Cohen, Ervin Teng, Vincent-Pierre Berges, Ruo-Ping Dong, Hunter Henry, Marwan Mattar, Alexander Zook, Sujoy Ganguly

Abstract: The creation and destruction of agents in cooperative multi-agent reinforcement learning (MARL) is a critically under-explored area of research. Current MARL algorithms often assume that the number of agents within a group remains fixed throughout an experiment. However, in many practical problems, an agent may terminate before their teammates. This early termination issue presents a challenge: th… ▽ More The creation and destruction of agents in cooperative multi-agent reinforcement learning (MARL) is a critically under-explored area of research. Current MARL algorithms often assume that the number of agents within a group remains fixed throughout an experiment. However, in many practical problems, an agent may terminate before their teammates. This early termination issue presents a challenge: the terminated agent must learn from the group's success or failure which occurs beyond its own existence. We refer to propagating value from rewards earned by remaining teammates to terminated agents as the Posthumous Credit Assignment problem. Current MARL methods handle this problem by placing these agents in an absorbing state until the entire group of agents reaches a termination condition. Although absorbing states enable existing algorithms and APIs to handle terminated agents without modification, practical training efficiency and resource use problems exist. In this work, we first demonstrate that sample complexity increases with the quantity of absorbing states in a toy supervised learning task for a fully connected network, while attention is more robust to variable size input. Then, we present a novel architecture for an existing state-of-the-art MARL algorithm which uses attention instead of a fully connected layer with absorbing states. Finally, we demonstrate that this novel architecture significantly outperforms the standard architecture on tasks in which agents are created or destroyed within episodes as well as standard multi-agent coordination tasks. △ Less

Submitted 6 June, 2022; v1 submitted 10 November, 2021; originally announced November 2021.

Comments: RL in Games Workshop AAAI 2022

Many nodal domains in random regular graphs

Authors: Shirshendu Ganguly, Theo McKenzie, Sidhanth Mohanty, Nikhil Srivastava

Abstract: Let $G$ be a random $d$-regular graph. We prove that for every constant $α> 0$, with high probability every eigenvector of the adjacency matrix of $G$ with eigenvalue less than $-2\sqrt{d-2}-α$ has $Ω(n/$polylog$(n))$ nodal domains. Let $G$ be a random $d$-regular graph. We prove that for every constant $α> 0$, with high probability every eigenvector of the adjacency matrix of $G$ with eigenvalue less than $-2\sqrt{d-2}-α$ has $Ω(n/$polylog$(n))$ nodal domains. △ Less

Submitted 24 October, 2021; v1 submitted 23 September, 2021; originally announced September 2021.

Comments: 18 pages. Minor changes to the introduction

MSC Class: 05C80; 60B20

Semi-supervised Dense Keypoints Using Unlabeled Multiview Images

Authors: Zhixuan Yu, Haozheng Yu, Long Sha, Sujoy Ganguly, Hyun Soo Park

Abstract: This paper presents a new end-to-end semi-supervised framework to learn a dense keypoint detector using unlabeled multiview images. A key challenge lies in finding the exact correspondences between the dense keypoints in multiple views since the inverse of the keypoint mapping can be neither analytically derived nor differentiated. This limits applying existing multiview supervision approaches use… ▽ More This paper presents a new end-to-end semi-supervised framework to learn a dense keypoint detector using unlabeled multiview images. A key challenge lies in finding the exact correspondences between the dense keypoints in multiple views since the inverse of the keypoint mapping can be neither analytically derived nor differentiated. This limits applying existing multiview supervision approaches used to learn sparse keypoints that rely on the exact correspondences. To address this challenge, we derive a new probabilistic epipolar constraint that encodes the two desired properties. (1) Soft correspondence: we define a matchability, which measures a likelihood of a point matching to the other image's corresponding point, thus relaxing the requirement of the exact correspondences. (2) Geometric consistency: every point in the continuous correspondence fields must satisfy the multiview consistency collectively. We formulate a probabilistic epipolar constraint using a weighted average of epipolar errors through the matchability thereby generalizing the point-to-point geometric error to the field-to-field geometric error. This generalization facilitates learning a geometrically coherent dense keypoint detection model by utilizing a large number of unlabeled multiview images. Additionally, to prevent degenerative cases, we employ a distillation-based regularization by using a pretrained model. Finally, we design a new neural network architecture, made of twin networks, that effectively minimizes the probabilistic epipolar errors of all possible correspondences between two view images by building affinity matrices. Our method shows superior performance compared to existing methods, including non-differentiable bootstrapping in terms of keypoint accuracy, multiview consistency, and 3D reconstruction accuracy. △ Less

Submitted 19 February, 2024; v1 submitted 20 September, 2021; originally announced September 2021.

Comments: Published as a conference paper at NeurIPS 2021

Unity Perception: Generate Synthetic Data for Computer Vision

Authors: Steve Borkman, Adam Crespi, Saurav Dhakad, Sujoy Ganguly, Jonathan Hogins, You-Cyuan Jhang, Mohsen Kamalzadeh, Bowen Li, Steven Leal, Pete Parisi, Cesar Romero, Wesley Smith, Alex Thaman, Samuel Warren, Nupur Yadav

Abstract: We introduce the Unity Perception package which aims to simplify and accelerate the process of generating synthetic datasets for computer vision tasks by offering an easy-to-use and highly customizable toolset. This open-source package extends the Unity Editor and engine components to generate perfectly annotated examples for several common computer vision tasks. Additionally, it offers an extensi… ▽ More We introduce the Unity Perception package which aims to simplify and accelerate the process of generating synthetic datasets for computer vision tasks by offering an easy-to-use and highly customizable toolset. This open-source package extends the Unity Editor and engine components to generate perfectly annotated examples for several common computer vision tasks. Additionally, it offers an extensible Randomization framework that lets the user quickly construct and configure randomized simulation parameters in order to introduce variation into the generated datasets. We provide an overview of the provided tools and how they work, and demonstrate the value of the generated synthetic datasets by training a 2D object detection model. The model trained with mostly synthetic data outperforms the model trained using only real data. △ Less

Submitted 19 July, 2021; v1 submitted 9 July, 2021; originally announced July 2021.

Comments: We corrected tasks supported by NVISII platform. For the Unity perception package, see https://github.com/Unity-Technologies/com.unity.perception

A Semantic Segmentation Network for Urban-Scale Building Footprint Extraction Using RGB Satellite Imagery

Authors: Aatif Jiwani, Shubhrakanti Ganguly, Chao Ding, Nan Zhou, David M. Chan

Abstract: Urban areas consume over two-thirds of the world's energy and account for more than 70 percent of global CO2 emissions. As stated in IPCC's Global Warming of 1.5C report, achieving carbon neutrality by 2050 requires a clear understanding of urban geometry. High-quality building footprint generation from satellite images can accelerate this predictive process and empower municipal decision-making a… ▽ More Urban areas consume over two-thirds of the world's energy and account for more than 70 percent of global CO2 emissions. As stated in IPCC's Global Warming of 1.5C report, achieving carbon neutrality by 2050 requires a clear understanding of urban geometry. High-quality building footprint generation from satellite images can accelerate this predictive process and empower municipal decision-making at scale. However, previous Deep Learning-based approaches face consequential issues such as scale invariance and defective footprints, partly due to ever-present class-wise imbalance. Additionally, most approaches require supplemental data such as point cloud data, building height information, and multi-band imagery - which has limited availability and are tedious to produce. In this paper, we propose a modified DeeplabV3+ module with a Dilated Res-Net backbone to generate masks of building footprints from three-channel RGB satellite imagery only. Furthermore, we introduce an F-Beta measure in our objective function to help the model account for skewed class distributions and prevent false-positive footprints. In addition to F-Beta, we incorporate an exponentially weighted boundary loss and use a cross-dataset training strategy to further increase the quality of predictions. As a result, we achieve state-of-the-art performances across three public benchmarks and demonstrate that our RGB-only method produces higher quality visual results and is agnostic to the scale, resolution, and urban density of satellite imagery. △ Less

Submitted 18 November, 2021; v1 submitted 2 April, 2021; originally announced April 2021.

Comments: 11 pages, 5 figures. Code available at https://github.com/aatifjiwani/rgb-footprint-extract/

Large deviations for the largest eigenvalue of Gaussian networks with constant average degree

Authors: Shirshendu Ganguly, Kyeongsik Nam

Abstract: Large deviation behavior of the largest eigenvalue $λ_1$ of Gaussian networks (Erdős-Rényi random graphs $\mathcal{G}_{n,p}$ with i.i.d. Gaussian weights on the edges) has been the topic of considerable interest. Recently in [6,30], a powerful approach was introduced based on tilting measures by suitable spherical integrals, particularly establishing a non-universal large deviation behavior for fi… ▽ More Large deviation behavior of the largest eigenvalue $λ_1$ of Gaussian networks (Erdős-Rényi random graphs $\mathcal{G}_{n,p}$ with i.i.d. Gaussian weights on the edges) has been the topic of considerable interest. Recently in [6,30], a powerful approach was introduced based on tilting measures by suitable spherical integrals, particularly establishing a non-universal large deviation behavior for fixed $p<1$ compared to the standard Gaussian ($p=1$) case. The case when $p\to 0$ was however completely left open with one expecting the dense behavior to hold only until the average degree is logarithmic in $n$. In this article we focus on the case of constant average degree i.e., $p=\frac{d}{n}$. We prove the following results towards a precise understanding of the large deviation behavior in this setting. 1. (Upper tail probabilities): For $δ>0,$ we pin down the exact exponent $ψ(δ)$ such that $$\mathbb{P}(λ_1\ge \sqrt{2(1+δ)\log n})=n^{-ψ(δ)+o(1)}.$$ Further, we show that conditioned on the upper tail event, with high probability, a unique maximal clique emerges with a very precise $δ$ dependent size (takes either one or two possible values) and the Gaussian weights are uniformly high in absolute value on the edges in the clique. Finally, we also prove an optimal localization result for the leading eigenvector, showing that it allocates most of its mass on the aforementioned clique which is spread uniformly across its vertices. 2. (Lower tail probabilities): The exact stretched exponential behavior of $\mathbb{P}(λ_1\le \sqrt{2(1-δ)\log n})$ is also established. As an immediate corollary, we get $λ_1 \approx \sqrt{2 \log n}$ typically, a result that surprisingly appears to be new. A key ingredient is an extremal spectral theory for weighted graphs obtained via the classical Motzkin-Straus theorem. △ Less

Submitted 16 February, 2021; originally announced February 2021.

Comments: 49 pages

Technology Readiness Levels for Machine Learning Systems

Authors: Alexander Lavin, Ciarán M. Gilligan-Lee, Alessya Visnjic, Siddha Ganju, Dava Newman, Atılım Güneş Baydin, Sujoy Ganguly, Danny Lange, Amit Sharma, Stephan Zheng, Eric P. Xing, Adam Gibson, James Parr, Chris Mattmann, Yarin Gal

Abstract: The development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. The lack of diligence can lead to technical debt, scope creep and misaligned objectives, model misuse and failures, and expensive consequences. Engineering systems, on the other hand, follow well-defined processes and testing standards t… ▽ More The development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. The lack of diligence can lead to technical debt, scope creep and misaligned objectives, model misuse and failures, and expensive consequences. Engineering systems, on the other hand, follow well-defined processes and testing standards to streamline development for high-quality, reliable results. The extreme is spacecraft systems, where mission critical measures and robustness are ingrained in the development process. Drawing on experience in both spacecraft engineering and ML (from research through product across domain areas), we have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" (MLTRL) framework defines a principled process to ensure robust, reliable, and responsible systems while being streamlined for ML workflows, including key distinctions from traditional software engineering. Even more, MLTRL defines a lingua franca for people across teams and organizations to work collaboratively on artificial intelligence and machine learning technologies. Here we describe the framework and elucidate it with several real world use-cases of developing ML methods from basic research through productization and deployment, in areas such as medical diagnostics, consumer computer vision, satellite imagery, and particle physics. △ Less

Submitted 29 November, 2021; v1 submitted 11 January, 2021; originally announced January 2021.

Parallel-beam X-ray CT datasets of apples with internal defects and label balancing for machine learning

Authors: Sophia Bethany Coban, Vladyslav Andriiashen, Poulami Somanya Ganguly, Maureen van Eijnatten, Kees Joost Batenburg

Abstract: We present three parallel-beam tomographic datasets of 94 apples with internal defects along with defect label files. The datasets are prepared for development and testing of data-driven, learning-based image reconstruction, segmentation and post-processing methods. The three versions are a noiseless simulation; simulation with added Gaussian noise, and with scattering noise. The datasets are base… ▽ More We present three parallel-beam tomographic datasets of 94 apples with internal defects along with defect label files. The datasets are prepared for development and testing of data-driven, learning-based image reconstruction, segmentation and post-processing methods. The three versions are a noiseless simulation; simulation with added Gaussian noise, and with scattering noise. The datasets are based on real 3D X-ray CT data and their subsequent volume reconstructions. The ground truth images, based on the volume reconstructions, are also available through this project. Apples contain various defects, which naturally introduce a label bias. We tackle this by formulating the bias as an optimization problem. In addition, we demonstrate solving this problem with two methods: a simple heuristic algorithm and through mixed integer quadratic programming. This ensures the datasets can be split into test, training or validation subsets with the label bias eliminated. Therefore the datasets can be used for image reconstruction, segmentation, automatic defect detection, and testing the effects of (as well as applying new methodologies for removing) label bias in machine learning. △ Less

Submitted 24 December, 2020; originally announced December 2020.

Comments: Data Descriptor, to be submitted, 21 pages, 12 figures

MSC Class: 68-11; 90-05; 90C90; 78A46 ACM Class: I.4.1; I.4.5; I.4.9; G.1.10

Building Reservoir Computing Hardware Using Low Energy-Barrier Magnetics

Authors: Samiran Ganguly, Avik W. Ghosh

Abstract: Biologically inspired recurrent neural networks, such as reservoir computers are of interest in designing spatio-temporal data processors from a hardware point of view due to the simple learning scheme and deep connections to Kalman filters. In this work we discuss using in-depth simulation studies a way to construct hardware reservoir computers using an analog stochastic neuron cell built from a… ▽ More Biologically inspired recurrent neural networks, such as reservoir computers are of interest in designing spatio-temporal data processors from a hardware point of view due to the simple learning scheme and deep connections to Kalman filters. In this work we discuss using in-depth simulation studies a way to construct hardware reservoir computers using an analog stochastic neuron cell built from a low energy-barrier magnet based magnetic tunnel junction and a few transistors. This allows us to implement a physical embodiment of the mathematical model of reservoir computers. Compact implementation of reservoir computers using such devices may enable building compact, energy-efficient signal processors for standalone or in-situ machine cognition in edge devices. △ Less

Submitted 6 July, 2020; originally announced July 2020.

Comments: To be presented at International Conference on Neuromorphic Systems 2020

Temporal Memory with Magnetic Racetracks

Authors: Hamed Vakili, Mohammad Nazmus Sakib, Samiran Ganguly, Mircea Stan, Matthew W. Daniels, Advait Madhavan, Mark D. Stiles, Avik W. Ghosh

Abstract: Race logic is a relative timing code that represents information in a wavefront of digital edges on a set of wires in order to accelerate dynamic programming and machine learning algorithms. Skyrmions, bubbles, and domain walls are mobile magnetic configurations (solitons) with applications for Boolean data storage. We propose to use current-induced displacement of these solitons on magnetic racet… ▽ More Race logic is a relative timing code that represents information in a wavefront of digital edges on a set of wires in order to accelerate dynamic programming and machine learning algorithms. Skyrmions, bubbles, and domain walls are mobile magnetic configurations (solitons) with applications for Boolean data storage. We propose to use current-induced displacement of these solitons on magnetic racetracks as a native temporal memory for race logic computing. Locally synchronized racetracks can spatially store relative timings of digital edges and provide non-destructive read-out. The linear kinematics of skyrmion motion, the tunability and low-voltage asynchronous operation of the proposed device, and the elimination of any need for constant skyrmion nucleation make these magnetic racetracks a natural memory for low-power, high-throughput race logic applications. △ Less

Submitted 21 May, 2020; originally announced May 2020.

Comments: 9 pages, 3 figures, submitted for review

An Empirical Study of Incremental Learning in Neural Network with Noisy Training Set

Authors: Shovik Ganguly, Atrayee Chatterjee, Debasmita Bhoumik, Ritajit Majumdar

Abstract: The notion of incremental learning is to train an ANN algorithm in stages, as and when newer training data arrives. Incremental learning is becoming widespread in recent times with the advent of deep learning. Noise in the training data reduces the accuracy of the algorithm. In this paper, we make an empirical study of the effect of noise in the training phase. We numerically show that the accurac… ▽ More The notion of incremental learning is to train an ANN algorithm in stages, as and when newer training data arrives. Incremental learning is becoming widespread in recent times with the advent of deep learning. Noise in the training data reduces the accuracy of the algorithm. In this paper, we make an empirical study of the effect of noise in the training phase. We numerically show that the accuracy of the algorithm is dependent more on the location of the error than the percentage of error. Using Perceptron, Feed Forward Neural Network and Radial Basis Function Neural Network, we show that for the same percentage of error, the accuracy of the algorithm significantly varies with the location of error. Furthermore, our results show that the dependence of the accuracy with the location of error is independent of the algorithm. However, the slope of the degradation curve decreases with more sophisticated algorithms △ Less

Submitted 7 May, 2020; originally announced May 2020.

Comments: Oral Presentation delivered at the 7th International Conference on Computers and Devices for Communication (CODEC) 2019. To appear in Lecture Notes in Networks and Systems (LNSS Springer)

Spectral Edge in Sparse Random Graphs: Upper and Lower Tail Large Deviations

Authors: Bhaswar B. Bhattacharya, Sohom Bhattacharya, Shirshendu Ganguly

Abstract: In this paper we consider the problem of estimating the joint upper and lower tail large deviations of the edge eigenvalues of an Erdős-Rényi random graph $\mathcal{G}_{n,p}$, in the regime of $p$ where the edge of the spectrum is no longer governed by global observables, such as the number of edges, but rather by localized statistics, such as high degree vertices. Going beyond the recent developm… ▽ More In this paper we consider the problem of estimating the joint upper and lower tail large deviations of the edge eigenvalues of an Erdős-Rényi random graph $\mathcal{G}_{n,p}$, in the regime of $p$ where the edge of the spectrum is no longer governed by global observables, such as the number of edges, but rather by localized statistics, such as high degree vertices. Going beyond the recent developments in mean-field approximations of related problems, this paper provides a comprehensive treatment of the large deviations of the spectral edge in this entire regime, which notably includes the well studied case of constant average degree. In particular, for $r \geq 1$ fixed, we pin down the asymptotic probability that the top $r$ eigenvalues are jointly greater/less than their typical values by multiplicative factors bigger/smaller than $1$, in the regime mentioned above. The proof for the upper tail relies on a novel structure theorem, obtained by building on estimates of Krivelevich and Sudakov (2003), followed by an iterative cycle removal process, which shows, conditional on the upper tail large deviation event, with high probability the graph admits a decomposition in to a disjoint union of stars and a spectrally negligible part. On the other hand, the key ingredient in the proof of the lower tail is a Ramsey-type result which shows that if the $K$-th largest degree of a graph is not atypically small (for some large $K$ depending on $r$), then either the top eigenvalue or the $r$-th largest eigenvalue is larger than that allowed by the lower tail event on the top $r$ eigenvalues, thus forcing a contradiction. The above arguments reduce the problems to developing a large deviation theory for the extremal degrees which could be of independent interest. △ Less

Submitted 1 April, 2020; originally announced April 2020.

Comments: 36 pages, 1 figure

On the Capacity Regions of Cloud Radio Access Networks with Limited Orthogonal Fronthaul

Authors: Shouvik Ganguly, Seung-Eun Hong, Young-Han Kim

Abstract: Uplink and downlink cloud radio access networks are modeled as two-hop K-user L-relay networks, whereby small base-stations act as relays for end-to-end communications and are connected to a central processor via orthogonal fronthaul links of finite capacities. Simplified versions of network compress-forward (or noisy network coding) and distributed decode-forward are presented to establish inner… ▽ More Uplink and downlink cloud radio access networks are modeled as two-hop K-user L-relay networks, whereby small base-stations act as relays for end-to-end communications and are connected to a central processor via orthogonal fronthaul links of finite capacities. Simplified versions of network compress-forward (or noisy network coding) and distributed decode-forward are presented to establish inner bounds on the capacity region for uplink and downlink communications, that match the respective cutset bounds to within a finite gap independent of the channel gains and signal to noise ratios. These approximate capacity regions are then compared with the capacity regions for networks with no capacity limit on the fronthaul. Although it takes infinite fronthaul link capacities to achieve these "fronthaul-unlimited" capacity regions exactly, these capacity regions can be approached approximately with finite-capacity fronthaul. The total fronthaul link capacities required to approach the fronthaul-unlimited sum-rates (for uplink and downlink) are characterized. Based on these results, the capacity scaling law in the large network size limit is established under certain uplink and downlink network models, both theoretically and via simulations. △ Less

Submitted 9 December, 2019; originally announced December 2019.

DeepSat V2: Feature Augmented Convolutional Neural Nets for Satellite Image Classification

Authors: Qun Liu, Saikat Basu, Sangram Ganguly, Supratik Mukhopadhyay, Robert DiBiano, Manohar Karki, Ramakrishna Nemani

Abstract: Satellite image classification is a challenging problem that lies at the crossroads of remote sensing, computer vision, and machine learning. Due to the high variability inherent in satellite data, most of the current object classification approaches are not suitable for handling satellite datasets. The progress of satellite image analytics has also been inhibited by the lack of a single labeled h… ▽ More Satellite image classification is a challenging problem that lies at the crossroads of remote sensing, computer vision, and machine learning. Due to the high variability inherent in satellite data, most of the current object classification approaches are not suitable for handling satellite datasets. The progress of satellite image analytics has also been inhibited by the lack of a single labeled high-resolution dataset with multiple class labels. In a preliminary version of this work, we introduced two new high resolution satellite imagery datasets (SAT-4 and SAT-6) and proposed DeepSat framework for classification based on "handcrafted" features and a deep belief network (DBN). The present paper is an extended version, we present an end-to-end framework leveraging an improved architecture that augments a convolutional neural network (CNN) with handcrafted features (instead of using DBN-based architecture) for classification. Our framework, having access to fused spatial information obtained from handcrafted features as well as CNN feature maps, have achieved accuracies of 99.90% and 99.84% respectively, on SAT-4 and SAT-6, surpassing all the other state-of-the-art results. A statistical analysis based on Distribution Separability Criterion substantiates the robustness of our approach in learning better representations for satellite imagery. △ Less

Submitted 14 November, 2019; originally announced November 2019.

Comments: This is an Accepted Manuscript of an article published by Taylor & Francis Group in Remote Sensing Letters. arXiv admin note: text overlap with arXiv:1509.03602

High-girth near-Ramanujan graphs with localized eigenvectors

Authors: Noga Alon, Shirshendu Ganguly, Nikhil Srivastava

Abstract: We show that for every prime $d$ and $α\in (0,1/6)$, there is an infinite sequence of $(d+1)$-regular graphs $G=(V,E)$ with girth at least $2α\log_{d}(|V|)(1-o_d(1))$, second adjacency matrix eigenvalue bounded by $(3/\sqrt{2})\sqrt{d}$, and many eigenvectors fully localized on small sets of size $O(|V|^α)$. This strengthens the results of Ganguly-Srivastava, who constructed high girth (but not ex… ▽ More We show that for every prime $d$ and $α\in (0,1/6)$, there is an infinite sequence of $(d+1)$-regular graphs $G=(V,E)$ with girth at least $2α\log_{d}(|V|)(1-o_d(1))$, second adjacency matrix eigenvalue bounded by $(3/\sqrt{2})\sqrt{d}$, and many eigenvectors fully localized on small sets of size $O(|V|^α)$. This strengthens the results of Ganguly-Srivastava, who constructed high girth (but not expanding) graphs with similar properties, and may be viewed as a discrete analogue of the "scarring" phenomenon observed in the study of quantum ergodicity on manifolds. Key ingredients in the proof are a technique of Kahale for bounding the growth rate of eigenfunctions of graphs, discovered in the context of vertex expansion and a method of Erdős and Sachs for constructing high girth regular graphs. △ Less

Submitted 10 August, 2019; originally announced August 2019.

Progressively Growing Generative Adversarial Networks for High Resolution Semantic Segmentation of Satellite Images

Authors: Edward Collier, Kate Duffy, Sangram Ganguly, Geri Madanguit, Subodh Kalia, Gayaka Shreekant, Ramakrishna Nemani, Andrew Michaelis, Shuang Li, Auroop Ganguly, Supratik Mukhopadhyay

Abstract: Machine learning has proven to be useful in classification and segmentation of images. In this paper, we evaluate a training methodology for pixel-wise segmentation on high resolution satellite images using progressive growing of generative adversarial networks. We apply our model to segmenting building rooftops and compare these results to conventional methods for rooftop segmentation. We present… ▽ More Machine learning has proven to be useful in classification and segmentation of images. In this paper, we evaluate a training methodology for pixel-wise segmentation on high resolution satellite images using progressive growing of generative adversarial networks. We apply our model to segmenting building rooftops and compare these results to conventional methods for rooftop segmentation. We present our findings using the SpaceNet version 2 dataset. Progressive GAN training achieved a test accuracy of 93% compared to 89% for traditional GAN training. △ Less

Submitted 12 February, 2019; originally announced February 2019.

Comments: Accepted too and presented at DMESS 2018 as part of IEEE ICDM 2018

Analog Signal Processing Using Stochastic Magnets

Authors: Samiran Ganguly, Kerem Y. Camsari, Avik W. Ghosh

Abstract: We present a low barrier magnet based compact hardware unit for analog stochastic neurons and demonstrate its use as a building-block for neuromorphic hardware. By coupling circular magnetic tunnel junctions (MTJs) with a CMOS based analog buffer, we show that these units can act as leaky-integrate-and fire (LIF) neurons, a model of biological neural networks particularly suited for temporal infer… ▽ More We present a low barrier magnet based compact hardware unit for analog stochastic neurons and demonstrate its use as a building-block for neuromorphic hardware. By coupling circular magnetic tunnel junctions (MTJs) with a CMOS based analog buffer, we show that these units can act as leaky-integrate-and fire (LIF) neurons, a model of biological neural networks particularly suited for temporal inferencing and pattern recognition. We demonstrate examples of temporal sequence learning, processing, and prediction tasks in real time, as a proof of concept demonstration of scalable and adaptive signal-processors. Efficient non von-Neumann hardware implementation of such processors can open up a pathway for integration of hardware based cognition in a wide variety of emerging systems such as IoT, industrial controls, bio- and photo-sensors, and Unmanned Autonomous Vehicles. △ Less

Submitted 19 December, 2018; originally announced December 2018.

Comments: 4 pages, 4 figures, under review

Journal ref: IEEE Access (2021)

Reservoir Computing based Neural Image Filters

Authors: Samiran Ganguly, Yunfei Gu, Yunkun Xie, Mircea R. Stan, Avik W. Ghosh, Nibir K. Dhar

Abstract: Clean images are an important requirement for machine vision systems to recognize visual features correctly. However, the environment, optics, electronics of the physical imaging systems can introduce extreme distortions and noise in the acquired images. In this work, we explore the use of reservoir computing, a dynamical neural network model inspired from biological systems, in creating dynamic i… ▽ More Clean images are an important requirement for machine vision systems to recognize visual features correctly. However, the environment, optics, electronics of the physical imaging systems can introduce extreme distortions and noise in the acquired images. In this work, we explore the use of reservoir computing, a dynamical neural network model inspired from biological systems, in creating dynamic image filtering systems that extracts signal from noise using inverse modeling. We discuss the possibility of implementing these networks in hardware close to the sensors. △ Less

Submitted 7 September, 2018; originally announced September 2018.

Comments: 5 pages, 4 figures, To appear in Conference Proceedings of The 44th Annual Conference of IEEE Industrial Electronics Society (2018): Special Session on Machine Vision, Control and Navigation

High Probability Frequency Moment Sketches

Authors: Sumit Ganguly, David P. Woodruff

Abstract: We consider the problem of sketching the $p$-th frequency moment of a vector, $p>2$, with multiplicative error at most $1\pm ε$ and \emph{with high confidence} $1-δ$. Despite the long sequence of work on this problem, tight bounds on this quantity are only known for constant $δ$. While one can obtain an upper bound with error probability $δ$ by repeating a sketching algorithm with constant error p… ▽ More We consider the problem of sketching the $p$-th frequency moment of a vector, $p>2$, with multiplicative error at most $1\pm ε$ and \emph{with high confidence} $1-δ$. Despite the long sequence of work on this problem, tight bounds on this quantity are only known for constant $δ$. While one can obtain an upper bound with error probability $δ$ by repeating a sketching algorithm with constant error probability $O(\log(1/δ))$ times in parallel, and taking the median of the outputs, we show this is a suboptimal algorithm! Namely, we show optimal upper and lower bounds of $Θ(n^{1-2/p} \log(1/δ) + n^{1-2/p} \log^{2/p} (1/δ) \log n)$ on the sketching dimension, for any constant approximation. Our result should be contrasted with results for estimating frequency moments for $1 \leq p \leq 2$, for which we show the optimal algorithm for general $δ$ is obtained by repeating the optimal algorithm for constant error probability $O(\log(1/δ))$ times and taking the median output. We also obtain a matching lower bound for this problem, up to constant factors. △ Less

Submitted 28 May, 2018; originally announced May 2018.

Comments: Extended Abstract to appear in Proceedings of ICALP 2018

Nearest neighbor density functional estimation from inverse Laplace transform

Authors: J. Jon Ryu, Shouvik Ganguly, Young-Han Kim, Yung-Kyun Noh, Daniel D. Lee

Abstract: A new approach to $L_2$-consistent estimation of a general density functional using $k$-nearest neighbor distances is proposed, where the functional under consideration is in the form of the expectation of some function $f$ of the densities at each point. The estimator is designed to be asymptotically unbiased, using the convergence of the normalized volume of a $k$-nearest neighbor ball to a Gamm… ▽ More A new approach to $L_2$-consistent estimation of a general density functional using $k$-nearest neighbor distances is proposed, where the functional under consideration is in the form of the expectation of some function $f$ of the densities at each point. The estimator is designed to be asymptotically unbiased, using the convergence of the normalized volume of a $k$-nearest neighbor ball to a Gamma distribution in the large-sample limit, and naturally involves the inverse Laplace transform of a scaled version of the function $f.$ Some instantiations of the proposed estimator recover existing $k$-nearest neighbor based estimators of Shannon and Rényi entropies and Kullback--Leibler and Rényi divergences, and discover new consistent estimators for many other functionals such as logarithmic entropies and divergences. The $L_2$-consistency of the proposed estimator is established for a broad class of densities for general functionals, and the convergence rate in mean squared error is established as a function of the sample size for smooth, bounded densities. △ Less

Submitted 4 February, 2022; v1 submitted 21 May, 2018; originally announced May 2018.

Comments: 43 pages, 4 figures. IEEE Transactions on Information Theory (to appear)

Hardware based Spatio-Temporal Neural Processing Backend for Imaging Sensors: Towards a Smart Camera

Authors: Samiran Ganguly, Yunfei Gu, Mircea R. Stan, Avik W. Ghosh

Abstract: In this work we show how we can build a technology platform for cognitive imaging sensors using recent advances in recurrent neural network architectures and training methods inspired from biology. We demonstrate learning and processing tasks specific to imaging sensors, including enhancement of sensitivity and signal-to-noise ratio (SNR) purely through neural filtering beyond the fundamental limi… ▽ More In this work we show how we can build a technology platform for cognitive imaging sensors using recent advances in recurrent neural network architectures and training methods inspired from biology. We demonstrate learning and processing tasks specific to imaging sensors, including enhancement of sensitivity and signal-to-noise ratio (SNR) purely through neural filtering beyond the fundamental limits sensor materials, and inferencing and spatio-temporal pattern recognition capabilities of these networks with applications in object detection, motion tracking and prediction. We then show designs of unit hardware cells built using complementary metal-oxide semiconductor (CMOS) and emerging materials technologies for ultra-compact and energy-efficient embedded neural processors for smart cameras. △ Less

Submitted 22 March, 2018; originally announced March 2018.

Comments: 11 pages, 5 figures. To be presented in SPIE DCS 2018: Image Sensing Technologies: Materials, Devices, Systems, and Applications V