Showing 1–50 of 1,153 results for author: Chang, S

LLMs generate structurally realistic social networks but overestimate political homophily

Authors: Serina Chang, Alicja Chaszczewicz, Emma Wang, Maya Josifovska, Emma Pierson, Jure Leskovec

Abstract: Generating social networks is essential for many applications, such as epidemic modeling and social simulations. Prior approaches either involve deep learning models, which require many observed networks for training, or stylized models, which are limited in their realism and flexibility. In contrast, LLMs offer the potential for zero-shot and flexible network generation. However, two key question… ▽ More Generating social networks is essential for many applications, such as epidemic modeling and social simulations. Prior approaches either involve deep learning models, which require many observed networks for training, or stylized models, which are limited in their realism and flexibility. In contrast, LLMs offer the potential for zero-shot and flexible network generation. However, two key questions are: (1) are LLM's generated networks realistic, and (2) what are risks of bias, given the importance of demographics in forming social ties? To answer these questions, we develop three prompting methods for network generation and compare the generated networks to real social networks. We find that more realistic networks are generated with "local" methods, where the LLM constructs relations for one persona at a time, compared to "global" methods that construct the entire network at once. We also find that the generated networks match real networks on many characteristics, including density, clustering, community structure, and degree. However, we find that LLMs emphasize political homophily over all other types of homophily and overestimate political homophily relative to real-world measures. △ Less

Submitted 29 August, 2024; originally announced August 2024.

Toward Robust Early Detection of Alzheimer's Disease via an Integrated Multimodal Learning Approach

Authors: Yifei Chen, Shenghao Zhu, Zhaojie Fang, Chang Liu, Binfeng Zou, Yuhe Wang, Shuo Chang, Fan Jia, Feiwei Qin, Jin Fan, Yong Peng, Changmiao Wang

Abstract: Alzheimer's Disease (AD) is a complex neurodegenerative disorder marked by memory loss, executive dysfunction, and personality changes. Early diagnosis is challenging due to subtle symptoms and varied presentations, often leading to misdiagnosis with traditional unimodal diagnostic methods due to their limited scope. This study introduces an advanced multimodal classification model that integrates… ▽ More Alzheimer's Disease (AD) is a complex neurodegenerative disorder marked by memory loss, executive dysfunction, and personality changes. Early diagnosis is challenging due to subtle symptoms and varied presentations, often leading to misdiagnosis with traditional unimodal diagnostic methods due to their limited scope. This study introduces an advanced multimodal classification model that integrates clinical, cognitive, neuroimaging, and EEG data to enhance diagnostic accuracy. The model incorporates a feature tagger with a tabular data coding architecture and utilizes the TimesBlock module to capture intricate temporal patterns in Electroencephalograms (EEG) data. By employing Cross-modal Attention Aggregation module, the model effectively fuses Magnetic Resonance Imaging (MRI) spatial information with EEG temporal data, significantly improving the distinction between AD, Mild Cognitive Impairment, and Normal Cognition. Simultaneously, we have constructed the first AD classification dataset that includes three modalities: EEG, MRI, and tabular data. Our innovative approach aims to facilitate early diagnosis and intervention, potentially slowing the progression of AD. The source code and our private ADMC dataset are available at https://github.com/JustlfC03/MSTNet. △ Less

Submitted 29 August, 2024; originally announced August 2024.

Comments: 5 pages, 2 figures

Chernoff Bounds for Tensor Expanders on Riemannian Manifolds Using Graph Laplacian Approximation

Authors: Shih-Yu Chang

Abstract: This paper addresses the advancement of probability tail bound analysis, a crucial statistical tool for assessing the probability of large deviations of random variables from their expected values. Traditional tail bounds, such as Markov's, Chebyshev's, and Chernoff bounds, have proven valuable across numerous scientific and engineering fields. However, as data complexity grows, there is a pressin… ▽ More This paper addresses the advancement of probability tail bound analysis, a crucial statistical tool for assessing the probability of large deviations of random variables from their expected values. Traditional tail bounds, such as Markov's, Chebyshev's, and Chernoff bounds, have proven valuable across numerous scientific and engineering fields. However, as data complexity grows, there is a pressing need to extend tail bound estimation from scalar variables to high-dimensional random objects. Existing studies often rely on the assumption of independence among high-dimensional random objects, an assumption that may not always be valid. Building on the work of researchers like Garg et al. and Chang, who employed random walks to model high-dimensional ensembles, this study introduces a more generalized approach by exploring random walks over manifolds. To address the challenges of constructing an appropriate underlying graph for a manifold, we propose a novel method that enhances random walks on graphs approximating the manifold. This approach ensures spectral similarity between the original manifold and the approximated graph, including matching eigenvalues, eigenvectors, and eigenfunctions. Leveraging graph approximation technique proposed by Burago et al. for manifolds, we derive the tensor Chernoff bound and establish its range for random walks on a Riemannian manifold according to the underlying manifold's spectral characteristics. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Imaging ferroelectric domains with soft X-ray ptychography at the oxygen K-edge

Authors: Tim A. Butcher, Nicholas W. Phillips, Chia-Chun Wei, Shih-Chao Chang, Igor Beinik, Karina Thånell, Jan-Chi Yang, Shih-Wen Huang, Jörg Raabe, Simone Finizio

Abstract: The ferroelectric domain structure of a freestanding BiFeO$_3$ film was visualized by ptychographic dichroic imaging with linearly polarized X-rays at the O K-edge around 530 eV. The dichroic contrast is maximized at the energy of the hybridization of the O 2p state and the Fe 3d orbitals, which is split by the octahedral crystal field of the perovskite structure. The thus obtained microscopy imag… ▽ More The ferroelectric domain structure of a freestanding BiFeO$_3$ film was visualized by ptychographic dichroic imaging with linearly polarized X-rays at the O K-edge around 530 eV. The dichroic contrast is maximized at the energy of the hybridization of the O 2p state and the Fe 3d orbitals, which is split by the octahedral crystal field of the perovskite structure. The thus obtained microscopy images compliment the ptychographic imaging of the antiferromagnetic contribution at the Fe L$_3$-edge. The approach is extendible to the separation of different ferroic contributions in other multiferroic oxides. △ Less

Submitted 18 August, 2024; originally announced August 2024.

Enhancing Exploratory Learning through Exploratory Search with the Emergence of Large Language Models

Authors: Yiming Luo, Patrick Cheong-Iao, Shanton Chang

Abstract: In the information era, how learners find, evaluate, and effectively use information has become a challenging issue, especially with the added complexity of large language models (LLMs) that have further confused learners in their information retrieval and search activities. This study attempts to unpack this complexity by combining exploratory search strategies with the theories of exploratory le… ▽ More In the information era, how learners find, evaluate, and effectively use information has become a challenging issue, especially with the added complexity of large language models (LLMs) that have further confused learners in their information retrieval and search activities. This study attempts to unpack this complexity by combining exploratory search strategies with the theories of exploratory learning to form a new theoretical model of exploratory learning from the perspective of students' learning. Our work adapts Kolb's learning model by incorporating high-frequency exploration and feedback loops, aiming to promote deep cognitive and higher-order cognitive skill development in students. Additionally, this paper discusses and suggests how advanced LLMs integrated into information retrieval and information theory can support students in their exploratory searches, contributing theoretically to promoting student-computer interaction and supporting their learning journeys in the new era with LLMs. △ Less

Submitted 9 August, 2024; originally announced August 2024.

Comments: 11 pages, 7 figures

RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation

Authors: Dongyu Ru, Lin Qiu, Xiangkun Hu, Tianhang Zhang, Peng Shi, Shuaichen Chang, Cheng Jiayang, Cunxiang Wang, Shichao Sun, Huanyu Li, Zizhao Zhang, Binjie Wang, Jiarong Jiang, Tong He, Zhiguo Wang, Pengfei Liu, Yue Zhang, Zheng Zhang

Abstract: Despite Retrieval-Augmented Generation (RAG) showing promising capability in leveraging external knowledge, a comprehensive evaluation of RAG systems is still challenging due to the modular nature of RAG, evaluation of long-form responses and reliability of measurements. In this paper, we propose a fine-grained evaluation framework, RAGChecker, that incorporates a suite of diagnostic metrics for b… ▽ More Despite Retrieval-Augmented Generation (RAG) showing promising capability in leveraging external knowledge, a comprehensive evaluation of RAG systems is still challenging due to the modular nature of RAG, evaluation of long-form responses and reliability of measurements. In this paper, we propose a fine-grained evaluation framework, RAGChecker, that incorporates a suite of diagnostic metrics for both the retrieval and generation modules. Meta evaluation verifies that RAGChecker has significantly better correlations with human judgments than other evaluation metrics. Using RAGChecker, we evaluate 8 RAG systems and conduct an in-depth analysis of their performance, revealing insightful patterns and trade-offs in the design choices of RAG architectures. The metrics of RAGChecker can guide researchers and practitioners in developing more effective RAG systems. This work has been open sourced at https://github.com/amazon-science/RAGChecker. △ Less

Submitted 16 August, 2024; v1 submitted 15 August, 2024; originally announced August 2024.

Comments: Under Review. Github Repo: https://github.com/amazon-science/RAGChecker

Imagen 3

Authors: Imagen-Team-Google, :, Jason Baldridge, Jakob Bauer, Mukul Bhutani, Nicole Brichtova, Andrew Bunner, Kelvin Chan, Yichang Chen, Sander Dieleman, Yuqing Du, Zach Eaton-Rosen, Hongliang Fei, Nando de Freitas, Yilin Gao, Evgeny Gladchenko, Sergio Gómez Colmenarejo, Mandy Guo, Alex Haig, Will Hawkins, Hexiang Hu, Huilian Huang, Tobenna Peter Igwe, Christos Kaplanis, Siavash Khodadadeh , et al. (227 additional authors not shown)

Abstract: We introduce Imagen 3, a latent diffusion model that generates high quality images from text prompts. We describe our quality and responsibility evaluations. Imagen 3 is preferred over other state-of-the-art (SOTA) models at the time of evaluation. In addition, we discuss issues around safety and representation, as well as methods we used to minimize the potential harm of our models. We introduce Imagen 3, a latent diffusion model that generates high quality images from text prompts. We describe our quality and responsibility evaluations. Imagen 3 is preferred over other state-of-the-art (SOTA) models at the time of evaluation. In addition, we discuss issues around safety and representation, as well as methods we used to minimize the potential harm of our models. △ Less

Submitted 13 August, 2024; originally announced August 2024.

Electrically-Driven Two-Dimensional Semiconductor Microcavity Laser

Authors: Zheng-Zhe Chen, Hsiang-Ting Lin, Chiao-Yun Chang, Adil Muhammad, Po-Cheng Tsai, Tsung Sheng Kao, Chi Chen, Shu-Wei Chang, Shih-Yen Lin, Min-Hsiung Shih

Abstract: Two-dimensional (2-D) monolayer transition-metal dichalcogenides (TMDCs) are promising materials for realizing ultracompact, low-threshold semiconductor lasers. And the development of the electrical-driven TMDC devices is crucial for enhancing the integration potential of practical optoelectronic systems. However, at current stage, the electrically-driven 2-D TMDC laser has never been realized. He… ▽ More Two-dimensional (2-D) monolayer transition-metal dichalcogenides (TMDCs) are promising materials for realizing ultracompact, low-threshold semiconductor lasers. And the development of the electrical-driven TMDC devices is crucial for enhancing the integration potential of practical optoelectronic systems. However, at current stage, the electrically-driven 2-D TMDC laser has never been realized. Herein, we have developed the first electrically-driven 2-D TMDC microcavity laser. △ Less

Submitted 13 August, 2024; originally announced August 2024.

RT-Surv: Improving Mortality Prediction After Radiotherapy with Large Language Model Structuring of Large-Scale Unstructured Electronic Health Records

Authors: Sangjoon Park, Chan Woo Wee, Seo Hee Choi, Kyung Hwan Kim, Jee Suk Chang, Hong In Yoon, Ik Jae Lee, Yong Bae Kim, Jaeho Cho, Ki Chang Keum, Chang Geol Lee, Hwa Kyung Byun, Woong Sub Koom

Abstract: Accurate patient selection is critical in radiotherapy (RT) to prevent ineffective treatments. Traditional survival prediction models, relying on structured data, often lack precision. This study explores the potential of large language models (LLMs) to structure unstructured electronic health record (EHR) data, thereby improving survival prediction accuracy through comprehensive clinical informat… ▽ More Accurate patient selection is critical in radiotherapy (RT) to prevent ineffective treatments. Traditional survival prediction models, relying on structured data, often lack precision. This study explores the potential of large language models (LLMs) to structure unstructured electronic health record (EHR) data, thereby improving survival prediction accuracy through comprehensive clinical information integration. Data from 34,276 patients treated with RT at Yonsei Cancer Center between 2013 and 2023 were analyzed, encompassing both structured and unstructured data. An open-source LLM was used to structure the unstructured EHR data via single-shot learning, with its performance compared against a domain-specific medical LLM and a smaller variant. Survival prediction models were developed using statistical, machine learning, and deep learning approaches, incorporating both structured and LLM-structured data. Clinical experts evaluated the accuracy of the LLM-structured data. The open-source LLM achieved 87.5% accuracy in structuring unstructured EHR data without additional training, significantly outperforming the domain-specific medical LLM, which reached only 35.8% accuracy. Larger LLMs were more effective, particularly in extracting clinically relevant features like general condition and disease extent, which closely correlated with patient survival. Incorporating LLM-structured clinical features into survival prediction models significantly improved accuracy, with the C-index of deep learning models increasing from 0.737 to 0.820. These models also became more interpretable by emphasizing clinically significant factors. This study shows that general-domain LLMs, even without specific medical training, can effectively structure large-scale unstructured EHR data, substantially enhancing the accuracy and interpretability of clinical predictive models. △ Less

Submitted 16 August, 2024; v1 submitted 9 August, 2024; originally announced August 2024.

Comments: 23 pages, 2 tables, 4 figures

Digital Avatars: Framework Development and Their Evaluation

Authors: Timothy Rupprecht, Sung-En Chang, Yushu Wu, Lei Lu, Enfu Nan, Chih-hsiang Li, Caiyue Lai, Zhimin Li, Zhijun Hu, Yumei He, David Kaeli, Yanzhi Wang

Abstract: We present a novel prompting strategy for artificial intelligence driven digital avatars. To better quantify how our prompting strategy affects anthropomorphic features like humor, authenticity, and favorability we present Crowd Vote - an adaptation of Crowd Score that allows for judges to elect a large language model (LLM) candidate over competitors answering the same or similar prompts. To visua… ▽ More We present a novel prompting strategy for artificial intelligence driven digital avatars. To better quantify how our prompting strategy affects anthropomorphic features like humor, authenticity, and favorability we present Crowd Vote - an adaptation of Crowd Score that allows for judges to elect a large language model (LLM) candidate over competitors answering the same or similar prompts. To visualize the responses of our LLM, and the effectiveness of our prompting strategy we propose an end-to-end framework for creating high-fidelity artificial intelligence (AI) driven digital avatars. This pipeline effectively captures an individual's essence for interaction and our streaming algorithm delivers a high-quality digital avatar with real-time audio-video streaming from server to mobile device. Both our visualization tool, and our Crowd Vote metrics demonstrate our AI driven digital avatars have state-of-the-art humor, authenticity, and favorability outperforming all competitors and baselines. In the case of our Donald Trump and Joe Biden avatars, their authenticity and favorability are rated higher than even their real-world equivalents. △ Less

Submitted 7 August, 2024; originally announced August 2024.

Comments: This work was presented during the IJCAI 2024 conference proceedings for demonstrations

MSC Class: 68 ACM Class: D.2.2; C.3

Journal ref: 2024 Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence Demo Track. Pages 8780-8783

A Study on Quantum Graph Neural Networks Applied to Molecular Physics

Authors: Simone Piperno, Andrea Ceschini, Su Yeon Chang, Michele Grossi, Sofia Vallecorsa, Massimo Panella

Abstract: This paper introduces a novel architecture for Quantum Graph Neural Networks, which is significantly different from previous approaches found in the literature. The proposed approach produces similar outcomes with respect to previous models but with fewer parameters, resulting in an extremely interpretable architecture rooted in the underlying physics of the problem. The architectural novelties ar… ▽ More This paper introduces a novel architecture for Quantum Graph Neural Networks, which is significantly different from previous approaches found in the literature. The proposed approach produces similar outcomes with respect to previous models but with fewer parameters, resulting in an extremely interpretable architecture rooted in the underlying physics of the problem. The architectural novelties arise from three pivotal aspects. Firstly, we employ an embedding updating method that is analogous to classical Graph Neural Networks, therefore bridging the classical-quantum gap. Secondly, each layer is devoted to capturing interactions of distinct orders, aligning with the physical properties of the system. Lastly, we harness SWAP gates to emulate the problem's inherent symmetry, a novel strategy not found currently in the literature. The obtained results in the considered experiments are encouraging to lay the foundation for continued research in this field. △ Less

Submitted 6 August, 2024; originally announced August 2024.

Comments: 20 pages, 10 figures, 3 tables

Impact of opinion dynamics on recurrent pandemic waves: balancing risk aversion and peer pressure

Authors: Sheryl L. Chang, Quang Dang Nguyen, Carl J. E. Suster, Christina M. Jamerlan, Rebecca J. Rockett, Vitali Sintchenko, Tania C. Sorrell, Alexandra Martiniuk, Mikhail Prokopenko

Abstract: Recurrent waves which are often observed during long pandemics typically form as a result of several interrelated dynamics including public health interventions, population mobility and behaviour, varying disease transmissibility due to pathogen mutations, and changes in host immunity due to recency of vaccination or previous infections. Complex nonlinear dependencies among these dynamics, includi… ▽ More Recurrent waves which are often observed during long pandemics typically form as a result of several interrelated dynamics including public health interventions, population mobility and behaviour, varying disease transmissibility due to pathogen mutations, and changes in host immunity due to recency of vaccination or previous infections. Complex nonlinear dependencies among these dynamics, including feedback between disease incidence and the opinion-driven adoption of social distancing behaviour, remain poorly understood, particularly in scenarios involving heterogeneous population, partial and waning immunity, and rapidly changing public opinions. This study addressed this challenge by proposing an opinion dynamics model that accounts for changes in social distancing behaviour (i.e., whether to adopt social distancing) by modelling both individual risk perception and peer pressure. The opinion dynamics model was integrated and validated within a large-scale agent-based COVID-19 pandemic simulation that modelled the spread of the Omicron variant of SARS-CoV-2 between December 2021 and June 2022 in Australia. Our study revealed that the fluctuating adoption of social distancing, shaped by individual risk aversion and social peer pressure from both household and workplace environments, may explain the observed pattern of recurrent waves of infections. △ Less

Submitted 26 July, 2024; originally announced August 2024.

Comments: 18 pages, 7 figures

Learning production functions for supply chains with graph neural networks

Authors: Serina Chang, Zhiyin Lin, Benjamin Yan, Swapnil Bembde, Qi Xiu, Chi Heem Wong, Yu Qin, Frank Kloster, Alex Luo, Raj Palleti, Jure Leskovec

Abstract: The global economy relies on the flow of goods over supply chain networks, with nodes as firms and edges as transactions between firms. While we may observe these external transactions, they are governed by unseen production functions, which determine how firms internally transform the input products they receive into output products that they sell. In this setting, it can be extremely valuable to… ▽ More The global economy relies on the flow of goods over supply chain networks, with nodes as firms and edges as transactions between firms. While we may observe these external transactions, they are governed by unseen production functions, which determine how firms internally transform the input products they receive into output products that they sell. In this setting, it can be extremely valuable to infer these production functions, to better understand and improve supply chains, and to forecast future transactions more accurately. However, existing graph neural networks (GNNs) cannot capture these hidden relationships between nodes' inputs and outputs. Here, we introduce a new class of models for this setting, by combining temporal GNNs with a novel inventory module, which learns production functions via attention weights and a special loss function. We evaluate our models extensively on real supply chains data, along with data generated from our new open-source simulator, SupplySim. Our models successfully infer production functions, with a 6-50% improvement over baselines, and forecast future transactions on real and synthetic data, outperforming baselines by 11-62%. △ Less

Submitted 26 July, 2024; originally announced July 2024.

SuperFlow: A Fully-Customized RTL-to-GDS Design Automation Flow for Adiabatic Quantum-Flux-Parametron Superconducting Circuits

Authors: Yanyue Xie, Peiyan Dong, Geng Yuan, Zhengang Li, Masoud Zabihi, Chao Wu, Sung-En Chang, Xufeng Zhang, Xue Lin, Caiwen Ding, Nobuyuki Yoshikawa, Olivia Chen, Yanzhi Wang

Abstract: Superconducting circuits, like Adiabatic Quantum-Flux-Parametron (AQFP), offer exceptional energy efficiency but face challenges in physical design due to sophisticated spacing and timing constraints. Current design tools often neglect the importance of constraint adherence throughout the entire design flow. In this paper, we propose SuperFlow, a fully-customized RTL-to-GDS design flow tailored fo… ▽ More Superconducting circuits, like Adiabatic Quantum-Flux-Parametron (AQFP), offer exceptional energy efficiency but face challenges in physical design due to sophisticated spacing and timing constraints. Current design tools often neglect the importance of constraint adherence throughout the entire design flow. In this paper, we propose SuperFlow, a fully-customized RTL-to-GDS design flow tailored for AQFP devices. SuperFlow leverages a synthesis tool based on CMOS technology to transform any input RTL netlist to an AQFP-based netlist. Subsequently, we devise a novel place-and-route procedure that simultaneously considers wirelength, timing, and routability for AQFP circuits. The process culminates in the generation of the AQFP circuit layout, followed by a Design Rule Check (DRC) to identify and rectify any layout violations. Our experimental results demonstrate that SuperFlow achieves 12.8% wirelength improvement on average and 12.1% better timing quality compared with previous state-of-the-art placers for AQFP circuits. △ Less

Submitted 25 July, 2024; originally announced July 2024.

Comments: Accepted by DATE 2024

Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspective

Authors: Yujian Liu, Yang Zhang, Tommi Jaakkola, Shiyu Chang

Abstract: This paper investigates Who's Harry Potter (WHP), a pioneering yet insufficiently understood method for LLM unlearning. We explore it in two steps. First, we introduce a new task of LLM targeted unlearning, where given an unlearning target (e.g., a person) and some unlearning documents, we aim to unlearn only the information about the target, rather than everything in the unlearning documents. We… ▽ More This paper investigates Who's Harry Potter (WHP), a pioneering yet insufficiently understood method for LLM unlearning. We explore it in two steps. First, we introduce a new task of LLM targeted unlearning, where given an unlearning target (e.g., a person) and some unlearning documents, we aim to unlearn only the information about the target, rather than everything in the unlearning documents. We further argue that a successful unlearning should satisfy criteria such as not outputting gibberish, not fabricating facts about the unlearning target, and not releasing factual information under jailbreak attacks. Second, we construct a causal intervention framework for targeted unlearning, where the knowledge of the unlearning target is modeled as a confounder between LLM input and output, and the unlearning process as a deconfounding process. This framework justifies and extends WHP, deriving a simple unlearning algorithm that includes WHP as a special case. Experiments on existing and new datasets show that our approach, without explicitly optimizing for the aforementioned criteria, achieves competitive performance in all of them. Our code is available at https://github.com/UCSB-NLP-Chang/causal_unlearn.git. △ Less

Submitted 24 July, 2024; originally announced July 2024.

SAM-Driven Weakly Supervised Nodule Segmentation with Uncertainty-Aware Cross Teaching

Authors: Xingyue Zhao, Peiqi Li, Xiangde Luo, Meng Yang, Shi Chang, Zhongyu Li

Abstract: Automated nodule segmentation is essential for computer-assisted diagnosis in ultrasound images. Nevertheless, most existing methods depend on precise pixel-level annotations by medical professionals, a process that is both costly and labor-intensive. Recently, segmentation foundation models like SAM have shown impressive generalizability on natural images, suggesting their potential as pseudo-lab… ▽ More Automated nodule segmentation is essential for computer-assisted diagnosis in ultrasound images. Nevertheless, most existing methods depend on precise pixel-level annotations by medical professionals, a process that is both costly and labor-intensive. Recently, segmentation foundation models like SAM have shown impressive generalizability on natural images, suggesting their potential as pseudo-labelers. However, accurate prompts remain crucial for their success in medical images. In this work, we devise a novel weakly supervised framework that effectively utilizes the segmentation foundation model to generate pseudo-labels from aspect ration annotations for automatic nodule segmentation. Specifically, we develop three types of bounding box prompts based on scalable shape priors, followed by an adaptive pseudo-label selection module to fully exploit the prediction capabilities of the foundation model for nodules. We also present a SAM-driven uncertainty-aware cross-teaching strategy. This approach integrates SAM-based uncertainty estimation and label-space perturbations into cross-teaching to mitigate the impact of pseudo-label inaccuracies on model training. Extensive experiments on two clinically collected ultrasound datasets demonstrate the superior performance of our proposed method. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Comments: ISBI 2024 Oral

Beyond Mask: Rethinking Guidance Types in Few-shot Segmentation

Authors: Shijie Chang, Youwei Pang, Xiaoqi Zhao, Lihe Zhang, Huchuan Lu

Abstract: Existing few-shot segmentation (FSS) methods mainly focus on prototype feature generation and the query-support matching mechanism. As a crucial prompt for generating prototype features, the pair of image-mask types in the support set has become the default setting. However, various types such as image, text, box, and mask all can provide valuable information regarding the objects in context, clas… ▽ More Existing few-shot segmentation (FSS) methods mainly focus on prototype feature generation and the query-support matching mechanism. As a crucial prompt for generating prototype features, the pair of image-mask types in the support set has become the default setting. However, various types such as image, text, box, and mask all can provide valuable information regarding the objects in context, class, localization, and shape appearance. Existing work focuses on specific combinations of guidance, leading FSS into different research branches. Rethinking guidance types in FSS is expected to explore the efficient joint representation of the coupling between the support set and query set, giving rise to research trends in the weakly or strongly annotated guidance to meet the customized requirements of practical users. In this work, we provide the generalized FSS with seven guidance paradigms and develop a universal vision-language framework (UniFSS) to integrate prompts from text, mask, box, and image. Leveraging the advantages of large-scale pre-training vision-language models in textual and visual embeddings, UniFSS proposes high-level spatial correction and embedding interactive units to overcome the semantic ambiguity drawbacks typically encountered by pure visual matching methods when facing intra-class appearance diversities. Extensive experiments show that UniFSS significantly outperforms the state-of-the-art methods. Notably, the weakly annotated class-aware box paradigm even surpasses the finely annotated mask paradigm. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Comments: Preprint under review

Feature Diversification and Adaptation for Federated Domain Generalization

Authors: Seunghan Yang, Seokeon Choi, Hyunsin Park, Sungha Choi, Simyung Chang, Sungrack Yun

Abstract: Federated learning, a distributed learning paradigm, utilizes multiple clients to build a robust global model. In real-world applications, local clients often operate within their limited domains, leading to a `domain shift' across clients. Privacy concerns limit each client's learning to its own domain data, which increase the risk of overfitting. Moreover, the process of aggregating models train… ▽ More Federated learning, a distributed learning paradigm, utilizes multiple clients to build a robust global model. In real-world applications, local clients often operate within their limited domains, leading to a `domain shift' across clients. Privacy concerns limit each client's learning to its own domain data, which increase the risk of overfitting. Moreover, the process of aggregating models trained on own limited domain can be potentially lead to a significant degradation in the global model performance. To deal with these challenges, we introduce the concept of federated feature diversification. Each client diversifies the own limited domain data by leveraging global feature statistics, i.e., the aggregated average statistics over all participating clients, shared through the global model's parameters. This data diversification helps local models to learn client-invariant representations while preserving privacy. Our resultant global model shows robust performance on unseen test domain data. To enhance performance further, we develop an instance-adaptive inference approach tailored for test domain data. Our proposed instance feature adapter dynamically adjusts feature statistics to align with the test input, thereby reducing the domain gap between the test and training domains. We show that our method achieves state-of-the-art performance on several domain generalization benchmarks within a federated learning setting. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: Accepted to ECCV 2024

Proceedings of the Thirteenth Workshop on Trends in Functional Programming in Education

Authors: Stephen Chang

Abstract: This volume of the Electronic Proceedings in Theoretical Computer Science (EPTCS) contains revised selected papers that were initially presented at the 13th International Workshop on Trends in Functional Programming in Education (TFPIE 2024). This workshop was held at Seton Hall University in South Orange, NJ, USA on January 9, 2024. It was co-located with the 25th International Symposium on Trend… ▽ More This volume of the Electronic Proceedings in Theoretical Computer Science (EPTCS) contains revised selected papers that were initially presented at the 13th International Workshop on Trends in Functional Programming in Education (TFPIE 2024). This workshop was held at Seton Hall University in South Orange, NJ, USA on January 9, 2024. It was co-located with the 25th International Symposium on Trends in Functional Programming (TFP 2024), which took place on January 10-12, 2024. The goal of TFPIE is to gather researchers, teachers, and professionals that use, or are interested in the use of, functional programming in education. TFPIE aims to be a venue where novel ideas, classroom-tested ideas, and works-in-progress on the use of functional programming in education are discussed. TFPIE workshops have previously been held in St Andrews, Scotland (2012), Provo, Utah, USA (2013), Soesterberg, The Netherlands (2014), Sophia-Antipolis, France (2015), College Park, MD, USA (2016), Canterbury, UK (2017), Gothenburg, Sweden (2018), Vancouver, Canada (2019), Krakow, Poland (2020), online due to COVID-19 (2021, 2022, with some talks from TFPIE 2022 also presented in person at the Lambda Days in Krakow, Poland), and Boston, MA, USA (2023, back in-person). △ Less

Submitted 8 July, 2024; originally announced July 2024.

Journal ref: EPTCS 405, 2024

Generalized Multivariate Hypercomplex Function Inequalities and Their Applications

Authors: Shih-Yu Chang

Abstract: This work extends the Mond-Pecaric method to functions with multiple operators as arguments by providing arbitrarily close approximations of the original functions. Instead of using linear functions to establish lower and upper bounds for multivariate functions as in prior work, we apply sigmoid functions to achieve these bounds with any specified error threshold based on the multivariate function… ▽ More This work extends the Mond-Pecaric method to functions with multiple operators as arguments by providing arbitrarily close approximations of the original functions. Instead of using linear functions to establish lower and upper bounds for multivariate functions as in prior work, we apply sigmoid functions to achieve these bounds with any specified error threshold based on the multivariate function approximation method proposed by Cybenko. This approach allows us to derive fundamental inequalities for multivariate hypercomplex functions, leading to new inequalities based on ratio and difference kinds. For applications about these new derived inequalities for multivariate hypercomplex functions, we first introduce a new concept called W-boundedness for hypercomplex functions by applying ratio kind multivariate hypercomplex inequalities. W-boundedness generalizes R-boundedness for norm mappings with input from Banach space. Additionally, we develop an approximation theory for multivariate hypercomplex functions and establish bounds algebra, including operator bounds and tail bounds algebra for multivariate random tensors. △ Less

Submitted 6 July, 2024; originally announced July 2024.

Comments: arXiv admin note: text overlap with arXiv:2404.11880, arXiv:2405.17203

YourMT3+: Multi-instrument Music Transcription with Enhanced Transformer Architectures and Cross-dataset Stem Augmentation

Authors: Sungkyun Chang, Emmanouil Benetos, Holger Kirchhoff, Simon Dixon

Abstract: Multi-instrument music transcription aims to convert polyphonic music recordings into musical scores assigned to each instrument. This task is challenging for modeling as it requires simultaneously identifying multiple instruments and transcribing their pitch and precise timing, and the lack of fully annotated data adds to the training difficulties. This paper introduces YourMT3+, a suite of model… ▽ More Multi-instrument music transcription aims to convert polyphonic music recordings into musical scores assigned to each instrument. This task is challenging for modeling as it requires simultaneously identifying multiple instruments and transcribing their pitch and precise timing, and the lack of fully annotated data adds to the training difficulties. This paper introduces YourMT3+, a suite of models for enhanced multi-instrument music transcription based on the recent language token decoding approach of MT3. We enhance its encoder by adopting a hierarchical attention transformer in the time-frequency domain and integrating a mixture of experts. To address data limitations, we introduce a new multi-channel decoding method for training with incomplete annotations and propose intra- and cross-stem augmentation for dataset mixing. Our experiments demonstrate direct vocal transcription capabilities, eliminating the need for voice separation pre-processors. Benchmarks across ten public datasets show our models' competitiveness with, or superiority to, existing transcription models. Further testing on pop music recordings highlights the limitations of current models. Fully reproducible code and datasets are available with demos at \url{https://github.com/mimbres/YourMT3}. △ Less

Submitted 1 August, 2024; v1 submitted 5 July, 2024; originally announced July 2024.

Comments: Accepted at IEEE International Workshop on Machine Learning for Signal Processing (MLSP) 2024, London

Anole: Adapting Diverse Compressed Models For Cross-Scene Prediction On Mobile Devices

Authors: Yunzhe Li, Hongzi Zhu, Zhuohong Deng, Yunlong Cheng, Liang Zhang, Shan Chang, Minyi Guo

Abstract: Emerging Artificial Intelligence of Things (AIoT) applications desire online prediction using deep neural network (DNN) models on mobile devices. However, due to the movement of devices, unfamiliar test samples constantly appear, significantly affecting the prediction accuracy of a pre-trained DNN. In addition, unstable network connection calls for local model inference. In this paper, we propose… ▽ More Emerging Artificial Intelligence of Things (AIoT) applications desire online prediction using deep neural network (DNN) models on mobile devices. However, due to the movement of devices, unfamiliar test samples constantly appear, significantly affecting the prediction accuracy of a pre-trained DNN. In addition, unstable network connection calls for local model inference. In this paper, we propose a light-weight scheme, called Anole, to cope with the local DNN model inference on mobile devices. The core idea of Anole is to first establish an army of compact DNN models, and then adaptively select the model fitting the current test sample best for online inference. The key is to automatically identify model-friendly scenes for training scene-specific DNN models. To this end, we design a weakly-supervised scene representation learning algorithm by combining both human heuristics and feature similarity in separating scenes. Moreover, we further train a model classifier to predict the best-fit scene-specific DNN model for each test sample. We implement Anole on different types of mobile devices and conduct extensive trace-driven and real-world experiments based on unmanned aerial vehicles (UAVs). The results demonstrate that Anole outwits the method of using a versatile large DNN in terms of prediction accuracy (4.5% higher), response time (33.1% faster) and power consumption (45.1% lower). △ Less

Submitted 9 May, 2024; originally announced July 2024.

VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMs

Authors: Qiucheng Wu, Handong Zhao, Michael Saxon, Trung Bui, William Yang Wang, Yang Zhang, Shiyu Chang

Abstract: Vision language models (VLMs) are an exciting emerging class of language models (LMs) that have merged classic LM capabilities with those of image processing systems. However, the ways that these capabilities combine are not always intuitive and warrant direct investigation. One understudied capability in VLMs is visual spatial planning -- the ability to comprehend the spatial arrangements of obje… ▽ More Vision language models (VLMs) are an exciting emerging class of language models (LMs) that have merged classic LM capabilities with those of image processing systems. However, the ways that these capabilities combine are not always intuitive and warrant direct investigation. One understudied capability in VLMs is visual spatial planning -- the ability to comprehend the spatial arrangements of objects and devise action plans to achieve desired outcomes in visual scenes. In our study, we introduce VSP, a benchmark that 1) evaluates the spatial planning capability in these models in general, and 2) breaks down the visual planning task into finer-grained sub-tasks, including perception and reasoning, and measure the LMs capabilities in these sub-tasks. Our evaluation shows that both open-source and private VLMs fail to generate effective plans for even simple spatial planning tasks. Evaluations on the fine-grained analytical tasks further reveal fundamental deficiencies in the models' visual perception and bottlenecks in reasoning abilities, explaining their worse performance in the general spatial planning tasks. Our work illuminates future directions for improving VLMs' abilities in spatial planning. Our benchmark is publicly available at https://github.com/UCSB-NLP-Chang/Visual-Spatial-Planning. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Cost-efficient Active Illumination Camera For Hyper-spectral Reconstruction

Authors: Yuxuan Zhang, T. M. Sazzad, Yangyang Song, Spencer J. Chang, Ritesh Chowdhry, Tomas Mejia, Anna Hampton, Shelby Kucharski, Stefan Gerber, Barry Tillman, Marcio F. R. Resende, William M. Hammond, Chris H. Wilson, Alina Zare, Sanjeev J. Koppal

Abstract: Hyper-spectral imaging has recently gained increasing attention for use in different applications, including agricultural investigation, ground tracking, remote sensing and many other. However, the high cost, large physical size and complicated operation process stop hyperspectral cameras from being employed for various applications and research fields. In this paper, we introduce a cost-efficient… ▽ More Hyper-spectral imaging has recently gained increasing attention for use in different applications, including agricultural investigation, ground tracking, remote sensing and many other. However, the high cost, large physical size and complicated operation process stop hyperspectral cameras from being employed for various applications and research fields. In this paper, we introduce a cost-efficient, compact and easy to use active illumination camera that may benefit many applications. We developed a fully functional prototype of such camera. With the hope of helping with agricultural research, we tested our camera for plant root imaging. In addition, a U-Net model for spectral reconstruction was trained by using a reference hyperspectral camera's data as ground truth and our camera's data as input. We demonstrated our camera's ability to obtain additional information over a typical RGB camera. In addition, the ability to reconstruct hyperspectral data from multi-spectral input makes our device compatible to models and algorithms developed for hyperspectral applications with no modifications required. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Project Management for Ground-based Telescope Array Development

Authors: Ji Hoon Kim, Myungshin Im, Hyung Mok Lee, Seo-Won Chang

Abstract: Center for the Gravitational-Wave Universe at Seoul National University has been operating its main observational facility, the 7-Dimensional Telescope (7DT) since October 2023. Located at El Sauce Observatory in Chilean Rio Hurtado Valley, 7DT consists of 20 50-cm telescopes equipped with 40 medium-band filters of 25 nm full width at half maximum along with a CMOS camera of 61 megapixels. 7DT pro… ▽ More Center for the Gravitational-Wave Universe at Seoul National University has been operating its main observational facility, the 7-Dimensional Telescope (7DT) since October 2023. Located at El Sauce Observatory in Chilean Rio Hurtado Valley, 7DT consists of 20 50-cm telescopes equipped with 40 medium-band filters of 25 nm full width at half maximum along with a CMOS camera of 61 megapixels. 7DT produces about 1 TB per night of spectral mapping image data including calibration, and the byproduct of the data reduction pipeline once our planned three layered surveys (Reference Imaging Survey, Wide Field Survey, and Intensive Monitoring Survey) start in 2024. We are expecting to generate 1 PB per year by combining raw data, reduced data, and data products (e.g. calibrated stacked images, spectral cubes, and object catalogs). To incorporate this huge amount of data, we now have a data storage for 1 PB which we will increment by 1 PB per year. We also have a high-performance computation facility that is equipped with 2 NVIDIA A100 GPU cards since we plan to carry out real-time data reduction and analysis for follow-up observation data of gravitational wave events. To incorporate this, we established a 400 Mbps network connection between the facilities in Korea and Chile. Taking advantage of the high-performance network, we have been carrying out fully remote operations since October 2023. In this talk, we present details of designing, planning, and executing the ground-based telescope facility project, especially within low-budget academic environments. While we cover as much ground as possible, we will emphasize human resource management, project risk management, and financial contingency management. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: 7 pages, 1 figures, Proceedings of the SPIE conference "Modeling, Systems Engineering, and Project Management for Astronomy XI" SPIE Astronomical Telescopes + Instrumentation 2024 (Paper No. 13099-77)

Introduction to the 7-Dimensional Telescope: Commissioning Procedures and Data Characteristics

Authors: Ji Hoon Kim, Myungshin Im, Hyung Mok Lee, Seo-Won Chang, Hyeonho Choi, Gregory S. H. Paek

Abstract: The 7-Dimensional Telescope (7DT) is a multi-telescope system designed to identify electromagnetic (EM) counterparts of gravitational-wave (GW) sources. Consisting of 20 50-cm telescopes along with 40 medium-band filters of 25 nm width, 7DT can obtain spectral mapping images for a large field of view (~1.25 square degrees). Along with flexible operation, real-time data reduction, and analysis, the… ▽ More The 7-Dimensional Telescope (7DT) is a multi-telescope system designed to identify electromagnetic (EM) counterparts of gravitational-wave (GW) sources. Consisting of 20 50-cm telescopes along with 40 medium-band filters of 25 nm width, 7DT can obtain spectral mapping images for a large field of view (~1.25 square degrees). Along with flexible operation, real-time data reduction, and analysis, the 7DT's spectral mapping capability enables 7DT to follow up GW events quickly and discover EM counterparts. Among 20 planned telescopes, 12 units are deployed at the El Sauce Observatory located at Rio Hurtado Valley in Chile. Since we obtained the first light of 7DT in October 2023, we started its commissioning procedures including examination of bias levels, master flat production, and spectrophotometric standardization. In this talk, we present 7DT instruments and their set-up, commissioning procedures, and data characteristics of 7DT along with our three-layered surveys which are assumed to be initiated in early 2024. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: 11 pages, 3 figures, Proceedings of the SPIE conference 13094 Ground-based and Airborne Telescope X, SPIE Astronomical Telescopes + Instrumentations 2024 (Paper No. 13094-034)

On the existence of conic Sasaki-Einstein metrics on log Fano Sasakian manifolds of dimension five

Authors: Shu-Cheng Chang, Fengjiang Li, Chien Lin, Chin-Tung Wu

Abstract: In this paper, we derive the uniform L^{4}-bound of the transverse conic Ricci curvature along the conic Sasaki-Ricci flow on a compact transverse log Fano Sasakian manifold M of dimension five and the space of leaves of the characteristic foliation is not well-formed. Then we first show that any solution of the conic Sasaki-Ricci flow converges in the Cheeger-Gromov sense to the unique singular o… ▽ More In this paper, we derive the uniform L^{4}-bound of the transverse conic Ricci curvature along the conic Sasaki-Ricci flow on a compact transverse log Fano Sasakian manifold M of dimension five and the space of leaves of the characteristic foliation is not well-formed. Then we first show that any solution of the conic Sasaki-Ricci flow converges in the Cheeger-Gromov sense to the unique singular orbifold conic Sasaki-Ricci soliton on M_{infinite} which is a S^{1}-orbibundle over the unique singular conic Keahler-Ricci soliton on a log del Pezzo orbifold surface. As a consequence, there exists a Keahler-Ricci soliton orbifold metric on its leave space which is a log del Pezzo orbifold surface. Second, we show that the conic Sasaki-Ricci soliton is the conic Sasaki-Einstein if M is transverse log K-polystable. In summary, we have the existence theorems of orbifold Sasaki-Ricci solitons and Sasaki-Einstein metrics on a compact quasi-regular Sasakian manifold of dimension five. △ Less

Submitted 14 August, 2024; v1 submitted 24 June, 2024; originally announced June 2024.

DCDILP: a distributed learning method for large-scale causal structure learning

Authors: Shuyu Dong, Michèle Sebag, Kento Uemura, Akito Fujii, Shuang Chang, Yusuke Koyanagi, Koji Maruhashi

Abstract: This paper presents a novel approach to causal discovery through a divide-and-conquer framework. By decomposing the problem into smaller subproblems defined on Markov blankets, the proposed DCDILP method first explores in parallel the local causal graphs of these subproblems. However, this local discovery phase encounters systematic challenges due to the presence of hidden confounders (variables w… ▽ More This paper presents a novel approach to causal discovery through a divide-and-conquer framework. By decomposing the problem into smaller subproblems defined on Markov blankets, the proposed DCDILP method first explores in parallel the local causal graphs of these subproblems. However, this local discovery phase encounters systematic challenges due to the presence of hidden confounders (variables within each Markov blanket may be influenced by external variables). Moreover, aggregating these local causal graphs in a consistent global graph defines a large size combinatorial optimization problem. DCDILP addresses these challenges by: i) restricting the local subgraphs to causal links only related with the central variable of the Markov blanket; ii) formulating the reconciliation of local causal graphs as an integer linear programming method. The merits of the approach, in both terms of causal discovery accuracy and scalability in the size of the problem, are showcased by experiments and comparisons with the state of the art. △ Less

Submitted 14 June, 2024; originally announced June 2024.

CliBench: Multifaceted Evaluation of Large Language Models in Clinical Decisions on Diagnoses, Procedures, Lab Tests Orders and Prescriptions

Authors: Mingyu Derek Ma, Chenchen Ye, Yu Yan, Xiaoxuan Wang, Peipei Ping, Timothy S Chang, Wei Wang

Abstract: The integration of Artificial Intelligence (AI), especially Large Language Models (LLMs), into the clinical diagnosis process offers significant potential to improve the efficiency and accessibility of medical care. While LLMs have shown some promise in the medical domain, their application in clinical diagnosis remains underexplored, especially in real-world clinical practice, where highly sophis… ▽ More The integration of Artificial Intelligence (AI), especially Large Language Models (LLMs), into the clinical diagnosis process offers significant potential to improve the efficiency and accessibility of medical care. While LLMs have shown some promise in the medical domain, their application in clinical diagnosis remains underexplored, especially in real-world clinical practice, where highly sophisticated, patient-specific decisions need to be made. Current evaluations of LLMs in this field are often narrow in scope, focusing on specific diseases or specialties and employing simplified diagnostic tasks. To bridge this gap, we introduce CliBench, a novel benchmark developed from the MIMIC IV dataset, offering a comprehensive and realistic assessment of LLMs' capabilities in clinical diagnosis. This benchmark not only covers diagnoses from a diverse range of medical cases across various specialties but also incorporates tasks of clinical significance: treatment procedure identification, lab test ordering and medication prescriptions. Supported by structured output ontologies, CliBench enables a precise and multi-granular evaluation, offering an in-depth understanding of LLM's capability on diverse clinical tasks of desired granularity. We conduct a zero-shot evaluation of leading LLMs to assess their proficiency in clinical decision-making. Our preliminary results shed light on the potential and limitations of current LLMs in clinical settings, providing valuable insights for future advancements in LLM-powered healthcare. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: Project page: https://clibench.github.io

Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference

Authors: Jiabao Ji, Yujian Liu, Yang Zhang, Gaowen Liu, Ramana Rao Kompella, Sijia Liu, Shiyu Chang

Abstract: As Large Language Models (LLMs) demonstrate extensive capability in learning from documents, LLM unlearning becomes an increasingly important research area to address concerns of LLMs in terms of privacy, copyright, etc. A conventional LLM unlearning task typically involves two goals: (1) The target LLM should forget the knowledge in the specified forget documents, and (2) it should retain the oth… ▽ More As Large Language Models (LLMs) demonstrate extensive capability in learning from documents, LLM unlearning becomes an increasingly important research area to address concerns of LLMs in terms of privacy, copyright, etc. A conventional LLM unlearning task typically involves two goals: (1) The target LLM should forget the knowledge in the specified forget documents, and (2) it should retain the other knowledge that the LLM possesses, for which we assume access to a small number of retain documents. To achieve both goals, a mainstream class of LLM unlearning methods introduces an optimization framework with a combination of two objectives - maximizing the prediction loss on the forget documents while minimizing that on the retain documents, which suffers from two challenges, degenerated output and catastrophic forgetting. In this paper, we propose a novel unlearning framework called Unlearning from Logit Difference (ULD), which introduces an assistant LLM that aims to achieve the opposite of the unlearning goals: remembering the forget documents and forgetting the retain knowledge. ULD then derives the unlearned LLM by computing the logit difference between the target and the assistant LLMs. We show that such reversed objectives would naturally resolve both aforementioned challenges while significantly improving the training efficiency. Extensive experiments demonstrate that our method efficiently achieves the intended forgetting while preserving the LLM's overall capabilities, reducing training time by more than threefold. Notably, our method loses 0% of model utility on the ToFU benchmark, whereas baseline methods may sacrifice 17% of utility on average to achieve comparable forget quality. Our code will be publicly available at https://github.com/UCSB-NLP-Chang/ULD. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 21 pages, 11 figures

Jet modification via $π^0$-hadron correlations in Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$ GeV

Authors: PHENIX Collaboration, N. J. Abdulameer, U. Acharya, A. Adare, S. Afanasiev, C. Aidala, N. N. Ajitanand, Y. Akiba, H. Al-Bataineh, J. Alexander, M. Alfred, K. Aoki, N. Apadula, L. Aphecetche, J. Asai, H. Asano, E. T. Atomssa, R. Averbeck, T. C. Awes, B. Azmoun, V. Babintsev, M. Bai, G. Baksay, L. Baksay, A. Baldisseri , et al. (510 additional authors not shown)

Abstract: High-momentum two-particle correlations are a useful tool for studying jet-quenching effects in the quark-gluon plasma. Angular correlations between neutral-pion triggers and charged hadrons with transverse momenta in the range 4--12~GeV/$c$ and 0.5--7~GeV/$c$, respectively, have been measured by the PHENIX experiment in 2014 for Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$~GeV. Suppression is obs… ▽ More High-momentum two-particle correlations are a useful tool for studying jet-quenching effects in the quark-gluon plasma. Angular correlations between neutral-pion triggers and charged hadrons with transverse momenta in the range 4--12~GeV/$c$ and 0.5--7~GeV/$c$, respectively, have been measured by the PHENIX experiment in 2014 for Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$~GeV. Suppression is observed in the yield of high-momentum jet fragments opposite the trigger particle, which indicates jet suppression stemming from in-medium partonic energy loss, while enhancement is observed for low-momentum particles. The ratio and differences between the yield in Au$+$Au collisions and $p$$+$$p$ collisions, $I_{AA}$ and $Δ_{AA}$, as a function of the trigger-hadron azimuthal separation, $Δφ$, are measured for the first time at the Relativistic Heavy Ion Collider. These results better quantify how the yield of low-$p_T$ associated hadrons is enhanced at wide angle, which is crucial for studying energy loss as well as medium-response effects. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 534 authors from 83 institutions, 12 pages, 7 figures. v1 is version submitted to Physical Review C. HEPdata tables for the points plotted in figures for this and previous PHENIX publications are (or will be) publicly available at http://www.phenix.bnl.gov/papers.html

Crayon: Customized On-Device LLM via Instant Adapter Blending and Edge-Server Hybrid Inference

Authors: Jihwan Bang, Juntae Lee, Kyuhong Shim, Seunghan Yang, Simyung Chang

Abstract: The customization of large language models (LLMs) for user-specified tasks gets important. However, maintaining all the customized LLMs on cloud servers incurs substantial memory and computational overheads, and uploading user data can also lead to privacy concerns. On-device LLMs can offer a promising solution by mitigating these issues. Yet, the performance of on-device LLMs is inherently constr… ▽ More The customization of large language models (LLMs) for user-specified tasks gets important. However, maintaining all the customized LLMs on cloud servers incurs substantial memory and computational overheads, and uploading user data can also lead to privacy concerns. On-device LLMs can offer a promising solution by mitigating these issues. Yet, the performance of on-device LLMs is inherently constrained by the limitations of small-scaled models. To overcome these restrictions, we first propose Crayon, a novel approach for on-device LLM customization. Crayon begins by constructing a pool of diverse base adapters, and then we instantly blend them into a customized adapter without extra training. In addition, we develop a device-server hybrid inference strategy, which deftly allocates more demanding queries or non-customized tasks to a larger, more capable LLM on a server. This ensures optimal performance without sacrificing the benefits of on-device customization. We carefully craft a novel benchmark from multiple question-answer datasets, and show the efficacy of our method in the LLM customization. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: ACL 2024 Main

A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagation

Authors: Bairu Hou, Yang Zhang, Jacob Andreas, Shiyu Chang

Abstract: This paper focuses on the task of hallucination detection, which aims to determine the truthfulness of LLM-generated statements. To address this problem, a popular class of methods utilize the LLM's self-consistencies in its beliefs in a set of logically related augmented statements generated by the LLM, which does not require external knowledge databases and can work with both white-box and black… ▽ More This paper focuses on the task of hallucination detection, which aims to determine the truthfulness of LLM-generated statements. To address this problem, a popular class of methods utilize the LLM's self-consistencies in its beliefs in a set of logically related augmented statements generated by the LLM, which does not require external knowledge databases and can work with both white-box and black-box LLMs. However, in many existing approaches, the augmented statements tend to be very monotone and unstructured, which makes it difficult to integrate meaningful information from the LLM beliefs in these statements. Also, many methods work with the binarized version of the LLM's belief, instead of the continuous version, which significantly loses information. To overcome these limitations, in this paper, we propose Belief Tree Propagation (BTProp), a probabilistic framework for LLM hallucination detection. BTProp introduces a belief tree of logically related statements by recursively decomposing a parent statement into child statements with three decomposition strategies, and builds a hidden Markov tree model to integrate the LLM's belief scores in these statements in a principled way. Experiment results show that our method improves baselines by 3%-9% (evaluated by AUROC and AUC-PR) on multiple hallucination detection benchmarks. Code is available at https://github.com/UCSB-NLP-Chang/BTProp. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: 26 pages, 18 figures

Latent Style-based Quantum GAN for high-quality Image Generation

Authors: Su Yeon Chang, Supanut Thanasilp, Bertrand Le Saux, Sofia Vallecorsa, Michele Grossi

Abstract: Quantum generative modeling is among the promising candidates for achieving a practical advantage in data analysis. Nevertheless, one key challenge is to generate large-size images comparable to those generated by their classical counterparts. In this work, we take an initial step in this direction and introduce the Latent Style-based Quantum GAN (LaSt-QGAN), which employs a hybrid classical-quant… ▽ More Quantum generative modeling is among the promising candidates for achieving a practical advantage in data analysis. Nevertheless, one key challenge is to generate large-size images comparable to those generated by their classical counterparts. In this work, we take an initial step in this direction and introduce the Latent Style-based Quantum GAN (LaSt-QGAN), which employs a hybrid classical-quantum approach in training Generative Adversarial Networks (GANs) for arbitrary complex data generation. This novel approach relies on powerful classical auto-encoders to map a high-dimensional original image dataset into a latent representation. The hybrid classical-quantum GAN operates in this latent space to generate an arbitrary number of fake features, which are then passed back to the auto-encoder to reconstruct the original data. Our LaSt-QGAN can be successfully trained on realistic computer vision datasets beyond the standard MNIST, namely Fashion MNIST (fashion products) and SAT4 (Earth Observation images) with 10 qubits, resulting in a comparable performance (and even better in some metrics) with the classical GANs. Moreover, we analyze the barren plateau phenomena within this context of the continuous quantum generative model using a polynomial depth circuit and propose a method to mitigate the detrimental effect during the training of deep-depth networks. Through empirical experiments and theoretical analysis, we demonstrate the potential of LaSt-QGAN for the practical usage in the context of image generation and open the possibility of applying it to a larger dataset in the future. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Comments: 28 pages, 22 figures

SCUBA-2 Ultra Deep Imaging EAO Survey (STUDIES). V. Confusion-limited Submillimeter Galaxy Number Counts at 450 $μ$m and Data Release for the COSMOS Field

Authors: Zhen-Kai Gao, Chen-Fatt Lim, Wei-Hao Wang, Chian-Chou Chen, Ian Smail, Scott C. Chapman, Xian Zhong Zheng, Hyunjin Shim, Tadayuki Kodama, Yiping Ao, Siou-Yu Chang, David L. Clements, James S. Dunlop, Luis C. Ho, Yun-Hsin Hsu, Chorng-Yuan Hwang, Ho Seong Hwang, M. P. Koprowski, Douglas Scott, Stephen Serjeant, Yoshiki Toba, Sheona A. Urquhart

Abstract: We present confusion-limited SCUBA-2 450-$μ$m observations in the COSMOS-CANDELS region as part of the JCMT Large Program, SCUBA-2 Ultra Deep Imaging EAO Survey (STUDIES). Our maps at 450 and 850 $μ$m cover an area of 450 arcmin$^2$. We achieved instrumental noise levels of $σ_{\mathrm{450}}=$ 0.59 mJy beam$^{-1}$ and $σ_{\mathrm{850}}=$ 0.09 mJy beam$^{-1}$ in the deepest area of each map. The co… ▽ More We present confusion-limited SCUBA-2 450-$μ$m observations in the COSMOS-CANDELS region as part of the JCMT Large Program, SCUBA-2 Ultra Deep Imaging EAO Survey (STUDIES). Our maps at 450 and 850 $μ$m cover an area of 450 arcmin$^2$. We achieved instrumental noise levels of $σ_{\mathrm{450}}=$ 0.59 mJy beam$^{-1}$ and $σ_{\mathrm{850}}=$ 0.09 mJy beam$^{-1}$ in the deepest area of each map. The corresponding confusion noise levels are estimated to be 0.65 and 0.36 mJy beam$^{-1}$. Above the 4 (3.5) $σ$ threshold, we detected 360 (479) sources at 450 $μ$m and 237 (314) sources at 850 $μ$m. We derive the deepest blank-field number counts at 450 $μ$m, covering the flux-density range of 2 to 43 mJy. These are in agreement with other SCUBA-2 blank-field and lensing-cluster observations, but are lower than various model counts. We compare the counts with those in other fields and find that the field-to-field variance observed at 450 $μ$m at the $R=6^\prime$ scale is consistent with Poisson noise, so there is no evidence of strong 2-D clustering at this scale. Additionally, we derive the integrated surface brightness at 450 $μ$m down to 2.1 mJy to be $57.3^{+1.0}_{-6.2}$~Jy deg$^{-2}$, contributing to (41$\pm$4)\% of the 450-$μ$m extragalactic background light (EBL) measured by COBE and Planck. Our results suggest that the 450-$μ$m EBL may be fully resolved at $0.08^{+0.09}_{-0.08}$~mJy, which extremely deep lensing-cluster observations and next-generation submillimeter instruments with large aperture sizes may be able to achieve. △ Less

Submitted 31 May, 2024; originally announced May 2024.

Comments: 29 pages, 14 figures, accepted for publication in ApJ

WIDIn: Wording Image for Domain-Invariant Representation in Single-Source Domain Generalization

Authors: Jiawei Ma, Yulei Niu, Shiyuan Huang, Guangxing Han, Shih-Fu Chang

Abstract: Language has been useful in extending the vision encoder to data from diverse distributions without empirical discovery in training domains. However, as the image description is mostly at coarse-grained level and ignores visual details, the resulted embeddings are still ineffective in overcoming complexity of domains at inference time. We present a self-supervision framework WIDIn, Wording Images… ▽ More Language has been useful in extending the vision encoder to data from diverse distributions without empirical discovery in training domains. However, as the image description is mostly at coarse-grained level and ignores visual details, the resulted embeddings are still ineffective in overcoming complexity of domains at inference time. We present a self-supervision framework WIDIn, Wording Images for Domain-Invariant representation, to disentangle discriminative visual representation, by only leveraging data in a single domain and without any test prior. Specifically, for each image, we first estimate the language embedding with fine-grained alignment, which can be consequently used to adaptively identify and then remove domain-specific counterpart from the raw visual embedding. WIDIn can be applied to both pretrained vision-language models like CLIP, and separately trained uni-modal models like MoCo and BERT. Experimental studies on three domain generalization datasets demonstrate the effectiveness of our approach. △ Less

Submitted 28 May, 2024; originally announced May 2024.

Multivariate Mond-Pecaric Method with Applications to Hypercomplex Function Sobolev Embedding

Authors: Shih-Yu Chang

Abstract: Mond and Pecaric introduced a method to simplify the determination of complementary inequalities for Jensen's inequality by converting it into a single-variable maximization or minimization problem of continuous functions. This principle has significantly enriched the field of operator inequalities. Our contribution lies in extending the Mond-Pecaric method from single-input operators to multiple-… ▽ More Mond and Pecaric introduced a method to simplify the determination of complementary inequalities for Jensen's inequality by converting it into a single-variable maximization or minimization problem of continuous functions. This principle has significantly enriched the field of operator inequalities. Our contribution lies in extending the Mond-Pecaric method from single-input operators to multiple-input operators. We commence by defining normalized positive linear maps, accompanied by illustrative examples. Subsequently, we employ the Mond-Pecaric method to derive fundamental inequalities for multivariate hypercomplex functions bounded by linear functions. These foundational inequalities serve as the basis for establishing several multivariate hypercomplex function inequalities, focusing on ratio relationships. Additionally, we present similar results based on difference relationships. Finally, we apply the derived multivariate hypercomplex function inequalities to establish Sobolev embedding via Sobolev inequality for hypercomplex functions with operator inputs. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Detecting Multimodal Situations with Insufficient Context and Abstaining from Baseless Predictions

Authors: Junzhang Liu, Zhecan Wang, Hammad Ayyubi, Haoxuan You, Chris Thomas, Rui Sun, Shih-Fu Chang, Kai-Wei Chang

Abstract: Despite the widespread adoption of Vision-Language Understanding (VLU) benchmarks such as VQA v2, OKVQA, A-OKVQA, GQA, VCR, SWAG, and VisualCOMET, our analysis reveals a pervasive issue affecting their integrity: these benchmarks contain samples where answers rely on assumptions unsupported by the provided context. Training models on such data foster biased learning and hallucinations as models te… ▽ More Despite the widespread adoption of Vision-Language Understanding (VLU) benchmarks such as VQA v2, OKVQA, A-OKVQA, GQA, VCR, SWAG, and VisualCOMET, our analysis reveals a pervasive issue affecting their integrity: these benchmarks contain samples where answers rely on assumptions unsupported by the provided context. Training models on such data foster biased learning and hallucinations as models tend to make similar unwarranted assumptions. To address this issue, we collect contextual data for each sample whenever available and train a context selection module to facilitate evidence-based model predictions. Strong improvements across multiple benchmarks demonstrate the effectiveness of our approach. Further, we develop a general-purpose Context-AwaRe Abstention (CARA) detector to identify samples lacking sufficient context and enhance model accuracy by abstaining from responding if the required context is absent. CARA exhibits generalization to new benchmarks it wasn't trained on, underscoring its utility for future VLU benchmarks in detecting or cleaning samples with inadequate context. Finally, we curate a Context Ambiguity and Sufficiency Evaluation (CASE) set to benchmark the performance of insufficient context detectors. Overall, our work represents a significant advancement in ensuring that vision-language models generate trustworthy and evidence-based outputs in complex real-world scenarios. △ Less

Submitted 25 May, 2024; v1 submitted 17 May, 2024; originally announced May 2024.

Joint Signal Detection and Automatic Modulation Classification via Deep Learning

Authors: Huijun Xing, Xuhui Zhang, Shuo Chang, Jinke Ren, Zixun Zhang, Jie Xu, Shuguang Cui

Abstract: Signal detection and modulation classification are two crucial tasks in various wireless communication systems. Different from prior works that investigate them independently, this paper studies the joint signal detection and automatic modulation classification (AMC) by considering a realistic and complex scenario, in which multiple signals with different modulation schemes coexist at different ca… ▽ More Signal detection and modulation classification are two crucial tasks in various wireless communication systems. Different from prior works that investigate them independently, this paper studies the joint signal detection and automatic modulation classification (AMC) by considering a realistic and complex scenario, in which multiple signals with different modulation schemes coexist at different carrier frequencies. We first generate a coexisting RADIOML dataset (CRML23) to facilitate the joint design. Different from the publicly available AMC dataset ignoring the signal detection step and containing only one signal, our synthetic dataset covers the more realistic multiple-signal coexisting scenario. Then, we present a joint framework for detection and classification (JDM) for such a multiple-signal coexisting environment, which consists of two modules for signal detection and AMC, respectively. In particular, these two modules are interconnected using a designated data structure called "proposal". Finally, we conduct extensive simulations over the newly developed dataset, which demonstrate the effectiveness of our designs. Our code and dataset are now available as open-source (https://github.com/Singingkettle/ChangShuoRadioData). △ Less

Submitted 29 April, 2024; originally announced May 2024.

MoDE: CLIP Data Experts via Clustering

Authors: Jiawei Ma, Po-Yao Huang, Saining Xie, Shang-Wen Li, Luke Zettlemoyer, Shih-Fu Chang, Wen-Tau Yih, Hu Xu

Abstract: The success of contrastive language-image pretraining (CLIP) relies on the supervision from the pairing between images and captions, which tends to be noisy in web-crawled data. We present Mixture of Data Experts (MoDE) and learn a system of CLIP data experts via clustering. Each data expert is trained on one data cluster, being less sensitive to false negative noises in other clusters. At inferen… ▽ More The success of contrastive language-image pretraining (CLIP) relies on the supervision from the pairing between images and captions, which tends to be noisy in web-crawled data. We present Mixture of Data Experts (MoDE) and learn a system of CLIP data experts via clustering. Each data expert is trained on one data cluster, being less sensitive to false negative noises in other clusters. At inference time, we ensemble their outputs by applying weights determined through the correlation between task metadata and cluster conditions. To estimate the correlation precisely, the samples in one cluster should be semantically similar, but the number of data experts should still be reasonable for training and inference. As such, we consider the ontology in human language and propose to use fine-grained cluster centers to represent each data expert at a coarse-grained level. Experimental studies show that four CLIP data experts on ViT-B/16 outperform the ViT-L/14 by OpenAI CLIP and OpenCLIP on zero-shot image classification but with less ($<$35\%) training cost. Meanwhile, MoDE can train all data expert asynchronously and can flexibly include new data experts. The code is available at https://github.com/facebookresearch/MetaCLIP/tree/main/mode. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: IEEE CVPR 2024 Camera Ready. Code Link: https://github.com/facebookresearch/MetaCLIP/tree/main/mode

Squeezed Displaced Schrödinger-cat state as a signature of the PT-symmetry phase transition

Authors: Yuetao Chen, Shoukang Chang, Shaoyan Gao

Abstract: Parity-time (PT ) symmetric systems are gain-loss systems whose dynamics are governed by non-Hermitian Hamiltonians with degeneracies at exceptional-points (EPs) and has been studied in various photonic, electrical, mechanical systems, and so on. However, it is still an open question how to capture PT symmetry phase transition in electronic system where the transport properties of electron will be… ▽ More Parity-time (PT ) symmetric systems are gain-loss systems whose dynamics are governed by non-Hermitian Hamiltonians with degeneracies at exceptional-points (EPs) and has been studied in various photonic, electrical, mechanical systems, and so on. However, it is still an open question how to capture PT symmetry phase transition in electronic system where the transport properties of electron will be dramatically effected. Fortunately, the hybridization between photon and electron offers a novel way not only to control but also probe material properties. Here, we investigate a cavity coupled to a non-Hermitian Su-Schrieffer-Heeger (SSH) chain within mean-field ansatzs. We find that Squeezed Displaced Schrodinger cat (SDSc) will emerge with high fidelity in cavity ground state when PT -symmetry is broken and the fidelity will experience a sharp drop from almost 1 to 0 as PT symmetry recovers. Additionally, in semiclassical limit, we find that there exists local extrema at two sides of $x=0$ in semiclassical photon Hamiltonian $H_{\rm eff}(x, p)$, a clear signature of the emergence of SDSc state in cavity ground state. Thus, the appearance of SDSc state can be used to capture PT-symmetry phase transition which can not be modified by cavity mode. Besides, we exploit the cavity ground state to estimate the phase in the optical interferometer, and show that the quantum Fisher information and nonclassicality will sharply decline at EPs. This reveals that PT-symmetry breaking in electronic materials can also be captured by the quantum Fisher information and nonclassicality in phase estimation. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Ultrasound Nodule Segmentation Using Asymmetric Learning with Simple Clinical Annotation

Authors: Xingyue Zhao, Zhongyu Li, Xiangde Luo, Peiqi Li, Peng Huang, Jianwei Zhu, Yang Liu, Jihua Zhu, Meng Yang, Shi Chang, Jun Dong

Abstract: Recent advances in deep learning have greatly facilitated the automated segmentation of ultrasound images, which is essential for nodule morphological analysis. Nevertheless, most existing methods depend on extensive and precise annotations by domain experts, which are labor-intensive and time-consuming. In this study, we suggest using simple aspect ratio annotations directly from ultrasound clini… ▽ More Recent advances in deep learning have greatly facilitated the automated segmentation of ultrasound images, which is essential for nodule morphological analysis. Nevertheless, most existing methods depend on extensive and precise annotations by domain experts, which are labor-intensive and time-consuming. In this study, we suggest using simple aspect ratio annotations directly from ultrasound clinical diagnoses for automated nodule segmentation. Especially, an asymmetric learning framework is developed by extending the aspect ratio annotations with two types of pseudo labels, i.e., conservative labels and radical labels, to train two asymmetric segmentation networks simultaneously. Subsequently, a conservative-radical-balance strategy (CRBS) strategy is proposed to complementally combine radical and conservative labels. An inconsistency-aware dynamically mixed pseudo-labels supervision (IDMPS) module is introduced to address the challenges of over-segmentation and under-segmentation caused by the two types of labels. To further leverage the spatial prior knowledge provided by clinical annotations, we also present a novel loss function namely the clinical anatomy prior loss. Extensive experiments on two clinically collected ultrasound datasets (thyroid and breast) demonstrate the superior performance of our proposed method, which can achieve comparable and even better performance than fully supervised methods using ground truth annotations. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Comments: Accepted by TCSVT

Noiseless linear amplification-based quantum Ziv-Zakai bound for phase estimation and its Heisenberg error limits in noisy scenarios

Authors: Wei Ye, Peng Xiao, Xiaofan Xu, Xiang Zhu, Yunbin Yan, Lu Wang, Jie Ren, Yuxuan Zhu, Ying Xia, Xuan Rao, Shoukang Chang

Abstract: In this work, we address the central problem about how to effectively find the available precision limit of unknown parameters. In the framework of the quantum Ziv-Zakai bound (QZZB), we employ noiseless linear amplification (NLA)techniques to an initial coherent state (CS) as the probe state, and focus on whether the phase estimation performance is improved significantly in noisy scenarios, invol… ▽ More In this work, we address the central problem about how to effectively find the available precision limit of unknown parameters. In the framework of the quantum Ziv-Zakai bound (QZZB), we employ noiseless linear amplification (NLA)techniques to an initial coherent state (CS) as the probe state, and focus on whether the phase estimation performance is improved significantly in noisy scenarios, involving the photon-loss and phase-diffusion cases. More importantly, we also obtain two kinds of Heisenberg error limits of the QZZB with the NLA-based CS in these noisy scenarios, making comparisons with both the Margolus-Levitin (ML) type bound and the Mandelstam-Tamm (MT) type bound. Our analytical results show that in cases of photon loss and phase diffusion, the phase estimation performance of the QZZB can be improved remarkably by increasing the NLA gain factor. Particularly, the improvement is more pronounced with severe photon losses. Furthermore in minimal photon losses, our Heisenberg error limit shows better compactness than the cases of the ML-type and MT-type bounds. Our findings will provide an useful guidance for accomplishing more complex quantum information processing tasks. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Comments: 10 pages, 9 figures

Advancing the Robustness of Large Language Models through Self-Denoised Smoothing

Authors: Jiabao Ji, Bairu Hou, Zhen Zhang, Guanhua Zhang, Wenqi Fan, Qing Li, Yang Zhang, Gaowen Liu, Sijia Liu, Shiyu Chang

Abstract: Although large language models (LLMs) have achieved significant success, their vulnerability to adversarial perturbations, including recent jailbreak attacks, has raised considerable concerns. However, the increasing size of these models and their limited access make improving their robustness a challenging task. Among various defense strategies, randomized smoothing has shown great potential for… ▽ More Although large language models (LLMs) have achieved significant success, their vulnerability to adversarial perturbations, including recent jailbreak attacks, has raised considerable concerns. However, the increasing size of these models and their limited access make improving their robustness a challenging task. Among various defense strategies, randomized smoothing has shown great potential for LLMs, as it does not require full access to the model's parameters or fine-tuning via adversarial training. However, randomized smoothing involves adding noise to the input before model prediction, and the final model's robustness largely depends on the model's performance on these noise corrupted data. Its effectiveness is often limited by the model's sub-optimal performance on noisy data. To address this issue, we propose to leverage the multitasking nature of LLMs to first denoise the noisy inputs and then to make predictions based on these denoised versions. We call this procedure self-denoised smoothing. Unlike previous denoised smoothing techniques in computer vision, which require training a separate model to enhance the robustness of LLMs, our method offers significantly better efficiency and flexibility. Our experimental results indicate that our method surpasses existing methods in both empirical and certified robustness in defending against adversarial attacks for both downstream tasks and human alignments (i.e., jailbreak attacks). Our code is publicly available at https://github.com/UCSB-NLP-Chang/SelfDenoise △ Less

Submitted 18 April, 2024; originally announced April 2024.

Comments: Accepted by NAACL 2024. Jiabao, Bairu, Zhen, Guanhua contributed equally. This is an updated version of the paper: arXiv:2307.07171

Generalized Converses of Operator Jensens Inequalities with Applications to Hypercomplex Function Approximations and Bounds Algebra

Authors: Shih-Yu Chang

Abstract: Mond and Pecaric proposed a powerful method, namd as MP method, to deal with operator inequalities. However, this method requires a real-valued function to be convex or concave, and the normalized positive linear map between Hilbert spaces. The objective of this study is to extend the MP method by allowing non-convex or non-concave real-valued functions and nonlinear mapping between Hilbert spaces… ▽ More Mond and Pecaric proposed a powerful method, namd as MP method, to deal with operator inequalities. However, this method requires a real-valued function to be convex or concave, and the normalized positive linear map between Hilbert spaces. The objective of this study is to extend the MP method by allowing non-convex or non-concave real-valued functions and nonlinear mapping between Hilbert spaces. The Stone-Weierstrass theorem and Kantorovich function are fundamental components employed in generalizing the MP method inequality in this context. Several examples are presented to demonstrate the inequalities obtained from the conventional MP method by requiring convex function with a normalized positive linear map. Various new inequalities regarding hypercomplex functions, i.e., operator-valued functions with operators as arguments, are derived based on the proposed MP method. These inequalities are applied to approximate hypercomplex functions using ratio criteria and difference criteria. Another application of these new inequalities is to establish bounds for hypercomplex functions algebra, i.e., an abelian monoid for the addition or multiplication of hypercomplex functions, and to derive tail bounds for random tensors ensembles addition or multiplication systematically. △ Less

Submitted 17 April, 2024; originally announced April 2024.

Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models

Authors: Haotian Zhang, Haoxuan You, Philipp Dufter, Bowen Zhang, Chen Chen, Hong-You Chen, Tsu-Jui Fu, William Yang Wang, Shih-Fu Chang, Zhe Gan, Yinfei Yang

Abstract: While Ferret seamlessly integrates regional understanding into the Large Language Model (LLM) to facilitate its referring and grounding capability, it poses certain limitations: constrained by the pre-trained fixed visual encoder and failed to perform well on broader tasks. In this work, we unveil Ferret-v2, a significant upgrade to Ferret, with three key designs. (1) Any resolution grounding and… ▽ More While Ferret seamlessly integrates regional understanding into the Large Language Model (LLM) to facilitate its referring and grounding capability, it poses certain limitations: constrained by the pre-trained fixed visual encoder and failed to perform well on broader tasks. In this work, we unveil Ferret-v2, a significant upgrade to Ferret, with three key designs. (1) Any resolution grounding and referring: A flexible approach that effortlessly handles higher image resolution, improving the model's ability to process and understand images in greater detail. (2) Multi-granularity visual encoding: By integrating the additional DINOv2 encoder, the model learns better and diverse underlying contexts for global and fine-grained visual information. (3) A three-stage training paradigm: Besides image-caption alignment, an additional stage is proposed for high-resolution dense alignment before the final instruction tuning. Experiments show that Ferret-v2 provides substantial improvements over Ferret and other state-of-the-art methods, thanks to its high-resolution scaling and fine-grained visual processing. △ Less

Submitted 11 April, 2024; originally announced April 2024.

Comments: Preprint. 14 pages, 4 figures

HyperCLOVA X Technical Report

Authors: Kang Min Yoo, Jaegeun Han, Sookyo In, Heewon Jeon, Jisu Jeong, Jaewook Kang, Hyunwook Kim, Kyung-Min Kim, Munhyong Kim, Sungju Kim, Donghyun Kwak, Hanock Kwak, Se Jung Kwon, Bado Lee, Dongsoo Lee, Gichang Lee, Jooho Lee, Baeseong Park, Seongjin Shin, Joonsang Yu, Seolki Baek, Sumin Byeon, Eungsup Cho, Dooseok Choe, Jeesung Han , et al. (371 additional authors not shown)

Abstract: We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment t… ▽ More We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment to responsible AI. The model is evaluated across various benchmarks, including comprehensive reasoning, knowledge, commonsense, factuality, coding, math, chatting, instruction-following, and harmlessness, in both Korean and English. HyperCLOVA X exhibits strong reasoning capabilities in Korean backed by a deep understanding of the language and cultural nuances. Further analysis of the inherent bilingual nature and its extension to multilingualism highlights the model's cross-lingual proficiency and strong generalization ability to untargeted languages, including machine translation between several language pairs and cross-lingual inference tasks. We believe that HyperCLOVA X can provide helpful guidance for regions or countries in developing their sovereign LLMs. △ Less

Submitted 13 April, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Comments: 44 pages; updated authors list and fixed author names

RAP: Retrieval-Augmented Planner for Adaptive Procedure Planning in Instructional Videos

Authors: Ali Zare, Yulei Niu, Hammad Ayyubi, Shih-fu Chang

Abstract: Procedure Planning in instructional videos entails generating a sequence of action steps based on visual observations of the initial and target states. Despite the rapid progress in this task, there remain several critical challenges to be solved: (1) Adaptive procedures: Prior works hold an unrealistic assumption that the number of action steps is known and fixed, leading to non-generalizable mod… ▽ More Procedure Planning in instructional videos entails generating a sequence of action steps based on visual observations of the initial and target states. Despite the rapid progress in this task, there remain several critical challenges to be solved: (1) Adaptive procedures: Prior works hold an unrealistic assumption that the number of action steps is known and fixed, leading to non-generalizable models in real-world scenarios where the sequence length varies. (2) Temporal relation: Understanding the step temporal relation knowledge is essential in producing reasonable and executable plans. (3) Annotation cost: Annotating instructional videos with step-level labels (i.e., timestamp) or sequence-level labels (i.e., action category) is demanding and labor-intensive, limiting its generalizability to large-scale datasets.In this work, we propose a new and practical setting, called adaptive procedure planning in instructional videos, where the procedure length is not fixed or pre-determined. To address these challenges we introduce Retrieval-Augmented Planner (RAP) model. Specifically, for adaptive procedures, RAP adaptively determines the conclusion of actions using an auto-regressive model architecture. For temporal relation, RAP establishes an external memory module to explicitly retrieve the most relevant state-action pairs from the training videos and revises the generated procedures. To tackle high annotation cost, RAP utilizes a weakly-supervised learning manner to expand the training dataset to other task-relevant, unannotated videos by generating pseudo labels for action steps. Experiments on CrossTask and COIN benchmarks show the superiority of RAP over traditional fixed-length models, establishing it as a strong baseline solution for adaptive procedure planning. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Comments: 23 pages, 6 figures, 12 tables

A characterization on trees $T$ with $m(T, λ)=p(T)-2$

Authors: Sarula Chang, Jianxi Li, Yirong Zheng

Abstract: Let $m(G,λ)$ be the multiplicity of an eigenvalue $λ$ of a connected graph $G$. Wang et al. [Linear Algebra Appl. 584(2020), 257-266] proved that for any connected graph $G\neq C_n$, $m(G, λ) \leq 2c(G) + p(G) -1$, where $c (G) = |E(G)| - |V (G)| + 1$ and $p(G)$ are the cyclomatic number and the number of pendant vertices of $G$, respectively. In the same paper, they proposed the problem to charac… ▽ More Let $m(G,λ)$ be the multiplicity of an eigenvalue $λ$ of a connected graph $G$. Wang et al. [Linear Algebra Appl. 584(2020), 257-266] proved that for any connected graph $G\neq C_n$, $m(G, λ) \leq 2c(G) + p(G) -1$, where $c (G) = |E(G)| - |V (G)| + 1$ and $p(G)$ are the cyclomatic number and the number of pendant vertices of $G$, respectively. In the same paper, they proposed the problem to characterize all connected graphs $G$ with eigenvalue $λ$ such that $m(G, λ) =2c (G)+ p(G)-1$. Wong et al. [Discrete Math. 347(2024), 113845] solved this problem for the case when $G$ is a tree by characterizing all trees $T$ with eigenvalue $λ$ such that $m(T , λ) = p(T )-1$. In this paper, we further provide the structural characterization on trees $T$ with eigenvalue $λ$ such that $m(T , λ) = p(T )-2$. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Enhanced Short Text Modeling: Leveraging Large Language Models for Topic Refinement

Authors: Shuyu Chang, Rui Wang, Peng Ren, Haiping Huang

Abstract: Crafting effective topic models for brief texts, like tweets and news headlines, is essential for capturing the swift shifts in social dynamics. Traditional topic models, however, often fall short in accurately representing the semantic intricacies of short texts due to their brevity and lack of contextual data. In our study, we harness the advanced capabilities of Large Language Models (LLMs) to… ▽ More Crafting effective topic models for brief texts, like tweets and news headlines, is essential for capturing the swift shifts in social dynamics. Traditional topic models, however, often fall short in accurately representing the semantic intricacies of short texts due to their brevity and lack of contextual data. In our study, we harness the advanced capabilities of Large Language Models (LLMs) to introduce a novel approach termed "Topic Refinement". This approach does not directly involve itself in the initial modeling of topics but focuses on improving topics after they have been mined. By employing prompt engineering, we direct LLMs to eliminate off-topic words within a given topic, ensuring that only contextually relevant words are preserved or substituted with ones that fit better semantically. This method emulates human-like scrutiny and improvement of topics, thereby elevating the semantic quality of the topics generated by various models. Our comprehensive evaluation across three unique datasets has shown that our topic refinement approach significantly enhances the semantic coherence of topics. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Comments: 6 pages, 4 figures