Showing 1–24 of 24 results for author: Cai, W

Amplitude-Phase Fusion for Enhanced Electrocardiogram Morphological Analysis

Authors: Shuaicong Hu, Yanan Wang, Jian Liu, Jingyu Lin, Shengmei Qin, Zhenning Nie, Zhifeng Yao, Wenjie Cai, Cuiwei Yang

Abstract: Considering the variability of amplitude and phase patterns in electrocardiogram (ECG) signals due to cardiac activity and individual differences, existing entropy-based studies have not fully utilized these two patterns and lack integration. To address this gap, this paper proposes a novel fusion entropy metric, morphological ECG entropy (MEE) for the first time, specifically designed for ECG mor… ▽ More Considering the variability of amplitude and phase patterns in electrocardiogram (ECG) signals due to cardiac activity and individual differences, existing entropy-based studies have not fully utilized these two patterns and lack integration. To address this gap, this paper proposes a novel fusion entropy metric, morphological ECG entropy (MEE) for the first time, specifically designed for ECG morphology, to comprehensively describe the fusion of amplitude and phase patterns. MEE is computed based on beat-level samples, enabling detailed analysis of each cardiac cycle. Experimental results demonstrate that MEE achieves rapid, accurate, and label-free localization of abnormal ECG arrhythmia regions. Furthermore, MEE provides a method for assessing sample diversity, facilitating compression of imbalanced training sets (via representative sample selection), and outperforms random pruning. Additionally, MEE exhibits the ability to describe areas of poor quality. By discussing, it proves the robustness of MEE value calculation to noise interference and its low computational complexity. Finally, we integrate this method into a clinical interactive interface to provide a more convenient and intuitive user experience. These findings indicate that MEE serves as a valuable clinical descriptor for ECG characterization. The implementation code can be referenced at the following link: https://github.com/fdu-harry/ECG-MEE-metric. △ Less

Submitted 15 April, 2024; originally announced April 2024.

Comments: 16 pages, 12 figures

ACM Class: I.5.2

Optimal coordination in Minority Game: A solution from reinforcement learning

Authors: Guozhong Zheng, Weiran Cai, Guanxiao Qi, Jiqiang Zhang, Li Chen

Abstract: Efficient allocation is important in nature and human society where individuals often compete for finite resources. The Minority Game is perhaps the simplest model that provides deep insights into how human coordinate to maximize the resource utilization. However, this model assumes the static strategies that are provided a priori, failing to capture their adaptive nature. Here, we turn to the par… ▽ More Efficient allocation is important in nature and human society where individuals often compete for finite resources. The Minority Game is perhaps the simplest model that provides deep insights into how human coordinate to maximize the resource utilization. However, this model assumes the static strategies that are provided a priori, failing to capture their adaptive nature. Here, we turn to the paradigm of reinforcement learning, where individuals' strategies are evolving by evaluating both the past experience and rewards in the future. Specifically, we adopt the Q-learning algorithm, each player is endowed with a Q-table that guides their decision-making. We reveal that the population is able to reach the optimal allocation when individuals appreciate both the past experience and rewards in the future, and they are able to balance the exploitation of their Q-tables and the exploration by randomly acting. The optimal allocation is ruined when individuals tend to use either exploitation-only or exploration-only, where only partial coordination and even anti-coordination are observed. Mechanism analysis reveals that a moderate level of exploration can escape local minimums of metastable periodic states, and reaches the optimal coordination as the global minimum. Interestingly, the optimal coordination is underlined by a symmetry-breaking of action preferences, where nearly half of the population choose one side while the other half prefer the other side. The emergence of optimal coordination is robust to the population size and other game parameters. Our work therefore provides a natural solution to the Minority Game and sheds insights into the resource allocation problem in general. Besides, our work demonstrates the potential of the proposed reinforcement learning paradigm in deciphering many puzzles in the socio-economic context. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: 10 pages, 7 figures, 1 table. A working paper, comments are welcome

Decoding trust: A reinforcement learning perspective

Authors: Guozhong Zheng, Jiqiang Zhang, Jing Zhang, Weiran Cai, Li Chen

Abstract: Behavioral experiments on the trust game have shown that trust and trustworthiness are universal among human beings, contradicting the prediction by assuming \emph{Homo economicus} in orthodox Economics. This means some mechanism must be at work that favors their emergence. Most previous explanations however need to resort to some factors based upon imitative learning, a simple version of social l… ▽ More Behavioral experiments on the trust game have shown that trust and trustworthiness are universal among human beings, contradicting the prediction by assuming \emph{Homo economicus} in orthodox Economics. This means some mechanism must be at work that favors their emergence. Most previous explanations however need to resort to some factors based upon imitative learning, a simple version of social learning. Here, we turn to the paradigm of reinforcement learning, where individuals update their strategies by evaluating the long-term return through accumulated experience. Specifically, we investigate the trust game with the Q-learning algorithm, where each participant is associated with two evolving Q-tables that guide one's decision making as trustor and trustee respectively. In the pairwise scenario, we reveal that high levels of trust and trustworthiness emerge when individuals appreciate both their historical experience and returns in the future. Mechanistically, the evolution of the Q-tables shows a crossover that resembles human's psychological changes. We also provide the phase diagram for the game parameters, where the boundary analysis is conducted. These findings are robust when the scenario is extended to a latticed population. Our results thus provide a natural explanation for the emergence of trust and trustworthiness without external factors involved. More importantly, the proposed paradigm shows the potential in deciphering many puzzles in human behaviors. △ Less

Submitted 26 November, 2023; v1 submitted 25 September, 2023; originally announced September 2023.

Comments: 12 pages, 11 figures. Comments are appreciated

Adaptive Sampling Strategies to Construct Equitable Training Datasets

Authors: William Cai, Ro Encarnacion, Bobbie Chern, Sam Corbett-Davies, Miranda Bogen, Stevie Bergman, Sharad Goel

Abstract: In domains ranging from computer vision to natural language processing, machine learning models have been shown to exhibit stark disparities, often performing worse for members of traditionally underserved groups. One factor contributing to these performance gaps is a lack of representation in the data the models are trained on. It is often unclear, however, how to operationalize representativenes… ▽ More In domains ranging from computer vision to natural language processing, machine learning models have been shown to exhibit stark disparities, often performing worse for members of traditionally underserved groups. One factor contributing to these performance gaps is a lack of representation in the data the models are trained on. It is often unclear, however, how to operationalize representativeness in specific applications. Here we formalize the problem of creating equitable training datasets, and propose a statistical framework for addressing this problem. We consider a setting where a model builder must decide how to allocate a fixed data collection budget to gather training data from different subgroups. We then frame dataset creation as a constrained optimization problem, in which one maximizes a function of group-specific performance metrics based on (estimated) group-specific learning rates and costs per sample. This flexible approach incorporates preferences of model-builders and other stakeholders, as well as the statistical properties of the learning task. When data collection decisions are made sequentially, we show that under certain conditions this optimization problem can be efficiently solved even without prior knowledge of the learning rates. To illustrate our approach, we conduct a simulation study of polygenic risk scores on synthetic genomic data -- an application domain that often suffers from non-representative data collection. We find that our adaptive sampling strategy outperforms several common data collection heuristics, including equal and proportional sampling, demonstrating the value of strategic dataset design for building equitable models. △ Less

Submitted 31 January, 2022; originally announced February 2022.

Comments: 15 pages, 2 figures

ACM Class: I.2.0; F.2.0

Detecting chaos in hurricane intensity

Authors: Chanh Kieu, Weiran Cai, Wai-Tong, Fan

Abstract: Determining the maximum potential limit in the accuracy of hurricane intensity prediction is important for operational practice. Using the phase-space reconstruction method for hurricane intensity time series, here we found that hurricane dynamics contain inherent low-dimensional chaos at the maximum intensity equilibrium. Examination of several chaotic invariants including the largest Lyapunov ex… ▽ More Determining the maximum potential limit in the accuracy of hurricane intensity prediction is important for operational practice. Using the phase-space reconstruction method for hurricane intensity time series, here we found that hurricane dynamics contain inherent low-dimensional chaos at the maximum intensity equilibrium. Examination of several chaotic invariants including the largest Lyapunov exponent, the Sugihara-May correlation, and the correlation dimension consistently captures an intrinsic dimension of the hurricane chaotic attractor in the range of 4-5. In addition, the error doubling time is roughly 1-5 hours, which accords with the decay time obtained from the Sugihara-May correlation. The confirmation of hurricane chaotic intensity as found in this study suggests a relatively short limit for intensity predictability of $\sim$18-24 hours after reaching the maximum intensity stage. So long as the traditional metrics for hurricane intensity such as the maximum surface wind or the minimum central pressure is used for intensity forecast, our results support that hurricane intensity forecast errors will not be reduced indefinitely in any modelling systems, even in the absence of all model and observational errors. As such, the future improvement of hurricane intensity forecast should be based on different intensity metric beyond the absolute intensity errors as in the current practice of intensity verification. △ Less

Submitted 6 October, 2021; originally announced October 2021.

Comments: 20 pages, 5 figures

Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications

Authors: Yang Ma, Chaoyi Zhang, Mariano Cabezas, Yang Song, Zihao Tang, Dongnan Liu, Weidong Cai, Michael Barnett, Chenyu Wang

Abstract: Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patient's neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and catego… ▽ More Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patient's neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings. △ Less

Submitted 27 January, 2022; v1 submitted 20 April, 2021; originally announced April 2021.

Comments: Accepted to appear in IEEE Journal of Biomedical And Health Informatics

Quantifying the Global Support Network for Non-State Armed Groups (NAGs)

Authors: Weiran Cai, Belgin San-Akca, Jordan Snyder, Grayson Gordon, Zeev Maoz, Raissa M. D'Souza

Abstract: Human history has been shaped by armed conflicts. Rather than large-scale interstate wars, low-intensity attacks have been more prevalent in the post-World War era. These attacks are often carried out by non-state armed groups (NAGs), which are supported by host states (HSs). We analyze the global bipartite network of NAG-HS support and its evolution over the period of 1945-2010. We find striking… ▽ More Human history has been shaped by armed conflicts. Rather than large-scale interstate wars, low-intensity attacks have been more prevalent in the post-World War era. These attacks are often carried out by non-state armed groups (NAGs), which are supported by host states (HSs). We analyze the global bipartite network of NAG-HS support and its evolution over the period of 1945-2010. We find striking parallels to ecological networks such as mutualistic and parasitic forms of support, and a nested and modular network architecture. The nestedness emerges from preferential behaviors: highly connected players are more likely to both gain and lose connections. Long-persisting major modules are identified, reflecting both regional and trans-regional interests, which show significant turnover in their membership, contrary to the transitory ones. Revealing this architecture further enables the identification of actor's roles and provide insights for effective intervention strategies. △ Less

Submitted 31 January, 2021; originally announced February 2021.

Comments: 13 pages, 6 figures

A Degradation Performance Model With Mixed-type Covariates and Latent Heterogeneity

Authors: Xuxue Sun, Wenjun Cai, Qiong Zhang, Mingyang Li

Abstract: Successful modeling of degradation performance data is essential for accurate reliability assessment and failure predictions of highly reliable product units. The degradation performance measurements over time are highly heterogeneous. Such heterogeneity can be partially attributed to external factors, such as accelerated/environmental conditions, and can also be attributed to internal factors, su… ▽ More Successful modeling of degradation performance data is essential for accurate reliability assessment and failure predictions of highly reliable product units. The degradation performance measurements over time are highly heterogeneous. Such heterogeneity can be partially attributed to external factors, such as accelerated/environmental conditions, and can also be attributed to internal factors, such as material microstructure characteristics of product units. The latent heterogeneity due to the unobserved/unknown factors shared within each product unit may also exists and need to be considered as well. Existing degradation models often fail to consider (i) the influence of both external accelerated/environmental conditions and internal material information, (ii) the influence of unobserved/unknown factors within each unit. In this work, we propose a generic degradation performance modeling framework with mixed-type covariates and latent heterogeneity to account for both influences of observed internal and external factors as well as unobserved factors. Effective estimation algorithm is also developed to jointly quantify the influences of mixed-type covariates and individual latent heterogeneity, and also to examine the potential interaction between mixed-type covariates. Functional data analysis and data augmentation techniques are employed to address a series of estimation issues. A real case study is further provided to demonstrate the superior performance of the proposed approach over several alternative modeling approaches. Besides, the proposed degradation performance modeling framework also provides interpretable findings. △ Less

Submitted 10 January, 2021; originally announced January 2021.

Comments: 24 pages

NP-ODE: Neural Process Aided Ordinary Differential Equations for Uncertainty Quantification of Finite Element Analysis

Authors: Yinan Wang, Kaiwen Wang, Wenjun Cai, Xiaowei Yue

Abstract: Finite element analysis (FEA) has been widely used to generate simulations of complex and nonlinear systems. Despite its strength and accuracy, the limitations of FEA can be summarized into two aspects: a) running high-fidelity FEA often requires significant computational cost and consumes a large amount of time; b) FEA is a deterministic method that is insufficient for uncertainty quantification… ▽ More Finite element analysis (FEA) has been widely used to generate simulations of complex and nonlinear systems. Despite its strength and accuracy, the limitations of FEA can be summarized into two aspects: a) running high-fidelity FEA often requires significant computational cost and consumes a large amount of time; b) FEA is a deterministic method that is insufficient for uncertainty quantification (UQ) when modeling complex systems with various types of uncertainties. In this paper, a physics-informed data-driven surrogate model, named Neural Process Aided Ordinary Differential Equation (NP-ODE), is proposed to model the FEA simulations and capture both input and output uncertainties. To validate the advantages of the proposed NP-ODE, we conduct experiments on both the simulation data generated from a given ordinary differential equation and the data collected from a real FEA platform for tribocorrosion. The performances of the proposed NP-ODE and several benchmark methods are compared. The results show that the proposed NP-ODE outperforms benchmark methods. The NP-ODE method realizes the smallest predictive error as well as generates the most reasonable confidence interval having the best coverage on testing data points. △ Less

Submitted 12 December, 2020; originally announced December 2020.

Comments: 40 pages

A Causal Framework for Observational Studies of Discrimination

Authors: Johann Gaebler, William Cai, Guillaume Basse, Ravi Shroff, Sharad Goel, Jennifer Hill

Abstract: In studies of discrimination, researchers often seek to estimate a causal effect of race or gender on outcomes. For example, in the criminal justice context, one might ask whether arrested individuals would have been subsequently charged or convicted had they been a different race. It has long been known that such counterfactual questions face measurement challenges related to omitted-variable bia… ▽ More In studies of discrimination, researchers often seek to estimate a causal effect of race or gender on outcomes. For example, in the criminal justice context, one might ask whether arrested individuals would have been subsequently charged or convicted had they been a different race. It has long been known that such counterfactual questions face measurement challenges related to omitted-variable bias, and conceptual challenges related to the definition of causal estimands for largely immutable characteristics. Another concern, which has been the subject of recent debates, is post-treatment bias: many studies of discrimination condition on apparently intermediate outcomes, like being arrested, that themselves may be the product of discrimination, potentially corrupting statistical estimates. There is, however, reason to be optimistic. By carefully defining the estimand -- and by considering the precise timing of events -- we show that a primary causal quantity of interest in discrimination studies can be estimated under an ignorability condition that may hold approximately in some observational settings. We illustrate these ideas by analyzing both simulated data and the charging decisions of a prosecutor's office in a large county in the United States. △ Less

Submitted 4 April, 2022; v1 submitted 22 June, 2020; originally announced June 2020.

Journal ref: Statistics and Public Policy, 9 (2022) 26-48

Targeting Learning: Robust Statistics for Reproducible Research

Authors: Jeremy R. Coyle, Nima S. Hejazi, Ivana Malenica, Rachael V. Phillips, Benjamin F. Arnold, Andrew Mertens, Jade Benjamin-Chung, Weixin Cai, Sonali Dayal, John M. Colford Jr., Alan E. Hubbard, Mark J. van der Laan

Abstract: Targeted Learning is a subfield of statistics that unifies advances in causal inference, machine learning and statistical theory to help answer scientifically impactful questions with statistical confidence. Targeted Learning is driven by complex problems in data science and has been implemented in a diversity of real-world scenarios: observational studies with missing treatments and outcomes, per… ▽ More Targeted Learning is a subfield of statistics that unifies advances in causal inference, machine learning and statistical theory to help answer scientifically impactful questions with statistical confidence. Targeted Learning is driven by complex problems in data science and has been implemented in a diversity of real-world scenarios: observational studies with missing treatments and outcomes, personalized interventions, longitudinal settings with time-varying treatment regimes, survival analysis, adaptive randomized trials, mediation analysis, and networks of connected subjects. In contrast to the (mis)application of restrictive modeling strategies that dominate the current practice of statistics, Targeted Learning establishes a principled standard for statistical estimation and inference (i.e., confidence intervals and p-values). This multiply robust approach is accompanied by a guiding roadmap and a burgeoning software ecosystem, both of which provide guidance on the construction of estimators optimized to best answer the motivating question. The roadmap of Targeted Learning emphasizes tailoring statistical procedures so as to minimize their assumptions, carefully grounding them only in the scientific knowledge available. The end result is a framework that honestly reflects the uncertainty in both the background knowledge and the available data in order to draw reliable conclusions from statistical analyses - ultimately enhancing the reproducibility and rigor of scientific findings. △ Less

Submitted 12 June, 2020; originally announced June 2020.

Comments: 25 pages, 3 figures

MSC Class: 62A01 ACM Class: G.3

DIHARD II is Still Hard: Experimental Results and Discussions from the DKU-LENOVO Team

Authors: Qingjian Lin, Weicheng Cai, Lin Yang, Junjie Wang, Jun Zhang, Ming Li

Abstract: In this paper, we present the submitted system for the second DIHARD Speech Diarization Challenge from the DKULENOVO team. Our diarization system includes multiple modules, namely voice activity detection (VAD), segmentation, speaker embedding extraction, similarity scoring, clustering, resegmentation and overlap detection. For each module, we explore different techniques to enhance performance. O… ▽ More In this paper, we present the submitted system for the second DIHARD Speech Diarization Challenge from the DKULENOVO team. Our diarization system includes multiple modules, namely voice activity detection (VAD), segmentation, speaker embedding extraction, similarity scoring, clustering, resegmentation and overlap detection. For each module, we explore different techniques to enhance performance. Our final submission employs the ResNet-LSTM based VAD, the Deep ResNet based speaker embedding, the LSTM based similarity scoring and spectral clustering. Variational Bayes (VB) diarization is applied in the resegmentation stage and overlap detection also brings slight improvement. Our proposed system achieves 18.84% DER in Track1 and 27.90% DER in Track2. Although our systems have reduced the DERs by 27.5% and 31.7% relatively against the official baselines, we believe that the diarization task is still very difficult. △ Less

Submitted 4 May, 2020; v1 submitted 23 February, 2020; originally announced February 2020.

Comments: Submitted to Odyssesy 2020

Sequential Selection for Accelerated Life Testing via Approximate Bayesian Inference

Authors: Ye Chen, Qiong Zhang, Mingyang Li, Wenjun Cai

Abstract: Accelerated life testing (ALT) is typically used to assess the reliability of material's lifetime under desired stress levels. Recent advances in material engineering have made a variety of material alternatives readily available. To identify the most reliable material setting with efficient experimental design, a sequential test planning strategy is preferred. To guarantee a tractable statistical… ▽ More Accelerated life testing (ALT) is typically used to assess the reliability of material's lifetime under desired stress levels. Recent advances in material engineering have made a variety of material alternatives readily available. To identify the most reliable material setting with efficient experimental design, a sequential test planning strategy is preferred. To guarantee a tractable statistical mechanism for information collection and update, we develop explicit model parameter update formulas via approximate Bayesian inference. Theories show that our explicit update formulas give consistent parameter estimates. Simulation study and a case study show that the proposed sequential selection approach can significantly improve the probability of identifying the material alternative with best reliability performance over other design approaches. △ Less

Submitted 15 January, 2020; originally announced January 2020.

Multi-scale Deep Neural Networks for Solving High Dimensional PDEs

Authors: Wei Cai, Zhi-Qin John Xu

Abstract: In this paper, we propose the idea of radial scaling in frequency domain and activation functions with compact support to produce a multi-scale DNN (MscaleDNN), which will have the multi-scale capability in approximating high frequency and high dimensional functions and speeding up the solution of high dimensional PDEs. Numerical results on high dimensional function fitting and solutions of high d… ▽ More In this paper, we propose the idea of radial scaling in frequency domain and activation functions with compact support to produce a multi-scale DNN (MscaleDNN), which will have the multi-scale capability in approximating high frequency and high dimensional functions and speeding up the solution of high dimensional PDEs. Numerical results on high dimensional function fitting and solutions of high dimensional PDEs, using loss functions with either Ritz energy or least squared PDE residuals, have validated the increased power of multi-scale resolution and high frequency capturing of the proposed MscaleDNN. △ Less

Submitted 25 October, 2019; originally announced October 2019.

Efficient Estimation of Pathwise Differentiable Target Parameters with the Undersmoothed Highly Adaptive Lasso

Authors: Mark J. van der Laan, David Benkeser, Weixin Cai

Abstract: We consider estimation of a functional parameter of a realistically modeled data distribution based on observing independent and identically distributed observations. We define an $m$-th order Spline Highly Adaptive Lasso Minimum Loss Estimator (Spline HAL-MLE) of a functional parameter that is defined by minimizing the empirical risk function over an $m$-th order smoothness class of functions. We… ▽ More We consider estimation of a functional parameter of a realistically modeled data distribution based on observing independent and identically distributed observations. We define an $m$-th order Spline Highly Adaptive Lasso Minimum Loss Estimator (Spline HAL-MLE) of a functional parameter that is defined by minimizing the empirical risk function over an $m$-th order smoothness class of functions. We show that this $m$-th order smoothness class consists of all functions that can be represented as an infinitesimal linear combination of tensor products of $\leq m$-th order spline-basis functions, and involves assuming $m$-derivatives in each coordinate. By selecting $m$ with cross-validation we obtain a Spline-HAL-MLE that is able to adapt to the underlying unknown smoothness of the true function, while guaranteeing a rate of convergence faster than $n^{-1/4}$, as long as the true function is cadlag (right-continuous with left-hand limits) and has finite sectional variation norm. The $m=0$-smoothness class consists of all cadlag functions with finite sectional variation norm and corresponds with the original HAL-MLE defined in van der Laan (2015). In this article we establish that this Spline-HAL-MLE yields an asymptotically efficient estimator of any smooth feature of the functional parameter under an easily verifiable global undersmoothing condition. A sufficient condition for the latter condition is that the minimum of the empirical mean of the selected basis functions is smaller than a constant times $n^{-1/2}$, which is not parameter specific and enforces the selection of the $L_1$-norm in the lasso to be large enough to include sparsely supported basis. We demonstrate our general result for the $m=0$-HAL-MLE of the average treatment effect and of the integral of the square of the data density. We also present simulations for these two examples confirming the theory. △ Less

Submitted 2 July, 2021; v1 submitted 14 August, 2019; originally announced August 2019.

Weight Normalization based Quantization for Deep Neural Network Compression

Authors: Wen-Pu Cai, Wu-Jun Li

Abstract: With the development of deep neural networks, the size of network models becomes larger and larger. Model compression has become an urgent need for deploying these network models to mobile or embedded devices. Model quantization is a representative model compression technique. Although a lot of quantization methods have been proposed, many of them suffer from a high quantization error caused by a… ▽ More With the development of deep neural networks, the size of network models becomes larger and larger. Model compression has become an urgent need for deploying these network models to mobile or embedded devices. Model quantization is a representative model compression technique. Although a lot of quantization methods have been proposed, many of them suffer from a high quantization error caused by a long-tail distribution of network weights. In this paper, we propose a novel quantization method, called weight normalization based quantization (WNQ), for model compression. WNQ adopts weight normalization to avoid the long-tail distribution of network weights and subsequently reduces the quantization error. Experiments on CIFAR-100 and ImageNet show that WNQ can outperform other baselines to achieve state-of-the-art performance. △ Less

Submitted 1 July, 2019; originally announced July 2019.

Comments: 10 pages, 5 figures

Nonparametric Bootstrap Inference for the Targeted Highly Adaptive LASSO Estimator

Authors: Weixin Cai, Mark van der Laan

Abstract: The Highly-Adaptive-LASSO Targeted Minimum Loss Estimator (HAL-TMLE) is an efficient plug-in estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance functional parameters (i.e., the relevant part of data distribution) are finite. It relies on an initial estimator (HAL-MLE) of the nuis… ▽ More The Highly-Adaptive-LASSO Targeted Minimum Loss Estimator (HAL-TMLE) is an efficient plug-in estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance functional parameters (i.e., the relevant part of data distribution) are finite. It relies on an initial estimator (HAL-MLE) of the nuisance functional parameters by minimizing the empirical risk over the parameter space under the constraint that the sectional variation norm of the candidate functions are bounded by a constant, where this constant can be selected with cross-validation. In this article, we establish that the nonparametric bootstrap for the HAL-TMLE, fixing the value of the sectional variation norm at a value larger or equal than the cross-validation selector, provides a consistent method for estimating the normal limit distribution of the HAL-TMLE. In order to optimize the finite sample coverage of the nonparametric bootstrap confidence intervals, we propose a selection method for this sectional variation norm that is based on running the nonparametric bootstrap for all values of the sectional variation norm larger than the one selected by cross-validation, and subsequently determining a value at which the width of the resulting confidence intervals reaches a plateau. We demonstrate our method for 1) nonparametric estimation of the average treatment effect based on observing on each unit a covariate vector, binary treatment, and outcome, and for 2) nonparametric estimation of the integral of the square of the multivariate density of the data distribution. In addition, we also present simulation results for these two examples demonstrating the excellent finite sample coverage of bootstrap-based confidence intervals. △ Less

Submitted 7 February, 2020; v1 submitted 23 May, 2019; originally announced May 2019.

Comments: arXiv admin note: substantial text overlap with arXiv:1708.09502

PhaseDNN - A Parallel Phase Shift Deep Neural Network for Adaptive Wideband Learning

Authors: Wei Cai, Xiaoguang Li, Lizuo Liu

Abstract: In this paper, we propose a phase shift deep neural network (PhaseDNN) which provides a wideband convergence in approximating a high dimensional function during its training of the network. The PhaseDNN utilizes the fact that many DNN achieves convergence in the low frequency range first, thus, a series of moderately-sized of DNNs are constructed and trained in parallel for ranges of higher freque… ▽ More In this paper, we propose a phase shift deep neural network (PhaseDNN) which provides a wideband convergence in approximating a high dimensional function during its training of the network. The PhaseDNN utilizes the fact that many DNN achieves convergence in the low frequency range first, thus, a series of moderately-sized of DNNs are constructed and trained in parallel for ranges of higher frequencies. With the help of phase shifts in the frequency domain, implemented through a simple phase factor multiplication on the training data, each DNN in the series will be trained to approximate the target function's higher frequency content over a specific range. Due to the phase shift, each DNN achieves the speed of convergence as in the low frequency range. As a result, the proposed PhaseDNN system is able to convert wideband frequency learning to low frequency learning, thus allowing a uniform learning to wideband high dimensional functions with frequency adaptive training. Numerical results have demonstrated the capability of PhaseDNN in learning information of a target function from low to high frequency uniformly. △ Less

Submitted 10 May, 2019; v1 submitted 3 May, 2019; originally announced May 2019.

MSC Class: 68Q32; 68T01

A nonparametric super-efficient estimator of the average treatment effect

Authors: David Benkeser, Weixin Cai, Mark J van der Laan

Abstract: Doubly robust estimators of causal effects are a popular means of estimating causal effects. Such estimators combine an estimate of the conditional mean of the outcome given treatment and confounders (the so-called outcome regression) with an estimate of the conditional probability of treatment given confounders (the propensity score) to generate an estimate of the effect of interest. In addition… ▽ More Doubly robust estimators of causal effects are a popular means of estimating causal effects. Such estimators combine an estimate of the conditional mean of the outcome given treatment and confounders (the so-called outcome regression) with an estimate of the conditional probability of treatment given confounders (the propensity score) to generate an estimate of the effect of interest. In addition to enjoying the double-robustness property, these estimators have additional benefits. First, flexible regression tools, such as those developed in the field of machine learning, can be utilized to estimate the relevant regressions, while the estimators of the treatment effects retain desirable statistical properties. Furthermore, these estimators are often statistically efficient, achieving the lower bound on the variance of regular, asymptotically linear estimators. However, in spite of their asymptotic optimality, in problems where causal estimands are weakly identifiable, these estimators may behave erratically. We propose two new estimation techniques for use in these challenging settings. Our estimators build on two existing frameworks for efficient estimation: targeted minimum loss estimation and one-step estimation. However, rather than using an estimate of the propensity score in their construction, we instead opt for an alternative regression quantity when building our estimators: the conditional probability of treatment given the conditional mean outcome. We discuss the theoretical implications and demonstrate the estimators' performance in simulated and real data. △ Less

Submitted 15 January, 2019; originally announced January 2019.

Application of the novel fractional grey model FAGMO(1,1,k) to predict China's nuclear energy consumption

Authors: Wenqing Wu, Xin Ma, Bo Zeng, Yong Wang, Wei Cai

Abstract: At present, the energy structure of China is shifting towards cleaner and lower amounts of carbon fuel, driven by environmental needs and technological advances. Nuclear energy, which is one of the major low-carbon resources, plays a key role in China's clean energy development. To formulate appropriate energy policies, it is necessary to conduct reliable forecasts. This paper discusses the nuclea… ▽ More At present, the energy structure of China is shifting towards cleaner and lower amounts of carbon fuel, driven by environmental needs and technological advances. Nuclear energy, which is one of the major low-carbon resources, plays a key role in China's clean energy development. To formulate appropriate energy policies, it is necessary to conduct reliable forecasts. This paper discusses the nuclear energy consumption of China by means of a novel fractional grey model FAGMO(1,1,k). The fractional accumulated generating matrix is introduced to analyse the fractional grey model properties. Thereafter, the modelling procedures of the FAGMO(1,1,k) are presented in detail, along with the transforms of its optimal parameters. A stochastic testing scheme is provided to validate the accuracy and properties of the optimal parameters of the FAGMO(1,1,k). Finally, this model is used to forecast China's nuclear energy consumption and the results demonstrate that the FAGMO(1,1,k) model provides accurate prediction, outperforming other grey models. △ Less

Submitted 28 September, 2018; originally announced September 2018.

Journal ref: Energy, 2018, vol. 165, pp. 223-234

TequilaGAN: How to easily identify GAN samples

Authors: Rafael Valle, Wilson Cai, Anish Doshi

Abstract: In this paper we show strategies to easily identify fake samples generated with the Generative Adversarial Network framework. One strategy is based on the statistical analysis and comparison of raw pixel values and features extracted from them. The other strategy learns formal specifications from the real data and shows that fake samples violate the specifications of the real data. We show that fa… ▽ More In this paper we show strategies to easily identify fake samples generated with the Generative Adversarial Network framework. One strategy is based on the statistical analysis and comparison of raw pixel values and features extracted from them. The other strategy learns formal specifications from the real data and shows that fake samples violate the specifications of the real data. We show that fake samples produced with GANs have a universal signature that can be used to identify fake samples. We provide results on MNIST, CIFAR10, music and speech data. △ Less

Submitted 13 July, 2018; originally announced July 2018.

Comments: 10 pages, 16 figures

One-step Targeted Maximum Likelihood for Time-to-event Outcomes

Authors: Weixin Cai, Mark J. van der Laan

Abstract: Current Targeted Maximum Likelihood Estimation (TMLE) methods used to analyze time-to-event data estimate the survival probability for each time point separately, which result in estimates that are not necessarily monotone. In this paper, we present an extension of TMLE for observational time-to-event data, the one-step Targeted Maximum Likelihood Estimator for the treatment-rule specific survival… ▽ More Current Targeted Maximum Likelihood Estimation (TMLE) methods used to analyze time-to-event data estimate the survival probability for each time point separately, which result in estimates that are not necessarily monotone. In this paper, we present an extension of TMLE for observational time-to-event data, the one-step Targeted Maximum Likelihood Estimator for the treatment-rule specific survival curve. We construct a one-dimensional universal least favorable submodel that targets the entire survival curve, and thereby requires minimal extra fitting with data to achieve its goal of solving the efficient influence curve equation. Through the use of a simulation study, we will show that this method improves on previously proposed methods in both robustness and efficiency, and at the same time respects the monotone decreasing nature of the survival curve. △ Less

Submitted 12 June, 2019; v1 submitted 26 February, 2018; originally announced February 2018.

Data-adaptive statistics for multiple hypothesis testing in high-dimensional settings

Authors: Weixin Cai, Nima S. Hejazi, Alan E. Hubbard

Abstract: Current statistical inference problems in areas like astronomy, genomics, and marketing routinely involve the simultaneous testing of thousands -- even millions -- of null hypotheses. For high-dimensional multivariate distributions, these hypotheses may concern a wide range of parameters, with complex and unknown dependence structures among variables. In analyzing such hypothesis testing procedure… ▽ More Current statistical inference problems in areas like astronomy, genomics, and marketing routinely involve the simultaneous testing of thousands -- even millions -- of null hypotheses. For high-dimensional multivariate distributions, these hypotheses may concern a wide range of parameters, with complex and unknown dependence structures among variables. In analyzing such hypothesis testing procedures, gains in efficiency and power can be achieved by performing variable reduction on the set of hypotheses prior to testing. We present in this paper an approach using data-adaptive multiple testing that serves exactly this purpose. This approach applies data mining techniques to screen the full set of covariates on equally sized partitions of the whole sample via cross-validation. This generalized screening procedure is used to create average ranks for covariates, which are then used to generate a reduced (sub)set of hypotheses, from which we compute test statistics that are subsequently subjected to standard multiple testing corrections. The principal advantage of this methodology lies in its providing valid statistical inference without the \textit{a priori} specifying which hypotheses will be tested. Here, we present the theoretical details of this approach, confirm its validity via a simulation study, and exemplify its use by applying it to the analysis of data on microRNA differential expression. △ Less

Submitted 23 April, 2017; originally announced April 2017.

When Doesn't Cokriging Outperform Kriging?

Authors: Hao Zhang, Wenxiang Cai

Abstract: Although cokriging in theory should yield smaller or equal prediction variance than kriging, this outperformance sometimes is hard to see in practice. This should motivate theoretical studies on cokriging. In general, there is a lack of theoretical results for cokriging. In this work, we provide some theoretical results to compare cokriging with kriging by examining some explicit models and specif… ▽ More Although cokriging in theory should yield smaller or equal prediction variance than kriging, this outperformance sometimes is hard to see in practice. This should motivate theoretical studies on cokriging. In general, there is a lack of theoretical results for cokriging. In this work, we provide some theoretical results to compare cokriging with kriging by examining some explicit models and specific sampling schemes. [arXiv:1507.08017] △ Less

Submitted 30 July, 2015; originally announced July 2015.

Comments: Published at http://dx.doi.org/10.1214/15-STS518 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-STS-STS518

Journal ref: Statistical Science 2015, Vol. 30, No. 2, 176-180