Sigmoid Functions Some Approximation and Modelling Aspects
Sigmoid Functions Some Approximation and Modelling Aspects
net/publication/308898939
Sigmoid Functions Some Approximation and Modelling Aspects: Some Moduli in Programming Environment MATHEMATICA
CITATIONS READS
11 1,546
2 authors:
All content following this page was uploaded by Nikolay Kyurkchiev on 24 July 2020.
SIGMOID FUNCTIONS:
SOME APPROXIMATION,
AND MODELLING ASPECTS
2015
3
CONTENTS
Preface 5
Chapter 1. Sigmoid Functions
1.1. Introduction 7
1.2. Sigmoid and step functions 8
1.2.1 Hausdorff continuity 8
1.2.2 The step function 8
1.2.3 The cut functions 9
1.2.4 Sums of sigmoid functions 9
1.2.5 Hausdorff distance 9
1.2.6 The logistic sigmoid functions 11
1.2.7 Approximation issues 12
1.2.8 Shifted logistic functions 16
1.3. Kinetic mechanisms yielding Verhulst model 16
1.3.1 A simple autocatalytic reaction 17
1.3.2 Autocatalytic reaction involving nutrient substrate 18
1.3.3 Nutrient supply as input function 20
1.3.4 Computational issues 20
1.4 Conclusions 21
1.5 Supplements 23
1.5.1 Supplement 1. More precise bounds for d(k) 23
1.5.2 Supplement 2. Approximation of continuous functions by multilayer perceptrons 26
Appendix 1. 30
Appendix 2. 31
Appendix 3. 32
Appendix 4. 32
Chapter 2. Approximation of the cut function
2.1 Introduction 33
2.2 Preliminaries 34
2.2.1 Sigmoid functions 34
2.2.2 Piece-wise linear sigmoid functions: cut function 34
2.2.3 Smooth sigmoid functions: logistic function 36
2.3 Approximation of the cut function by logistic functions 36
2.4 Approximation of the cut function by squashing functions 39
Appendix 45
Chapter 3. Log–logistic and transmuted log–logistic functions
3.1 Introduction 47
3.2 Approximation of the step function by log–logistic functions 49
4
[1] Alt, R., S. Markov, Theoretical and computational studies of some bioreactor models,
Computers and Mathematics with Applications 64 (2012), 350–360.
[2] Anguelov, R., S. Markov, Hausdorff Continuous Interval Functions and Approxima-
tions, Mathematics and Computers in Simulation (accepted), 2015.
[4] Costarelli, D., R. Spigler, Approximation results for neural network operators acti-
vated by sigmoidal functions, Neural Networks 44, 101–106 (2013).
[7] Hausdorff, F., Set theory (2 ed.), New York, Chelsea Publ.,1962 [1957], ISBN 978-
0821838358 (Republished by AMS-Chelsea 2005).
[8] Markov, S., Biomathematics and Interval Analysis: A Prosperous Marriage, in: Chris-
tov, Ch. and Todorov, M. D. (eds), Proc. of the 2nd Intern. Conf. on Application of
Mathematics in Technical and Natural Sciences (AMiTaNS’10), AIP Conf. Proc. 1301,
26–36. 2010.
[9] Markov, S., Cell Growth Models Using Reaction Schemes: Batch Cultivation, Biomath
2/2 (2013), 1312301
http://dx.doi.org/10.11145/j.biomath.2013.12.301
[10] Markov, S., On the Use of Computer Algebra Systems and Enclosure Methods in the
Modelling and Optimization of Biotechnological Processes, Bioautomation (Intern.
Electronic Journal) 3, 2005, 1–9.
[11] Markov, S., On the mathematical modelling of microbial growth: some computational
aspects, Serdica J. Computing 5 (2), 153–168 (2011).
101
102 BIBLIOGRAPHY
[15] Verhulst, P.-F., Notice sur la loi que la population poursuit dans son accroissement,
Correspondance mathematique et physique 10: 113–121 (1838).
[17] Verhulst, P.-F., Deuxieme memoire sur la loi d’accroissement de la population, Mem-
oires de l’Academie Royale des Sciences, des Lettres et des Beaux-Arts de Belgique
20: 1–32 (1847).
[18] Sendov Bl., V. Popov, The exact asymptotic behavior of the best approximation by
algebraic and trigonometric polynomials in the Hausdorff metric, Mat. Sb. (N.S.), 89,
1972, 138–147.
[19] Traub, J., Iterative Methods for the Solution of Equations, Chelsea Publishing Com-
pany, New York, N.Y., 1982.
[21] Anguelov R., S. Markov, B. Sendov, On the Normed Linear Space of Hausdorff Con-
tinuous Functions. In: Lirkov, I., et al (eds): Lecture Notes in Computer Science 3743,
Springer, (2006), 281–288.
[22] Anguelov R., S. Markov, B. Sendov, Algebraic Operations on the Space of Hausdorff
Continuous Functions. In: Boyanov, B. (ed.): Constructive Theory of Functions, Prof.
M. Drinov Academic Publ. House, Sofia (2006), 35–44.
[23] Anguelov R., S. Markov, B. Sendov, The Set of Hausdorff Continuous Functions - the
Largest Linear Space of Interval Functions. Reliable Computing 12, 337–363 (2006).
[24] Rosenblat F., The Perceptron: A Probabilistic Model for Information Storage and
Organization in the Brain, Cornell Aeronautical Laboratory, Psychological Review,
65 (6), 386–408 (1958).
[26] Dombi J., Z. Gera, The approximation of piecewise linear membership functions and
Likasiewicz operators, Fuzzy Sets and Systems, 154, 275–286 (2005).
BIBLIOGRAPHY 103
[27] Kyurkchiev N., S. Markov, On the Hausdorff distance between the Heaviside step
function and Verhulst logistic function, J. of Math. Chemistry, 2015 (submitted).
[28] Fisk, P. R., The Graduation of Income Distribution, Econometrica, 29 (2), 1961,
171–185.
[29] Collett, D., Modelling Survival Data in Medical Research,(2nd. ed.), 2003, CRC press,
ISBN 1-58488-325-1.
[30] Shaw, W., I. Buckley, The alchemy of probability distributions: beyond Gram-Charlier
expansion and a skew-kurtotic-normal distribution from rank transmutation map,
Research report (2007).
[31] Gupta, R. C., O. Acman, S. Lvin, A sudy of log–logistic model in survival analysis,
Biometrica Journal, 41 (4), 1999, 431–443.
[32] Iliev, A., N. Kyurkchiev, S. Markov, On the approximation of the cut function by
smooth sigmoidal functions, Int. J. Math. Models Biosci., 4, 2015 (submitted).
[33] Albano, G., V. Giono, On the first exit time problem for a Gompertz-type tumor
growth, Lecture Notes in Computer Science 5717, 2009, 113–120.
[34] Cao, F. L., Z. Chen, Z., The approximation operators with sigmoidal functions, Com-
puter & Mathematics with Applications, 58:4 (2009) 758–765.
[36] Chen, Zh., F. Cao, J. Hu, Approximation by network operators with logistic activation
functions, Applied Mathematics and Computation 256 (2015), 565–571.
[39] Dombi, J., Membership function as an evaluation, Fuzzy Sets and Systems, 35, 1–21
(1990).
[40] Dombi, J., Zs. Gera, Rule based fuzzy classification using squashing functions, In:
Journal of Intelligent & Fuzzy Systems 19 (2008), 3–8.
[41] Gera, Zs., Computationaly efficient reasoning using approximated fuzzy intervals,
Fuzzy Sets and Systems, 158 (7), 689–703, (2007).
[42] Gera, Zs., J. Dombi, Genetic Algorithm with Gradient Based Tuning for Constructibg
Fuzzy Rules, Proc. of the 6th Intern. Symposium of Hungarian Researchers on Com-
putational Intelligence, Budapest, 2005.
http://www.academia.edu/10987580/
104 BIBLIOGRAPHY
[43] Gompertz, B., On the nature of the function expressive of the law of human mortality,
and on a new mode of determining the value of the life contingencies, Philos. Trans.
R. Soc. London, 115, 1825, 513–585.
[44] Henri, V., Ueber das Gesetz der Wirkung des Invertins. Z. Phys. Chem., 39 (1901),
194–216.
[47] Kramer, W., J. Wolff v. Gudenberg, Scientific Computing, Validated Numerics, Inter-
val Methods, Proceedings of the conference Scan-2000/Interval-2000, Kluwer/Plenum,
2001.
[49] Thomas, J., A. Runkler, Function approximation with polynomial membership func-
tions and alternating cluster information, Fuzzy Sets and Systems, 101, 207–218
(1999).
[50] van der Walt, J. H., The Linear Space of Hausdorff Continuous Interval Functions.
Biomath 2 (2013), 1311261
http://dx.doi.org/10.11145/j.biomath.2013.11.261
[53] Han, J., C. Morag, The influence of the sigmoid function parameters on the speed of
backpropagation learning, In: Mira, J., Sandoval, F. (Eds) From Natural to Artificial
Neural Computation, 195201, 1995.
[54] Richards, F. J., A flexible growth function for empirical use, J. Exp. Bot., 10, 1959,
290–300.
[56] Schnute, J., A versatile growth model with statistically stable parameters, Can. J.
Fish. Aquat. Sci., 38, 1981, 1128–1140.
[59] Sas, A. A., H. Snieder, J. Korf, Gompertz’ survivorship law as an intrinsic principle
of aging, Medical Hypotheses, 78 (5), 2012, 659–663.
[60] Berger, R. D., Comparison of the Gompertz and logistic equation to describe plant
disease progress, Phytopathology 71, 1980, 716–719.
[64] Carey, E., Using Calculus to Model the Growth of L.Plantarum Bacteria, Undergrad-
uate J. of Math. Modeling, 1 (2), Article 2, 2009, 11 pp.
[66] Rickles, R., A. Scheuerlein, Biological Imlications of the Weibull and Gompertz Models
of Aging, J. of Gerontology: BIOLOGICA SCIENCES, 57 (2), 2002, B69–B76.
[67] Wu, J. W., W. L. Hung, Ch. H. Tsai, EStimation of parameters of the Gompertz
distribution using the least squares method, Appl. Math. Comp., 158, 2004, 133–147.
[68] Makany, R., A theoretical basis of Gompertz’s curve, Biometrica Journal, 33, 1991,
121–128.
[69] Franses, P. H., Fitting a Gompertz curve, J. of Operational Research Society, 45,
1994, 109–113.
[70] Chen, Z., Parameter estimation of the Gompertz population, Biometrical Journal, 39,
1997, 117–124.
[72] Abrams, P. A., D. Ludwig, Optimality theory, Gompertz’law, and the disposable some
theory of senescence, Evolution., 49, 1995, 1055–1066.
106 BIBLIOGRAPHY
[73] Wilson, D. L., The analysis of survival (mortality) data: fitting Gompertz, Weibull,
and logistic functions, Mech Ageing Devel., 74, 1994, 15–33.
[74] Akanuma, A., Parameter analysis of Gompertz function drowth model in clinical tu-
mors, European J. of Cancer, 14, 1978, 681–688.
[75] Winsor, C. P., The Gompertz curve as a growth curve, Proceedings of the National
Academy of Sciences, 18, 1932, 1–8.
[78] Merovci, F., Transmuted Rayleigh Distribution, Austrian Journal of Statistics, 4 (1),
2013, 21–31.
[80] Mahdi, S., Improved parameter estimation in Rayleigh model, Metodoloski zvezki, 3,
2006, 63–74.
[81] Manesh, S. F., B. Khaledi, On the likelihod ratio order for convolutions of independent
generalized Rayleigh random variables, Statistics and Probability Letters, 78, 2008,
3139–3144.
[82] Merovci, F., Transmuted Generalized Rayleigh Distribution, J. of Stat. Appl. Proba-
bility, 3 (1), 2014, 9–20.
[84] Lindley, D. V., Fiducial distributions and Bayes theorem, Journal of the Royal Sta-
tistical Society, Series B, 20, 1958, 102–107.
[85] Lindley, D. V., Introduction to Probability and Statistics from a Bayesian Viewpoint,
Part II: Inference, Cambridge University Press, New York, 1965.
[86] Merovci, F., Transmuted Lindley Distribution, Int. J. Open Problems Compt. Math.,
6 (2), 2013, 63–72.
[88] Aryal, G. R., C. P. Tsokos, On the transmuted extreme value distribution with appli-
cation, Nonlinear Analysis: Theory, Methods amd Applications, 71, 2009, 1401–1407.
BIBLIOGRAPHY 107
[90] Ashor, S. and et al., Transmuted Exponential Lomax Distribution, Austrilian Journal
of Basic and Applied Sciences, 7, 2013, 658–667.
[91] Voda, R., On the inverse Rayleigh variable, Rep. Stat. Res. Juse, 19 (4), 1972, 15–21.
[92] Soliman, A., and et al., Estimatiom and Prediction from the inverse Rayleigh distri-
bution based on lower record values, Appl. Math. Sci., 4 (62), 2010, 3057–3066.
[95] Ghitany, M. E., B. Atieh, S. Nadarajah, Lindley distribution and its application,
Math. Comput. Simulation, 78, 2008, 493–506.
[97] Merovci, F., I. Elbatal, Transmuted Lindley geometric and its application, J. Stat.
Appl., 3 (1), 2014, 77–91.
[103] Jodra, J., Computer generation of random variables with Lindley or Poisson–Lindley
distribution via the Lambert Wfunction, Math. Comput. Simulation, 81, 2010, 851–
859.
[106] Nadarajah, S., Exponential Pareto distribution, Statistics, 39 (3), 2005, 255–260.
[107] Nadarajah, S., The exponentiated Gumbel distribution with climate application, En-
vironmetrics, 17 (3), 2006, 13–23.
[108] Sankaran, M., The discrete Poisson–Lindley distribution, Biometrics, 26, 1970, 145–
149.
[110] Singh, S., G. Maddala, A function for the size distribution of incomes, Econometrica,
44 (5), 1976, 963–970.
[112] Tadikamalla, Pandu R., A Look at the Burr and Related Distributions, International
Statistical Review, 48 (3), 1980, 337–344.
[113] Kliber, C., S. Kotz, Statistical Size Distributions in Economics and Actuarial Sciences,
New York; Wiley, 2003.
[115] Rodriguez, R. V., A guide to Burr distributions, Biometrika, 64 (1), 1977, 129–134.
[116] Rodriguez, R. V., Burr Distributions, Enciklopedia of Statistical Sciences, John Wiley
and Sons, Inc., 2004.
[117] Al-Hussaini, K. Essam, A characterization of the Burr XII distribution, Appl. Math.
Lett., 4 (1), 1991, 59–61.
[118] Gramming, J., Kai-Oliver Mauer, Non–monotonic hazard functions and the autore-
gressive conditional duration model, Econometrics J,, 3, 2000, 16–38.
[119] Raqab, M. Z., Order statistics from Burr type X model, Computers and Mathematics
with Applications, 36 (4), 1998, 111–120.
[120] Fry, T. R. L., Univariate and multivariate Burr distributions: A survey, Pak. J.
Statist., 9 (1), 1993, 1–29.
[121] Sartawi, H. A., M. S. Abu–Salih, Bayesian prediction pounds for the Burr type X
model, Commun. Statist.–Theory Meth., 20 (7), 1991, 2307–2330.
[122] Ahmad, N., A. Islam, Optimal accelerated life test designs for Burr type XII distri-
butions under periodic inspection and type I censoring, Naval Res. Logistics, 43 (8),
1996, 1049–1077.
BIBLIOGRAPHY 109
[123] Domma, F., Some properties of the bivariate Burr type III distribution, Statistics: A
J. of Theoretical and Appl. Statistics, 44 (2), 2014, 203–215.
[124] Domma, F., New developments on the use of bivariate Rodrigues–Burr III distribution
in reliability studies, Working Paper 07–2009, 12pp., Dipartimento di Economica e
Statistica, Univ. della Calabria.
[125] Gove, J. H., M. J. Ducey, W. B. Leak, L. Zhang, Rotated sigmoid structures in man-
aged un even–aged northern hardwood stands: a look at the Burr type distributions,
Forestry, 5, 2008.
[126] Lomax, K. S., Business Failures: Another example of the analysis of failure data,
Journal of the American Statistica Association, 49, 1954, 847–851.
[130] Ahsanullax, M., Record values of Lomax distribution, Statistica Nederlandica, 41 (1),
1991, 21–29.
[132] Lemonte, A., G. Cordeiro, An extended Lomax distribution, Statistics, iFirst, Taylor
and Francis, 2011, 1–17.
[133] Balkema, A., L. de Haan, Residual life time at great age, Annals of Probability, 2 (5),
1974, 792–804.
[135] Cramer, E., A. B. Schmiedt, Progressively type–II censored competong rosks data
from Lomax distributions, Computational Statistics and Data Analysis, 55 (3), 2011,
1285–1303.
[137] Petropoulos, C., S. Kourouklis, Improved estimation of extreme quantiles in the mul-
tivariate Lomax (Pareto II) distribution, Metrika, 60, 2004, 15–24.
110 BIBLIOGRAPHY
[138] Eugene, N., C. Lee, F. Fanaoye, Beta–normal distribution and its applications, Com-
mun. Stat. Theory Methods, 31, 2002, 497–512.
[140] Gumbel, E. J., Les valeurs extremes des distributions statistiques, Ann. Inst. Henri
Poincare, 5 (2), 1935, 115–158.
[141] Gumbel, E. J., The return period of flood flows, The Annals of Mathematical Statis-
tics, 12, 1941, 163–190.
[142] Weibul, W., A statistical distribution function of wide applicability, J. Appl. Mech.–
Trans. ASUE, 18 (3), 1951, 293–297.