This book introduces the gamma-divergence, a measure of distance between probability distributions that was proposed by Fujisawa and Eguchi in 2008. The gamma-divergence has been extensively explored to provide robust estimation when the power index γ is positive. The gamma-divergence can be defined even when the power index γ is negative, as long as the condition of integrability is satisfied. Thus, the authors consider the gamma-divergence defined on a set of discrete distributions. The arithmetic, geometric, and harmonic means for the distribution ratios are closely connected with the gamma-divergence with a negative γ. In particular, the authors call the geometric-mean (GM) divergence the gamma-divergence when γ is equal to -1.

The book begins by providing an overview of the gamma-divergence and its properties. It then goes on to discuss the applications of the gamma-divergence in various areas, including machine learning, statistics, and ecology. Bernoulli, categorical, Poisson, negative binomial, and Boltzmann distributions are discussed as typical examples. Furthermore, regression analysis models that explicitly or implicitly assume these distributions as the dependent variable in generalized linear models are discussed to apply the minimum gamma-divergence method.

In ensemble learning, AdaBoost is derived by the exponential loss function in the weighted majority vote manner. It is pointed out that the exponential loss function is deeply connected to the GM divergence. In the Boltzmann machine, the maximum likelihood has to use approximation methods such as mean field approximation because of the intractable computation of the partition function. However, by considering the GM divergence and the exponential loss, it is shown that the calculation of the partition function is not necessary, and it can be executed without variational inference.

 

Les mer
In particular, the authors call the geometric-mean (GM) divergence the gamma-divergence when γ is equal to -1.

The book begins by providing an overview of the gamma-divergence and its properties.

Les mer

1. Introduction.- 2. Framework of gamma-divergence.- 2.1. Scale invariance.- 2.2 GM divergence and HM divergence.- 3. Minimum divergence methods for generalized linear models.- 3.1. Bernoulli logistic model.- 3.2. Poisson log-linear model.- 3.3. Poisson point process model.- 4. Minimum divergence methods in machine leaning.- 4.1. Multi-class AdaBoost.- 4.2. Boltzmann machine.- 5. gamma-divergence for real valued functions.- 6. Discussion.

Les mer

This book introduces the gamma-divergence, a measure of distance between probability distributions that was proposed by Fujisawa and Eguchi in 2008. The gamma-divergence has been extensively explored to provide robust estimation when the power index γ is positive. The gamma-divergence can be defined even when the power index γ is negative, as long as the condition of integrability is satisfied. Thus, the authors consider the gamma-divergence defined on a set of discrete distributions. The arithmetic, geometric, and harmonic means for the distribution ratios are closely connected with the gamma-divergence with a negative γ. In particular, the authors call the geometric-mean (GM) divergence the gamma-divergence when γ is equal to -1.

The book begins by providing an overview of the gamma-divergence and its properties. It then goes on to discuss the applications of the gamma-divergence in various areas, including machine learning, statistics, and ecology. Bernoulli, categorical, Poisson, negative binomial, and Boltzmann distributions are discussed as typical examples. Furthermore, regression analysis models that explicitly or implicitly assume these distributions as the dependent variable in generalized linear models are discussed to apply the minimum gamma-divergence method.

In ensemble learning, AdaBoost is derived by the exponential loss function in the weighted majority vote manner. It is pointed out that the exponential loss function is deeply connected to the GM divergence. In the Boltzmann machine, the maximum likelihood has to use approximation methods such as mean field approximation because of the intractable computation of the partition function. However, by considering the GM divergence and the exponential loss, it is shown that the calculation of the partition function is not necessary, and it can be executed without variational inference.
 

Les mer
Introduces gamma-divergence, a measure of distance between probability distributions Discusses applications of gamma-divergence in machine learning, statistics, and ecology Provides a unified framework for understanding exponential loss and minimum gamma-divergence
Les mer
GPSR Compliance The European Union's (EU) General Product Safety Regulation (GPSR) is a set of rules that requires consumer products to be safe and our obligations to ensure this. If you have any concerns about our products you can contact us on ProductSafety@springernature.com. In case Publisher is established outside the EU, the EU authorized representative is: Springer Nature Customer Service Center GmbH Europaplatz 3 69115 Heidelberg, Germany ProductSafety@springernature.com
Les mer

Produktdetaljer

ISBN
9789819788798
Publisert
2025-03-12
Utgiver
Vendor
Springer Nature
Høyde
235 mm
Bredde
155 mm
Aldersnivå
Research, P, 06
Språk
Product language
Engelsk
Format
Product format
Heftet

Forfatter

Biografisk notat

Shinto Eguchi received his master degree from Osaka University in 1979 and a Ph.D. from Hiroshima University, Japan, in 1984. His working career started as Assistant Professor of Hiroshima University, 1984, Associate Professor of Shinamne University, 1986, and Professor of The Institute of Statistical Mathematics, 1995-2020. He is currently Emeritus Professor at the Institute of Statistical Mathematics and Graduate University of Advanced Studies. His research interest is primarily statistics, including statistical machine learning, bioinformatics, information geometry, statistical ecology and parametric/semiparametric inference and robust statistics.
 
His recent publication:
 
-A generalized quasi-linear mixed-effects model, Y Saigusa, S Eguchi, O Komori, Statistical Methods in Medical Research, 31 (7), 1280-1291, 2022.
 
-Robust self-tuning semiparametric PCA for contaminated elliptical distribution, H Hung, SY Huang, S Eguchi, IEEE Transactions on Signal Processing 70, 5885-5897, 2022.
 
-Minimum information divergence of Q-functions for dynamic treatment resumes. S Eguchi, Information Geometry, 1-21, 2022.