Finite Mixture Models Estimation with a Credal EM Algorithm. Estimation de Modèles de Mélanges Finis par un Algorithme EM Crédibiliste

Finite Mixture Models Estimation with a Credal EM Algorithm

Estimation de Modèles de Mélanges Finis par un Algorithme EM Crédibiliste

Patrick Vannoorenberghe

Laboratoire de Télédétection à Haute résolution, ERT43 Université Paul Sabatier,Toulouse 3

Centre de Télédétection Spatiale, 118 route de Narbonne 31062 Toulouse cedex 4 – France

Page: 
103-113
|
Received: 
11 May 2006
|
Accepted: 
N/A
|
Published: 
30 April 2007
| Citation

OPEN ACCESS

Abstract: 

This paper is concerned with finite mixture models estimation in the framework of Transferable Belief Model.This model relies on a non probabilistic formalism for representing and manipulating imprecise and uncertain information with belief functions.Within this framework,a credal EM algorithm,a variant of classical EM algorithm based on belief functions,is introduced for finite mixture parameters learning.This algorithm can be applied in several situations where available information on the data generation model is partially known.In the learning problem,this knowledge is represented with belief functions which allow to represent as better as possible the uncertainty on the component from where each observation has been generated.Several experimentations highlight situations where the algorithm is applied when available information on the learning set is imprecise (partially supervised learning where the actual component of each sample is only known as belonging to a subset of components),and/or uncertain (unsupervised learning where the knowledge about the actual sample is represented by a belief function).Synthetic data sets allow us to demonstrate the good performance of the proposed approach based on estimated parameters analysis and learning with gaussian finite mixture models.

Résumé

Dans cet article,l’estimation d’un modèle de mélange fini est abordée dans le cadre du Modèle des Croyances Transférables (MCT). Ce modèle constitue le socle d’un formalisme non probabiliste pour la représentation d’informations imprécises et incertaines par des fonctions de croyances. Dans ce contexte,un algorithme EM crédibiliste,une extension de l’algorithme EM aux fonctions de croyance,est introduit pour l’apprentissage des paramètres du modèle de mélange fini. Nous montrons comment cet algorithme peut être appliqué dans plusieurs contextes où l’information sur le modèle de génération des données n’est que partiellement disponible. Cette information est représentée,dans le problème d’apprentissage,par des fonctions de croyance qui permettent de modéliser la connaissance disponible sur la composante ayant servie à générer chaque observation de manière la plus fine possible. Plusieurs simulations mettent en évidence des situations où le modèle de génération des données n’est connu que de manière imprécise (apprentissage partiellement supervisé) et où l’on ne posséde auncune information sur la composante d’appartenance de chaque observation (apprentissage non supervisé). Des jeux de données synthétiques permettent de démontrer les bonnes performances de l’approche proposée en terme d’estimation mais également en terme d’apprentissage sur des modèles de mélanges gaussiens.

Keywords: 

Finite mixture models,Transferable Belief Model,EM algorithm,Learning.

Mots clés 

Modèle de mélange fini,Modèle des croyances transférables,Algorithme EM,Apprentissage.

1. Introduction
2. Fonctions de croyance
3. Apprentissage de Modèles de Mélanges Finis
4. Algorithme EM Crédibiliste
5. Résultats Expérimentaux
6. Conclusions – Perspectives
  References

[1] C. AMBROISE, T. DENŒUX, G. GOVAERT, and P. SMETS, Learning from an imprecise teacher: probabilistic and evidential approaches. In Proccedings of ASMDA’2001, volume 1, pp. 100-105 Compiègne, France, 2001. 

[2] F. DELMOTE and P. SMETS, Target identification based on the Transferable Belief Model interpretation of Dempster-Shafer. IEEE Transactions on Systems, Man and Cybernetics,A 34:457-471, 2004. 

[3] A.P. DEMPSTER, N.M. LAIRD, and D.B. RUBIN, Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, series B, 39:1-38, 1977. 

[4] Mario A.T. FIGUEIREDO and Anil K. JAIN, Unpervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(3):381-396, 2002. 

[5] G. Mc LACHLAN and D. PEEL,Finite Mixture Models. Wiley Series in Probability and Statistics. John Wiley & Sons, New-York, 2000. 

[6] J.M. MARIN, K. MENGERSEN, and C.P. ROBERT, Bayesian modelling and inference on mixtures of distributions. Handbook of Statistics, 25, 2005. 

[7] G. McLACHLAN and K. BASFORD, Mixture Models: Inference and Application to Clustering. Marcel Dekker, New-York, 1988. 

[8] G. McLACHLAN and T. KRISHNAN, The EM algorithm and extensions. John Wiley, New-York, 1997. 

[9] R. NEAL and G. HINTON,A view of the EM algorithm that justifies incrememtal, sparse, and other variants. In M.I. Jordan, editor, Learning in Graphical Models. Kluwer, 1998. 

[10] G. SHAFER, A Mathematical Theory of Evidence. Princeton univ. Princeton, NJ, 1976. 

[11] Ph. SMETS,Belief functions:the disjunctive rule of combination and the genralized Bayesian theorem. International Journal of Approximate Reasoning, 9:1-35, 1993. 

[12] Ph. SMETS, The application of the matrix calculus to belief functions. International Journal of Approximate Reasoning, 31:1-30, 2002. 

[13] Ph. SMETS, Decision making in the TBM: the necessity of the pignistic transformation. International Journal of Approximate Reasoning, 38:133-147, 2005. 

[14] Ph. SMETS and R. KENNES,The transferable belief model. Artifical Intelligence, 66:191-234, 1994. 

[15] P. VANNOORENBERGHE and Ph. SMETS, Partially supervised learning by a credal EM approach. In Lluis Godo, editor, Symbolic and Quantitative Approaches to Reasoning with uncertainty, 8th European Conference, ECSQARU 2005, pp. 956-967, Spain, July 6-8 2005. Springer, 2005. 

[16] Z. ZIVKOVIC and F. VAN DER HEIJDEN, Recursive unsupervised learning of finite mixture models. IEEE Transictions on Pattern Analysis and Machine intelligence, 26(5):651-656, May 2004.