Estimation de signaux par noyaux d'ondelettes
Estimating signals using multiple wavelet kernels
OPEN ACCESS
This paper addresses the problem of regression in the case of non-uniform sampled signals. Our method is based on supervised learning theory, we propose to use L2 estimation with wavelet kernels combined with L1 multiscale regularization. The use of Least Angle Regression as solver enable us to propose new solutions to set the regularization parameter.
Résumé
Cet article présente une méthode de régression pour les signaux non uniformément échantillonnés basée sur les ondelettes. Nous utilisons une formulation issue de l'apprentissage supervisé et des méthodes à noyaux qui combine une fonction coût L2 et une régularisation L1 multi-échelles. L'utilisation de l'algorithme Least Angle Regression pour la résolution du problème est à la fois efficace et intéressante, elle permet de calculer le chemin complet de régularisation et d'introduire de nouvelles solutions pour régler le compromis biais-variance.
Regularization L1, Multiple Kernels, Wavelets, Regression
Mots clés
Régularisation L1, Noyaux multiples, Ondelettes, Régression
[AAP04] U. AMATO, A. ANTONIADIS, and M. PENSKY, Wavelet kernel penalized estimation for non-equispaced design regression. Technical report, Istituto per le Applicazioni del Calcolo Mauro Picone, 2004.
[BBE+03] J. BI, K. BENNETT, M. EMBRECHTS, C. BRENEMAN, and M. SONG, Dimensionality reduction via sparse support vector machines. Journal of Machine Learning Research, 3:1229-1243, 2003.
[BTJ04] F.R. BACH, R. THIBAUX, and M.I. JORDAN, Computing regularization paths for learning multiple kernels. In Advances in Neural Information Processing Systems, volume 17, 2004.
[CDS98] S.S. CHEN, D.L. DONOHO, and M.A. SAUNDERS, Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 20(1):33-61, 1998.
[DDM04] I. DAUBECHIES, M. DEFRISE, and C. DE MOL, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Pure and Applied Mathematics, 2004.
[DJ94] D. DONOHO and I. JOHNSTONE, Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81:425-455, 1994.
[EHJT04] B. EFRON, T. HASTIE, I. JOHNSTONE, and R. TIBSHIRANI, Least angle regression. Annals of statistics, 32(2):407-499, 2004.
[EPP00] T. EVGENIOU, M. PONTIL, and T. POGGIO, Regularization Networks and Support Vector Machines. Advances in Computational Mathematics, 13(1):1-50, 2000.
[Gra98] Y. GRANDVALET, Least absolute shrinkage is equivalent to quadratic penalization. In ICANN, pages 201-206, 1998.
[GRC05] V. GUIGUE, A. RAKOTOMAMONJY, and S. CANU, Kernel basis pursuit. In 16th European Conference on Machine Learning, Porto, 2005.
[Gui05] V. GUIGUE, Méthodes à noyaux pour la représentation et la discrimination de signaux non-stationnaires. PhD thesis, INSA de Rouen, 2005.
[HT86] T. HASTIE and R. TIBSHIRANI, Generalized additive models. Statistical Science, 1:297-318, 1986.
[HT90] T. HASTIE and R. TIBSHIRANI, Generalized Additive Models. Chapman and Hall, London, 1990.
[KS00] A. KOVAC and B.W. SILVERMAN, Extending the scope of wavelet regression methods by coefficient-dependent thresholding. Journal of the American Statistical Association, 95:172-183, 2000.
[KW71] G. KIMELDORF and G. WAHBA, Some results on Tchebycheffian spline functions. J. Math. Anal. Applic., 33:82-95, 1971.
[LCV+04] G. LOOSLI, S. CANU, S.V.N. VISHWANATHAN, A. J. SMOLA, and M. CHATTOPADHYAY, Une boîte à outils rapide et simple pour les svm. In Michel Liquière and Marc Sebban, editors, CAp, pages 113-128. Presses Universitaires de Grenoble, 2004.
[Lju87] L. LJUNG, System Identification – Theory for the User. 1987.
[Mal97] S. MALLAT. A Wavelet Tour Of Signal Processing. Academic Press, 1997.
[MLD05] A. MAITROT, M.F. LUCAS, and C. DONCARLI, Design of wavelets adapted to signals and application. In IEEE Internantional Conference an Acoustics, Speech and Signal Processing (ICASSP), Philadelphie, USA, March 2005.
[Ng04] A.Y. NG, Feature selection, l1 vs. l2 regularization, and rotational invariance. In International Conference on Machine Learning, 2004.
[RC05] A. RAKOTOMAMONJY and S. CANU, Frame, reproducing kernel, regularization and learning. JMLR, 6:1485-1515, 2005.
[SS02] B. SCHÖLKOPF and A.J. SMOLA, Learning with kernels. MIT Press, 2002.
[Tib96] R. TIBSHIRANI, Regression shrinkage and selection via the lasso. J. Royal. Statist., 58(1):267-288, 1996.
[VB02] P. VINCENT and Y. BENGIO, Kernel matching pursuit. Machine Learning Journal, 48(1):165-187, 2002.