Arabic and Latin Handwritting Recognition Using Multi-Stream Hidden Markov Models
Reconnaissance de L’Écriture Manuscrite Arabe et Latine par des Modèles de Markov Cachés Multi-Flux
OPEN ACCESS
In this paper,we present a multi-stream approach for off-line handwritten word recognition.The multi-stream formalism presents many advantages:it can combine several kinds of independent features.The combination can be adaptive: some sources of information can be weighted,or even rejected if they are not reliable.The topology of the HMM can be adapted to each source of information.It also allows asynchronous modelling of streams.
The proposed approach combines low level feature streams namely,density based features extracted from 2 different sliding windows with different widths,and contour based features extracted from upper and lower contours. Significant experiments have been carried out on two publicly available word databases:IFN/ENIT benchmark database (Arabic script) and IRONOFF database (Latin script).
In order to model the Latin characters,we built 26 uppercase character models and 26 lowercase character models).In the case of Arabic characters,we built up to 159 character models. An Arabic character may actually have different shapes according to its position within the word (beginning,middle,end word position).Other models are specified with additional marks such as “shadda”.In both Latin and Arabic script,each character model is composed of 4 emitting states.The observation probabilities are modelled with Gaussian Mixtures (3 per state).Embedded training is used where all character models are trained in parallel using Baum-Welch algorithm applied on word examples.The system builds a word HMM by concatenation of the character HMM corresponding to the word transcription of the training sample.
The recognition step is doing allowing the HMM-recombination algorithm that consists in building the product HMM and using a classical Viterbi decoding algorithm.We investigate the extension of 2-stream approach to N streams (N=2,...,4) and analyze the improvement in the recognition performance.The computational cost of this extension is discussed. The developed system has been tested on two publicly available databases.For both scripts the results show significant improvement while using a multi-stream approach.The comparison of the multi-stream performances to the classical combination strategies namely,fusion of features and fusion of decisions shows the superiority of the multi-stream approach.Moreover,the proposed recognition system provides significant results comparable to the best results reported in the literature on both databases.
Résumé
Dans cet article nous proposons une approche de reconnaissance de l’écriture manuscrite. L’objectif étant de proposer un système indépendant de la nature du script,nous procédons alors sans segmentation. Des caractéristiques bas niveaux,basées sur les directions des contours et les densités de pixels,sont combinées à travers une approche multi-flux. Nous évaluons l’apport de l’approche multi-flux ainsi proposée et nous la comparons aux approches classiques de combinaison par fusion de représentations et par fusion de décisions. Pour valider l’approche proposée nous avons effectué des expérimentations sur deux bases de données de référence,la base de mots arabes IFN/ENIT et la base IRONOFF de mots latins. Les résultats montrent que le système proposé donne de bons résultats comparables aux meilleurs approches rapportées dans la litérature,aussi bien sur le Latin que sur l’Arabe.
Off-Line handwriting recognition,Hidden Markov Models,Latin script,Arabic script,multi-stream,information combination.
Mots clés
Reconnaissance de l’écriture manuscrite hors-ligne,écriture arabe,écriture latine,combinaison d’information, modèles de Markov cachés multi-flux.
[ART 03] T. ARTIÈRES, N. GAUTHIER, P. GALLINARI, B. DORIZZI, A Hidden Markov Models combination framework for handwriting recognition, International Journal on Document Anlysis and Recognition (IJDAR), Vol 5, N° 4, p. 233-243, July (2003).
[BEL 97] A. BELAID,G. SAON,Utilisation des Processus Markoviens en Reconnaissance de l’Ecriture, Traitement du Signal, Vol 14, N° 2, p. 161-177 (1997).
[BOU 97] H. BOURLARD, S. DUPONT, Sub-band-based Speech Recognition. In IEEE Int. Conf. on Acoust., Speech, and Signal Processing, p. 1251-1254 (1997).
[BOU 96] H. BOURLARD, S. DUPONT, and C. RISS, Multi-stream speech recognition. Technical Report IDIAP-RR 96-07, IDIAP, 1996.
[BOZ 89] R. M. BOZINOVIC and S. N. SRIHARI. «Off-line Cursive Script Word Recognition», IEEE Transactions on PAMI, Vol 11, N° 1, p. 68-83, January (1989).
[EL-Y 02] M.A. EL-YACOUBI, M. GILLOUX & J.M. BERTILLE. A Statistical Approach for Phrase Location and Recognition within a Text Line:An Application to Street Name Recog nition. IEEE Trans. on PAMI, vol. 24, no. 2, p. 172-188 (2002).
[GAU 01] N. GAUTHIER,T. ARTIÈRES,B. DORIZZI,and P. GALLINARI, Strategies for combining on-line and off-line information in an online handwriting recognition system. International Conference on Document Analysis and Recognition (ICDAR), p. 412-416 (2001).
[HAJ 05] R. EL-HAJJ, L. LIKFORMAN-SULEM, C. MOKBEL, Arabic handwriting recognition using baseline dependent features and Hidden Markov Modeling, ICDAR 2005, Seoul, Corée du Sud.
[KES 08] Y. KESSENTINI, T. PAQUET, A. BENHAMADOU. Combinaison d’Information pour la Reconnaissance de l’Ecriture Manuscrite Hors-Ligne. RFIA’2008,Amiens, France.
[KIM 94] F. KIMURA, S. TSURUOKA,Y. MIYAKE & M. SHRIDHAR, «A Lexicon Directed Algorithm for Recognition of Un-constrained Handwritten Words». IEICE Trans. Inf. &Syst.,vol. E77-D,no. 7 (1994).
[KIM 00] J. KIM, K. KIM, C. NADAL, and C. SUEN, A methodology of combining hmm and mlp classifiers for cursive word recognition. International Conference Document Analysis and Recognition (ICDAR), vol 2, p. 319-322 (2000).
[KNE 98] S. KNERR, E. AUGUSTIN, O. BARET and D. PRICE, Hidden Markov Model Based Word Recognition and Its Application to Legal Amount Reading on French Checks. Computer Vision and Image Understanding 70(3): 404-419 (1998).
[KOE 02] A. KOERICH,Large vocabulary off-line handwritten word recognition. PhD Thesis (2002).
[OKA 98] S. OKAWA, E. BOCCHIERI, and A. POTAMIANOS, Multiband speech recognition in noisy environments. In Proceedings of IEEE International Conference on Acoustic, Speech, and Signal Processing, p. 641.644, Seattle, Washington (1998).
[PEC 02] M. PECHWITZ,S.S. MADDOURI,V. MAERGNER,N. ELLOUZE, and H. AMIRI, IFN/ENIT-database of handwritten Arabic words. In Proc. of CIFED 2002, p. 129-136, Hammamet, Tunisia, October (2002).
[POT 98] G. POTAMIANOS, H.P. GRAF, Discriminative training of HMM flux exponents for audio-visual speech recognition. Proceeding IEEE Conference Acoustics, Speech, and Signal Processing. vol.6, p. 3733-3736, Seattle, WA (1998).
[PRE 03] L. PREVOST, C. MICHEL-SENDIS, A. MOISES, L. OUDOT, and M. MILGRAM, Combining model-based and discriminative classifiers: application to handwritten character recognition. In 7th International Conference on Document Analysis and Recognition, volume 1, p. 31-35 (2003).
[RAB 90] RABINER, L.R.,A tutorial on hidden markov models and selected applications in speech recognition. Readings in speech recognition, pp. 267-296, 1990.
[SAK 79] H. SAKOE, Two-level DP matching – A dynamic programming – based pattern matching algorithm for connected word recognition, IEEE Transactions of the IECE of Japan, Vol 27, p. 588- 595, 1979.
[SME 94] Ph. SMETS and R. KENNES, The Transferable Belief Model. Artificial Intelligence, vol 66, n° 2, p. 191-234 (1994).
[SRI 00] N.S. SRIHARI, Handwritten Address Interpretation: A Task of Many Pattern Recognition Problems. IJPRAI 14(5), p. 663-674, (2000).
[TAY 01] Y. H. TAY,P.M. LALLICAN,M. KHALID,C. VIARD-GAUDIN and S. KNERR, An Offline Cursive Handwritten Word Recognition System, IEEE Region 10 International Conference on Electrical and Electronic Technology (TENCON), vol. 2, p. 519- 524 (2001).
[VAR 90] A. P. VARGA and R. K. MOORE, Hidden Markov model decomposition of speech and noise. Proc. IEEE Internat. Conf. Acoust. Speech and Signal Process, p. 845-848 (1990).
[VIA 99] C. VIARD-GAUDIN,P.M. LALLICAN,S. KNERR,P. BINTER, The IRESTE On/Off (IRONOFF) Dual Handwriting Database, ICDAR’99, p. 455-459 (2009).
[VIN 04] A. VINCIARELLI and S. BENGIO and H. BUNKE. Offline Recognition of Unconstrained Handwritten Texts Using {HMM}s and Statistical Language Models, IEEE Transactions on PAMI, Vol26, N° 6, p. 709-720, 2004.
[VIA 05] C. VIARD-GAUDIN , P. LALLICAN , S. KNERR, Recognitiondirected recovering of temporal information from handwriting images, Pattern Recognition Letters, v.26 n.16, p. 2537-2548, December (2005).
[WEL 98] C.J. WELLEKENS, J. KANGASHARJU, C. MILESI, The use of meta-HMM in multiflux HMM training for automatic speech recognition, Proc. of Intl. Conference on Spoken Language Processing (Sydney), p. 2991-2994, December (1998).