Evaluation of Automatic Classification of Handwritten Signatures for Online Authentication. Évaluation sur la Création Automatique de Classes de Signatures Manuscrites pour L’Authentification En Ligne

In online handwritten signature verification systems[PLAMONDON 89,LECLERC 94,JAIN 02],adaptation to a specific user or a group of users is a key point since the parameters that determine the acceptation or the rejection of a user are quite sensible.Indeed,biometric data are behavioral and not physical which makes them more variable.Moreover, few learning data are available and the biometric profile of a user can only be determined from a limited number of signature acquisitions.This is why,in the same way as it is done in handwriting recognition,we wanted to study both how classes of handwritten signatures can be automatically determined and what is the impact of these classes on a verification system. 

The verification system,on which this study is based,is not new itself.It is a Coarse To Fine approach that uses the Dynamic Time Warping algorithm (DTW)[WIROTIUS 05a].This is a ligth system that was specifically designed to be embedded on smartphones or PDAs.During the enrolment step,5 reference signatures are acquired from the user.One should notice here that the system forces the user to provide 5 signatures that are roughly of the same style (with similar total length and total duration).This is coherent with the database we used for our experiments and with the classification methods employed in our study.After the preprocessing (normalization – translation,rotation and scale – and removal of the unnecessary points:only the points with minimal local speed are kept[WIROTIUS 05b]),these signatures represent the biometric profile of the user.The authentication itself is based on the Coarse To Fine approach. In the coarse step,signatures that are too dissimilar from the profile are rejected.This assessment is peformed from a simple comparison based on the total length and the total duration of the signatures.For the signatures that are not rejected at this first step,a finer comparison is performed using the Dynamic Time Warping algorithm.The final acceptation or rejection of a signature depends on a threshold m that is learnt from an experimental database to reach the equal error rate (EER). 

The main drawback of the previous system,considering its parameters,is the threshold m.This is the reason why we tried to find an automatic mechanism to determine classes of signatures and to adapt the system (m) to these classes. In fact,we used clustering algorithms and we studied the impact on the performances of the system of:the number of classes; the clustering feature space; the algorithm used. 

The process to determine the classes consists in using a clustering algorithm on a learning dataset that contains the biometric profiles of several signers.Then,for each of the k classes obtained,we find a local threshold mk that enables to obtain the EER on the class k.Next,during the enrolment of a new signer,the class to which the signer belongs is considered to be the one in which most of the signatures of his profile are (this is why it is better that all signatures of the profile should be of the same style).The threshold that will be used during the authentication step is the one corresponding to that class. 

The experiments were performed using the SVC database using the leave-one-out protocol to learn the thresholds and to test the system on different data.Without using the classes of signatures,the system can achieve 1.94% of EER on this dataset.1 We next conducted several experiments.Firstly,we used the K-means algorithm with several values for K and several feature subspaces based on the global characteristics of the signatures (static and dynamic).The results show that the EER can be decreased to 1.84% for some small values of K.For most of the feature subspaces used, over K = 3,the results are not interesting (equal or worse to those obtained with the original system).Moreover,the results do not seem to be stable depending on the value of K.In fact,observing the result of the clustering,it seems that for too many signers,the different signatures in their profile end up in different classes,not in a unique one,which indicates that the clustering is not stable,despite our constraint on the signatures of the profile.The best results – considering the compromise between the EER performance and the stability of the clustering,i.e.the standard deviation of FAR and FRR– are obtained using the two principal axes (using a principal component analysis) and 2 classes.The second experiment was equivalent but using the fuzzy C-means instead of the K-means.The results show a great improvement in terms of stability of the clustering and a general improvement of the EER for small values of K.The best result was obtained using two classes and again the two principal axis.In this case,the EER fall to 1.66% which represents an improvement of 14.4%.Nevertheless,deeper studies of the results of the clustering show that we are still unable to find a classification that is valid for all signers:there are still signatures of a same signer that belong to different classes.This phenomenom of course increases with higher values of K.Then our perspectives are either to consider that a signer could belong to several classes (even if he always signs with the same style) or to operate several classifications,using different feature spaces,and to choose for a signer the most pertinent classification.

Résumé

Dans ce papier,nous avons cherché à évaluer l’intérêt de la création de classes de signatures manuscrites pour un système d’authentification en ligne. Notre objectif est d’étudier:comment créer automatiquement des classes de signatures (ou styles de signature); comment prendre en compte ces classes pendant l’authentification afin de spécialiser le système lors de l’enrôlement d’un utilisateur; quelles améliorations nous pouvons en attendre. Dans notre étude,la création des classes s’effectue grâce à deux algorithmes de clustering et en se basant sur différents sous-espaces de description des signatures. La spécialisation du système consiste à déterminer non plus un seuil de décision (acceptation ou rejet) global au système (i.e. le même pour toutes les personnes qui s’enrôleront) mais un seuil adapté à chacune des classes. En termes d’évaluation,nous nous sommes plus particulièrement attachés à étudier l’impact de la classification (en fonction de l’algorithme de classification,du nombre de classes,de l’espace de description) sur les performances d’un système d’authentification basé sur une approche Coarse To Fine et l’algorithme Dynamic Time Warping. Les résultats expérimentaux sur la base SVC montrent que l’on peut améliorer les performances en diminuant le taux d’erreurs égales de 14,4%. Cependant la sensibilité de la classification est très grande et la notion de classe unique pour un signataire semble trop restrictive.