Evaluation and calibration of agents behavior for immersive simulations

Evaluation and calibration of agents behavior for immersive simulations

Kévin Darty Julien Saunier Nicolas Sabouret

LIMSI-CNRS, UPR 3251, Univ. Paris-Sud, Orsay, France

LITIS, INSA de Rouen, France

LIMSI-CNRS, UPR 3251, Univ. Paris-Sud, Orsay, France

Corresponding Author Email: 
30 April 2016
| Citation

In the context of agent-based simulation, a major issue is to define relevant parameters of the agent model and calibrate them. This issue is yet harder in immersive virtual environments, where intelligent agents reproduce human behaviour and interact with users. We propose to log and analyse agents behaviour to evaluate their similarity to humans behaviour in an immersive virtual environment. The behaviour archetypes obtained by clustering are studied in order to identify agent lacks, capacities and errors. This study enables to 1) dismiss invalid parameter sets, 2) calibrate valid simulations and 3) explain lacks in the agent models.


multi-agent simulation, evaluation, parameters calibration, clustering.

1. Introduction
2. Travaux connexes
3. Méthode d’évaluation des comportements d’agents
4. Cycle d’amélioration
5. Évaluation
6. Conclusion et perspectives

Bosse T., Hoogendoorn M., Klein M. C., Treur J., Van Der Wal C. N. (2011). Agent-based analysis of patterns in crowd behaviour involving contagion of mental states. In Modern approaches in applied intelligence, p. 566–577. Springer.

Caillou P., Gil-Quijano J. (2012). Simanalyzer: Automated description of groups dynamics in agent-based simulations. , p. 1353–1354.

Cali´nski T., Harabasz J. (1974). A dendrite method for cluster analysis. Communications in Statistics-theory and Methods, vol. 3, no 1, p. 1–27.

Campano S., Sabouret N., Sevin E. de, Corruble V. (2013). An evaluation of the cor-e computational model for affective behaviors. In Proceedings of the 2013 international conference on autonomous agents and multi-agent systems, p. 745–752.

Champion A., Éspié S., Auberlet J. M. (2001). Behavioral road traffic simulation with archisim. In Summer computer simulation conference, p. 359–364.

Darty K., Saunier J., Sabouret N. (2014a). Agents behavior semi-automatic analysis through their comparison to human behavior clustering. In Intelligent virtual agents, p. 154–163.

Darty K., Saunier J., Sabouret N. (2014b). Analyse des comportements agents par agrégation aux comportements humains. In 22èmes journées francophones sur les systèmes multi-agents (jfsma 2014). Cépaduès.

Darty K., Saunier J., Sabouret N. (2014c). A method for semi-automatic explicitation of agent’s behavior: application to the study of an immersive driving simulator. In The 6th international onference on agents and artificial intelligence (icaart 2014), p. 81–91. SciTePress.

Delaherche E., Chetouani M., Mahdhaoui A., Saint-Georges C., Viaux S., Cohen D. (2012). Interpersonal synchrony: A survey of evaluation methods across disciplines. Affective Computing, IEEE Transactions on, vol. 3, no 3, p. 349–365.

Doniec A., Mandiau R., Piechowiak S., Espié S. (2008). A behavioral multi-agent model for road traffic simulation. Engineering Applications of Artificial Intelligence, vol. 21, no 8, p. 1443–1454.

Drogoul A., Corbara B., Fresneau D. (1995). Manta: New experimental results on the emergence of (artificial) ant societies. Artificial Societies: the computer simulation of social life, p. 190–211.

Espié S., Auberlet J. M. (2007). Archisim: A behavioral multi-actors traffic simulation model for the study of a traffic system including its aspects. International Journal of ITS Research, vol. 1.

Fehler M., Klügl F., Puppe F. (2005). Techniques for analysis and calibration of multi-agent simulations. , p. 305–321.

Fehler M., Klügl F., Puppe F. (2006). Approaches for resolving the dilemma between model structure refinement and parameter calibration in agent-based simulations. , p. 120–122.

Gonçalves J., Rossetti R. J. F. (2013). Extending sumo to support tailored driving styles. 1st SUMO User Conference, DLR, Berlin - Adlershof, Germany, vol. 21, p. 205–211.

Gratch J., Marsella S. (2005). Evaluating a computational model of emotion. Autonomous Agents and Multi-Agent Systems, vol. 11, no 1, p. 23–43.

Guyot P., Drogoul A. (2005). Multi-agent based participatory simulations on various scales. In Massively multi-agent systems i, p. 149–160. Springer.

Huraux T. (2015). Simulation multi-agent d’un système complexe : combiner des domaines d’expertise par une approche multi-niveau. le cas de la consommation électrique résidentielle [PhD thesis].

Lacroix B., Mathieu P., Kemeny A. (2012). Formalizing the construction of populations in multi-agent simulations. Engineering Applications of Artificial Intelligence.

Lester J. C., Converse S. A. et al. (1997). The persona effect: affective impact of animated pedagogical agents. In Proceedings of the sigchi conference on human factors in computing systems, p. 359–366.

Likert R. (1932). A technique for the measurement of attitudes. Archives of psychology.

Mathieu P., Brandouy O. (2010). A generic architecture for realistic simulations of complex financial dynamics. , p. 185–197.

Pelachaud C. (2009). Modelling multimodal expression of emotion in a virtual agent. Phil. Trans. R. Soc. B: Biological Sciences, vol. 364, no 1535, p. 3539–3548.

Rand W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association, vol. 66, no 336, p. 846–850.

Reason J., Manstead A., Stradling S., Baxter J., Campbell K. (1990). Errors and violations on the roads: a real distinction? Ergonomics, vol. 33, no 10-11, p. 1315–1332.

Salvador S., Chan P. (2007). Toward accurate dynamic time warping in linear time and space. Intelligent Data Analysis, vol. 11, no 5, p. 561–580.

Six L. (2014). Vers un modèle de comportements de véhicules lourds en utilisant une méthode incrémentale basée sur la vérification et l’hystérésis: le modèle archipl. Thèse de doctorat non publiée, Università c Pierre et Marie Curie.

Taillandier P., Drogoul A. (2010). Supervised feature evaluation by consistency analysis: application to measure sets used to characterise geographic objects. In Knowledge and systems engineering (kse), 2010 second international conference on, p. 63–68.

Veremme A., Lefevre É., Morvan G., Dupont D., Jolly D. (2012). Evidential calibration process of multi-agent based system: An application to forensic entomology. Expert Systems with Applications, vol. 39, no 3, p. 2361–2374.