© 2021 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
The validation of the operationality of models is considered a crucial step in the model development process. Recent developments in Digital Twinning (DT) enable the online availability of operational data from the physical asset required for operational validation. The benefits of DT in situations where operational validation has formed a basis for model adaptation has also been demonstrated. However, these benefits within DT have not been fully utilized due to the lack of an approach for benchmarking the required quantity, quality and diversity of validation data and performance metrics for online model validation and adaptation. Therefore, there is a need for a framework for benchmarking validation data and metrics requirements during model validation in different domains. An approach for bench-marking the required quantity, quality and variability of validation data and performance metric(s) for online model adaptation within DT is proposed. The approach is focused on addressing the problem of parameter(s) uncertainty of a predictive model within its uncertainty boundary. It involves generating virtual test models, a primary and another reference model for the performance evaluation of one compared to the another with the benchmarked validating data and metrics within DT. This process is repeated until the dataset and/or metric(s) are promising enough to validate primary model against the reference model. The proposed approach is demonstrated using BEASY – a simulator designed to pre- dict protection provided by a cathodic protection system to an asset. In this case, a marine structure is the focus of the study, where the protection potentials to prevent corrosion are predicted over the life of the structure. The algorithm(s) for the approach are provided within a Scientific Software (MATLAB) and integrated to the simulator-based cathodic-protection model.
Adaptive Simulation Validation, Cathodic-Protection Digital Twin, Validating Framework
[1] Sargent, R.G., Verification and validation of simulation models. Journal of Simulation, 7(1), pp. 12–24, 2013. https://doi.org/10.1057/jos.2012.20
[2] Oliveira, H.L. & Leonel, E.D., Constitutive relation error formalism applied to the solution of inverse problems using the BEM. Engineering Analysis with Boundary Elements, 108, pp. 30–40, 2019. https://doi.org/10.1016/j.enganabound.2019.07.023
[3] Sehgal, S. & Kumar, H., Structural dynamic model updating techniques: a state of the art review. Archives of Computational Methods in Engineering, 23(3), pp. 515–33, 2016. https://doi.org/10.1007/s11831-015-9150-3
[4] Liu, Y., Chen, W., Arendt, P. & Huang, H.Z., Toward a better understanding of model validation metrics. Journal of Mechanical Design, 133(7), 2011. https://doi.org/10.1115/1.4004223
[5] Pace, D.K., Modeling and simulation verification and validation challenges. Johns Hopkins APL Technical Digest, 25(2), pp. 163–72, 2004.
[6] Madni, A.M., Madni, C.C., & Lucero, S., Leveraging digital twin technology in model- based systems engineering. Systems, 7(1), p. 7, 2019. https://doi.org/10.3390/sys- tems7010007
[7] Wright, L., & Davidson, S., How to tell the difference between a model and a digital twin. Advanced Modeling and Simulation in Engineering Sciences, 7(1), pp. 1–3, 2020. https://doi.org/10.1186/s40323-020-00147-4
[8] Barricelli, B.R., Casiraghi, E., & Fogli, D., A survey on digital twin: Definitions, char- acteristics, applications, and design implications. IEEE Access, 14(7), pp. 167653–71, 2019. https://doi.org/10.1109/ACCESS.2019.2953499
[9] El Haouzi, H.B., Thomas, A., & Charpentier, P., Toward adaptive modelling & simulation for IMS: The Adaptive Capability Maturity Model and future challenges. IFAC Proceedings Volumes, 46(7), pp. 174–179, 2013. https://doi.org/10.3182/20130522-3- br-4036.00104
[10] Brynjarsdóttir, J., & O’Hagan, A., Learning about physical parameters: The importance of model discrepancy. Inverse Problems, 30(11), p. 114007, 2014. https://doi.org/ 10.1088/0266-5611/30/11/114007
[11] Higdon, D., Gattiker, J., Williams, B. & Rightley, M., Computer model calibration using high-dimensional output. Journal of the American Statistical Association, 103(482), pp. 570–583, 2008. https://doi.org/10.1198/016214507000000888
[12] Sarin, H., Kokkolaras, M., Hulbert, G., Papalambros, P., Barbat, S. & Yang, R. J., Comparing time histories for validation of simulation models: error measures and metrics. Journal of Dynamic Systems, Measurement, and Control, 132(6), 2010. https://doi.org/ 10.1115/1.4002478
[13] Hora, J. & Campos, P., A review of performance criteria to validate simulation models. Expert Systems, 32(5), pp. 578–595, 2015. https://doi.org/10.1111/exsy.12111
[14] Goller, B., Broggi, M., Calvi, A., & Schuëller, G.I., A stochastic model updating technique for complex aerospace structures. Finite Elements in Analysis and Design, 47(7), pp. 739–752, 2011. https://doi.org/10.1016/j.finel.2011.02.005
[15] Glaessgen, E. & Stargel, D., The digital twin paradigm for future NASA and US Air Force vehicles. In 53rd AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics and materials conference 20th AIAA/ASME/AHS adaptive structures conference 14th AIAA (p. 1818), 2012.
[16] Ye, Y., Yang, Q., Yang, F., Huo, Y. & Meng, S., Digital twin for the structural health management of reusable spacecraft: A case study. Engineering Fracture Mechanics, 234, p. 107076, 2020. https://doi.org/10.1016/j.engfracmech.2020.107076
[17] Fuller, A., Fan, Z., Day, C. & Barlow, C., Digital twin: Enabling technologies, challenges and open research. IEEE Access, 8, p. 108952-71, 2020. https://doi.org/10.1109/ access.2020.2998358
[18] Gabor, T., Belzner, L., Kiermeier, M., Beck, M.T. & Neitz, A., A simulation-based architecture for smart cyber-physical systems. In 2016 IEEE international conference on autonomic computing (ICAC) (pp. 374–379). IEEE, 2016.
[19] Park, B. & Schneeberger, J.D., Microscopic simulation model calibration and validation: case study of VISSIM simulation model for a coordinated actuated signal sys- tem. Transportation Research Record, 1856(1), pp. 185–192, 2003. https://doi.org/ 10.3141/1856-20
[20] Tahmasebi, F., Zach, R., Schuß, M., & Mahdavi, A., Simulation model calibration: An optimization-based approach. In Proceedings of Fourth German-Austrian IBPSA Conference, BauSIM (pp. 386–391), 2012.
[21] Adey, R.A., Modelling of Cathodic Protection Systems. United Kingdom, WIT, 2006.
[22] Ni, D., Leonard, J.D., Guin, A. & Williams, B.M., Systematic approach for validating traffic simulation models. Transportation Research Record, 1876(1), pp. 20–31, 2004.
[23] Adey, R., Peratta, C. & Baynham, J., Corrosion Data Management Using 3D Visualisation and a Digital Twin. In NACE International Corrosion Conference Proceedings (pp. 1–13). NACE International, 2020.
[24] Kear, G. & Walsh, F.C., The characteristics of a true Tafel slope. Corrosion and Materials, 30(6), pp. 51–55, 2005.
[25] Higham, D.J. & Higham, N.J., MATLAB guide. Society for Industrial and Applied Mathematics, 2016.