Application of Multilayer Perceptron Neural Network in Geological Modeling of Categorical Variables: A Case Study in Peru

The objective of this research was to use an Artificial Neural Network Multilayer Perceptron (ANN-MLP) for geological modeling of categorical variables, using a database of 5,654 samples obtained from a diamond drilling campaign in a mine in northern Peru. The ANN-PML architecture consisted of an input layer of 5 neurons, three hidden layers with 100, 50 and 20 neurons respectively, and an output layer with 5 neurons, optimized with the Adam algorithm. Variables analyzed included geographic coordinates (east, north and elevation), copper and molybdenum concentrations


INTRODUCTION
In the mining field, geostatistics plays a crucial role in the identification and estimation of mineral resources, as well as in the definition of essential rock characteristics [1].In this context, categorical geological models stand out for their importance by directly correlating with metallurgical processes and determining plant dispatch strategies in mine planning [2].Although various neural network techniques, such as deep neural networks [3] and graphical networks [4], have been explored in this field, the application of Multi-Layer Perceptron Artificial Neural Networks (ANN-MLP) emerges as a significant innovation.The ANN-MLP [5] is distinguished by its ability to efficiently process and classify large volumes of geological data, surpassing conventional methods and playing a crucial role in optimizing mining planning and operation.
Fundamentally, geological modeling focuses on extrapolations and estimates related to the geological particularities of the study area, based on a limited set of information [20].Usually, such modeling perceives geological entities as discrete variables, such as rock types or layers, making the modeling task align with a categorization activity based on such variables [21].At its core, machine learning, a branch of artificial intelligence, aspires to acquire knowledge from existing data to, based on this, develop estimates and insights into unexplored scenarios [22,23].This congruence in objectives has led to machine learning playing a significant role in the construction of three-dimensional geological models.
As for the prominent tools in this field, Artificial Neural Networks (ANNs), especially the multilayer perceptron model, have established themselves as central methods, standing out for their ability to emulate biological neural processing and for their ability to interconnect elements via synaptic weights [32].
Although various machine learning techniques have been explored in geological modeling, there is a gap in the application of ANN-MLP specifically in the geological modeling of categorical variables.This study aimed to fill this gap, demonstrating the efficacy of the ANN-MLP in the accurate classification of rock types and in enhancing threedimensional geological modeling.
In this article, the organization of the content is arranged as follows: Section 2 sets out in detail the methodology adopted, elucidating the underlying fundamentals of the multilayer artificial neural network and its specific suitability for geological modeling.Subsequently, in Section 3, this methodology is contextualized and applied to a specific case study, providing a pragmatic demonstration of the versatility and applicability of this technique.We conclude in Section 4, where the main conclusions drawn from this work are outlined, highlighting the contributions and advances achieved, as well as proposing directions for further research.

MATERIALS AND METHODS
The collection of geological data began with obtaining diamond drilling records, rock samples, and other relevant reports that evidence the categorical variables of interest.A rigorous selection process was implemented for these records, prioritizing locations with significant geological diversity and high mineral potential, typical of northern Peru.This ensured that the dataset reflected a wide range of geological contexts, essential for the efficacy of the ANN-MLP model.After collection, a data cleansing process was carried out, focusing on the identification and removal of missing values or inconsistencies, thus ensuring the quality and accuracy of the processed information.
Once the data were cleaned, categorical variables like rock type were transformed into numerical representations using techniques such as one-hot encoding, to convert categorical variables like rock types into binary numerical formats.This technique is crucial for enabling the neural network to efficiently process the data [33].In addition, other transformation techniques were applied, such as the normalization of continuous variables, to standardize the scale of the data and improve the accuracy of the model [34].Table 1 shows the database used, which includes data such as coordinates (east, north, and elevation), grades of copper (Cu) and molybdenum (Mo), and the type of rock.The encoding of rock types was done by assigning numerical values to each category, such as Magnetite Skarn (1), Granodiorite (2), Dacite Porphyry (3), Limestone Sediments (4), and Catalina Volcanics (5), providing a clear and systematic approach for subsequent analysis.
The neural network design focused on determining an optimal architecture.A specific number of hidden layers and neurons per layer was established based on the characteristics and complexity of the data.Tanh activation functions were chosen for the hidden layers, while for the output layer, due to the multi-category nature of the problem, the "softmax" function was chosen.The initialization of weights was performed considering strategies that favored convergence during training [35].Within multilayer neural networks, the forward propagation of a neuron is generally described by Dowd and Pardo-Igúzquiza [36], and Henao et al. [37]: where,  () are the weights,  () in the input,  () is the bias, and  is the activation function.
Prior to training, the data set was divided into three subsets: training, validation, and testing, ensuring adequate representation of all classes in each subset.The optimizer selected for the training process was "adam", accompanied by a learning rate appropriate to the behavior of the loss function.The cost function "categorical_crossentropy" was used to guide the training.During this process, the validation set was monitored using techniques such as "early stopping" to prevent overfitting.Training is performed by iteratively adjusting the weights and biases using the backpropagation algorithm [38].
The cost function J is optimized with respect to w and b using the optimizer "adam" the iterative process is described by: where, α is the learning rate.
Once the neural network was trained, its performance on the test set was evaluated using accuracy, completeness, and F1score metrics.Additionally, a confusion matrix was made to visualize and analyze the errors and successes of the model in each of the geological categories [39,40].The architecture of the Multi-Layer Perceptron Artificial Neural Network (ANN-MLP) used in this study consists of an input layer with 5 neurons, followed by three hidden layers with 20, 15, and 10 neurons respectively.The Tanh activation function was used in the hidden layers, while the softmax function was employed in the output layer.The Adam optimizer was utilized for network optimization during the training process.The network was trained with a total of 1000 epochs and a batch size of 64, and training was halted when the network achieved optimal results as shown in Table 2.
The graphical representation of the neural network used in this research is shown in Figure 1.

Exploratory data analysis
Table 3 shows the statistics of the database to be worked, where the total amount, mean, standard deviation (Std), minimum, quartile 1 (Q1), quartile 2 (Q2), quartile 3 (Q3) and maximum of the variables present in the database are shown.Likewise, there is a categorical variable which is the type of rock (Rock) whose values are minimum 1 and maximum 5. Figure 2 shows the East-North view of the diamond drill holes with respect to copper grade, molybdenum grade and rock type.Figure 3 shows the comparison of ore grade and frequency with respect to rock type, where it can be seen that for copper grade the maximum grade is found in lithology Granodiorite and for molybdenum the highest grade is located in lithology Calcareous Sediments, likewise, the highest amount of data in rock type is located in lithology Granodiorite for copper grade and molybdenum grade.
Table 4 shows the statistics of the rock type with respect to the copper grade.In which, for rock Magnetite Skarn there is a total of 906.0 data and the maximum copper grade is 1.873%, in rock Granodiorite there is a total of 3317.0 data with a maximum copper grade of 2.949%, in rock Dacite Porphyry there is a total of 1079.0 data with a maximum copper grade of 1.600%, rock Calcareous Sediments has a total of 307.0 data with a maximum copper grade of 1.894% and rock Catalina Volcanics has a total of 45.0 data with a maximum copper grade of 0.570%.
Figure 4 shows the distribution of copper and molybdenum grades, it is observed that copper grades have a mean of 0.43% and its grades are distributed in higher amounts in the ranges of 0.0% to 1.0%.Molybdenum grades have a mean of 0.02% and are distributed in greater quantities in the range of 0.0% to 0.05%.Figure 5 shows a graph of copper grade probability with respect to rock type, where it is observed that the highest probability of higher copper grades is in rock type Granodiorite and the lowest copper grades are found in rock type Catalina Volcanics.
Figure 6 shows a scatter graph showing the correlation between the variables of interest present in the database.It is observed that the correlation between the copper grade and the molybdenum grade is 0.14 (14%), which indicates that it is a very low correlation.Figure 8 showcases the confusion matrix for the test data, revealing high accuracy in the classification of Geological Model 2, with 600 instances accurately predicted out of 820.
Similarly, Class 1 achieved 69 correct predictions out of 114, while Class 3 recorded 85 correct predictions out of 154.The other classes also exhibit variations in classification accuracy.Table 5 presents the classification metrics for each class generated in the geological modeling.It is observed that Class 5 achieved the highest accuracy in classification at 78%, followed by Class 2 with 73%.Furthermore, the highest recall was observed for Class 2 at 87%, indicating the model's adept ability to correctly identify instances within the class.When comparing the accuracy of the Multilayer Perceptron Artificial Neural Network model with prior studies, it is noted that while it does not surpass the Deep Neural Network's performance at 84.4%, it does show considerable improvement over Naï ve Bayes at 54% and aligns with Support Vector Machines at 65%.Moreover, its accuracy approaches that of Random Forest at 76%, underscoring the competitiveness of this method in geological modeling as shown in Table 7.

Results of geological models generated with the ANN-MLP
In order to perform the geological modeling, it is necessary to create a block model.Table 8 shows the dimensions of the block model, where the origin and maximum coordinates of east, north and elevation are included, the block size was 20.0 meters, generating a total of 80 blocks in the X coordinate, 108 blocks in the Y coordinate, and 43 blocks in the Z coordinate.An estimated total volume of 8000 m³ was calculated.Using a density of 2.30g/cm³ , the total tonnage of material (either ore or waste) within the block model amounts to approximately 6835.968 million tons, with each block contributing 18400 tons.Table 9 outlines the tonnage of material for each geological model, with class 2 having the largest amount.
Figure 10 presents the visualization of the highlighted geological model of the neural network in a general isometric view with the block model, where the five types of geology present are observed.
Figure 11 shows separate representations of the geological modeling for each specific geological category, with geology 2 having the largest number of blocks present, followed by geology 1, 3, 4 and 5.

CONCLUSIONS
This study employs a Multi-Layer Perceptron Artificial Neural Network (ANN-MLP) in geological modeling, contributing notable advancements to the field.The performance of the ANN-MLP, with an average accuracy of 66% and an F1-Score of 58%, demonstrates its robustness in classifying geological variables.
However, the study is not without limitations.The potential for data biases and the risk of model overfitting necessitates further investigation.Future research should aim at enhancing data diversity, refining the model to prevent overfitting, and incorporating a broader spectrum of geological variables.
The practical applications of this research are significant, offering a scalable and efficient approach to geological modeling.This can lead to more accurate resource estimations and potentially transform mining operations, underscoring the importance of artificial intelligence in advancing geological sciences.

( a )
Diamond drill holes with respect to copper grade (b) Diamond drill holes with respect to molybdenum grade (c) Diamond drill holes with respect to rock type

Figure 2 .
Figure 2. East-North view of the diamond drill holes

( a )Figure 3 .
Figure 3.Comparison of ore grade with respect to rock type frequency

Figure 5 .
Figure 5. Probability graph of copper grades with respect to rock type 3.2 Neural network application Figure 7 displays the performance evolution of the Multi-Layer Perceptron Artificial Neural Network (ANN-MLP) model throughout the training epochs.A consistent decrease in the loss function is observed until it stabilizes around 0.54, while the categorical accuracy reaches a peak of 0.79.Regarding validation, the validation loss shows a decreasing trend until a minimum of 0.83, and the validation categorical accuracy remains at 0.66.These results indicate that the model improves its predictive capability with each epoch.Figure8showcases the confusion matrix for the test data,

Figure 9
Figure 9 shows the ROC (receiver operating characteristic) curve demonstrating the model's ability to distinguish each class from all others.The micro-averaged ROC curve, with an area of 0.92, highlights the overall high classification ability of the model.Meanwhile, individual class curves reveal specific variations in sensitivity and specificity, indicating how well each class is identified relative to the rest.

Table 1 .
Database to be worked

Table 2 .
Hyperparameters of the artificial neural network Figure 1.Multilayer Artificial Neural Network (MLP)

Table 5 .
Classification metrics for each geological modeling class

Evaluation of the neural network trough cross validationTable 6
provides an overview of the average crossvalidation metrics for a neural network model applied in geological classification.The table indicates an accuracy of 0.66, suggesting that the model correctly predicts 66% of the instances.The recall rate of 0.56 implies that the model can identify 56% of the relevant instances across all classes.The F1 score of 0.58 reflects a balanced measure of model precision and recall, indicating a moderate level of prediction accuracy.

Table 8 .
Dimensions of the block model

Table 9 .
Material tonnage for each geological model