Remote Organoleptic Testing System Using Pairwise Comparison Scale in Sensory Evaluation of Food Products

In the food industry or food service companies, product quality is a critical factor in determining market competitiveness [1]. The quality of food products is evaluated not only based on physical properties, chemical composition, and contamination levels (both microbiological and toxic substances) but also on sensory attributes such as taste, aroma, texture, and colour [2, 3]. Therefore, sensory evaluation is a crucial tool to ensure that food products meet consumer expectations and quality standards, reflecting the value for money.

Sensory evaluation is the science of measuring, analysing, and interpreting consumer reactions to products as perceived by the senses [4]. One method used to assess these sensory attributes is organoleptic testing, which heavily relies on human sensory perception [3].

However, the primary challenge in organoleptic testing is the subjectivity of assessment, which can affect the consistency and accuracy of results [5]. Each individual has different sensory perceptions, leading to potential variability in evaluations conducted by panelists [6]. This creates a need for more structured and objective methods in the execution of organoleptic tests. Moreover, the traditional approach to sensory evaluation requires panelists to be physically present in the same location, which can be logistically challenging and costly, especially when handling large-scale testing or testing across different geographic locations [7, 8]. The COVID-19 pandemic has further highlighted the limitations of in-person testing, driving the search for alternative approaches that enable sensory evaluation to continue remotely while maintaining data accuracy and reliability.

Remote sensory evaluation systems offer a promising solution by leveraging web-based platforms to allow panelists to perform evaluations from any location. Such systems reduce logistical burdens, minimize costs associated with travel and facility usage, and enable broader participation from geographically dispersed panelists. Remote systems also offer scalability, allowing for the recruitment of larger and more diverse samples, which can enhance the representativeness of sensory data. These practical applications are vital for food companies aiming to accelerate product development, optimize sensory testing workflows, and maintain business continuity during disruptions like pandemics.

Several previous studies have examined remote sensory evaluation. Dinnella et al. [7] systematically compared laboratory-based and remote testing settings across various sensory evaluation techniques, generally providing support for the use of remote testing due to its flexibility and convenience for participants. Shi et al. [9] compared liking scores from home-based testing and central location testing, using repeated measures with the same panel to test the validity of cross-location comparisons. The comparison results indicated that higher scores were generally observed in home-based testing. This suggests the feasibility of conducting sensory research in remote locations, facilitated by courier services and smart technology, while considering standard practices for remote sensory testing to ensure consistent comparisons across investigations [10].

1.1 Proposed solution

Based on the challenges encountered and supported by several studies on remote sensory evaluation, the authors designed a web-based organoleptic testing system that can be accessed remotely. This system implements the pairwise comparison scale method in organoleptic testing. This method allows for a more objective evaluation by directly comparing each pair of attributes [11], thereby reducing subjectivity and producing more accurate data. This study aims to develop and assess the feasibility of using a remote organoleptic testing system with the implementation of the pairwise comparison scale.

The remote organoleptic testing system was developed using PHP and JavaScript programming languages, with MySQL as the database. The system was tested using questionnaires distributed to Sensory Experts and Web Development Experts.

1.2 Theoretical study

Organoleptic testing is a sensory evaluation method used to assess the quality of a product based on human sensory perceptions, such as taste, aroma, texture, and colour [12]. This testing is typically conducted by panelists who use their senses to evaluate the sensory attributes of a product, particularly in the food industry [13, 14]. Sensory quality significantly influences consumer satisfaction and the competitiveness of a product in the market [15]. Consumers tend to prefer products that meet their expectations in terms of taste, aroma, texture, and colour. Additionally, organoleptic testing aids in the development of new products, the reformulation of existing products, and the monitoring of product quality consistency over time [16].

The pairwise comparison scale is a method used to directly compare two elements or attributes to determine which is superior or has a higher intensity [11]. This method is widely applied in various fields, such as decision-making, psychometrics, and criteria assessment in more complex evaluation processes like the Analytical Hierarchy Process (AHP) [17].

The principles of the pairwise comparison scale are based on the concept of relative comparison, where each element is evaluated against the other elements to establish a hierarchy of importance or intensity. This approach allows for more precise discrimination between attributes, compared to an absolute rating scale where panellists assign a value to each attribute separately. The scope of application of the pairwise comparison scale extends beyond sensory evaluation to areas such as decision-making, education, and psychology, where comparative judgement is essential. In sensory evaluation, the scale offers a systematic approach to identify the relative importance of sensory attributes (e.g., taste, aroma, colour, texture) in a structured manner.

Compared to other assessment methods such as hedonic scales or Likert scales, the pairwise comparison scale has unique advantages. While hedonic scales rely on subjective preferences and absolute ratings (e.g., ‘like’ or ‘dislike’), pairwise comparisons force a direct relative judgement between two elements, reducing ambiguity and cognitive bias. This direct comparison method can reveal more subtle differences in sensory attributes, which may be missed in traditional judgement systems. However, this method also has limitations, such as increased time requirements for multiple comparisons and cognitive load on panellists as the number of attributes increases.

In the sensory evaluation of food products, more commonly used methods include structured rating scales such as the hedonic scale or intensity scale [18]. However, the pairwise comparison scale offers a more direct and systematic approach to comparing sensory elements, although its use has not yet become standard in traditional organoleptic testing.

The development of a web-based system involves the use of technologies such as PHP, JavaScript, and MySQL, which are core components in building dynamic and interactive web applications. PHP (Hypertext Preprocessor) is a server-side programming language used for managing data and executing business logic in web applications. JavaScript functions as a client-side programming language, enabling the creation of an interactive and responsive user interface. MySQL, as a relational database management system (RDBMS), is used to store and manage data [19].

The remote organoleptic testing system was developed using PHP programming language, JavaScript for user interactivity, and MySQL for database management. This system is designed to facilitate the remote organoleptic testing process, enabling panelists to work from home.

Figure 3 illustrates the system design using a use case diagram, which is the result of gathering user requirements and technical specifications through interviews with sensory experts and web developers. The system involves two actors: admin and user. Users can register, input responses, and edit their profiles. Meanwhile, the admin can view respondent results and manage user data, such as editing passwords, resetting responses, and deleting data.

2-3.png

Figure 3. Use case diagram

Figure 4 illustrates the account registration page, where the user is required to enter important information such as full name, occupation, email address, gender, address, respondent/panellist category, age, and password. This information is required to create a user profile that will be used during the sensory evaluation process.

2-4.png

Figure 4. User account registration page

Once the registration process is complete, users can proceed to login using the email and password they provided during registration. Figure 5 shows the login page, where users must enter their registered email and password to access the system.

Figure 6 shows the organoleptic testing system using four criteria, namely: (1) taste, (2) aroma, (3) colour, and (4) texture. A pairwise comparison scale is then used to compare these four criteria.

2-5.png

Figure 5. User login page

2-6.png

Figure 6. Organoleptic test with pairwise comparison scale

In pairwise comparison, each score provided by the respondent reflects the relative importance of one element (e.g., flavour) compared to another (e.g., colour). The scale used includes the values detailed in Table 5.

Table 5. Rating scale

The scale ranges from 9 to 1 for positive ratings, and from 1/2 to 1/9 for negative ratings. If a respondent compares the flavour criterion with the colour criterion and assigns a score of 7 on the rating scale, it indicates that the flavour criterion is considered seven times more important than the colour criterion. However, if the comparison is made in the opposite direction, i.e., comparing colour with flavour, the score would be 1/7, approximately 0.143.

Figure 7 shows the admin menu interface. The admin can view all respondents from the organoleptic test. The collective results from the respondents will later be weighted to determine which criterion is the most important among the four criteria tested for the food product.

2-7.png

Figure 7. Organoleptic test respondent results

The system feasibility assessment was conducted using a structured questionnaire designed to evaluate various aspects of the system. The assessment focused on seven main criteria: System Accessibility (SA), Questionnaire Filling with Comparison Scale (CS), Questionnaire Submission and Storage (QS), User Interface (UI), Functionality (FU), Performance (PE), and User Satisfaction (US). Each criterion was assessed using several statements that were rated by respondents on a five-point Likert scale, ranging from ‘Strongly Disagree’ (1) to ‘Strongly Agree’ (5).

The rationale behind the design of this questionnaire is to ensure an evaluation that covers functionality, usability, and overall user satisfaction. Each criterion was selected based on its relevance to the success of the remote sensing evaluation system. For example, ‘System Accessibility’ assesses whether users can access the system on various devices, while ‘Questionnaire Delivery and Storage’ ensures that user responses are safely stored and retrievable.

Table 6 shows the results of the questionnaire assessment from 7 respondents, with a total score of 457 out of a maximum of 525, resulting in an overall percentage of 87.0%. According to the Feasibility Scale in Table 4, this result places the system in the "Very Decent" category (81-100%). The assessment results indicate that the tested system has a very high level of feasibility. The "User Satisfaction (US2)" aspect received the highest score with a percentage of 97.1%, indicating that the majority of users are very satisfied with the system. The "Questionnaire Completion with Comparison Scale (CS1)" aspect also showed very positive results with a percentage of 94.3%, reflecting the ease of use and clarity of the comparison scale.

Table 6. Questionnaire assessment results

Other aspects such as "User Interface (UI1)" and "Questionnaire Submission and Storage (QS1)" also demonstrated strong results, with percentages above 90%, indicating that the user interface is considered intuitive and the system reliable in managing questionnaire data. Some aspects, such as "System Accessibility (SA2)" with a percentage of 77.1%, still require some improvement. Overall, the assessment indicates that this system is highly feasible for implementation in real-world operational scenarios.

The use of technology in sensory evaluation, particularly in remote organoleptic testing, offers several advantages. One key advantage is the increased efficiency and accuracy in data collection and processing. A web-based system allows researchers to collect data from a large number of panelists simultaneously, without the logistical challenges typically associated with conventional testing. Flexibility is also a significant benefit of using technology in organoleptic testing. This study aligns with references [7, 9], where the system can be accessed from anywhere, and panelists are not required to be in a specific location to conduct assessments. This flexibility opens up opportunities to involve panelists from diverse backgrounds and geographical locations, enriching the variety of data obtained and enabling more representative testing of a broader population.

However, the implementation of this technology also faces challenges, consistent with the findings [10], which require strict technical and procedural considerations. These include ensuring that the devices used by panelists meet the necessary standards for sensory evaluation. For instance, screen quality for visual assessments or audio devices for sound evaluation must be consistent among panelists to avoid biased results. Additionally, factors such as internet connectivity, system reliability, and the panelists' ability to use the system must be taken into account. Therefore, training and support for panelists in using the web-based system are essential to ensure that the organoleptic testing process runs smoothly and yields accurate data.