Towards Sustainable Water Consumption: Developing and Validating a Proposed Behavioral Measurement Instrument in the Context of Malaysia

The fundamentals of ethical, methodologically sound, and transparent research depend on its capacity to accurately describe and appropriately measure the phenomena it aims to explore. This capacity is fundamentally represented by an instrument comprising domains that may be adopted, adapted, or newly developed, all anchored in the concept of validity [1, 2]. Validity, in turn, is a critical factor in the selection and application of an instrument both in practice and research. It can be defined as “the degree to which evidence and theory support the interpretation of test scores entailed by the proposed use of tests” or the extent to which the instrument measures what it intends to measure [3-5]. Comprehending the concept of validity enables researchers to appreciate its complexity and its critical role in upholding research integrity [1]. However, validity is not one-dimensional and cannot be assessed in isolation or through a single method. In this context, the study underwent an expert pretest, which provided an initial validity check on the appropriateness of the measure by evaluating whether it appears to assess what it is intended to, based on subjective judgment. Consequently, the quantification of content validity, as the next layer of the research validation process, ensures that the instrument is both conceptually sound and operationally captures all relevant domains, producing findings that genuinely reflect the phenomena under study, as judged by experts [1, 3, 6, 7].

The instrument was designed to assess water consumption behavior in Malaysia, particularly within the context of water demand management. This area has become a crucial policy strategy for addressing issues related to water stress and environmental sustainability [8]. In fact, the Organization for Economic Cooperation and Development (OECD) projected in 2012 that water demand would increase by more than 50% between 2000 and 2050, driven by economic industrialization and development, as well as a parallel increase in population, which would elevate domestic consumption by 130% [9]. A key issue is striking a balance between the accessibility and affordability of water, recognized as a basic human right, while aligning economic benefits with long-term sustainability goals [10]. The economic aspect of water, as both a commodity and a public good, inevitably comes into play to facilitate a more effective allocation of water resources. Achieving sustainable development indeed requires the efficient use of resources, along with the advancement of technologies and practices that are both environmentally sustainable and economically viable [11].

The increasing gap between water supply and demand in recent years underscores the growing severity of water stress. None of the targets under Sustainable Development Goal (SDG) 6, specifically Target 6.1, which aims to “achieve universal and equitable access to safe and affordable drinking water for all” by 2030, are on track to be achieved [12]. In 2015, when the SDGs were introduced, approximately 70% of the global population had safe drinking water, and this figure has gradually increased recently [13] (see Figure 1). However, if progress continues at this slow pace, we will not achieve the target of universal and equitable access to safe and affordable drinking water by 2030 [13]. Water stress is also affecting several developing countries, endangering their survival and economies [10]. In this regard, the crisis is caused by two main challenges: in certain nations, the primary issue is economic scarcity, where water is costly for potable purposes, while in other countries, the major barrier is the limited availability of water resources [14].

Malaysia, as a water-rich country, has raw water extraction totaling 17,299 million liters per day, with sources primarily from surface water, constituting 80.2% from rivers and 17.5% from dams, while groundwater accounts for 2.3% [15]. Most Malaysians have access to basic amenities and facilities for clean and safe water, with piped water availability in households reaching almost universal coverage at 97% in both urban and rural areas as of 2023 [15]. Despite living in a water-rich country, consumers lack strong internalized norms for conservation [16]. The average daily water consumption is 225 liters per capita per day (LCD), exceeding the United Nations’ recommended limit of 165 LCD [15]. Furthermore, the water tariffs for domestic water in Malaysia are highly subsidized, making it affordable for the average household [17]. Although these subsidized tariffs enable consumers to access services at a minimal expense, they inadvertently cultivate the perception that water is abundant and of little worth. As a result, consumers view water as “cheap” relative to its true cost of sustainable service provision, diminishing their initiatives to conserve [18]. Conversely, water operators face financial constraints that impede their capacity to update deteriorating water pipelines, leading to excessive amounts of non-revenue water; thus, water operators compromise on maintenance due to financial constraints [19]. Undeniably, the discrepancy between price sensitivity and available funding is a significant obstacle, as low tariff revenues deprive water operators of necessary resources, resulting in service failures.

To effectively manage water demand, consumers need to transform their water usage habits to be more environmentally friendly [20]. In this context, managing household water consumption emerges as a critical strategy for easing the burdens of water stress and alleviating growing pressure on the environment [21]. However, a persistent challenge undermines this effort: the "behavioral gap" where high awareness of water issues often fails to translate into consistent sustainable consumption actions [22]. Addressing this issue requires a strategic shift from supply-side solutions to demand-side governance, prioritizing long-term sustainability through consumer engagement and equitable policy design. A key question arises: What undermines consumers’ motivation to conserve water? A critical concern is how perceived service quality fails to enhance consumer satisfaction and cooperation with sustainability initiatives. This behavioral gap suggests that current messaging does not effectively translate consumers’ perceptions into actions. Therefore, this study aims to fill this gap by proposing a conceptual model that explores the intricate interplay between perceived service quality and satisfaction and the moderating factor of price sensitivity in shaping sustainable water consumption behavior, particularly within the underexplored context of developing nations like Malaysia.

This study presents a novel conceptual framework that integrates perceived service quality, assessed using the SERVPERF scale, as an antecedent to satisfaction, which is hypothesized to subsequently influence sustainable water consumption behavior in a proposed framework. It also explores price sensitivity as a moderator. Effective water tariffs can increase consumer awareness, enabling individuals to recognize their consumption patterns and adjust their usage based on cost signals, where higher prices typically lead to reduced consumption and promote water resource sustainability [23]. Although SERVPERF was originally developed for commercial marketing [24], this study highlights its strategic applicability to the public water utility sector, providing a robust metric for assessing how performance perceptions impact consumer behavior. To guide this investigation, the study employs the Stimulus-Organism-Response (S-O-R) framework [25], which provides a useful theoretical lens for hypothesizing environmental behavior [26]. This theoretical application offers a structured method for analyzing how external stimuli influence internal states, ultimately driving behavioral responses, with price sensitivity serving as a significant boundary condition.

The relevance of this study lies in its provision of essential knowledge for advancing environmental management by moving beyond mere awareness campaigns to tackle the complex factors that influence consumer behavior. Additionally, the conceptual model developed in this study links service marketing theory with environmental psychology, offering significant transferability. This enhances its relevance to a broader audience and makes it applicable to other public goods and sustainable consumption behaviors beyond the immediate focus of this study. Furthermore, the proposed instrument, following future full-scale validation to establish its reliability and construct validity, could form a crucial foundation for developing effective policies for water demand management. The conceptual model proposes a direct correlation between the behavioral gap that contributes to per capita consumption and its tangible environmental consequences, a relationship that warrants empirical testing in future research. Consequently, if this proposed model is supported by subsequent empirical research, policymakers could use such an instrument to create targeted interventions, provide incentives, and enforce regulations that promote water conservation at the household level.

When a new instrument is designed, measuring and reporting its content validity is fundamentally important [38]. Content validity is a prerequisite for other types of validity and should be prioritized in the earliest phase of instrument development [7, 39]. If an instrument lacks content validity, establishing its reliability becomes impossible. Therefore, allocating more resources to a content validity study at the outset can reduce the need for resources in future reviews of the instrument during the psychometric process [40-42]. Consequently, this study utilized a practical methodology for instrument development and content validity evaluation, drawing from Almanasreh et al. [3]. The content validation process also included a pretest, as suggested by Lim [1]. This involved getting feedback from experts within pertinent academic disciplines and subsequently implementing modifications informed by the feedback received. Thus, the instrument development and content validation process in this study consists of four stages: the development stage, the judgment and quantification stage, the pretest stage, and the content validation quantification stage, as illustrated in Figure 4.

Figure 4. The flow diagram illustrates the process of instrument development and content validation for this study

3.1 Development stage

Following an extensive literature review of the theoretical background, a conceptual research framework was developed utilizing the S-O-R framework and the SERVPERF model. The operational definitions of the domains were clearly defined to avoid incongruity [3, 27], as discussed in Section 2. Subsequently, the instrument was developed through determining the domains, item generations, and instrument construction [3, 40, 42]. A total of 36 items were generated by adapting items from previous studies, which collectively formed the instrument for this study (see Table 1).

Table 1. Domains, sources and number of items

3.2 Judgment and quantification stage

This stage involves the designation of the content validity assessment form and the invitation to a group of experts to evaluate how effectively the operationalization of the items captures the domains and how well they have been incorporated into the instrument. Additionally, experts assess the extent to which the instrument is designed to measure the specific concept of interest [1, 3, 40]. The content validity assessment form was created to gather responses from experts regarding the adequacy of the items’ operationalization in capturing the construct. Consequently, selected experts with the necessary expertise, experience, and exposure the study [1] in the water sector were invited to participate in the assessment.

There is no consensus on the number of experts required to review the instrument. Lynn [39] suggested a minimum of three experts, while Muhamad Jamil et al. [48] recommended at least five to ensure that the consensus reflects the experts’ professional knowledge and understanding rather than mere coincidence. However, Yusoff [49] asserted that the number of experts for content validation should be at least six but should not exceed 10. In this study, the selection of experts adhered to the principle of representativeness, including policymakers, regulators, academics, and industry representatives for a comprehensive assessment of the instrument. The experts selected to review the instrument were chosen based on well-defined criteria adapted from guidelines established by the study of Grant and Davis [50], along with quantitative scoring adaptations from previous studies [51-53]. The criteria for participation included a solid educational background with at least a bachelor’s degree (Q1), a relevant professional background (Q2), a minimum of five years of practical experience in the water sector (Q3), and familiarity with the concepts evaluated in this study (Q4). These criteria ensured that all participants possessed substantial experience and insight, making them suitable for accurate judgments.

Table 2. Experts’ profile criteria

The content validity assessment form was emailed to 15 experts, accompanied by a cover letter and a comparison between the original and adapted items for this study. The cover letter outlined the study’s purpose, provided a brief description of the domains and research conceptual framework, and explained the content validity assessment form the study of Rubio et al. [41]. After two weeks, the primary author sent a reminder to all the experts. Five experts agreed to complete the form in person, while five provided feedback via email. After six weeks, feedback from 10 experts with diverse backgrounds was gathered (see Table 2).

The overall authority of the experts was evaluated by quantifying the combined influence of their experience and insight in the water sector through the calculation of an Expert Authority Coefficient (Cr) according to the Delphi method. The equation utilized for this assessment is represented as Cr = (Ca + Cs) / 2, wherein Ca denotes the Expert Judgment Basis Coefficient and Cs signifies the Expert Familiarity Score [53]. The Ca encompasses three distinct criteria, each classified into three tiers: high, medium, and low, with coefficients of 0.4, 0.3, 0.2 (Q1), 0.2, 0.1, 0.1 (Q2), and 0.4, 0.3, 0.2 (Q3). Cs represents Q4, rated over three tiers: high (1.0), medium (0.7), and low (0.5). Consequently, Cr serves as an indicator of the degree of expert authority, which is denoted by the mean of Ca and Cs. Prediction accuracy increases with the level of Cr; a Cr of ≥ 0.70 is considered acceptable [53].

3.3 Pretest stage

Lim [1] explained that achieving content validity typically involves a pretest, during which a cyclical process of critique and refinement occurs, guided by feedback from a panel of experts. These experts, selected from relevant academic and professional fields, possess the necessary expertise to evaluate the instrument’s alignment with the domains. While he suggests that starting with a minimum of two experts is advisable, he notes that the ideal number may vary depending on the complexity of the domains and the scope of the study. For this study, two experts with academic backgrounds in quantitative research methodology were selected. Their feedback led to revisions of the instrument, specifically in terms of wording clarity and adjustments to the response.

3.4 Content validation quantification stage

There is no uniform approach for examining the content validity of an instrument, nor is there a single statistical method [3]. In this study, the instrument was quantitatively assessed using the content validity index (CVI) to measure the relevance of each item (the level of importance of the item [6, 54, 55], the content validity ratio (CVR) to evaluate the essentiality of each item (how necessary the item is) [56], and modified Kappa (K) to determine the level of agreement among experts. Microsoft Excel was utilized for data entry and tabulation of measurements related to relevance, essentiality, and expert agreement. After analyzing the quantitative data, the experts’ comments and suggestions were considered to decide whether to retain, revise, delete or add new items to the instrument.

3.4.1 Content Validity Index

The CVI is a widely used method for assessing the relevance of items on the instrument [7]. Following recommendations from the study of Polit et al. [54] that reference the method outlined by the study of Waltz and Bausell [55], a 4-point Likert scale was utilized to rate the relevance of each item (1 = Not Relevant, 2 = Somewhat Relevant, 3 = Quite Relevant, and 4 = Highly Relevant). The CVI was measured at both the item level (I-CVI) and the scale level (S-CVI). Before calculating the CVI, relevance ratings were recorded as 1 (for ratings of 3 or 4) or 0 (for ratings of 1 or 2). The I-CVI was determined by dividing the number of experts who rated an item as 3 or 4 by the total number of experts [39]. With 10 experts participating in this study, the acceptable cut-off score for the I-CVI value is 0.78, indicating that items rated above this threshold are considered relevant [39, 54]. Items with an I-CVI value lower than 0.78 but above 0.70 require revisions, while items with an I-CVI value below 0.70 are eliminated [39]. 

$I-C V I=\frac{\begin{array}{c}\text { number of experts agree on items rated as } \\ 3 \text { or } 4\end{array}}{\text { total number of experts }}$     (1)

Meanwhile, the S-CVI indicates the overall CVI of the instrument. The literature presents varying acceptable standards for the S-CVI; generally, the minimum acceptable range is between 0.8 and 0.9 [3, 6, 48]. The average S-CVI (S-CVI/Ave) in this study was computed by dividing the sum of the I-CVIs by the total number of items [3].

S-CVI $/$ Ave $=\frac{\sum(\text { I }- \text { CVIs })}{\text { total number of items }}$    (2)

3.4.2 Content Validity Ratio

The CVR measures the essentiality of each item, as established by the study of Lawshe [56]. According to Lawshe [56], a minimum CVR of 0.62 is typically required for 10 experts to establish statistical significance. Experts evaluate each item by utilizing a 3-point scale: 1 = not essential, 2 = useful but not essential, and 3 = essential. Ratings of 1 and 2 signify content that is deemed “not essential,” whereas a rating of 3 denotes content that is considered “essential” [48, 57]. The CVR is computed by subtracting half the total number of experts from the count of experts who have rated an item as “essential,” followed by dividing this result by half the total number of experts. CVR values range from 1 (indicating perfect agreement) to -1 (indicating perfect disagreement), with a value above 0 signifying more than half of the experts agree on the item’s essentiality. However, given the preliminary nature of this instrument development and to ensure comprehensive domain coverage, this study adopted a relaxed threshold of CVR > 0 as a baseline for item retention or revision, provided that the item also achieved excellent relevancy (I-CVI > 0.78), and excellent agreement (K > 0.74), as recommended by research [54]. Items falling below CVR 0.62 but above 0 were not automatically retained but were strictly revised based on qualitative expert feedback.

$C V R=\frac{n_e-(N / 2)}{N / 2}$  (3)

3.4.3 Modified Kappa

Polit et al. [54] have developed a modified Kappa (K) to overcome a major weakness of the I-CVI. It is derived by translating the I-CVI into a Kappa statistic to measure the level of agreement among experts while accounting for chance. The evaluation criteria for the K value are established as fair (0.40 - 0.59), good (0.60 - 0.74) and excellent (> 0.74). After calculating the value of I-CVI for all the items, K can be computed by applying the value of probability of chance agreement and I-CVI through the equation: 

$K=\frac{I-C V I-P_c}{1-P_c}$  (4)

This study proposes a preliminarily developed instrument to assess the content relevance of items related to the interplay between service performance, satisfaction, and price sensitivity in shaping sustainable water consumption behavior. It emphasizes that effective demand management may need to consider the nuanced interactions between concrete service evaluations and economic perceptions. This concept is presented in the conceptual framework, which suggests potential applicability to water governance in Malaysia and similar contexts within developing countries. By documenting this process, the study enables readers to systematically understand and replicate the content validation procedure.

While further empirical validation is needed, this instrument, after future full-scale validation, could serve as a tool for policymakers and utilities seeking to design equitable, efficient, and conservation-oriented pricing strategies. Ultimately, its future application could help translate national roadmaps, such as Air 2040, into tangible progress and promote a financially sustainable water sector that is resilient and aligned with broader environmental goals. However, given this study’s limitation to content validity, future research must assess its reliability, factor structure, criterion validity, and association with actual water use across diverse populations. Additionally, validation across various populations and settings is necessary to enhance the instrument’s generalizability, reliability, and validity, thereby supporting resilient and conservation-oriented water pricing.