Overall assessment of environmental sustainability: The Portuguese water industry case

Similarly to what happens worldwide, environmental sustainability is one of the pillars of the Portuguese water industry regulatory model. Taking into account the criteria and preference profiles defined by the Portuguese regulatory authority, the overall sustainability performance of the Portuguese water utilities is assessed using multicriteria decision analysis methodologies. The results showed that about half of the utilities have an overall unsatisfactory performance, which is not shown by the item-by-item assessment by the regulatory authority. The overall assessment allows not only highlighting and exposing the best practices of the sector, in line with the sunshine regulation model, as it proves appropriate to the valorization, in each regulatory period, of the priorities established in the sectoral plans for the sector.


IntroductIon
Climate change is increasingly becoming one of the greatest environmental threats and no one will escape exposure to this risk. Progressively longer and more intense cycles of flood and drought translate into environmental and economic losses that must be mitigated.
Simultaneously, demand for drinking water is increasing, due in part to the severity of the impact of climate change as well as the intrinsic characteristics of the water industry. These challenges demand coordinated management policies for the full range of stakeholders in this sector. Greater water management efficiency also requires new management tools that are capable of mitigating the effects of climate change. These mechanisms imply change in the way water is used, managed and shared.
It is not surprising, though, that one of the pillars of the water industry regulatory model in terms of the quality of service provided is environmental sustainability. In Portugal, the Water and Waste Services Regulation Authority, ERSAR, has been responsible for regulating the water industry since 2004. It uses a model composed of a set of benchmarks that evaluates the water supply, urban wastewater management, and urban waste management services. The indicators are divided into three groups: protection of user interests, operator sustainability, and environmental sustainability. This model generates a partial criterion by criterion analysis, which does not, however, differentiate between operators that perform well and those who are unsatisfactory.
This paper aims to assess the overall performance of these operators, in place of the item by item evaluation carried by the regulator. To this end, the criteria and preference profiles defined by ERSAR are taken into account and multicriteria decision analysis methodologies (MCDA) are used to adjust the weight of each criterion according to established sustainability goals, for a specific regulatory period, by applying different scale factors to each indicator. Pinto et al. (2017) propose to aggregate the performance indicators through the ELECTRE TRI-nC method in order to define quality of service categories.
The main advantage of aggregating factors in the performance assessment is to provide incentives for the operators to integrate the water industry priorities set out in strategic plans as well as those of the national and international directives. By allowing the classification and ranking of operators according to their performance, the MDCA methodologies are in line with the e model of sunshine regulation, where exposing performance and identifying benchmarks induces the search for best practices. This paper is structured as follows. Section 2 describes the methodological approach regarding regulatory mechanisms and performance evaluation and it also provides theoretical references about the MCDA methodologies. Section 3 looks at the environmental sustainability of Portuguese bulk water suppliers and then presents results and sensitivity analysis. Section 4 discusses the conclusions.

mEthodologIcal aPProach
Regulation by comparison is an incentive-based mechanism of economic regulation based on yardstick competition. It compares productivity and quality of service criteria of different operators to encourage competition in the market and raises up the most efficient companies as the reference for a particular industry (Shleifer, 1985). It is compatible with sunshine regulatory instruments and it uses performance indicators as benchmarks and then publishes the results so as to promote competitiveness among operators so as to increase individual and industry performance, and thereby promote the implementation of industry strategies (Artley and Stroh, 2001;Marques, 2005;Martins, 2007).
It is consensual in the international context that water quality should be regulated. Moreover, the best known regulators evaluate operators' performance, using benchmarking tools that include measures of environmental sustainability.

Multiple criteria decision aid methodologies
Among the tools for decision-making, sorting (P.β) and ranking (P.y) ones are the most adequate to the context of analysis. The sorting problematic allows to sort alternatives (water operators) according to some predefined norms (Roy, 1996) which corresponds to the ERSAR's approach to assess the operators' performance (by attribute or criterion). The ranking problematic highlighting the operators with the best performance seems to be suited to implement benchmarking regulation.
Multiattribute Utility Theory (MAUT) and Multiattribute Value Theory (MAVT) are the most practical MCDA methods for aggregating criteria (Riabacke et al., 2012;Zopounidis and Doumpos, 2002). Considering the deterministic characteristic of the data, we used MAVT to sort the data (P.β). The Elimination and Choice Translating Algorithm (ELECTRE) family of methods are the most commonly used outranking methods for the partial ordering of alternatives. In this exploratory study, the ELECTRE TRI method was used to sort the operators. We used both methods and to compare the results and we also applied these methods to evaluate the environmental sustainability of bulk water suppliers.

Multiattribute value theory
MAVT is a compensatory method of additive aggregation that allows for the verification of criteria independence. It yields an overall score to for each operator and so enables researchers to produce a complete ranking of operators (Keeney and Raifa, 1976). This method allows poor performance on one criterion to be offset by high performance in another, not valuing balanced performances. However, allowing small advantages on many criteria may add up and outweigh significant advantages of any single criterion (Keeney and Raifa, 1976;Belton and Stewart, 2002).The model adds the values for each criterion to an additive value function v(a i ) the values for each criterion for each operator a i , using a weighted sum constituted by n scale coefficients w 1 , …, w n , which are also known as weights, that is: where v(a i ) is the overall value of an alternative a i ; w j is the weight or scale coefficient of criterion j; v j (a i ) that reflects the performance of alternative i for the criterion j. Criteria weights or scale coefficients are central to this method. They show the relative importance of the evaluation criteria and describe the significance of each in any specific decision, while also taking into consideration the scale of each criterion (Danielson and Ekenberg, 2016). MAVT can be used to verify the additive independence between criteria so that, even if two operators have the same value in a criterion, the difference in attractiveness between them will not be affected. The researcher is responsible for allocating values to scale coefficients that define the trade-offs between criteria.
The advantages of this method include its compensatory nature and its intuitiveness. The fact that decision makers can easily understand the problem helps them set up the value functions that represent their preferences.

Outranking method electre tri
The Elimination and Choice Translating Algorithm TRI (ELECTRE TRI) is a multicriteria outranking sorting method that aids the assignment of alternatives to a predetermined set of categories (Dezert and Tacnet, 2012). It does not compare alternatives; instead they are classified by comparing each one with a reference action defined to delimit the category (Figueira et al., 2005).
Given the set of alternatives A={a 1 , a 2 , …, a m } assessed in n criteria of the set G={g 1 , g 2 , …, g n } which are intended to affect each alternative a i of A the set of categories K={k 1 , k 2 , …, k k } in which k k is the best category and k 1 is the worst. These categories are defined by two reference actions which enable researchers to establish higher (b h ) and lower (b h-1 ) limits for each category and to identify the reference profiles represented by Each alternative is allocated to a particular category based on successive comparisons of alternatives to references. Alternatives are classified based on the allocation of a i to the higher category, so that a i outranked b h-1 .
ELECTRE TRI assigns alternatives to categories based on the set up of an outranking relation S that characterizes how alternatives compare to the limits of categories and on using relation S to allocate each alternative to a category (Mousseau et al., 2000).
a i S j b h is said to be based on the conditions of concordance rather than discordance. For the concordance condition to exist, a significant majority of criteria should be in favour of the statement that a i "is at least as good as" b h . It should be verified that none of the minority criteria significantly oppose the outrank. In order to set up the outranking relation, the parameters of the criteria defining the importance coefficients (w j ) the performance of each alternative in a given criterion and the preference (p j ), indifference (q j ) and veto (v j ) thresholds are necessary. The veto threshold is determinant because it establishes that if one alternative is much worse than another regarding one criterion, it cannot be better than the other independently of the performance in all other criteria (Lourenço and Costa, 2004).
When these parameters are defined, partial concordance indexes c j (a i ,b h ) can be calculated for the statement a i S j b h from: The global concordance index C(a i ,b h ) is calculated based on the aggregation of the partial concordance for all criteria and it requires that the relative importance of each criterion (w j ) be defined from The veto thresholds establish discordance indexes that prevent the assertion "a i outranks b h ". The global discordance index D j (a i ,b h ) is calculated from the partial discordance indexes: If there are significant discordances, the global concordance must either be adjusted or reduced. The global concordance combined with the discordance allows for the calculation of the credibility index σ (    Credibility is a measure along a range of [0.5; 1], so in order to set outranking relation a i Sb h or a i Sb h , a cutting level λ must be established when σ (a i ,b h ) ≥ λ, a i outranks b h .

Identification and characterization of the criteria
The environmental sustainability of bulk water suppliers is assessed according to two criteria: 1) efficient use of environmental resources is measured by three indicators (real water loss, fulfilment of the water abstraction licensing, and standardized energy consumption); and 2) pollution prevention efficiency is determined by a sludge destination indicator. As not all Portuguese operators have sludge treatment plants, pollution prevention efficiency was not assessed in this work.
Real water loss (RWL) assesses the environmental sustainability of a service by measuring the actual water loss throughout the system. The fulfilment of water abstraction licensing (FWAL) indicator evaluates the safety of water abstractions and environmental protection. This indicator is defined as the volume of water abstracted in licensed water extractions in accordance with the Water Law (Diário da República, 2005). Finally, the standardized energy consumption (SEC) indicator assesses energy efficiency as defined by the average power consumption of normalized pumping facilities.
The performance categories and reference values for each indicator, defined by ERSAR (Table 1), were used to calculate the preference profiles b 1 and b 2 (Table 2).

Application and discussion
Knowing the decision alternatives and attributes related to the assessment of environmental sustainability of the operators, the performance matrix was based on the ERSAR service quality evaluation sheets (Table 3). When no data was available for a given criterion, we assigned it the worst performance (lower limit of the underperforming category). The decision problem comprised two decreasing functions (RWL and SEC) and an increasing one (FWAL). In order to minimize issues arising from the use of different scales and conflicting goals, we standardized both the performance data matrix and the profiles and adjusted the value scales for the same range [0;1] and then set an increasing value for the decreasing value functions, thereby normalizing the highest value.
Once the additive independence condition of criteria was checked (Keeney and Raifa, 1976;Watson and Buede, 1987), we set the value function for each criterion, so that the alternative satisfaction level could be expressed for each criterion. Subsequently, we evaluated the relative importance of each criterion in order to assign their weights. In this case, we took the role of the decision maker and chose to assign the same importance to each criterion. The value functions were then added up to obtain the aggregate value of each alternative, which was the sum of the value functions, duly weighted, which permitted their sorting (Table 4). The literature suggests that, for attributes with different ranges of scale, it is necessary to adjust the weights in order to maintain the ratio scale value judgments without violating the range of sensitivity principle (Riabacke et al., 2012). Therefore, we calculated the weights that were better adjusted to the range. In order to obtain a sorting order, b 1 and b 2 profiles were calculated (respectively 0.770495827 and 0.657142857) as adjusted by scale factors, from the values of the profiles extracted from the reference ranges.
We used MAVT to rank the environmental sustainability alternatives. Those operators with higher overall values were the AdNoroeste and AdTMAD. The performance of most operators was found to be unsatisfactory. The compensatory nature of the method penalized those alternatives with unsatisfactory partial water loss, as in the case of EPAL (34.88 m 3 per km/day) and AdSAndré (8.42 m 3 per km/day), the licensing level of abstraction, in the cases of AdZCoa and AdCentro, AdPAlentejo and AdSAndré; and energy efficiency for ICOVI. João Paulo Costa Overall assessment Of envirOnmental sustainability: the POrtuguese water industry case 55 ELECTRE TRI was applied to identify performance profiles based on reference values for each criterion to reflect the preferences of the decision maker (ERSAR) and to set the indifference (q j ), preference (p j ) and veto (v j ) thresholds, as well as the weight to be given to each criterion (w j ). As the weights and the thresholds were outside of the scope of the ERSAR model, they had to be obtained by elicitation. We played the part of the decision-maker and considered q j = 0 and p j = 0. Thus, a zone of weak preference was not considered, and we went directly from the area where the alternative a was preferred to b to that where b was preferred to a (aPb to bPa).
The veto threshold permitted us to clearly differentiate the outranking logic from the compensation logic (Vincke, 1992). Based on the performance of the operator for each criterion and its established ranges, we set the following veto thresholds: 10 to RWL indicator, 55 to FWAL and +∞ to SEC. All criteria were assigned equal weights, and we used the same profiles of b 1 and b 2 as in the previous method. Once the method parameters were defined, we calculated the criteria concordance, the global concordance, and the discordance indexes, until the degree of credibility. The outranking relation was finally established through the cutting level. For a cutting level λ=0.5, the results yielded a score of good environmental sustainability performance to AdTMAD, AdAlgarve and AdNoroeste (Table 5). About 46.6% of the operators were judged to provide unsatisfactory performance while 33.3% were found to be acceptable.

Sensitivity analysis
In order to assess the impact of parameters on the results, we performed a sensitivity analysis.
For MAVT, the lack of a performance indicator for operators ICOVI and AdOeste (Table 3) related to the licensing justified such an analysis. We found that ICOVI would have received an added value that would have characterized its category as acceptable if the percentage of licensing had been 100%. In the case of the AdOeste, 74% of licensed abstraction would have altered its category. The lack of AdPAlentejo energy efficiency data did not penalize its rank. Even had its performance been the best possible, the overall value function would not have ranked far above unsatisfactory.
When we carried out a sensitivity analysis of the aggregation model, the results would have been better if we had assigned a higher weight to water loss: 75% for this criterion (and 12.5% for the other two). The most favourable scenario with seven operators receiving the best classification and only four in the worst would have resulted had the weights 90% be allocated for water loss and the other 10% divided into the other two criteria. The worst case resulted when a higher weight was assigned to the licensing criterion. However, this change only had an impact when 90% was attributed to this criterion.
When we used ELECTRE TRI and set λ=0.5, the lack of data regarding licensing water abstraction for AdOeste only failed to impact the classification of operators when performance was less than 49% in which case the operator could be placed in the acceptable category. For ICOVI to be scored into a higher category, it would have needed to score 90% performance for this criterion.
The results obtained by changing the criteria weights were more favourable when water loss and energy efficiency criteria were weights greater than or equal to 50%. In turn, increasing the abstraction licensing weight had no impact on the classification of the alternatives. When λ ≥ 0.57, the result worsened significantly and 80% operators were rated unsatisfactory.
The sensitivity analysis of indifference thresholds for RWL only led to changes in the q j = 1. For fulfilment of water abstraction licensing we only found differences for q j = 4.27 and while, finally, for energy efficiency there were changes when q j = 0.09.

conclusIons
Water is an essential resource for human survival and economic development, and efficient management tools are increasingly needed to mitigate the effects of climate change. Environmental sustainability, the focus of this paper, is one of the dimensions of water utilities' performance to be assessed. It is a key component of the Portuguese water industry regulatory model, as in the case of most international regulatory models.
We assessed Portuguese water utilities based on global indicators rather than those used by the Portuguese regulator. This approach was not only able to accommodate the priorities for the water industry in each regulatory period but it also took advantage of the potential of MCDA (MAVT and ELECTRE TRI) methodologies. These techniques were merely used to look at environmental sustainability, among the three dimensions used by ERSAR to assess the performance of Portuguese water services operators. The profile reference ranges were set in accordance with those of ERSAR. Other parameters (the importance coefficients, preference, indifference and veto thresholds) were necessary in order to use the outranking method. However, they were not part of the ERSAR evaluation model and had to be elicited, so we took the role of decision maker and tested appropriate values that reflected different analyses and priorities.
The overall sustainability performance of 46% of the operators was found to be unsatisfactory, which is not shown by the item-by-item assessment by the regulatory authority. The percentage of operators with good performance is around 13% when the compensatory method was MAVT and 20% with the ELECTRE TRI.
Both methods highlighted the good performance of two operators, AdNoroeste and AdTMAD. ELECTRE TRI was less strict, as it awarded good performance to one more operator, AdAlgarve, than did MAVT. The results obtained through the global performance indicator allow to identify operators with the best practices and thus make them a benchmark for the sector, in accordance with the sunshine regulatory model.
The MAVT and ELECTRE TRI methods produced similar results. These results suggest that the profiles of the alternatives are not influenced by the compensatory nature of the MAVT neither by the veto threshold. When equal importance was attributed to the indicators, the results were slightly more favourable in the ELECTRE TRI, and they revealed a high number of operators with poor overall performance, a fact that was not uncovered in the performance analysis of these operators.
The change in weights for each indicator also had an impact on operators' scores. Regarding overall assessment, the most favourable scenarios took place when water loss and energy efficiency were weighted greater than or equal to 50%. In this specific case, increasing the weight of the water abstraction licensing criterion had no impact on the classification.
Regarding operators' overall assessment, the approach permitted us to rank performance thus fulfilling the regulatory goals. This allows the exposure of the best performance practices, which according to the sunshine regulation model, should encourage the adoption of good practices.
With this exploratory approach, we intend to encourage reflection and discussion about the environmental sustainability assessment of Portuguese water utilities in general and their performance, in particular. We believe that this work may contribute to the definition of water policies, as it allows us to bring management objectives together with the priorities and recommendations for the water industry.