Lack of transparency in urban sustainability rankings

February 10, 2020

"The last two decades have seen significant growth in the spread of tools to classify and measure urban performance (rankings, indexes, etc.) across both the public and private institutions that use them, in response to different types of pressures encouraging uniformity. Naturally, all these tools are useful for guiding and assessing the policies implemented by local authorities in various fields of action, and are particularly prolific in the area of sustainability. Yet there is a lack of knowledge about the actual methodological base underpinning them and which is supposed to legitimize their use," explained Lucía Sáez-Vegas, PhD holder in the UPV/EHU's Department of Financial Economics II.

"With the aim of analysing and assessing quality and good practices in urban measuring and monitoring, and while devoting special attention to the methodological aspects, we took hundreds of measuring tools and selected a set of 21 similar rankings, indexes and tools designed to rank and monitor urban sustainability (understood in a very broad sense) so that we could study them in depth and thus adapt and apply an analysis methodology tested in another field, that of university rankings," added Dr Sáez.

The significance of methodological aspects

In each of the similar rankings, indexes and tools analysed, the researchers explored the following four main principles: aim and target group they are geared towards; methodology and weighting used in their design; transparency related to data gathering and information processing; and finally, the presentation of the results. As Dr Saéz explained, "of these four aspects the information on the first and the last is the most accessible, in other words, the descriptive information. That is specified by all the classifications analysed; yet that does not happen when it is about accessing all the information on the methodological aspects, data gathering and information processing; this results in what is known as the black box, an artefact whose results are studied and disseminated without its inner workings being thrown into doubt".

That is how the researchers confirmed various methodological weaknesses in all the rankings analysed. The researcher insists that "tools of this type tend to neglect complex causalities and lack transparency with respect to data gathering, weighting and aggregation process in their design, they tend to be biased and, as a result, tend to ignore badly ranked cities and to reinforce existing stereotypes".

"The possibility of ranking and comparing cities of different dimensions may help to spot those that appear to perform better in various urban aspects. That is why these tools are used on occasions by urban managers and public decision-makers to develop an action plan, even though one has to have a clear idea about how the ranking or index has been drawn up, and exercise caution when using it, above all if insufficient information is provided about the methodological aspects and the robustness of its results. We understand that these tools should be used more as a source of information and even inspiration and less as a road map for action," she added. "These rankings attract the interest of the general public because they measure concepts of a complex nature which are presented by means of a ranking, generally of a numerical type, which can be understood very easily. From our academic viewpoint, the fact that the results are presented in the form of a final ranking with a mention of the principal findings, but with little or no regard for the methodological aspects which, at the end of the day, are the ones that underpin the score or ranking, signifies a clear weakness of these tools when used to measure and monitor urban performance," said Sáez.
-end-
Additional information

The study was conducted by the following researchers in the UPV/EHU's Faculty of Economics and Business: Lucía Saéz-Vegas, Iñaki Heras-Saizarbitoria and Estibaliz Rodríguez-Núñez (of the Department of Financial Economics II and the Department of Business Organisation), who belong to the Basque Government's GIC 15/176 consolidated research group.

Bibliographical reference

Lucía Saéz-Vegas, Iñaki Heras-Saizarbitoria and Estibaliz Rodríguez-Núñez

Sustainable city rankings, benchmarking and indexes: Looking into the black box

Sustainable Cities and Society

DOI: 10.1016/j.scs.2019.101938

University of the Basque Country

Related Sustainability Articles from Brightsurf:

New method adds and subtracts for sustainability's true measure
Policies across the world seek clear paths to sustainability, but it takes a broad look to know their true impact.

Striving and stumbling towards sustainability amongst pandas and people
Understanding how achieving one of the 17 UN Sustainable Development Goals spins off more SDG success -- or sabotages progress on another goal across spatial and administrative boundaries.

Is less more? How consumers view sustainability claims
Communicating a product's reduced negative attribute might have unintended consequences if consumers approach it with the wrong mindset.

Innovations for sustainability in a post-pandemic future
The COVID-19 pandemic has thrust the world into turmoil and disrupted the status quo, but it is also providing opportunities for innovation in the way we live and work.

Thematic package: Corona and sustainability
The COVID-19 pandemic remains an important topic throughout the world.

Digital agriculture paves the road to agricultural sustainability
In a study published in Nature Sustainability, researchers outline how to develop a more sustainable land management system through data collection and stakeholder buy-in.

Lack of transparency in urban sustainability rankings
UPV/EHU researchers have looked at the quality and good methodological practices employed and published in 21 rankings, indexes and similar tools used for classifying and monitoring urban sustainability.

New research shows sustainability can be a selling point for new ingredients
The first UK consumer study on the use of Bambara Groundnut as an ingredient in products has shown that sharing information on its sustainable features increased consumers' positive emotional connection to food.

Sustainability strategies more successful when managers believe in them
New research from Cass Business School has found that business sustainability strategies can succeed alongside mainstream competitive strategies when managers believe in them.

Sustainability claims about rubber don't stick
Companies work hard to present an environmentally responsible image. How well do these claims stack up?

Read More: Sustainability News and Sustainability Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.