Nav: Home

Lack of transparency in urban sustainability rankings

February 10, 2020

"The last two decades have seen significant growth in the spread of tools to classify and measure urban performance (rankings, indexes, etc.) across both the public and private institutions that use them, in response to different types of pressures encouraging uniformity. Naturally, all these tools are useful for guiding and assessing the policies implemented by local authorities in various fields of action, and are particularly prolific in the area of sustainability. Yet there is a lack of knowledge about the actual methodological base underpinning them and which is supposed to legitimize their use," explained Lucía Sáez-Vegas, PhD holder in the UPV/EHU's Department of Financial Economics II.

"With the aim of analysing and assessing quality and good practices in urban measuring and monitoring, and while devoting special attention to the methodological aspects, we took hundreds of measuring tools and selected a set of 21 similar rankings, indexes and tools designed to rank and monitor urban sustainability (understood in a very broad sense) so that we could study them in depth and thus adapt and apply an analysis methodology tested in another field, that of university rankings," added Dr Sáez.

The significance of methodological aspects

In each of the similar rankings, indexes and tools analysed, the researchers explored the following four main principles: aim and target group they are geared towards; methodology and weighting used in their design; transparency related to data gathering and information processing; and finally, the presentation of the results. As Dr Saéz explained, "of these four aspects the information on the first and the last is the most accessible, in other words, the descriptive information. That is specified by all the classifications analysed; yet that does not happen when it is about accessing all the information on the methodological aspects, data gathering and information processing; this results in what is known as the black box, an artefact whose results are studied and disseminated without its inner workings being thrown into doubt".

That is how the researchers confirmed various methodological weaknesses in all the rankings analysed. The researcher insists that "tools of this type tend to neglect complex causalities and lack transparency with respect to data gathering, weighting and aggregation process in their design, they tend to be biased and, as a result, tend to ignore badly ranked cities and to reinforce existing stereotypes".

"The possibility of ranking and comparing cities of different dimensions may help to spot those that appear to perform better in various urban aspects. That is why these tools are used on occasions by urban managers and public decision-makers to develop an action plan, even though one has to have a clear idea about how the ranking or index has been drawn up, and exercise caution when using it, above all if insufficient information is provided about the methodological aspects and the robustness of its results. We understand that these tools should be used more as a source of information and even inspiration and less as a road map for action," she added. "These rankings attract the interest of the general public because they measure concepts of a complex nature which are presented by means of a ranking, generally of a numerical type, which can be understood very easily. From our academic viewpoint, the fact that the results are presented in the form of a final ranking with a mention of the principal findings, but with little or no regard for the methodological aspects which, at the end of the day, are the ones that underpin the score or ranking, signifies a clear weakness of these tools when used to measure and monitor urban performance," said Sáez.
-end-
Additional information

The study was conducted by the following researchers in the UPV/EHU's Faculty of Economics and Business: Lucía Saéz-Vegas, Iñaki Heras-Saizarbitoria and Estibaliz Rodríguez-Núñez (of the Department of Financial Economics II and the Department of Business Organisation), who belong to the Basque Government's GIC 15/176 consolidated research group.

Bibliographical reference

Lucía Saéz-Vegas, Iñaki Heras-Saizarbitoria and Estibaliz Rodríguez-Núñez

Sustainable city rankings, benchmarking and indexes: Looking into the black box

Sustainable Cities and Society

DOI: 10.1016/j.scs.2019.101938

University of the Basque Country

Related Sustainability Articles:

Striving and stumbling towards sustainability amongst pandas and people
Understanding how achieving one of the 17 UN Sustainable Development Goals spins off more SDG success -- or sabotages progress on another goal across spatial and administrative boundaries.
Is less more? How consumers view sustainability claims
Communicating a product's reduced negative attribute might have unintended consequences if consumers approach it with the wrong mindset.
Innovations for sustainability in a post-pandemic future
The COVID-19 pandemic has thrust the world into turmoil and disrupted the status quo, but it is also providing opportunities for innovation in the way we live and work.
Thematic package: Corona and sustainability
The COVID-19 pandemic remains an important topic throughout the world.
Digital agriculture paves the road to agricultural sustainability
In a study published in Nature Sustainability, researchers outline how to develop a more sustainable land management system through data collection and stakeholder buy-in.
Lack of transparency in urban sustainability rankings
UPV/EHU researchers have looked at the quality and good methodological practices employed and published in 21 rankings, indexes and similar tools used for classifying and monitoring urban sustainability.
New research shows sustainability can be a selling point for new ingredients
The first UK consumer study on the use of Bambara Groundnut as an ingredient in products has shown that sharing information on its sustainable features increased consumers' positive emotional connection to food.
Sustainability strategies more successful when managers believe in them
New research from Cass Business School has found that business sustainability strategies can succeed alongside mainstream competitive strategies when managers believe in them.
Sustainability claims about rubber don't stick
Companies work hard to present an environmentally responsible image. How well do these claims stack up?
Reconnecting with nature key for sustainability
People who live in more built up areas and spend less free-time in nature are also less likely to take actions that benefit the environment, such as recycling, buying eco-friendly products, and environmental volunteering.
More Sustainability News and Sustainability Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Debbie Millman: Designing Our Lives
From prehistoric cave art to today's social media feeds, to design is to be human. This hour, designer Debbie Millman guides us through a world made and remade–and helps us design our own paths.
Now Playing: Science for the People

#574 State of the Heart
This week we focus on heart disease, heart failure, what blood pressure is and why it's bad when it's high. Host Rachelle Saunders talks with physician, clinical researcher, and writer Haider Warraich about his book "State of the Heart: Exploring the History, Science, and Future of Cardiac Disease" and the ails of our hearts.
Now Playing: Radiolab

Insomnia Line
Coronasomnia is a not-so-surprising side-effect of the global pandemic. More and more of us are having trouble falling asleep. We wanted to find a way to get inside that nighttime world, to see why people are awake and what they are thinking about. So what'd Radiolab decide to do?  Open up the phone lines and talk to you. We created an insomnia hotline and on this week's experimental episode, we stayed up all night, taking hundreds of calls, spilling secrets, and at long last, watching the sunrise peek through.   This episode was produced by Lulu Miller with Rachael Cusick, Tracie Hunte, Tobin Low, Sarah Qari, Molly Webster, Pat Walters, Shima Oliaee, and Jonny Moens. Want more Radiolab in your life? Sign up for our newsletter! We share our latest favorites: articles, tv shows, funny Youtube videos, chocolate chip cookie recipes, and more. Support Radiolab by becoming a member today at Radiolab.org/donate.