Nav: Home

Lack of transparency in urban sustainability rankings

February 10, 2020

"The last two decades have seen significant growth in the spread of tools to classify and measure urban performance (rankings, indexes, etc.) across both the public and private institutions that use them, in response to different types of pressures encouraging uniformity. Naturally, all these tools are useful for guiding and assessing the policies implemented by local authorities in various fields of action, and are particularly prolific in the area of sustainability. Yet there is a lack of knowledge about the actual methodological base underpinning them and which is supposed to legitimize their use," explained Lucía Sáez-Vegas, PhD holder in the UPV/EHU's Department of Financial Economics II.

"With the aim of analysing and assessing quality and good practices in urban measuring and monitoring, and while devoting special attention to the methodological aspects, we took hundreds of measuring tools and selected a set of 21 similar rankings, indexes and tools designed to rank and monitor urban sustainability (understood in a very broad sense) so that we could study them in depth and thus adapt and apply an analysis methodology tested in another field, that of university rankings," added Dr Sáez.

The significance of methodological aspects

In each of the similar rankings, indexes and tools analysed, the researchers explored the following four main principles: aim and target group they are geared towards; methodology and weighting used in their design; transparency related to data gathering and information processing; and finally, the presentation of the results. As Dr Saéz explained, "of these four aspects the information on the first and the last is the most accessible, in other words, the descriptive information. That is specified by all the classifications analysed; yet that does not happen when it is about accessing all the information on the methodological aspects, data gathering and information processing; this results in what is known as the black box, an artefact whose results are studied and disseminated without its inner workings being thrown into doubt".

That is how the researchers confirmed various methodological weaknesses in all the rankings analysed. The researcher insists that "tools of this type tend to neglect complex causalities and lack transparency with respect to data gathering, weighting and aggregation process in their design, they tend to be biased and, as a result, tend to ignore badly ranked cities and to reinforce existing stereotypes".

"The possibility of ranking and comparing cities of different dimensions may help to spot those that appear to perform better in various urban aspects. That is why these tools are used on occasions by urban managers and public decision-makers to develop an action plan, even though one has to have a clear idea about how the ranking or index has been drawn up, and exercise caution when using it, above all if insufficient information is provided about the methodological aspects and the robustness of its results. We understand that these tools should be used more as a source of information and even inspiration and less as a road map for action," she added. "These rankings attract the interest of the general public because they measure concepts of a complex nature which are presented by means of a ranking, generally of a numerical type, which can be understood very easily. From our academic viewpoint, the fact that the results are presented in the form of a final ranking with a mention of the principal findings, but with little or no regard for the methodological aspects which, at the end of the day, are the ones that underpin the score or ranking, signifies a clear weakness of these tools when used to measure and monitor urban performance," said Sáez.
-end-
Additional information

The study was conducted by the following researchers in the UPV/EHU's Faculty of Economics and Business: Lucía Saéz-Vegas, Iñaki Heras-Saizarbitoria and Estibaliz Rodríguez-Núñez (of the Department of Financial Economics II and the Department of Business Organisation), who belong to the Basque Government's GIC 15/176 consolidated research group.

Bibliographical reference

Lucía Saéz-Vegas, Iñaki Heras-Saizarbitoria and Estibaliz Rodríguez-Núñez

Sustainable city rankings, benchmarking and indexes: Looking into the black box

Sustainable Cities and Society

DOI: 10.1016/j.scs.2019.101938

University of the Basque Country

Related Sustainability Articles:

Digital agriculture paves the road to agricultural sustainability
In a study published in Nature Sustainability, researchers outline how to develop a more sustainable land management system through data collection and stakeholder buy-in.
Lack of transparency in urban sustainability rankings
UPV/EHU researchers have looked at the quality and good methodological practices employed and published in 21 rankings, indexes and similar tools used for classifying and monitoring urban sustainability.
New research shows sustainability can be a selling point for new ingredients
The first UK consumer study on the use of Bambara Groundnut as an ingredient in products has shown that sharing information on its sustainable features increased consumers' positive emotional connection to food.
Sustainability strategies more successful when managers believe in them
New research from Cass Business School has found that business sustainability strategies can succeed alongside mainstream competitive strategies when managers believe in them.
Sustainability claims about rubber don't stick
Companies work hard to present an environmentally responsible image. How well do these claims stack up?
Reconnecting with nature key for sustainability
People who live in more built up areas and spend less free-time in nature are also less likely to take actions that benefit the environment, such as recycling, buying eco-friendly products, and environmental volunteering.
Towards sustainability -- from a by-product of the biodiesel industry to a valuable chemical
Scientists at Tokyo Institute of Technology (Tokyo Tech) and the National Taiwan University of Science and Technology (Taiwan Tech) develop a cheap and efficient copper-based catalyst that can be used to convert glycerol, one of the main by-products of the biodiesel industry, into a valuable compound called dihydroxyacetone.
All global sustainability is local
Groundbreaking ways to quantify progress towards sustainable development goals find sustainability, like politics, is local.
Focus on food security and sustainability
The number of malnourished people is increasing worldwide. More than two billion people suffer from a lack of micronutrients.
Assigning workers to new networks boosts sustainability
Innovation comes from people in different units who have new knowledge, and a new study about conservation organizations suggests encouraging employees to think and act outside network boxes from time to time.
More Sustainability News and Sustainability Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Climate Mindset
In the past few months, human beings have come together to fight a global threat. This hour, TED speakers explore how our response can be the catalyst to fight another global crisis: climate change. Guests include political strategist Tom Rivett-Carnac, diplomat Christiana Figueres, climate justice activist Xiye Bastida, and writer, illustrator, and artist Oliver Jeffers.
Now Playing: Science for the People

#562 Superbug to Bedside
By now we're all good and scared about antibiotic resistance, one of the many things coming to get us all. But there's good news, sort of. News antibiotics are coming out! How do they get tested? What does that kind of a trial look like and how does it happen? Host Bethany Brookeshire talks with Matt McCarthy, author of "Superbugs: The Race to Stop an Epidemic", about the ins and outs of testing a new antibiotic in the hospital.
Now Playing: Radiolab

Speedy Beet
There are few musical moments more well-worn than the first four notes of Beethoven's Fifth Symphony. But in this short, we find out that Beethoven might have made a last-ditch effort to keep his music from ever feeling familiar, to keep pushing his listeners to a kind of psychological limit. Big thanks to our Brooklyn Philharmonic musicians: Deborah Buck and Suzy Perelman on violin, Arash Amini on cello, and Ah Ling Neu on viola. And check out The First Four Notes, Matthew Guerrieri's book on Beethoven's Fifth. Support Radiolab today at Radiolab.org/donate.