In a new method for searching image databases, a hand-drawn sketch is all it takes

May 31, 2016

Computer scientists at the University of Basel have developed a new method for conducting image and video database searches based on hand-drawn sketches. The user draws a sketch on a tablet or interactive paper, and the system searches for a matching image in the database. The new method is free to access for researchers.

People today are increasingly confronted with the challenge of having to find their way around vast collections of photos and videos, both in their work lives and at home. Although search engines such as Google and Bing make it easy to find documents or websites quickly and efficiently using search terms, the options for searching collections of multimedia objects are more limited.

A broadly defined similarity concept

Researchers at the Department of Mathematics and Computer Science at the University of Basel have developed a system known as 'vitrivr', which allows a search for images and videos by means of a sketch. The user creates a sketch of the desired object on a tablet or interactive paper, and the program delivers the images and video clips that most resemble it. For videos, the user can even specify on the sketch in which direction an object is moving in the searched sequence.

In designing the system, the researchers deliberately set a very broad similarity concept and adapted it to different types of sketch; for example, similar colors, shapes or directions of movement.

Individual searches can then be augmented by a range of other query types - search terms, examples of images and videos, or combinations of all these. An important feature of the new system is its scalability, a feature that means it can be used even with very large multimedia collections.

Freely available system

The vitrivr system is entirely open source and is therefore freely available to the international research community. It is already being used for a wide range of purposes, from discerning patterns of movement in sports videos for the Federal Office of Sport to searching collections of digital watermarks in a collaboration with the Basel Paper Mill.

Researchers around the world are currently working on developing the system, often with the support of well-known programs, such as the Google Summer of Code. vitrivr is also the basis for the iMotion research project, which is funded by the EU and the Swiss National Science Foundation, and is set to be used as a search engine for large-scale video collections as part of a collaboration with Red Hen Labs in the US.
-end-


University of Basel

Related Collaboration Articles from Brightsurf:

Collaboration sparks new model for ceramic conductivity
As insulators, metal oxides - also known as ceramics - may not seem like obvious candidates for electrical conductivity.

Trustful collaboration critical for outcome of therapy
A trusting therapeutic relationship and outcome-oriented collaboration between therapist and patient are critical for the successful treatment of mental illness.

Collaboration is key to rebuilding coral reefs
The most successful and cost-effective ways to restore coral reefs have been identified by an international group of scientists, after analyzing restoration projects in Latin America.

Classification-coordination-collaboration
Fu et al. outlined a systems approach, Classification-Coordination-Collaboration (3C), to advance Sustainable Development Goals (SDGs).

Collaboration lets researchers 'read' proteins for new properties
A collaboration between the McKelvey School of Engineering and St.

Collaboration yields insights into mosquito reproduction
As carriers for diseases like dengue and Zika, mosquitoes kill more than 1 million people each year and sicken hundreds of millions more.

Achieving optimal collaboration when goals conflict
New research suggests that, when two people must work together on a physical task despite conflicting goals, the amount of information available about each other's actions influences how quickly and optimally they learn to collaborate.

Leipzig primate researchers initiate global collaboration
In order to investigate evolutionary questions, scientists require the largest and most versatile samples possible.

Dynamic collaboration behind new research into best way of using biologging tags
Methods used to design F1 cars and spacecraft have played a crucial role in new research into the tags used to track animal movements.

Imaging collaboration sheds new light on cancer growth
Walter and Eliza Hall Institute researchers have uncovered new insights into how the normal controls on cell growth are lost in cancer cells, leading to rapid tumour expansion.

Read More: Collaboration News and Collaboration Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.