Nav: Home

Is your big data messy? We're making an app for that

February 16, 2017

BUFFALO, N.Y. -- Like a teenager's bedroom, big data is often messy.

Malfunctioning computers, data entry errors and other hard-to-spot problems can skew datasets and mislead people -- everyone from data scientists to data hobbyists -- trying to draw conclusions from raw data.

Vizier, a software tool under development by a University at Buffalo-led research team, aims to proactively catch those errors.

The project, backed by a $2.7 million National Science Foundation grant, launched in January. Like Excel and other spreadsheet software, Vizier will allow users to interactively work with datasets. For example, it will help people explore, clean, curate and visualize data in meaningful ways, as well as spot errors and offer solutions.

But unlike spreadsheet software, Vizier is intended for much larger datasets; it will be used to examine millions or billions of data points, as opposed to hundreds or thousands typically plugged into spreadsheet software.

"We are creating a tool that'll let you work with the data you have, and also unobtrusively make helpful observations like 'Hmm... have you noticed that two out of a million records make a 10 percent difference in this average?'" says Oliver Kennedy, PhD, assistant professor of computer science and engineering at UB, and the grant's principal investigator.

Co-principal investigators include Juliana Freire, professor of computer science and engineering at New York University, and Boris Glavic, assistant professor in the Department of Computer Science at the Illinois Institute of Technology. The award is from NSF's Data Infrastructure Building Blocks (DIBBs) program.

For years, companies like Google, Microsoft and Apple have utilized big data to improve their products and services. That same power is now spreading to the masses as government agencies in the United States and elsewhere publish massive amounts of public data on the internet.

For example, New York City and the federal government have open data portals making it possible for anyone with an internet connection to download information and ask questions about their government. When properly used, these portals can shed light on issues relating to health code violations, discrimination, bias and other matters, Kennedy said. Vizier will be released as free, open-source software.

"We want to make it easier for data scientists -- and eventually data hobbyists -- to discover and communicate not only what the data says, but why the data says that," he said.
-end-
Contact: Cory Nealon, cmnealon@buffalo.edu, University at Buffalo

University at Buffalo

Related Engineering Articles:

Engineering a new cancer detection tool
E. coli may have potentially harmful effects but scientists in Australia have discovered this bacterium produces a toxin which binds to an unusual sugar that is part of carbohydrate structures present on cells not usually produced by healthy cells.
Engineering heart valves for the many
The Wyss Institute for Biologically Inspired Engineering and the University of Zurich announced today a cross-institutional team effort to generate a functional heart valve replacement with the capacity for repair, regeneration, and growth.
Geosciences-inspired engineering
The Mackenzie Dike Swarm and the roughly 120 other known giant dike swarms located across the planet may also provide useful information about efficient extraction of oil and natural gas in today's modern world.
Engineering success
Academically strong, low-income would-be engineers get the boost they need to complete their undergraduate degrees.
HKU Engineering Professor Ron Hui named a Fellow by the UK Royal Academy of Engineering
Professor Ron Hui, Chair Professor of Power Electronics and Philip Wong Wilson Wong Professor of Electrical Engineering at the University of Hong Kong, has been named a Fellow by the Royal Academy of Engineering, UK, one of the most prestigious national academies.
Engineering a better biofuel
The often-maligned E. coli bacteria has powerhouse potential: in the lab, it has the ability to crank out fuels, pharmaceuticals and other useful products at a rapid rate.
Pascali honored for contributions to engineering education
Raresh Pascali, instructional associate professor in the Mechanical Engineering Technology Program at the University of Houston, has been named the 2016 recipient of the Ross Kastor Educator Award.
Scaling up tissue engineering
A team at the Wyss Institute for Biologically Inspired Engineering at Harvard University and the Harvard John A.
Engineering material magic
University of Utah engineers have discovered a new kind of 2-D semiconducting material for electronics that opens the door for much speedier computers and smartphones that also consume a lot less power.
Engineering academic elected a Fellow of the IEEE
A University of Bristol academic has been elected a Fellow of the world's largest and most prestigious professional association for the advancement of technology.

Related Engineering Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Digital Manipulation
Technology has reshaped our lives in amazing ways. But at what cost? This hour, TED speakers reveal how what we see, read, believe — even how we vote — can be manipulated by the technology we use. Guests include journalist Carole Cadwalladr, consumer advocate Finn Myrstad, writer and marketing professor Scott Galloway, behavioral designer Nir Eyal, and computer graphics researcher Doug Roble.
Now Playing: Science for the People

#530 Why Aren't We Dead Yet?
We only notice our immune systems when they aren't working properly, or when they're under attack. How does our immune system understand what bits of us are us, and what bits are invading germs and viruses? How different are human immune systems from the immune systems of other creatures? And is the immune system so often the target of sketchy medical advice? Those questions and more, this week in our conversation with author Idan Ben-Barak about his book "Why Aren't We Dead Yet?: The Survivor’s Guide to the Immune System".