Can big data yield big ideas? Blend novel and familiar, new study findsJanuary 09, 2017
CATONSVILLE, MD, January 9, 2017 - Struggling to get your creative juices flowing for a new idea or project? A study in an upcoming edition of the INFORMS journal Marketing Science sheds light on the secret sauce to developing creative ideas, and it all comes down to word choice.
The study authors, Olivier Toubia and Oded Netzer of the Columbia Business School, based their research on the concept that ideas are considered more creative when they have a good balance between novelty and familiarity. By analyzing and text mining thousands of previously generated ideas, they identified common patterns in ideas that are considered creative.
"You can think about idea generation as creating a recipe where you to choose the appropriate ingredients and mix them together" said Netzer. "By analyzing the text in a large number of ideas across different domains, we were able to link an idea's judged creativity to its set of 'ingredients.' We found that what makes an idea creative as judged by both consumers and firms' executives is a mix of ingredients (words) that includes a balance between words that commonly appear together (familiar combinations) and words that do not (novel combinations)."
Thus, a creative idea is not simply an idea that includes novel ingredients, but combinations of words that are novel when appearing together balanced with combinations of ingredients that are more familiar. For example, if one generates an idea for an app that would help people live healthier lives and includes the words "running", "counting" and "steps," this would not classify the idea as creative, as all three of these words are fairly frequently used together. However, including the word "calendar" as an additional ingredient would take the idea in a more creative direction, for example the creation of a calendar app in which you could track daily movement and accomplishments. Even though the word "calendar" may not be novel with respect to the topic of health apps in and of itself, its combination with "running," "counting," and "steps" is novel.
Using insights gained from this research, the authors have developed a tool that can help people come up with better ideas.
"The tool analyzes in real time word combinations included in the idea and recommends words that help the innovator improve her idea" says Netzer. "If the idea is too familiar the algorithm will offer words that would make it more novel. On the other hand, if the idea is too novel, it will offer words that will make the idea more familiar."
The authors have found that their tool helps people improve their ideas and generate more balanced and creative ideas. The tool appears more helpful in improving the novelty of ideas that have too much familiarity than in improving the familiarity of ideas that have too much novelty.
Toubia noted: "By leveraging tools from the world of Big Data such as text mining and semantic network analysis we were able, for the first time, to scientifically analyze a large set of ideas and identify successful patterns that can help people improve their ideas. In a way, our tool turns the computer into a "sous chef" that helps the user assemble the right ingredients to combine in order to form creative ideas."
While only time will tell if computerized tools will surpass humans in generating creative ideas, the present research pours some systematic analysis and science into a domain - creativity and ideation - that has been largely based on intuition and hunch.
To view the public tool developed by the authors, visit: http://newtopic.protoideation.org/
With more than 12,500 members from across the globe, INFORMS is the leading international association for professionals in operations research and analytics. More information about INFORMS is available at http://www.informs.org or @informs.
Institute for Operations Research and the Management Sciences
Related Big Data Articles:
A special issue of Big Data presents a series of insightful articles that focus on Big Data and Social and Technical Trade-Offs.
At Los Alamos National Laboratory, home to more than 100 supercomputers since the dawn of the computing era, elegance and simplicity of programming are highly valued but not always achieved.
Big Data has imposed great challenges for machine learning. Bayesian methods provide a profound framework for characterizing the intrinsic uncertainty and performing probabilistic inference and decision-making.
When a group of researchers in the Undiagnosed Disease Network at Baylor College of Medicine realized they were spending days combing through databases searching for information regarding gene variants, they decided to do something about it.
Researchers at University of Helsinki, Finland and University at Buffalo, USA have analyzed the evolutionary history of silver birch using big data from the genomes of 150 birches.
Researchers from the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) and the Environmental Defense Fund (EDF) have developed an online tool that incorporates 21 years of night-time lights data to understand and compare changes in human activities in countries around the world.
Nothing works without proteins in the body, they are the molecular all-rounders in our cells.
Vizier, software under development by a University at Buffalo-led research team, aims to proactively catch big data errors.
Astronomers at Lomonosov Moscow State University in cooperation with their French colleagues and with the help of citizen scientists have released 'The Reference Catalog of galaxy SEDs,' which contains value-added information about 800,000 galaxies.
An enzyme found in many bacteria, including the bacterium that gives us strep throat, has given mankind a cheap and effective tool with which to edit our own genes.
Related Big Data Reading:
Big Data: A Revolution That Will Transform How We Live, Work, and Think
by Viktor Mayer-Schönberger (Author), Kenneth Cukier (Author)
Financial Times Business Book of the Year Finalist
“Illuminating and very timely . . . a fascinating — and sometimes alarming — survey of big data’s growing effect on just about everything: business, government, science and medicine, privacy, and even on the way we think.”
—New York Times
It seems like “big data” is in the news every day, as we read the latest examples of how powerful algorithms are teasing out the hidden connections between seemingly unrelated things. Whether it is used by the NSA to fight terrorism or by online... View Details
Big Data: Principles and best practices of scalable realtime data systems
by Nathan Marz (Author), James Warren (Author)
Big Data teaches you to build big data systems using an architecture that takes advantage of clustered hardware along with new tools designed specifically to capture and analyze web-scale data. It describes a scalable, easy-to-understand approach to big data systems that can be built and run by a small team. Following a realistic example, this book guides readers through the theory of big data systems, how to implement them in practice, and how to deploy and operate them once they're built.
Purchase of the print book includes a free eBook in PDF, Kindle,... View Details
Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are
by Seth Stephens-Davidowitz (Author)
New York Times Bestseller
Foreword by Steven Pinker, author of The Better Angels of our Nature
Blending the informed analysis of The Signal and the Noise with the instructive iconoclasm of Think Like a Freak, a fascinating, illuminating, and witty look at what the vast amounts of information now instantly available to us reveals about ourselves and our world—provided we ask the right questions.
By the end of an average day in the early twenty-first century, human beings searching the internet will... View Details
Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
by Martin Kleppmann (Author)
Data is at the center of many challenges in system design today. Difficult issues need to be figured out, such as scalability, consistency, reliability, efficiency, and maintainability. In addition, we have an overwhelming variety of tools, including relational databases, NoSQL datastores, stream or batch processors, and message brokers. What are the right choices for your application? How do you make sense of all these buzzwords?
In this practical and comprehensive guide, author Martin Kleppmann helps you navigate this diverse landscape by examining the pros and cons of various... View Details
Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking
by Foster Provost (Author), Tom Fawcett (Author)
Written by renowned data science experts Foster Provost and Tom Fawcett, Data Science for Business introduces the fundamental principles of data science, and walks you through the "data-analytic thinking" necessary for extracting useful knowledge and business value from the data you collect. This guide also helps you understand the many data-mining techniques in use today.
Based on an MBA course Provost has taught at New York University over the past ten years, Data Science for Business provides examples of real-world business problems to illustrate these principles. You’ll... View Details
Big Data For Dummies
by Judith Hurwitz (Author), Alan Nugent (Author), Fern Halper (Author), Marcia Kaufman (Author)
Find the right big data solution for your business or organization
Big data management is one of the major challenges facing business, industry, and not-for-profit organizations. Data sets such as customer transactions for a mega-retailer, weather patterns monitored by meteorologists, or social network activity can quickly outpace the capacity of traditional data management tools. If you need to develop or manage big data solutions, you'll appreciate how these four experts define, explain, and guide you through this new and often confusing concept. You'll learn what it is,... View Details
Data Strategy: How to Profit from a World of Big Data, Analytics and the Internet of Things
by Bernard Marr (Author)
Less than 0.5 per cent of all data is currently analysed and used. However, business leaders and managers cannot afford to be unconcerned or sceptical about data. Data is revolutionizing the way we work and it is the companies that view data as a strategic asset that will survive and thrive. Bernard Marr's Data Strategy is a must-have guide to creating a robust data strategy. Explaining how to identify your strategic data needs, what methods to use to collect the data and, most importantly, how to translate your data into organizational insights for improved business... View Details
Big Data: Using SMART Big Data, Analytics and Metrics To Make Better Decisions and Improve Performance
by Bernard Marr (Author)
Convert the promise of big data into real world results
There is so much buzz around big data. We all need to know what it is and how it works - that much is obvious. But is a basic understanding of the theory enough to hold your own in strategy meetings? Probably. But what will set you apart from the rest is actually knowing how to USE big data to get solid, real-world business results - and putting that in place to improve performance. Big Data will give you a clear understanding, blueprint, and step-by-step approach to building your own big data strategy. This is a... View Details
Big Data at Work: Dispelling the Myths, Uncovering the Opportunities
by Thomas H. Davenport (Author)
Go ahead, be skeptical about big data. The author wasat first.
When the term big data” first came on the scene, bestselling author Tom Davenport (Competing on Analytics, Analytics at Work) thought it was just another example of technology hype. But his research in the years that followed changed his mind.
Now, in clear, conversational language, Davenport explains what big data meansand why everyone in business needs to know about it. Big Data at Work covers all the bases: what big data means from a technical, consumer, and management... View Details
The Rise of Big Data Policing: Surveillance, Race, and the Future of Law Enforcement
by Andrew Guthrie Ferguson (Author)
The consequences of big data and algorithm-driven policing and its impact on law enforcement In a high-tech command center in downtown Los Angeles, a digital map lights up with 911 calls, television monitors track breaking news stories, surveillance cameras sweep the streets, and rows of networked computers link analysts and police officers to a wealth of law enforcement intelligence. This is just a glimpse into a future where software predicts future crimes, algorithms generate virtual “most-wanted” lists, and databanks collect personal and biometric information. ... View Details