Introduction

Like the famous quote by Mark Twain, “It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that ain’t so.”

The Streetlight Effect is the tendency for individuals and organizations to search for answers in the easiest of places. This leads them to convincing results that are far from the truth.

The streetlight effect can be likened to how individuals surfing the web rely on only one search engine and usually limit themselves to the first page to get results because it is the seemingly easiest way to get an answer. 

In the post, we will explain the concept of the Streetlight effect, its importance, implications, some examples, the origin of this phenomenon, and ways to manage it.

What is the Streetlight Effect?

The streetlight effect, otherwise known as the drunkard search principle, is a type of observational bias that happens when people only search for something where it is most convenient to look.

The streetlight effect is derived from an old story about a drunk man who purportedly lost his keys in the car park and was seen searching for them under the streetlights, some distance away from the car park. When asked why he was looking there, he mentioned that the light was brighter there.

So the streetlight effect became the term to describe a situation where people look for answers in the easiest ways. However, ease does not always mean accurate results.

Importance and Implications of the Streetlight Effect

Concerning the streetlight story analogy, it is clear that groping under the streetlight provides no clue as to where the drunkard’s keys are. However, it depicts one thing, the truth is difficult to find when the streetlight effect is the only source of illumination. For science and other disciplines, searching under the streetlight is a good place to start as long as there is a willingness to go further and push into the shadows in the right direction.

Big data from sources classified under the streetlight effect scenarios have shown limited viability. In the same way, using big data in the Ebola epidemic in West Africa led to overestimating the spread of the disease while underestimating the role of local initiatives.

We have established that the street light effect is the propensity of researchers to research seemingly easy studies. The implication is that many limitations would be evident from research employing the streetlight effect method.

For instance, in the course Research Design and Ethnographic Methods, it was stated that considerable research had been done on disparities in educational outcomes. It was carried out in classrooms repeatedly, instead of the students’ homes, because it was convenient. After all, educational bodies like schools are the preferred focus because of how much data they generate compared to students’ homes.

This streetlight effect depicted above is one of the main reasons big data studies are not useful in the real world. This is because most studies analyze readily available data from the internet based on the assumption that it provides insight into reality, which is not always the case.

The implication of the streetlight effect is also highlighted in a research study by Harvard professor Joseph Henrich and colleagues. The discoveries were based on research conducted with undergraduates at American universities – described as “some of the most psychologically unusual people on Earth.”

This study is limiting and applies only to that population and cannot make any claims about other human populations, including other Americans because a small number of the world’s population cannot be used to represent the true picture of the rest of the world.

Examples of the Streetlight Effect

Some examples of the streetlight effect exist across the globe. For example, many folks in America who experienced job losses during the 2 recessions during the ‘lost decade’ of the 2000s still went ahead to seek jobs in the same communities where the factories closed. It was easier for them to do so than to relocate to other viable parts of America then because they had established roots in the community, built their homes there, and had their kids in the community schools. It was more convenient for them to hope for a revival in their community and endure the hardship of the recession.

Another example is Hurricane Sandy which occurred in 2012. Due to the number of tweets from Manhattan, it was assumed that Manhattan was hit the hardest. Meanwhile, the New Jersey shore suffered the worst damage based on meteorological data.

Another example is the since-discarded  Google Flu Trends, 2013, which tracked and used online searches relating to flu symptoms to predict doctor visits. The result was that estimates of the spread of the disease were 2 times higher than the reports from the Centre for Disease Control and Prevention.

In the marketing industry, we also see viable examples of the streetlight effect, owing mainly to the rise of technology. With the internet, access to data can be achieved at the click of a button. There has been so much pressure to achieve results in the marketing space, however, the results have not been commensurate with marketing activities, and the disconnect is the use of easily accessible data, which is a result of the street light effect.

More often than not, marketing managers focus on readily available data instead of going through the pain of getting personalized data and analytics that would help drive growth. Everyone is buying data from data research companies, not realizing that reasonable solutions cannot be achieved by a one-size-fits-all approach.

Why the Streetlight Effect Dims Knowledge Discovery

The streetlight effect can cause major missteps and allows most organizations or individuals to rest on their oars. Valued knowledge, just like gold, cannot be achieved by scratching the surface.

Just like the story that gave rise to the Streetlight effect. Leveraging big data for small problems or, even worse, neglecting critical information altogether dims knowledge discovery.

In a study by the American Diabetes Association on the streetlight effect in Type 1 diabetes, we see how the streetlight effect dims knowledge discovery.

Over 100 years since the discovery of therapeutic insulin, efforts have been made to find the true cause of type 1 diabetes and a cure for the ailment, albeit unsuccessfully.

The pressure from the funders of the research has inhibited the drive for real in-depth analysis, and rather, the research has been reduced to the science of replication that evades the real questions. Thus little effort has been made to discredit the existing hypotheses. 

Also, scientists and researchers find it easier to replicate previous research than to cover new unknown grounds, all in a bid to play safe. The great tragedy of this science is that it inhibits deep research and solutions, especially as it is easier to get published when the subject is a study that can be easily corroborated.

This is further depicted in how the effort has been targeted at clinical diagnosis of Type 1 diabetes in a bid to find a cure, as this is where the light is and seemingly the easiest place to start. 

Type 1 diabetes is a deadly disease, and the streetlight effect is a factor that hampers the desired progress of understanding its causes. The right approach, seemingly more difficult, is the long and tedious progress of longitudinal studies from the birth of children, which is the key to a true breakthrough.

The streetlight effect, depicted by the over-reliance on social media for data, also inhibits deep knowledge discovery. How does this dim knowledge? It points out that researchers are always likely to focus on measurable quantities (i.e., where the light is good), regardless of whether they’re the most important. 

An example is the shards of excitement that hit the cardiology field during the early 1980s. Popular anti-arrhythmia drugs were discovered to help steady the heartbeats of people prone to heart attacks characterized by unsteady heartbeats. Thus a medication that could steady the rhythm seemed like the way out, and this drug was adopted as the standard care to manage heart palpitations. Sadly, the drug was fatal, killing 56,000 heart attack patients annually.

Then it became clear that the Cardiologists had focused on the readily measurable arrhythmia medication and unknowingly overlooked the long-term side effect, loss of life.

History of the Streetlight Effect

The history of the streetlight effect is attributed to Nasreddin Hodja, a fictional character, and hero of short satirical stories and folklore of Muslim ancestry. The history of the streetlight effect dates back to the 1920s.

“Nasreddin, also called Mullah, had lost his ring in the living room. He searched for it for a while, but since he could not find it, he went out into the yard and began to look there. His wife, who saw what he was doing, asked: “Mullah, you lost your ring in the room. Why are you looking for it in the yard?” Mullah stroked his beard and said: “The room is too dark, and I can’t see very well. I came out to the courtyard to look for my ring because there is much more light out here”.

The parables about Nasreddin Hodja are considered the earliest form of the streetlight effect story. Many Sufis used these stories to share about people who sought exotic sources of enlightenment. The streetlight effect metamorphosed into the principle of the drunkard’s search by Abraham  Kaplan in 1964.

The story since then has come in various forms, with the most popular being the drunk who was searching for car keys he had lost in the car park under a street light and when asked by a policeman, said he preferred that sport because the light was better.

How do you stop or manage the Streetlight effect?

To mitigate the impact of the street light effect, the first step is to not only search for the truth in clear or visible areas of study but also search for hidden and new grounds for the truth. This requires patience, as there might be some stumbling blocks as you stumble in the proverbial dark; just like Albert Einstein, you will find your own eureka moment and illumination not powered by streetlights.

The second step would be to put discoveries powered by the streetlight effect into proper context. Big data overestimated the potential spread of the Ebola virus in West Africa in 2014 and disregarded local initiatives in managing the outbreak. This overestimation occurred because the metrics taat were applied were suited to more developed nations. This indicates that putting information in the appropriate context is vital to combat the bias created by the streetlight effect.

For researchers who want to stop or manage the streetlight effect, the key would be to research a wider representation of the population or subject being studied rather than use a minority to generate majority results.

Conclusion

In a nutshell, we see that individuals, historians, mystics, scientists, and drunks have something in common: the propensity to seek solutions in the most convenient manner rather than finding answers where they lie. The street phenomenon highlights this fact, however being aware of this phenomenon and how it plays out in our daily lives is one of the first steps in managing its effects.