Unraveling the Mystery of Intuition

Before you begin, here's a quick summary:

• Intuition, those gut feelings that arise from nowhere, can profoundly influence our decisions - but where do they come from? The Bayesian Brain Theory illuminates intuition's mysterious origins.

• While intuition can profoundly guide experts and pivotal decisions when honed, cognitive biases also mean instincts aren't infallible.

• How can we then hone our "sixth sense" and get better at using our intuition?
  

---

Changing My Career Based on Gut Feeling: A Personal Story

When it comes to decision making, we usually advocate for careful analysis, planning, and deliberation before acting. However, there are often moments where an idea, thought, or answer appears out of nowhere. This mysterious sensation comes with many names—intuition, gut feeling, or the sixth sense. Whatever we call it, this thought that comes from nowhere is exceptionally convincing. When it hits us, even though we cannot quite identify where it comes from, we must follow our intuition. Sometimes, it could even propel us to make what appears to be an irrational decision.

About two years ago, I made one of the biggest decisions in my life by following my intuition. I decided to leave the bank and join a blockchain startup. It was not a welcomed decision. I was cautioned against it by my spouse, parents, and some closer colleagues, and they made a convincing case. For starters, I was on a great career trajectory in the bank; it would have been a waste to give it up. The bank also offered better fringe benefits, including a very significant discount on mortgage rates, which became even more important when interest rates started to rise. Moreover, the bank could offer a much higher level of job security, whereas a startup could vanish any day. My spouse said the salary increment was insufficient to cover the risk premium.

At first, I also shared this sentiment and only accepted the interview to practice my interview skills and gauge my value in the market. While I had some rudimentary understanding of blockchain technology, I knew the enterprise world did not really care about blockchain benefits. However, after meeting the CEO in person, I knew right away that the switch would be a brilliant career move. It wasn't because he changed my mind about blockchain applicability but because of intuition. Despite the odds, I felt he was the kind of leader that could truly transform the supply chain industry and I could see myself thriving with this nascent startup.

My husband said I was biased and must really want to leave the bank, after all it was just a one-hour meeting. But I made up my mind. Soon after, I received an offer letter from this startup and tendered my resignation immediately. Today, I’ve been with the firm for two years. My intuition proved accurate. In terms of the firm's prospects, we have been gaining traction quickly while our direct competitor ceased operations. For my career path, I was a valued member and received a promotion in my first year.

Lately, there are more and more discussions advocating for the power of intuitions. Author of “Thinking, Fast and Slow” - Daniel Kahneman, discusses the idea of expert intuition and says practitioners who have a lot of experience in a field can make very accurate intuitive decisions. Mihaly Csikszentmihalyi, author of international best seller - "Flow: The Psychology of Optimal Experience," suggests that intuition can lead to innovative ideas and creative breakthroughs. Malcolm Gladwell, another international best selling author, also advocated the value of intuition intelligence, or an ability for us to glean information from limited data.

However, there’s not much explanation as to where intuition comes from. We all know that it comes from the subconscious of our brain, a black box in the brain that we have no control or visibility on, but exactly how does it originate? I believe the Bayesian Brain Theory may be able to help us peek behind the curtain and understand the formation of intuition.

The Root of Our Intuition - Bayesian Brain Theory

The Bayesian Brain Theory dated back to the 19th century by a scientist called Hermann von Helmholtz, who proposed that the brain makes inferences about the world through a process similar to statistical prediction. However, it was only during the 1990s and 2000s that the theory began to be fleshed out. The theory suggests that the brain is constantly taking everything that we’ve learned and experienced in the past to generate predictions of what’s about to happen, and our perceptions are essentially our brain’s construction of what’s most possible. Here’s a very simple test to allow you to experience the predictive power of your brain. Try to understand what I am trying to say in the following paragraph:

“Aoccdrnig to rscheearch at Cmabrigde Uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoatnt tihng is that the frist and lsat ltteer be in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe. Amzanig, huh?”

Even though the spellings are off, I trust that you’re able to decode my gibberish message because of the predictive power in your brain. By seeing the first and last letter, and deploying this into the context of what we’re discussing, our brain would tap into its wealth of past data and prior belief and generate what makes the most sense to you. It is truly amazing, isn’t it? The best thing is, it can do this without any of your conscious effort. The predictive coding feature of our brain is not something we can turn off; it runs basically 24/7 even during our sleep.

Why is that? Because other than giving us perceptions, more importantly, it helps guide us what to do. Making sense of what's going on is only the first step. To survive, we need to know what we should do at any given moment. Let's say we encounter an unknown creature that has sharp claws and teeth. It also looks very muscular. Even though we can't pinpoint what that is, based on what we know about lions and tigers, our brain predicts that this unknown creature can potentially eat us alive and recommends that we should run away as fast as we can.

This is very much like the case of our intuitive response. There is some incoming sensory information that triggers a prediction, and that prediction informs us what we should do. The only difference between intuition and the scenario I described above is that, in the case of gut feeling, we can't quite pinpoint what the triggers are and we are only conscious of the recommended decision by the brain. As we don't know what the triggers are, it makes it hard for us to make sense of the origin of the thought, and as such, we can only describe this as “gut feeling”.

The Power of Our Intuition

Now it makes sense why Daniel Kahneman says experts can make very accurate decisions based on intuition alone. They have accumulated such a wealth of data in the back of their brain, and that vast diversity of patterns and examples allow them to make an accurate prediction, even with just a quick glance. In fact, there are many accounts from experts from different fields attesting to the power of intuition. Take medical practitioners working in the ER as an example, they don't have the luxury to examine fully and deduce what has happened. So instead, they have to tap into their experience and make the best guess of what's going on. The more experienced one is, the faster the diagnosis, because it has changed from a deliberate process to an instinctive response. Same applies to firemen, or a soldier on the battlefields, more often than not, they won't not have full details of what's going on and so they have to tap into the predictive power of their brain to guess what's most likely would happen next. Again, the more experienced one is, the response is almost reflexive and thus, comes off as intuition.

An experiment in Stanford University has also confirmed that intuition can lead to the best outcome 68% of the time, almost three times higher than the detail oriented approach. More shockingly, this result held for both objective and subjective aspects of the decision tested.

The Limits of Our Intuition

However, before we fully embrace the power of intuition and relieve ourselves from the burden of careful analysis and painstaking planning process, we must also look at the limits to our gut feelings.

For starters, given our intuition draws from our unconscious database, it's obvious that our gut feeling can only be accurate if we have enough data to support it. So unless we are an expert making a decision in our field, then the so-called “gut feeling” may be nothing more than just a blind guess.

Moreover, our memory cannot be trusted. The way memory works in the brain is nothing like a static database in a computer. The memory is basically represented by the connection of neurons, and due to the properties of neural plasticity, these neural pathways can fade or change overtime. In other words, memory is not a perfect account of what had actually happened. The Mandela Effect is an excellent example. In case you aren't familiar, the Mandela Effect was a phenomenon coined by Fiona Broome. She, along with other people, distinctly remembered Mandela died in prison in the 1980s, but in reality, Nelson Mandela was released from prison in 1990 and went on to become the President of South Africa.

Even if we have eidetic memory and do have expertise in the field, our intuition can still be thrown way off balance by a wide array of cognitive biases. To be concise, I'd list out a few primary ones.

• Availability heuristics: not all memory is retrieved at the same rate. Just now we've already established that memory is essentially a connection of neurons; the stronger the connection, the faster it can reach our Bayesian Brain. Certain types of memory tend to have stronger connections, I.e. memory that is recent, and memory that is visceral. Anyhow, because they are the first to touch our brain, we may overestimate the likelihood of the events that come to mind.

• Affective heuristics: depending on our mood, we may also retrieve a different set of memories. If we are generally happy and confident at the time, we may naturally recall successful past experiences and paint a rosier picture than our reality presents. However, if we are stressed or down in the dumps, it's more likely that we'd recall past failures that lead us to err on the side of caution. Either way, the presence of emotions always skews our perceptions, and naturally, our predictions.

• Confirmation bias: predictive coding is only the first part of the process of our Bayesian Brain. After the prediction, our brain is constantly checking its accuracy against reality. When there's no gap between the two, then the brain continues its business. But if there's a gap, or prediction error, the dopamine level will drop to signal below baseline a change. We can see this as a punishment of a sort because dopamine plays a significant role in motivation and reward. In other words, our brain gets to keep its happy hormones. On the contrary, its reward is taken away and it sinks into anxiety. As we are wired to pursue reward, it's only natural that we'd prioritize and accept information that confirms our prediction, even if it's not factual.

Conclusion

Because our intuition is a manifestation of the predictive process of our brain, and that the predictive process is essentially an outcome of what we can think of at the time, whether consciously or unconsciously, there are many more biases that can negatively impact the accuracy of our intuition. While it has proven itself useful on many occasions, particularly for experts who need to make quick decisions during urgent situations, we should not put all our faith in it. Intuition is the best, or rather, the only option when we don't have enough time. But when we do have time to deliberate, we should take advantage of it and reflect on our assumptions. This is particularly important when we are facing complex or high stakes decisions. Instead of purely relying on our intuition, we can combine our gut feeling with the analytical approach and see if the data supports our initial reaction. Beware of confirmation bias during the process and seek feedback or a second opinion when necessary. Above all, if you fancy or need the power of intuition, spend the time to build up the expertise so you can unconsciously tap into a wealth of data when you need to.

Epilogue 

Circling back to my decision to join the startup, while I wasn't able to explain my optimism and resolution that filled me at that moment, I evaluated the risk of switching and exit plan. If my hunch was way off and I didn't succeed in the startup (or that it just got shut down), I knew I could always go back to the field of banking and continue as a digital product lead, so it wasn't really an impulsive decision. I combined intuition with a risk analytic approach, and decided that there's little downside for me to go try, so I made the switch. Without a Time Stone, I couldn't possibly know if it was the best outcome, but I am very happy with my job, my teammates, and my boss. I'm glad it worked out the way I hoped it to be.

In the end, while intuition should not replace careful analysis altogether, it carries tremendous benefits when properly utilized. For experts making urgent decisions, intuition proves invaluable by unconsciously accessing vast experience banks. In complex scenarios, pairing intuition with reflection challenges assumptions and seeks objective perspectives. Personally, relying on intuition two years ago led me to a highly fulfilling position and career stage. Rather than distrusting gut instincts, we can optimize decision making by understanding intuition's strengths and limitations. With continuous learning over time, intuition naturally sharpens to reliably guide even our most pivotal choices.