Thursday, May 17. 2012
The inspiration for today's blog post is a chapter from Daniel Kahneman's "Thinking, Fast and Slow" where he explored how our brains often figure out how to solve difficult questions by – sometimes unwittingly – substituting an easier question, and answering that instead.
For instance, if someone asks us “How popular will the president be six months from now” most of us don’t actually go through the detailed analysis to project how changing trends will impact the president’s popularity down the road. Instead, our brains will tend to take a shortcut, like looking at how popular the president is now, and simply projecting that into the future (perhaps with some very small adjustments). Similarly, if someone asks you how happy you are with your overall life right now, you’re more likely to simply reflect what your current mood is right now – it’s an easier, more readily available question your brain knows how to answer. If your mood is good, you'll probably say life is pretty good; if your mood is bad, you'll answer accordingly, too.
The research shows that for some types of questions, the easier question we substitute may simply be one that reflects the current situation or environment, such as our mood for our overall life or the current popularity of the president for an estimate in the future. With some kinds of abstract questions, however, we have to go a step further.
Relating to Intensity
It turns out that one thing our brains are especially good at is thinking in terms of intensity, and relating things on different scales to a similar one based on intensity. For example, Kahneman’s book gives the example of a situation: “Julie read fluently when she was four years old.” If a researcher were then to ask you a relative question, such as “How tall is a man who is as tall as Julie was precocious” it turns out that most people give a fairly consistent response – in this context, we relate it to some height that is as remarkable as a four-year-old who can read is also remarkable.
In fact, within a consistent cultural environment – where we all tend to relate to the context similarly – we could also answer questions like “What level of income in your profession matches Julie’s reading achievement?” or “Which crime is as severe as Julie is precocious?” or “Which graduating GPA in an Ivy League college matches Julie’s reading?” The point, simply put – we’re quite good at relating comparative intensities on different scales and translating between the two.
Combining Substitution and Intensity Questions
In some situations, our brains will attempt to answer difficult questions by using a combination of substitution, for an easier question, along with intensity to answer that question. For instance, Kahneman's book notes that when we are asked a question like “How much would you contribute to save an endangered species?” what most people really answer is the question “How much emotion do I feel when I think of dying dolphins?” We grade the answer on our emotional intensity scale, relate it to an associated dollar amount, and answer the question accordingly.
In fact, in one experiment (which has since been confirmed repeatedly) first conducted not long after the Exxon Valdez spill, researchers asked various groups of participants about their willingness to pay for nets to cover oil ponds in which migratory birds often drowned. The participants were asked to state how much they would pay to save either 2,000, 20,000, or 200,000 birds. Logically, saving 200,000 birds should be worth radically more than saving just 2,000 birds, but the researchers found in fact that the average contributions of the three groups were $80, $78, and $88 respectively. The number of birds made remarkably little different at all. In this case, it was because the participants first substituted the question “how much would you pay to save XXX birds” for the easier question “how intensely do you feel about the image of a helpless bird drowning because its feathers are soaked in thick oil” and then related the intensity of their thoughts about a single bird into a dollar amount. While our brains are good at relating intensities, the substitution effect meant that the participants had virtually entirely disregarded the part of the question about the number of birds (without even realizing it) and the magnitude of the project and its success or failure.
Substitution and Intensity Questions in Financial Planning
Reading through this discussion on how our brains evaluate difficult questions was striking to me, because we often ask similarly challenging questions of clients as a part of the financial planning process. And given the research on how our brains think, it’s almost certain our clients engage in a similar process – which is somewhat concerning, as the consequences can lead to very distorted conclusions, such as the study participants who were only willing to pay 10% more to save 100 times as many birds.
For instance, imagine the situation where the client is asked to decide what probability of success is acceptable for his/her retirement plan. As the conversation often goes, the client is asked which plan is preferable: one that has an 85% probability of success, a 95% probability, or if the client would like to save more/spend less/retire later so that the plan can have a 99% probability.
In practice, our brains have little framework to really evaluate such probabilities; in the end, we don’t really know how to evaluate a retirement that has a 90% probability of succeeding. Instead, the research suggests that we probably substitute an easier question, such as “how intensely bad would you feel about running out of money in retirement?” Given our ease of converting intensity questions on different scales, the brain can easily answer this question, evaluating the intensity of negative feelings about the potential adverse outcome, and then converting them to a 1% - 100% scale. Clients who have intensely bad feelings about a potential retirement “failure” will give higher required probabilities of success, while clients who are less emotionally distressed at the thought will answer lower probabilities. Thus, notwithstanding the original question, clients who suggest a preferred probability of success are probably not actually indicating how much risk (of failure) they wish to expose themselves to, but instead are indicating how distressing in their minds that failure would be.
Framing the Consequences
The reason that this substitution effect matters – where clients answer the question “how much risk would you like to take” with the easier “how intensely bad would you feel if the adverse risky event happened” is that as planners, we often do a poor job of effectively defining exactly what the risky outcome would be.
For instance, think again about the scenario where a client is asked what probability of success would be preferable for a retirement plan: 85%, 95%, or 99%. In asking this question, we generally leave it up to the client to imagine what failure would look like. Without any other information, the logical conclusion – and in fact, the one sometimes implied by the planner – is that failure means a total loss of assets. Lifestyle and enjoyment ends. The family home is sold. From now on dinner is dog food, and you can never afford to see the grandchildren again.
Yet when we look at the realities of a retirement plan and how the financial planning process is executed, this is really a gross mis-statement. As discussed previously on this blog, the reality is that the probability of “failure” would be more actually characterized as a probability of adjustment instead. It represents the odds that the client would be heading down an adverse path that, through monitoring, might require a mid-course correction to get back on track. As research by Guyton in the Journal of Financial Planning has shown, “mere” spending cuts of 10% in difficult times can be effective to get clients back on track, and in fact they generally can make up the spending cuts and more in the future when the good returns come back.
So imagine a world where two clients are asked a similar but different question:
Client A: “What probability of success would be preferable for your retirement plan, 85%, 95%, or 99%?”
Client B: “What probability of success would be preferable for your retirement plan, 85%, 95%, or 99%, where a ‘failure’ means you would have to engage in a 1-5 year spending where your spending is cut by 10% until the market recovers?”
In reality, both scenarios describe the same situation, at least how it would likely play out with a planner engaging in an ongoing monitoring process with a typical client (who could intervene with a mid-course correction if the client was heading towards a danger zone). Except the reality is that because the scenarios have a very different implied outcome – client B faces “just” a potential 10% spending cut for a few years, while client A is left to his/her own imagination about how catastrophic the failure must be (given no other information) – the clients may convey very different comfort levels and risk preferences, even though it’s actually the same planning scenario, because they're expressing different intensities around what they think are different outcomes!
Which means the bottom line is that in situations where clients are invited to make a decision about how much risk to take, it is crucial to define what a risky event or an adverse outcome really means. Is it a retirement plan that requires some mid-course corrections with moderate spending cuts, or total destitution? Is it a portfolio that could experience a 20% pullback, or a 100% total loss? If the consequences aren’t defined clearly, the client at best will simply infer whatever consequence he/she thinks would be the result, judge the intensity accordingly, and make a decision about risk taking. At worst, the client infers the wrong risky outcome, leading to an entirely inappropriate conclusion about risk taking. Because like it or not, the research – as discussed in Kahneman’s book – clearly shows that this is how our brains operate.
So what do you think? Do you always clearly define the consequences of a potential risk decision? Do you frame for clients that probabilities of success are about success versus total failure? Or success versus moderate mid-course corrections? Do you think it would change their decisions about which retirement plan they choose?
Tracked: May 22, 12:23
Tracked: Jun 12, 13:50
Monte Carlo analysis has become an increasingly popular arrow in the financial planner's quiver, as an improvement over the oversimplified traditional straight-line projection. Unfortunately, though, use of Monte Carlo analysis has begun to focus excessiv
Tracked: Jul 26, 14:17
Tracked: Apr 17, 09:02
My opinion is that planners should focus more on helping clients make reasonable adjustments when things change and focus less on false assurances.
Understood, but at some point we need SOME understanding of what paths are likely to lead to no adjustments, some adjustments, or intolerable adjustments?
How else would we know whether $20k, $40k, $60k, or $80k/year of spending is prudent at all from a $1,000,000 portfolio without some form of probabilistic analysis?
I also agree that some form of probabilistic analysis is needed, what I'm not so sure about is if we should even talk to our clients about the raw numerical output? I don't know if this is true or not, but it seems we are taking a technical tool and placing it on a pedestal.
Maybe, and this is a hypothesis, we should instead use monte carlo, along with our understanding of what changes in spending might be needed to get back on track and how that might impact their life (risk capacity of the goal), and account for our understanding of their risk attitude, and derive our recommendation. I guess what I'm saying is, if clients aren't capable of making these decisions we're asking them to make then perhaps we should take more responsibility to come to a rational conclusion and recommend it. As the advisor, put a stake in the ground. The alternative, which you gave an example of Michael, is to figure out a different way to both ask and frame the question. I'm not so sure I know a good way to do that.
But I also feel the conversation should be very broad and should never be about confidence levels, etc.. Simply asking a question "what probability will you tolerate at 85%, 95% or 99%.... is a bad idea because it may give the client a feeling that there is more control than is actually the case.
Investing is not physics. Over the last twenty years, we have experienced several financial events that were supposed to happen only once in a lifetime. Trying to produce models and actually showing them to clients is a disservice, in my humble opinion.
With all due respect, there have NOT been any "once in a lifetime" events in the past twenty years on an ANNUAL or longer basis. These "black swan" events have been DAILY events that evened out over the span of just a few weeks and months, and given that clients don't need to make material liquidations on a daily or weekly basis, had NO black-swan-style impact. (The consequences for over-leveraged hedge funds, on the other hand, would be quite different.)
You might see some of the prior posts I've made on this, such as http://www.kitces.com/blog/archives/255-Are-Black-Swans-Just-A-Short-Term-Distraction.html - which shows that on an annual basis, the only "black swans" we've seen in the past century were impossibly GOOD returns, and not a single impossibly BAD one.
Our industry has grossly mischaracterized entirely-probable-and-expected big bear markets with black swan events. Individual companies have plenty of black swans; markets in the aggregate have not over any time period longer than minutes, days, or weeks.
In other words, as long as we're running Monte Carlo analysis on an annual basis, the primary error has been underestimating very GOOD years, not very bad ones.
When using Monte Carlo, we need to make sure clients understand that all we simulate is how a fixed spending scenario survives over a certain period under a large number of varied portfolio returns. The percentage of success is a pass/fail for that test, and doesn't show what percentage of the trials would have permitted more spending, nor what portion could have passed with moderate spending reductions (as you cite from Guyton's article). Change any of the spending assumptions, and the MC pass rate changes.
I hope I have educated my clients that MC "results" are helpful educational aids to indicate if our current direction is acceptable, or if a course correction (saving rate, timing and size of spending) should change. Sadly, many people seem to take the mental shortcut of thinking the result is a chance of success, just like your chances of winning the lottery.
Modern financial modeling is an incredibly powerful example of how our industry takes shortcuts instead of answering the real question (at least when we do not provide the proper context and framing). This problem is extraordinarily pervasive in the investment realm, as James Montier recently pointed out in his keynote presentation at the CFA Institute Annual Meeting: http://bit.ly/Iw7KS2, and it is beginning to creep into planning as well.
As Montier explains, “mathematics is ordinarily considered as producing precise, dependable results. But in the stock market, the more elaborate and obtuse the mathematics, the more uncertain and speculative the conclusions we draw therefrom. Whenever calculus is brought in, or higher algebra, you can take it as a warning signal that the operator is trying to substitute theory for experience...If you were to give a bunch of monkeys a CAPM pricing model or a VAR risk model, you will end up creating a financial crisis,” he said. “In fact, I am pretty sure that is just what we have done.”
Human beings feel more confident when we eliminate variables and uncertainty, which is something we attempt to do through this type of analysis. However, unless we give this type of analysis its proper respect and truly recognize its limitations, we are going to do every bit as much harm as good.
All very good points. I very much agree with you. I am in no way trying to mischaractarize our past bear markets as black swans. You mention that the bear markets were both probable and expected, and I completely agree.
I also agree that the term black swan has been abused. I think it was Taleb who coined the term and he meant it to be related to fat tails, which of course could be both bad and good, and which both occur often in the financial markets. The exceptionally good results you mention should also be construed as black swans.
My point was that from a client's perspective I feel it tends to send a wrong message.
Anyway, thanks for the post. Your blog is a pleasure.
Indeed, no question results are often misconstrued. I question whether all advisors even fully understand the implications of the Monte Carlo tools they are using, much less translate them effectively for clients.
That doesn't necessarily make the tool bad. It is a huge concern regarding effective advice, though. :/
It was kind of Monte Carlo simulation as a sales technique!
I do feel it can be a very good "behind the scenes" tool if used well and can help advisors in recommendations to clients.
We absolutely have the equipment to think probabilistically, but that requires us to think. Tools can too easily become a crutch that eliminates the need to think. If we focus instead on guiding people toward decisions that yield the highest probability of success given an unquantifiable amount of uncertainty, our clients will be far better off.
I think it depends on the client and the nature of their spending.
We have clients where a 5% spending cut would be traumatic. We have others where a 20% cut would be easy - they'd just trim or downscale the number of lavish cruises and vacations they take for a year or two.
I do think we could use a better framework for discussing with clients the flexibility of their spending, though. It's not a conversation we're typically trained to do.
I, like Khaneman, believe the most important # an advisor needs to know is the client's "Pain #". Knowing the point where a client's emotional state will overrun their rational self is the goal.