In November of 2016, a joint 18-month study from the State Street Center for Applied Research and the CFA Institute, which combined 200 in-depth interviews with global industry leaders and a survey of 3,300 “investment professionals”, was released, and claimed to have discovered “the hidden variable of [organizational] performance”. Dubbed “Phi”, the research suggested that a combination of the motivational forces of Purpose, Habits, and Incentives was associated with superior organizational outcomes; specifically, the results showed that a one-point increase in Phi was associated with 28% greater odds of excellent organizational performance, 55% greater odds of excellent client satisfaction, and 57% greater odds of excellent employee engagement.
In this guest post, Derek Tharp – our new Research Associate at Kitces.com, and a Ph.D. candidate in the financial planning program at Kansas State University – analyzes the Phi study and delves deeper into exactly what it is, how it is measured, and what conclusions should (and shouldn’t) be drawn from this study.
Because while the researchers did find a statistically significant relationship between Phi (as measured by self-reported purpose, habits, and incentives), and organizational outcomes, client satisfaction, and employee engagement, it’s not entirely clear what the results actually prove.
The challenge, in part, is that the study itself didn’t actually evaluate organizational performance, client satisfaction, or employee engagement! Instead, the researchers evaluated self-reported assessments of organizational performance, client satisfaction, and employee engagement, and compared them to self-reported statistics on Phi (for the entire organization). And this distinction is meaningful, because it isn’t exactly clear that employees make valid or reliable assessments of company-wide motivational forces, nor the entire company’s organizational performance, client satisfaction, or employee engagement — and especially when these employees may not actually even know if clients’ goals are being met (how “organizational performance” was defined), whether clients or satisfied, or whether other employees within a large multinational financial services firm are actually even engaged!
In addition, there’s the fundamental challenge that a correlation between two data points doesn’t necessarily prove causation either. In other words, the fact that high-performing organizations also report higher Phi doesn’t mean the higher Phi caused the better results. Perhaps organizations that are already outperforming simply attract more motivated employees (which means the causality goes the other direction).
In the end, the Phi study did ask some interesting questions and set out with some lofty ambitions to evaluate the impact of motivational forces – a question that is admittedly not easy to study. But until further research can tie Phi to actual outcomes or provide evidence that the self-reported measures are assessing something meaningful in a valid and reliable way, then the conclusions we can actually draw from the Phi research are limited.