We received many responses to Michael Edesess’ article,Why DFA’s New Research is Flawed, which appeared last week. Below are responses from individuals who disagreed with Edesess’ findings, followed by Edesess’ response and then by responses in agreement with his findings. There has also been an extensive discussion of this article on the “Bogleheads” forum here.
Dear Editor,
Folks have been lighting up my inbox with questions and comments about this article (link above for the three of you who may not have seen the piece … you three may also not be aware that Miley Cyrus appeared on the MTV Video Music Awards … link not included).
The article is critical of DFA’s recent work on profitability. I’ll focus most of my comments on the contents of Edesess’ section entitled “How the DFA argument is flawed.”
Don’t shoot the messenger
One point the article fails to mention is that Eugene Fama, Ken French and DFA are not alone on an island researching profitability. While Fama and French have no doubt made significant contributions in several peer-reviewed journal articles dating back to 2006, arguably the most important piece on the topic was done by professor Robert Novy-Marx at the University of Rochester in a Journal of Financial Economics article entitled The Other Side of Value: The Gross Profitability Premium. Professor Novy-Marx has no association with DFA. The empirical work that Dimensional has recently done is simply confirming Novy-Marx’s findings (and to some extent, confirming elements of Benjamin Graham and Warren Buffett's approaches to investing).
Bigger picture, I think it’s also relevant to note how infrequently papers submitted to journals such as the Journal of Financial Economics are actually published. For that particular journal, which also contained the 2006 Fama and French piece, approximately 10% of submissions are published. So, if a piece is published in this journal or journals of similar quality, it has undergone an extremely rigorous process. Further, articles published in these types of journals have led to innovations in thinking and understanding of financial markets that have undoubtedly led to better investment practices.
Scientism or empiricism?
I’m confident that both Fama and French are well aware that financial research isn’t the scientific equivalent of physics. The vast majority of the work that both are famous for is empirical and not theoretical in nature. In that empirical realm, they use the most advanced method available: out-of-sample tests. If you find that a strategy has worked in one market historically (e.g., the U.S. stock market) but not in another market (e.g., the international stock market), then your original finding could be bogus and a result of data mining. If you find the same result in both places, then your original finding may have merit.
However, Fama and French and others at DFA would also freely admit that no amount of empirical or theoretical work (or out-of-sample tests) can guarantee that the future returns of any strategy will look like past returns. There is a limit to what financial research can show, and I’ve heard both acknowledge that limit. The fact that there is a limit, however, doesn’t mean that financial research by Fama and French and others is of no value (or destroys value).
On the theoretical side, they would also be the first to say that theory isn’t reality and that theory can never fully describe reality. We can posit, for example, why a size premium or value premium might exist, but we have no way of confirming whether those explanations are 100% accurate or even 50% percent accurate. Again, this doesn’t mean theoretical models are a complete waste of time. It simply means there are natural limits as to what they can tell us.
Re-examining profitability
Now, let’s move specifically to Edesess’ critiques and questions about the profitability work. He states:
“Of course, just as the price of a bond or stock changes as the expected income stream changes, the price in Fama and French’s model cannot be fixed — not in the real world — as expected future earnings change.”
This statement is basically the entire basis for Edesess’ critique, as he argues that all of DFA's analysis that follows is based upon a flimsy model, essentially making the case that it’s unreasonable to construct a model where profitability is positively related with stock returns. What he misses here is that both practitioners and academics have long suspected there could be a link between profitability and stock returns. This quote from Benjamin Graham in 1973 is included in a working paper by Novy-Marx:
According to Graham, an equity investor should “… apply a set of standards to each [stock] purchase, to make sure that he obtains (1) a minimum of quality in the past performance and current financial position of the company, and also (2) a minimum of quantity in terms of earnings and assets per dollar of price.”
You don’t have to read too deeply to realize that Graham was basically saying investors should purchase stocks at relatively low valuations (i.e., value stocks) with attractive earnings potential (i.e., profitable stocks). In my mind, it’s impossible to argue that any area where both Graham and Fama and French (not to mention Novy-Marx) explored or are exploring is flimsy work or thinking.
Now, Edesess could argue that the positive link between profitability and stock returns in U.S., international and emerging markets is simply a coincidence and unlikely to repeat again (maybe this is what he is hinting at with his stock versus long-term bonds analogy).
He can’t, however, critique the entirety of DFA’s work on profitability by arguing that Fama and French’s theoretical model is absurd when multiple other researchers and practitioners have suggested the profitability and returns linkage and that result is confirmed by the data.
Edesess also states: “However, in another article by Jay D. Franklin and Mark Hebner of Index Fund Advisors, an advisory firm that uses DFA funds, they show a premium for the high-profitability portfolio of only about half a percent.”
Edesess then wonders why this result is different from the roughly 5% advantage documented by DFA’s Gerard O’Reilly and Savina Rizova in a separate piece. The explanation for this is relatively straightforward. O’Reilly and Rizova calculated the difference in returns of a portfolio of high profitability stocks compared with a portfolio of low profitability stocks. From 1975–2012, this return difference was about 5% per year.
Franklin and Hebner instead presented the difference in returns of a U.S. large-value portfolio compared with the returns of a U.S. large-value portfolio that modestly tilts toward more profitable U.S. large stocks over a different period of time. In short, the Franklin-Hebner analysis wasn’t comparing the returns of high-profitability stocks with low-profitability stocks like O’Reilly and Rizova was. Franklin and Hebner compared the returns of two very similar portfolios, with the only difference being that one has a slight tilt toward higher profitability stocks. As a result, the return difference is smaller.
The last two critiques relate to 1) what exactly is the explanation for why more profitable stocks have outperformed less profitable stocks and 2) whether current profitability truly is a good proxy for the long-term profitability of a company.
On the first count, I’m sympathetic. No one can really know exactly why more profitable stocks have outperformed less profitable stocks and whether they will continue to do so with certainty. Could it be that more profitable stocks earn higher returns because higher profits tend to encourage competitors to enter the same business, and the market is pricing in a return premium for this risk? Or could it be that this is a behavioral phenomenon? Relating back to the limits of financial research, we may never know the answer, and this is why financial research has to rely so heavily on empirical findings and testing those findings on other data periods and other samples of data.
On the second count, DFA shows that current profitability is significantly positively related to profitability at least seven years out. I’m not sure how Edesess would define long term, but it seems like seven years would certainly qualify.
Yours truly,
Jared Kizer, CFA
Director of Investment Strategy
The BAM Alliance
St. Louis, MO
Dear Editor,
I use DFA funds for client portfolios. Your article piqued my interest. It makes sense to go after the much bigger and better-known DFA rather than AQR, assuming you want to get noticed. Both derived similarly-screened profitability funds at about the same time.
Here are a few problems I see with the article.
Bad analogy
Edesess wrote:
A Parable: Bonds have a higher expected return than stocks
I posit that the expected return on bonds is greater than the expected return on stocks. Here is my reasoning:
A discount model shows that the current price of a security is equal to its expected future income stream discounted to the present by its expected return. Hence, holding the security’s price constant, the greater the expected income stream the greater the expected return. Bonds produce more income than stocks. Therefore they have a higher expected return.
The empirical data confirm this theoretical result. Bonds have outperformed stocks in the last 40 years. A regression of monthly bond returns against equity returns in the past 40 years shows a large and statistically significant positive alpha for bonds.
All of these things are true. Therefore, the expected return on bonds is higher than the expected return on stocks.
Here are the problems I see with his analogy:
- His hypothetical model ignores growth and focuses solely on income – think Gordon growth a.k.a. the dividend-discount model – and thus the analogy isn't a good one.
- There's no out-of-sample testing done on his analogy covering various time periods and other countries. Doing so surely would show the last 40 years, given whatever bond index he chose, was abnormal for stocks versus bonds. Further, stocks typically yielded more than bonds pre-1960, so the out-of-sample test would have challenged his conclusion even assuming his model was sound.
Fama and French’s supposition is a common one, done since the advent of efficient-market theory. Edesess doesn't like Fama and French's model:
“Suppose we fix the price Mt and the values of everything else in equation (3) except for expected future earnings and the discount rate (the expected stock return). Then the equation tells us that higher expected future earnings imply higher expected stock returns. This is the motivation for tests of a positive relation between expected stock returns and expected profitability.”
This is the basis for efficient-market theory – that the current price is a fair estimate (not a perfect estimate) of the future price. Efficient-market theory certainly isn't perfect, but in practice and for those who advocate active management, it is darn tough to beat.
All theories have simplifying assumptions, but should be representative and sufficiently describe the real world. Perhaps Edesess' problem is more so with financial theory and its assumptions and limitations in general. Yet, no other asset pricing theory has consistently yielded better, real-life investment results than efficient-market theory, as measured by scores of active verse passive studies over the years.
Further, Fama and French’s supposition allows expected future earnings and expected returns to vary, because they obviously do. They're just difficult, if not impossible, to forecast consistently. Applying various factors or screens helps to better identify those companies likely to have characteristics that will lead to some level of outperformance.
If you consult the various profitability research (not just DFA's), the profitability factor shows up in out-of-sample tests and across asset classes – pretty robust. You can see much of this very quickly from this short IFA article and particularly the last chart.
No one is completely objective, but I personally try to get information from various sources and continually challenge my beliefs and stay open to new ideas. That's why I read Edesess' article. It simply falls short.
Kevin Kroskey, CFP®, MBA
True Wealth Design, LLC
Akron, OH
Dear Editor,
I thought Edesess' article showed a shocking lack of knowledge about most things DFA as well as the research on profitability and implication for DFA’s funds.
Edesess was wrong about when DFA incorporated "patient trading" into their Micro Cap (formerly 9-10 fund) – it was there from the start, not "later."
Edesess was wrong about when financial advisors began gaining access to DFA’s strategies – it was almost 25 years ago, not "a little more than 10 years ago."
Edesess was wrong about his "scientism" reference as it relates to financial economics. In fact, the asset pricing research that serves as the backbone of DFA’s mutual funds is rooted in the scientific method, whereby we seek to acquire new knowledge about how capital markets function and where returns come from by forming a hypothesis based on well-researched financial models, test that hypothesis for economic and statistical significance, check for out-of-sample confirmation, introduce the findings into real-world investment portfolios, and observe and monitor their ongoing process to confirm or refute the original findings while incorporating enhancements along the way.
Edesess was wrong that DFA is a "large company." By assets under management, its almost-$300 billion is substantial, but with a little more than 700 employees, they are 20-times smaller than the largest mutual fund family – Vanguard, which employs over 13,500 people.
Edesess' parable made no sense, as it attempted to apply the dividend-discount model to a comparison of stocks and bonds without acknowledging that the different securities carry different costs-of-capital and cash-flow growth rates and therefore are completely separate matters.
Edesess was wrong about the ability to hold variables constant in an asset-pricing model. Holding price constant in a basic price-to-book value formula, when reconstituting the Fama-French value/growth indices, is at the heart of the historical measurement of the value premium. In real-world portfolios, securities are bought and sold more than just annually to reflect dynamically changing prices relative to fundamentals, but the price when the security is bought or sold is all that matters (and is fixed at that point in time). And as the +2.1% and 5.1% annual outperformance for DFA’s U.S. large-value and small-value funds (relative to the Russell 1000 and 2000 growth indices) shows this can be performed with a high degree of precision and success.
Edesess exaggerated the outperformance of bonds over stocks. There have been two rolling 40-year periods where long-term government bonds have outperformed the S&P 500 on a monthly basis (n= almost 600) since 1926. Comparing that fickle result with the statistical significance of the profitability premium is clearly flawed and doesn't make Edesess’ point. The actual time for which the chosen bond index outperformed stocks is irrelevant – it did so by 0.12% and 0.09% per year in the two 40-year periods where this happened (and did not in over 500 other rolling periods we can observe). The magnitudeof the outcome is clearly both statistically and economically insignificant (while the profitability premium was highly significant). The result was not pervasive across stock/bond asset classes – for example if we substitute short-bonds for long-bonds or small stocks or value stocks for large stocks, the outcome changes (the profitability premium was pervasive across asset classes), and to my knowledge we did not see the same long-bond premium in all other countries of the world (as the profitability premium is positive and significant in every region we have data).
Edesess was wrong that bonds, during either of the 40-year periods where they beat the S&P 500, showed "highly positive and statistically significant alpha." Bond regressions are done either based on two well-documented fixed-income factors (term and credit), or by also incorporating the three equity factors (market, size, and value) to pick up any covariance with stocks. Whichever model you choose, bonds had no alpha during either of these periods; they simply benefited from the large but anomalous term premium during this period, which is in fact a priced dimension of risk and not something deserving of "alpha."
Edesess was wrong to call the dividend-discount model (used in the Fama-French research papers) "flimsy." The model has been used for over 50 years as a basic arbiter of expected stock prices based on the relationship between cash flows and stock price growth rates as well as the cost-of-capital of the underlying companies. Either Edesess believed the historical data to be flimsy, which clearly showed the existence of a return premium, or the basis on which that premium should materialize: the dividend-discount model was flimsy. Of course, the first cannot be disputed unless you are proposing a conspiracy of the CRSP data, so I assumed Edesess was referring to the model itself as flimsy.
The dividend-discount model is used for valuation of equity securities, which is the only reasonable way you can propose to hold certain variables constant in an attempt to isolate the impact of others (it's only reasonable that two competing equities would have the same cost-of-capital or growth rates). Basic bond valuation says that the price of a bond equals the sum of the present value of its future coupon payments discounted at the market rate of interest. The dividend-discount model says a stock return is the sum of future cash flows discounted by the required rate of return for owning the stock (cash-flow growth rate minus equity cost-of-capital). A bond doesn't have a cash-flow growth rate nor does the equity cost of capital apply, so holding these constant is meaningless.
There is a widespread precedent for the practice of holding variables constant in the dividend-discount model or other financial equations applied to the appropriate securities. For example, we could set aside the profitability conversation and instead discuss the value premium, which the dividend-discount model predicts would exist if we rearrange the original model and hold everything in the equation fixed but the stock price – which of course says that lower prices equate to higher expected returns. Or that for a given price, a higher book ratio equates to a higher expected return.
Edesess was wrong when he referred to financial economics as "nothing but playing around with formulas." In fact, financial economics does use basic formulas as building blocks to derive testable or implementable policies from assumptions or findings. People make the mistake of assuming models are reality, which is clearly not the case. However, to imply that making financial decisions based on hunches or intuition, instead of adhering to a process and a set of rules-based assumptions, is clearly flawed.
Edesess was wrong when he assumed that the size of the U.S. long/short profitability factor was the "confirmation" of the financial economics derived from the Fama-French work. In fact, the size and statistical significance of the premium, the existence of a similar sized premium in large and small U.S. stocks, non-U.S. stocks and emerging-market stocks for as long as we have data also contributed to that confirmation. DFA’s finding that a financial variable that had statistically significant forecasting power in identifying highly profitable companies today that would remain highly profitable in the future all led to the conclusion that profitability was a significant determinate of future expected returns, as the dividend discount model would imply.
Edesess showed no understanding of the difference between the returns of a long/short expected return factor (which is how Fama and French derived their 5% profitability premium) and how DFA incorporated these findings into value and small-cap indices for illustration purposes. DFA's research included "value plus profitability indices" that simply underweighted or excluded a small percentage of unprofitable companies from existing large and small value stock indexes while extending the universe of acceptable value stocks (which lowered expected returns, while maintaining acceptable diversification). This of course has a smaller impact on expected returns (about 0.5% and as high as 0.9% for international large value). It also lowered the risk of the traditional value portfolios, decreased their tracking error relative to the market and traditional value indices, and increased the statistical significance of the outperformance.
DFA has also published research that applied the profitability factor in a more broad-based manner to the large-cap market and the large-growth market. Those indices produced results 1.2% and 1.3% greater than the S&P 500 and Russell 1000 growth indices, respectively, with less volatility. This is still not the 5% shown in long-short analysis; that result could only be achieved in a long-short portfolio because long-only investment indices that over/underweight securities based on their exposure to direct profitability don't "load" 100% on the premium, just as traditional value indexes don't "load" 100% on the value premium.
Edesess was wrong when he attempted to explain why DFA's metric for direct profitability is meaningless. First, regressions show that even basic profitability ratios (sales/book, income/book, etc.) do an admirable job of forecasting future profitability three to seven years out – explaining as much as 42% of future profitability over a seven-year period. Further, the variable DFA uses, "sales minus cost of goods sold minus selling and admin expenses minus interest expense divided by book value," addresses a number of outstanding issues that had previously surfaced when trying to identify future profitability using simpler proxies: it's comprehensive, comparable across sectors and applicable across countries, and excludes non-recurring items.
We've known about the existence of a high return for profitable stocks that is unexplained by prevailing asset pricing models for some time (Fama/French 2004; Novy-Marx more recently). The most basic equity valuation model in finance predicts that this should be the case. Regressions on basic profitability ratios find a high level of explainablity about future profitability going out five to seven years. Basic long/short profitability portfolios have produced large levels of return dispersion in every asset class over every time period we can measure, even after the original papers on the subject were published. We'd have a hard time showing that any of these facts would happen by pure chance or are irrelevant. That they all are true, and that the covariation of the profitability premium with the value and (to a lesser extent) the size premium means this is a very significant research breakthrough which should continue to be scrutinized but also taken very seriously.
Sincerely,
Eric D. Nelson, CFA
Managing Principal
Servo Wealth Management
Oklahoma City, OK
Michael Edesess replies:
I’m a little chagrined to hear from Jared Kizer that my article has gone as viral as the Miley Cyrus story. Nevertheless I’m gamely waiting by the phone for the requests for TV appearances and interviews. Meanwhile, I’ll respond to the voluminous critiques sent in by readers.
First, my account of the timing of DFA’s activities was derived from anecdotal information and a few short-term interactions with the company and its principals spread over a number of years – all of which incidentally have been wholly positive. I wouldn’t have wished to single DFA out, but it’s their analysis that crossed my desk, and unfortunately I think it is representative of an industry-wide problem.
I applied the word “flimsy” to the Fama/French argument that higher profitability implies higher expected return, not to the dividend discount model itself. I make the point in my article that Fama/French's inference from the dividend discount formula is wrong. Perhaps I need to make the point more clearly. Consider a simple three-variable formula, for example x=y+z. If you hold z constant then higher y implies higher x. But from this Fama/French draw the illogical conclusion that higher y simply implies higher x. But that's not at all true if you don't hold z constant – there are an unlimited number of real-world counterexamples. In fact the dividend discount model itself is a counterexample if you assume efficient markets – an assumption for which, of course, Professor Fama is famous: as expected future earnings increase, that expectation will be priced in so that expected return remains constant; hence expected return will not increase with increased earnings.
If this weren’t taken too seriously as mathematics it wouldn’t be of overriding importance. But that's not how it's treated in most of the financial world, and unfortunately not how it’s treated by DFA in its follow-on papers. It's elevated to the status of "science" which it most certainly is not. Meanwhile, stripped of this flawed, supposedly theoretical justification, there's no reason to believe that expected return is higher for higher profitability companies except that this was the case historically. But that history could, perhaps, equally well be interpreted to imply not that it will persist but that a reversal is coming – as it is with bonds.
In my parable I applied the dividend discount model as a formal mathematical model, deliberately making exactly the same mistake that Fama and French make in their SSRN article. Variables not represented in the model were irrelevant in their exercise, as they were in my parable.
I did not exaggerate the outperformance of bonds over stocks. Bonds outperformed stocks over a 40-year period ending recently, longer than the time period over which DFA observed outperformance of high-profitability stocks in the paper I cited. One can debate how alpha should be defined in this context, but there is no question that the Sharpe ratio of bonds was much higher than that of stocks over this period.
Neither the Fama/French formulaic analysis taken alone, nor the observation of historical outperformance of higher profitability stocks taken alone, nor both together can justify assigning a higher expected return to high-profitability stocks. For that, a better explanation and theoretical justification is needed. The same exact methodology that Fama/French/DFA use would require assigning a higher expected return to bonds than to stocks. This should suffice to raise suspicion about it.
Jared Kizer explains that the historical 5% high-profitability premium in one of the DFA papers was for portfolios entirely composed of high profitability stocks as compared to those entirely composed of low profitability stocks, while the half-percent premium in the Franklin-Hebner analysis was for a portfolio modestly tilted toward high profitability stocks, and over a different time period. That is helpful information, some of which I was not able to glean by only reading the DFA papers, which struck me as somewhat opaque.
On the bogleheads.org forum Larry Swedroe, a colleague of Mr. Kizer, observes that “no matter the value metric used, if you buy stocks with a low [ratio of] price to pretty much anything, you get a premium.” This is indeed a relationship that has been observed to be robust historically. It also coincided with the investing philosophies of the legendary Benjamin Graham and his eminent student Warren Buffett. Shedding light on this historical relationship is something that many investment theoreticians would like to do. It is important to do it because investors want to know if it will persist, or if it will reverse when too many people catch onto it. Personally I don’t think regressions performed on historical data are adding very much anymore to this inquiry – what we need are explanations, and sound ones, not more dry data studies, since as Mr. Swedroe notes they just keep coming up with the same thing.
Bill Bernstein further notes on the same site that whatever the cause – he invokes pragmatism – DFA’s numbers have been excellent. Broadly speaking, the data show that “passive” investment strategy providers, including Vanguard and DFA, are closely competitive with each other and produce good numbers. The passive nature of their processes and the low fees seem to be responsible. How much can be attributed to “tilts” is difficult to determine in a risk-aware evaluation framework, when the historical findings themselves have opened the accepted measure of risk, standard deviation, to question.
Perhaps we should ask whether it’s worth the additional cost of all this data mining, however carefully done, to eke out a few small additional speculative fractions of a percent in performance. A friend with whom I was discussing this came up with this bizarre but ingenious suggestion: that the cost of running regressions should be added as a factor in regressions of returns against factors of performance. What if it turned out that performance did indeed increase with the cost of running regressions, but by less than the cost?
John Bogle has said that investors have been given the equivalent of a "gold ingot" for portfolio construction, which is the low-cost diversified index fund. There have been many attempts to improve on it, while maintaining a passive framework. Advisors must decide whether a supposed improvement is a gimmick that serves to enrich product providers, or whether it is a genuine improvement that enriches investors. Empirical evidence for value tilts has been strong though highly period-dependent; furthermore a cogent and compelling argument for why it should persist is still lacking. Given how skeptical we remain as to the explanation for these empirical findings, we should be all the more skeptical about additional fine-tuning. And our statements about the state of scientific advancement of the industry should reflect that modesty.
Michael Edesess
Following are letters in support of Edesess’ claims:
Dear Editor,
Wow! Regarding Edesess' evaluation of DFA's new research, this man's bravery is only exceeded by his intellect. I do not have the training to evaluate the essential nature of his critique of DFA's research, but his points seem logical, cogent and are well presented. I found myself agreeing with much he had to say. I believe that such criticism and debate is not only healthy for the investment industry, but essential if it is to ever become a true profession. Edesess is to be lauded for taking on icons of the industry in such a smart, courteous and well meaning way.
I want to do whatever small thing I can to encourage others to be as brave as Edesess.
Clark M. Blackman II, M.A., CFA, CPA/PFS, CFP, CIMA, AIF
President
Alpha Wealth Strategies, LLC
Houston, TX
Dear Editor,
Thank you for making Edesess' critique of the DFA research available. Its observations on scientism are spot-on and says many things that need to be said, both factual and by way of hypothesis.
The standards of rigor, confirmation and accountability in most financial research I’ve encountered are very poor compared with what I experienced in earning and (briefly) using my PhD in theoretical chemistry. It’s almost a cliché to describe the elaborate mathematical edifices that are built on crude foundations and simplifying assumptions, leading to results that are (1) couched in language and mathematical notation that impresses and/or intimidates otherwise intelligent professionals, who are thus discouraged from questioning the work and might not even know where to start asking questions about it. As a result the conclusions, assertions and claims are (2) inadequately tested against reality, or can’t be tested because there’s no way to set up a proper experiment. This, more than anything else, is where economics and finance fail in comparison to the physical sciences whose mathematical rigor and technical apparatus they ape. It’s not that the PhDs in those disciplines are necessarily stupid or dishonest; it’s the simple lack of appropriate testing of inappropriately broad knowledge claims. Thus it’s no surprise that little or none of this research has any usable predictive power.
There’s also the lack of attention to details such as data quality and programming errors, viz Reinhart and Rogoff, which is surely not the only such case lying in wait.
I hope and expect that DFA will produce a straightforward response to Edesess’ claims, which are themselves straightforward enough to confirm or refute, and that a proper debate can ensue.
Eric J. Bruskin, Ph.D, CFP®
Read more articles by Various
