by Charles Schwartz, Professor Emeritus, University of California, Berkeley
firstname.lastname@example.org October 2, 2000
>> This series is available on the Internet at http://ocf.berkeley.edu/~schwrtz
At the July meeting of the Board of Regents there were a few speakers, during the Public Comment Period, who asked pointed questions and voiced sharp complaints about the regents' handling of the UCRP (Retirement Plan) investments. They cited a front page newspaper story which suggested political payoffs in the awarding of a UC consulting contract and a backstage move to oust Patricia Small, the University's Treasurer who had an outstanding record of success in managing the investments. It is rare that regents respond to these public critics; but on this occasion, Regents S. Sue Johnson, Chairman of the Board, and Judith L. Hopkinson, Chair of the Committee on Investments, issued a formal Joint Statement (July 19, 2000):
My first reaction, upon reading this Statement, was that the regents made no mention of the Treasurer and that looked like an ominous sign. Sure enough, within a month UC announced that Patricia Small was resigning. A generous severance package was given to this career public servant - 28 years of service to the University, 19 years as the Regents' Treasurer and Associate Treasurer - and she has been silent since then.
"We feel compelled to respond to this morning's public comments, stemming from recent misleading news reports on UC investment policies.
"First, we want to assure employees that their pension funds are safe and that they will continue to be safe. Safety of the pension funds is the uppermost concern of the regents, and it will continue to be. Safety was a major impetus in deciding to undertake a comprehensive review of investment activities.
"Here is some background on the issue.
... [The full text may be found at http://www.ucop.edu /bencom/news/regentnewitem.html ] ...
"The Wilshire firm [Wilshire Associates Inc.] was ultimately selected as the most qualified. The consultant has functioned solely as reviewers, advisers and monitors, and not as investment managers. Wilshire is a firm of international repute in the investment field. Its clients include a number of large international businesses, as well as other public funds. ...
"Throughout the process, the consultants' review and recommendations, and asset allocation plan recommendation, were reviewed on a continuous basis by the Commission, the Investment Committee, the Treasurer's Office, and the full Board of Regents. The plan was unanimously approved by the Regents in a public meeting in March 2000."
My second reaction, reading the last sentence
in the regents' Statement, was this bit of cynicism: After more than a
year of meeting in secret to work it all out, the regents held one brief
open session, in March, to formally ratify the plan, and they act as if
that should satisfy the public interest. When I checked into this
later, however, I discovered that no such "public meeting in March
2000" ever took place.
On July 27, I wrote the following letter to UC's president, Richard Atkinson.
I am trying to make sense of the recent public outcry concerning the University's investment policies and performance, following the article published in the San Francisco Examiner on July 16. I have a copy of the July 19 Joint Statement issued by Regents S. Sue Johnson and Judith L. Hopkinson, but this seems to raise more questions than it answers.
Please send me copies of all reports, letters of opinion and advice, etc., fromI will return later to tell more about this quest for official documents and the continuing cloak of secrecy over this whole matter. Now, let's get to some substance.
1) the Commission to Review the Office of the Treasurer;
2) Wilshire Associates;
3) Treasurer Patricia Small;
and any other documents that may help me obtain a fully informed understanding of these matters.
Digging Into the Wilshire Report
By mid August, a friend had pointed my browser to www.ucop.edu/bencom /news/revisedasset.html, where one can download a 30-page document: "Investment Strategy Study", dated March 16, 2000, addressed to The Regents of the University of California. This is the authoritative and (as far as we in the public are allowed to know) complete analysis and recommendations prepared by Wilshire Associates. I will limit myself to Section 2 of this report, pages 3-11, that concerns the UC Retirement Plan (UCRP), with $38 Billion in assets as of 6/30/99.
This is all about numbers. There are six tables of data (Exhibits 4 to 9) in this section of the report and plenty more numbers throughout the text. Here is Wilshire's description of their work (on page 6):
"Forecasting the long-range performance - periods exceeding 20 years - of different types of investments is a critical function of investment consulting firms like Wilshire Associates. Short-term market predictions can vary significantly and are almost always wrong. But, fortunately, there is a much better chance of forecasting the long-term return on financial markets. A number of academic studies examining the 200 year history of the U.S. financial markets have shown that equity and bond returns are more predictable over 20 year periods than any individual year. For example, while the one-year return on the U.S. stock market has ranged from -43% to +54%, the 20-year range in return is a much tighter +2% to +17%."
I have never studied Econometrics, which is, I think, the name of the academic specialty they are talking about in this paragraph. I do know a fair bit of mathematics, however, and I have the research scientist's habit of inquiry - that is, the nerve to be skeptical, to ask questions and to poke around for possible answers.
Let me first give a couple of
for any readers who may need it.
Return on Investment: If I invest $100. in some stock (and leave any dividends to be reinvested) and one year later this investment has a market value of $110., then I have realized a gain of $10., which is 10% of my original investment. We say that the annual return on this investment was 10%.
Compound Asset Growth: If I leave this investment untouched for 5 years and the rate of return continues to be 10% per year, then my total assets at the end of this time will be
$100. (1+10%)(1+10%)(1+10%)(1+10%)(1+10%) = $100. (1.10)5 = $161.05.
Here is the nub of the Wilshire report, their
5 - UCRP Asset Allocation Policies.
The top part of Exhibit 5 shows how the total
investment is to be divided among four different asset classes - U.S.
(domestic stocks), Non-U.S. Equity (foreign stocks), Fixed Income (bonds)
and Private Equity (venture capital partnerships, etc.) - for each of the
different policies considered, A, B, C and D (the
former Regents' policy.)
The bottom part of Exhibit 5 shows the results of Wilshire's calculations: the Expected Return and the Expected Risk for each of the four policies considered. Their conclusion: "Overall, Policy C is superior to the current policy because it both increases future return and reduces risk." How were those numbers for Expected Return and Expected Risk calculated? The report does not say; but in Part 2 of this series I will talk about this important question. What I want to do here is this: Accept the numbers as given by Wilshire in Exhibit 5 and ask, What does this mean?
A first (and healthy) reaction is to say, That doesn't look like much of a difference between the Expected Returns, comparing policies C and D: 8.5% - 8.3% = 0.2%. How can we judge if that 0.2% difference really matters? Here is where the calculation of Compound Asset Growth comes in, especially when we look at a many-year projection. Take the $38 Billion current assets of UCRP and multiply it by the factor (1.085) 30 times over and see what you get 30 years from now; do the same using the factor (1.083) and compare results. Wilshire does a part of this calculation and shows the result in their Exhibit 9, the relevant parts of which I reproduce below.
Exhibit 9 - Projected Assets, Liabilities, etc., under Recommended
| Assets at
Actually, they do a more realistic calculation
for the growth of assets in UCRP, replacing the 8.5% return on investments
by a 7.1% annual rate to allow for the fact that the fund is continuously
making benefit payments (to me, among the thousands of UC retirees)
that amount to 1.4% per year. Here, then, is my comparison
38.1 (1.071)30 = 298 for Policy C
38.1 (1.069)30 = 282 for Policy D.
The difference is 16 Billion dollars - not exactly chicken feed. (Wilshire's Exhibit 9 gives the result 296 for Policy C after 30 years, which is close enough to my number 298; strangely, they give no comparable numbers for Policy D.) If I said that this $16B difference amounted to only 5% of the total assets and maybe not worth the fuss of changing policies, I'd get laughed at as a silly absent minded professor.
But you don't need to be a professor to ask the next question: How reliable is the forecast that gave us those numbers in the first place? Nobody knows the future (I am talking about rational, not religious, things); any predictions must always be shrouded in uncertainty. How, indeed, did Wilshire arrive at those numbers (8.5% and 8.3% Expected Returns) and what uncertainty should we assign to them?
This Wilshire report does not describe what mathematics they use and no references are given. However, there are some clues one can pick up in the text; the most useful one is a single sentence (in parentheses) on page 7: "Risk is measured as the annualized standard deviation of return, a measure of return volatility."
Aha! Any scientist or engineer or anyone else who is at all educated in the subject of Statistics will recognize that phrase "Standard Deviation". This is how we describe uncertainty in using the theory of probability. So look back at the bottom row of data in Exhibit 5; take those "Expected Risk" numbers and interpret them as the Standard Deviation (uncertainty) to be assigned to each value of Expected Return. With this new information, I have a knee-jerk reaction: If there is a 12% standard deviation for each of the results for policies C and D, and the difference between them (8.5% - 8.3% = 0.2%) is so much smaller than this standard deviation, then it seems absurd to claim that there is any significant difference between these two policies. But this, too, is a mistake, which I will leave to the interested reader to figure out.
In order to get a clear view of what is going on here, we need a bit more mathematics. I'll redo the calculation of compound investment growth introducing some random variables to allow for the unpredictable market fluctuations that Wilshire talked about in the paragraph I quoted earlier.
UCRP Assets after 30 years = 38.1 (1+r+x1) (1+r+x2) (1+r+x3) ... (1+r+x30)
where r is the net rate of increase which we used before (0.071 or 0.069 for policies C and D respectively) and the x's are a set of independent random variables which have the specified standard deviation (0.123 or 0.124 for C and D respectively). For this calculation I have to use my computer, which has a built-in random number generator; and I do the calculation many times over. Each time the computer gives me a different result for the total assets after 30 years; and this simulates the many possible futures for how the financial markets might behave over the next 30 years. I average this large amount of data and present it in a scatter-graph (probability distribution) shown below.
Projection of Wilshire Associates'
Policy "C" for UCRP - 30 Years in the Future
* * *
* * * Each Point * is a Likely Future Outcome
* * *
* * * *
* * * *
* * * * * This is a Probability Distribution with:
* * * * * Average = 298
* * * * * * * Standard Deviation = 207
* * * * * * *
* * * * * * * *
* * * * * * * * *
* * * * * * * * * *
* * * * * * * * * * * *
* * * * * * * * * * * * * * * * *
0 100 200 | 300 400 500 600 700 800 900 Assets in $ Billions
marks the Accrued Liability of $250 Billion in 2029
The average of this distribution (298) is the same as the result from our earlier calculation. (This distribution is quite asymmetrical; there is a long tail which isn't shown.) The basic thing to understand about probability theory is this: Every single point (*) shown in this graph is as likely to happen in our real (but unknown) future as any other point. There are more points clustered about the central region; and so we see that an outcome in the range between 100 and 400 is more likely to occur than a result either below or above this range. The average value, 298, is no more likely to be actually realized than is any other value in the range from 100 to 300.
To my mind, the right question to ask is this: What is the probability that UCRP, if it follows Wilshire's recommended policy C, will be able to meet its Accrued Liabilities (that means pay all future benefits it will have promised) 30 years in the future? The answer, from my calculations, is: There is a 49% probability that UCRP's Assets will fall above the required $250 Billion mark; or put another way, there is a 51% probability that it will be necessary to make further payments into the UCRP fund in order to meet its obligations in 30 years. I find it inexcusable that Wilshire Associates would not have done this calculation or that they would not include this essential piece of information in their Study for The Regents. (For an example of how this is usually done, see www.ucop.edu/regents/regmeet/sep00/507.pdf page 5.) As it stands, Wilshire's Exhibit 9 presents completely misleading "forecasts" for the unwary reader.
Naturally, I have done the calculation for the current policy D in the same way: the scatter-graph looks almost identical to the one above; the numbers are: Average = 282; Standard Deviation = 198; probability that the Assets will be above 250 = 45%.
The chief conclusion is that there is a HUGE range of uncertainty in trying to predict the outcome of Wilshire's recommended policy C (or of policy D) 30 years into the future. The standard deviation seen in this probability distribution (207) is 69% of the predicted average value of the Assets (298). Compare this with the one-year prediction, where the standard deviation was only about 11% of the expected assets (.123/1.071); and we see that the accumulated uncertainty gets larger as we try to make predictions farther into the future. This sounds commonsensical; and it is mathematically correct.
But this brings us to a serious problem. The conclusion I just stated seems to be in direct contradiction to the authoritative statement made by Wilshire Associates in their opening paragraph about Investment Assumptions, quoted earlier:
"A number of academic studies examining the 200 year history of the U.S. financial markets have shown that equity and bond returns are more predictable over 20 year periods than any individual year." (emphasis added)I can guess how a certain mathematical trick, which may be used by experts in this field, could resolve this apparent contradiction (if you know what the meaning of "are" is). But, here, again, I find that the Wilshire report appears to be very misleading to an average reader.
Conclusions to Part 1
The advantage of choosing Policy C (with its 8.5% expected return) over Policy D (with its 8.3% expected return) is placed into considerable doubt. The most relevant way to present the comparison - accepting for now everything else about Wilshire's model and input data - is to ask this question:
What is the probability that UCRP will be able to meet its Accrued Liabilities in future years without the necessity of additional payments into the retirement fund?
Here are the results I calculate, using Wilshire's numbers.
|Year||Policy C||Policy D|
Did the regents ever get shown data like this? If the "Investment Strategy Study" by Wilshire Associates is the whole story (it is the only available one), then the answer is, No. When you, reader, look at this data now, do you feel that this makes a significant argument in favor of changing to Policy C from Policy D?
When I showed you a $16,000,000,000. difference between the 30-year projections using only the "Expected Returns" - without acknowledging the uncertainty inherent in those numbers - you were quickly drawn to see the advantage of C over D. Now, with a better appreciation of probabilistics in the making of forecasts, this appears to be a much more dubious choice.
What about comparing the Expected Risk numbers (12.3% for Policy C and 12.4% for Policy D)? Wilshire cites this difference of 0.1% as part of the reason for choosing C over D. Is this really a significant difference? I have not been able to find any reason to say so. In the calculations that give the numbers shown in Table 1 above, almost all of the difference between results for C and D comes from the Returns and not from the Risk inputs.
The next installment - Part 2, which I hope to release in about a week - will continue with critiques of more bad math and missing logic in the Wilshire report.
I want to thank Professors D. Brillinger and S.L. Messinger for some very helpful discussions.
Readers: Your feedback is invited.