>> This series is available on the Internet at   http:// ocf.berkeley.edu/~schwrtz

----------------------------------------------------------------------- ---------------------
Richard C. Atkinson, President                                   September 1, 2000
University of California

     RE:  my study of UC Investment policies, etc.

     While waiting for the General Counsel's office to provide the various documents I requested from you in my letter of July 27, 2000, I have been looking at some relevant documents found on the ucop [University of California Office of the President] websites.

     One key document is the March 16, 2000, "Investment Strategy Study", written (presumably) by Wilshire Associates for The Regents.   I am having trouble being sure that I understand some of the things contained here - mostly related to technical aspects of their computations and the interpretations that follow from the numbers they present.  Some things in this report appear to me to be of dubious validity; but I am not a professional expert in this area and rely upon my general level of knowledge.  It would be most helpful, therefore, if I could meet with someone knowledgeable about this subject and try to get my questions resolved.

     I am writing you now to request that you refer me to someone, perhaps a member of your staff, who could provide the intelligence I seek.  Thank you, in anticipation.
-------------------------------------------------------------------------------------------
     I am still waiting for a reply.
 

     To find out more about Wilshire Associates, I copied this from their web site:

"The Company:  Little did Wilshire know that when it got its start in the early 1970s, it would revolutionize the way investments were made. As the world began entering the computer age, Wilshire integrated computers with engineering and investment concepts, to provide some of the first data to money managers, ultimately shaping modern portfolio management theories.

"With the strong quantitative skills of its founder, Wilshire developed advanced portfolio analytical tools that helped investment managers, plan sponsors, and other institutional investors do their job of managing money more effectively. We complemented our technology with databases that were unparalleled in scope. Applying rigorous analysis and objectivity to our work, Wilshire became a leader in portfolio analytics, investment consulting, and asset management."

     Mathematics, often aided nowadays by computers, is a very powerful tool that we humans use in trying to understand the world, and often to manage it.  The world is very complex, however, and most mathematics that we use is very simple by comparison.  Scientists, engineers and many other technical professionals use mathematics applied to models - simplified models - rather than upon the real world.  Therefore, the conscientious scientist or engineer always takes a critical attitude toward the numbers that come out from the calculations.

     There are several steps in assessing the validity of one's results:

1. Step One - Acknowledge the extent of random uncertainty in the numerical output.
2. Step Two - Carry out a sensitivity analysis, i.e., vary the input data.
3. Step Three - Compare the results with those of competing theories/experiments.

     This is the approach I am following to assess the validity of Wilshire Associates' Investment Strategy Study prepared for the UC Regents.  It continues to surprise me how far I am able to go in this project, given that I have no prior expertise in their special field - analysis, consulting and management of financial investments.  The point I want to emphasize here is that I do not offer my opinion about professional choices or judgment on financial investments to oppose theirs;  I only look for logical or technical flaws in their analytical and quantitative work.  What I find, in summary, is that the Wilshire Report fails in every step of professional competence listed above.

     In Part 1, I showed how they failed Step One: to understand and to convey the huge degree of statistical uncertainty that must attach to their key calculations on investment returns projected many years into the future.  Now I proceed to the other steps.

     The primary numerical claim in the Wilshire report, as discussed in Part 1, was the comparison of Expected Return and Expected Risk calculated for two competing asset allocation policies, called Policy D (the former UC distribution of assets among stocks, bonds, etc.) and Policy C (a different distribution, favored by Wilshire because it purports to provide a higher return and a lower risk).  Wilshire's numbers are:

• Policy C:  Expected Return = 8.5%;  Expected Risk = 12.3%
• Policy D:  Expected Return = 8.3%;  Expected Risk = 12.4%

     Where do these numbers come from?  The relevant input data is provided in Wilshire's Exhibit 4 - Long Term Expected Return Assumptions:
 

Exhibit 4 - Long Term Expected Return Assumptions

	Annualized Expected Return
U.S. Equity	8.75%
Non-U.S. Equity	8.75
Fixed Income	6.25
Private Equity	11.75

These numbers, combined with the asset allocation numbers (previously given in Wilshire's Exhibit 5), lead to the quoted Expected Return for each of the portfolios called Policy C and Policy D.  When you hire Wilshire Associates to advise you on investment strategy, you are paying them for these numbers in Exhibit 4.  This is where their presumed expert data collection and market analysis comes in - to give an estimate of how the markets will behave in the years ahead.

     The questions I now ask are: How reliable are these input numbers?  If you change these input numbers a bit, how much will the outputs (8.5% for C and 8.3% for D) change?  This is what Sensitivity Analysis is all about; and I could find no indication at all of this concern in the Wilshire report!  This is a very serious flaw.

     Luckily, I have found a way to do this sensitivity analysis myself, at least approximately.  Among the pile of documents which I got from UC headquarters is a "Proposal to Provide Investment Consulting Services" to the UC Regents, prepared by Callan Associates, Inc., of San Francisco.  This firm, which apparently has extensive experience and credentials, was one of four, including Wilshire, that were invited to submit proposals to UC in early 1999.  Obviously, they did not get the job; but I found that their proposal contained extensive discussions and illustrations of the mathematical methods used in Asset Allocation Analysis.

     This resource provides a credible source of alternative input data: see Table 1.
 

Table 1.    Callan's 1999 Capital Market Projections

Asset Class	Expected Return
Broad Domestic Equity	9.40%
Non-US Equity	10.00
Domestic Fixed Income	5.60
Alternative Investments	12.30

     These numbers predicted by Callan differ somewhat from the numbers predicted by Wilshire (see the previous table).  I am happy to let those two industry leaders argue over which one has the better prediction; I do not take sides.  These numbers let me carry out the sensitivity analysis.  I use Callan's numbers to predict the outcomes for Policies C & D and compare with Wilshire's results:

• Expected Return =  8.26% for Policy C   (Wilshire gave 8.5%)
• Expected Return =  8.13% for Policy D   (Wilshire gave 8.3%).

     These Expected Returns, from Callan's input data, are "in the same ballpark" as Wilshire's numbers but the gap between Callan's and Wilshire's Expected Returns is not negligible - especially if you believe, as Wilshire does, that the difference between their results for C and for D (that 0.2% difference in Expected Return) is important enough to choose one policy over the other.  Perhaps the most interesting result of this calculation is that the difference between the Return for C and the Return for D - using the Callan input data - has now shrunk from 0.20% to 0.13%.

     The Callan resource lets me do more. I can calculate the Expected Risk for each policy using their input data.  The new results are:

• Expected Risk = 11.54% for Policy C    (Wilshire gave 12.3%)
• Expected Risk = 11.46% for Policy D    (Wilshire gave 12.4%).

     These new calculations further diminish one's confidence in Wilshire's work.
 

     Oops, we forgot something: Step Zero - check your results for simple arithmetic errors. Given the Expected Return data for each asset class (Wilshire's Exhibit 4) and the fraction of assets put into each asset class for the various policies considered (see Wilshire Exhibit 5, reproduced in Part 1) I can use a simple formula to check their calculation of the Expected Return numbers.  Here is what I find, for each of the four policlies considered (A.B,C,D) in Wilshire's Exhibit 5:

• Wilshire's calculated Returns:  (6.30% for A;  8.40% for B;  8.50% for C;  8.30% for D)
• Charlie's calculated Returns:    (6.25% for A;  7.90% for B;  8.03% for C;  7.94% for D).

     The calculation I just did can be challenged: How do you know what formula to use in doing this calculation?  Maybe my simple formula fails to include some sophisticated details that Wilshire Associates - the experienced professionals in this field - have incorporated into their computer codes.  Maybe so; and I invite Wilshire to correct me and instruct me.  But I do find confirmation in the documents from Callan Associates; I use my formulas on their displays of data and it checks out perfectly consistent with the Expected Return values that they show.

     Since it seems that Wilshire's numbers for Expected Returns have somehow gotten botched, I also wanted to check their number for Expected Risk.  But their report does not provide the necessary input data (although the Callan documents do provide such data). I wrote the following letter to Stephen Nesbitt, head of Wilshire's Consulting Division and the leader of the group that did the job for the UC Regents.

--------------------------------------------------------------------- ---------------------------
Dear Mr. Nesbitt;                                               September 19, 2000

     I have been studying the document "Investment Strategy Study," dated March 16, 2000, which you prepared for The Regents of University of California; and I have a number of questions which I am trying to clarify.

Firstly, on page 6 you say, "Wilshire annually publishes its long-term return and risk forecasts for a number of investment types."  I have looked at your website but was unable to find this information.  Could you please provide me with this data, for each of the last few years?

I will appreciate your assistance.
--------------------------------------------------------------------- ---------------------------
     He replied very promptly:

Dear Professor Schwartz,

The document you reference is provided only to Wilshire clients and therefore does not appear on our website.  Thank you for your interest.

 Regards, Steve Nesbitt
---------------------------------------------------------------------- --------------------------

      I then wrote to UC President Atkinson, informing him of the apparent arithmetic error in the Wilshire numbers, asking his advice, commenting on this abortive communication with Mr. Nesbitt and closing with the following:
---------------------------------------------------------------------- --------------------------
The central question in all of this is: How much confidence should we (current and future beneficiaries of the University's pension plan) place in the judgment of the officials who are nominally in charge of managing these monies. As you know, I have written to you before, with various questions relating to the recent changes in UC investment policies. When may I expect some intelligent reply?
----------------------------------------------------------------------- -------------------------

     That was over two weeks ago; and still, all I get from UC officials is silence.
 

     Wilshire Associates recommended (and the Regents approved)  the following changes in asset allocations for UC's pension and endowment funds (and if these seem like small changes, note that 10% of $50 Billion is a lot of money.)

Reduce investments in U.S. Equity by 10%  (from 63% to 53%);
Move 7% into Non-U.S. Equity;
Increase allocation to Private Equity by 3% (from 2% to 5%);
Leave Fixed Income allocation at 35%.

     Wilshire states (on page 1) that  the purpose of these changes is to increase return opportunity and to reduce risk by a broader diversification of the portfolio.  However, according to my foregoing study of the mathematics used in the Wilshire report:

• The increase in Return that they claim (0.2%) is perhaps only 0.13% or even 0.09%, and even at best this advantage is mostly swamped by the large statistical uncertainty;
• The decrease in Risk that they claim (0.1%) appears to be insignificant in magnitude, and it might even be negative, i.e., favoring the original UC policy.

     This leads us to the distressing conclusion that there is no rational basis for the changes in UC's asset allocation policy recommended in the Wilshire report - contradicting Wilshire's advertisement that they apply "rigorous analysis and objectivity to our work."
 

     There is no analysis at Step Three to be found in the Wilshire report, just as there was none at Step Two.  There are, however, other significant recommendations made in that document - without any analysis to justify them - which may be understood in terms of competing theories on investment strategy.

     The old fashioned way is described in the last (1999) issue of the UC Treasurer's Annual Report (page i):

"[Our] investment philosophy is to invest strategically with a global emphasis, taking a long-term perspective with regard to both asset allocation decisions and individual security selection. ... [I]nvesting in superior companies with above-average total return prospects provides superior investment results over the long term. However, the degree to which equity returns exceed those of fixed-income securities will vary given the economic environment at any particular time.  Importantly, this philosophy is backed by original in-depth research.  The Treasurer's Office uses a "top-down" approach to identify secular themes that can produce superior returns over a multi-year time horizon and a "bottom-up" approach to individual security selection."

     The newer way to guide investments, in which Wilshire Associates has been a leader, is called "index funds"; and they are very much in fashion.  In this approach one doesn't bother to study and evaluate individual companies in order to decide which stocks to buy ("stock-picking").  You just follow the average of any given market or portion of a market - using the S&P 500 index, or the Russell 2000 index, or the Russell 3000 index, or the Wilshire 5000 index, or... .  Rather than trying to look forward (which is difficult and full of doubt) you look backwards, gather lots of data into your computer, juggle it around, and make statistical predictions about the future.  Instead of telling the customer that your method is swamped with uncertainties, you say that Standard Deviation = Risk and claim that you can manage this risk to suit the customer's taste.

     It is no simple matter to decide what is the best way to invest - for an individual or for an institution.  The job of a professional consultant should be to present an objective analysis, with all necessary explanations and a fair comparison of alternatives, so that the client can make a fully informed and intelligent choice.

     In their "Investment Strategy Study" for the UC Regents, Wilshire Associates make a large number of recommendations beyond those of asset allocation, which I have discussed above.  Their most significant recommendations are that:

• 30% of U.S. equity investments be put into an external index fund;
• 85% of the non-U.S. equity investments be put into an external index fund; and
• specific benchmarks (Russell 3000, etc.) be used for those portions of the funds left to the management of the UC Treasurer.

     No justifications are given by Wilshire to support these recommendations (other than the general suggestion that they are supposed to reduce risk) and there is no analysis on these issues to be found in their report.  What I would expect of an objective expert is something like the following.

Look at the record of the UC Treasurer's investments over the past years.  The last Annual Report gives annualized total returns for UCRP over various time periods:
 

Table 2.   UCRP Investment Performance - annualized total returns in the past, from 1999

1-year	5-years	10-years	15-years	20-years
12.2%	21.3%	15.7%	17.0%	15.6%

The Wilshire report acknowledges the "exceedingly strong fiscal status" of UCRP (page 4) and adds (page 5) "it is worth noting that today's circumstances were created because a 15 year bull market in stocks and bonds, combined with the Treasurer's Office management, produced investment returns of 15% annually ..."  Such a cavalier remark (crediting "a 15 year bull market") is no substitute for an objective evaluation of the UC Treasurer's past investment performance.
 
     It may not be fair to make a direct comparison between Wilshire's predicted 8.5% return and the 15% return achieved in past years by the UC Treasurer. But I would like to see some intelligent analysis here.  For example, go back in time and use Wilshire's formulas for analysis, prediction and reliance on index funds over those same years; and compare the performance of the two investment philosophies.

     Another instructive comparison would be to look at some large pension fund that has followed Wilshire Associates' advice on their investments.  CalPERS - California's Public Employees Retirement System - is one possibility.  I have looked at the available data and have found the following: UCRP has shown substantially higher returns than PERS and PERS has shown a substantially lower standard deviation of returns.  How to make this comparison more precise is a difficult task, probably best assigned to experts in this field.

     Index funds are attractive for several reasons. They are cheap, compared to paying for the services of a staff of experienced stock-pickers; they are an easy way to go if you have not been fully in the market before (CalPERS, among other pension funds, had such a history); they offer the promise of, "you will never fall much below the average of all other investors"; they may seem a safe choice for trustees who are afraid of being sued if their investment returns sometimes lag behind. But are index funds the best advice for all institutional investors?

     That Wilshire fails the test of adequate objectivity and thoroughness in their study for The Regents is understandable, perhaps, since they are in business for themselves; they have their biases and they have a set of products to sell.  What I am most upset about is that the regents, apparently, failed to notice that they were being cheated in terms of the quality of advice that they bought.
 

     It was a prudent idea that The Regents should undertake a professional review of their investment policies and practices; but it now appears that what they got from Wilshire Associates was a truly bad piece of work - full of flaws in math, logic and objectivity.  How could such a thing happen?

     The inordinate secrecy surrounding this whole history is probably a part of the answer,  and that will be the topic of my next installment - due out in about a week.  After that, we will consider other theories.
 

Readers:  Your feedback is invited.