Home | Contact Us | About Us
 
 
 

Benchmarking and Survey Use

  1. Problems with Surveys and Benchmarking
  2. Types of Bias in Surveys
  3. How to Correct Survey Bias
  4. Key Peer Group Questions that Compensation Committees Should Be Asking
  5. A Practical Tool for Boards to Implement: Internal Pay Equity
  6. Practice Pointers
  7. Firm Memos
  8. Media Articles
  9. Benchmarking Disclosures Practice Area
  1. Problems with Surveys and Benchmarking

    The common practice when reviewing a CEO’s compensation is to refer to executive compensation surveys. Unfortunately, as compensation consultants now readily acknowledge, surveys are not just flawed and biased, they can be massaged and cherry picked to come up with pre-ordained results that a CEO has either explicitly requested or that the consultant has derived to ingratiate him/herself with the client. The consultant might deliver a report to the compensation committee with an incongruent pair of "peer" companies: one set for purposes of executive compensation and another for corporate performance.

    In these surveys, it is easy to inflate numbers by including extraordinary or one-time payments and grants, or by relying on surveys conducted for other companies that included such items. This reliance might even be unintentional. On top of all this, as more companies adopted the practice of "benchmarking"—taking the position that their above-average CEO should be paid in the top 25th percentile of all CEOs—a slippery slope resulted. Compounded each year, benchmarking quickly became an out-of-control ratcheting-up phenomenon that resulted in exponential growth in compensation, as this year’s top 25th percentile becomes next year’s 50th, requiring another increase to get back into the top 25th, and so on.

    Another problem is that these surveys often do not take into account that some companies may put greater emphasis on cash compensation than stock options, while other companies may place greater emphasis on long-term incentives. By cherry picking from each category, a CEO can have his cake and eat it too.

    All of this can help largely explain how at some companies CEO salary and bonus and stock grants and other components of CEO compensation have gotten so far out of line—not just from the median workforce salaries generally, but also from other senior and mid-level compensation levels at the CEO’s own company.

    It still is troubling that many compensation consultants, who acknowledge how surveys have brought CEO compensation into the stratosphere, still use those inflated survey numbers—that have years of compounded inflated numbers built into them—to help clients determine what is the appropriate level of salary, bonus and other compensation. This continues to perpetuate all the mistakes of the past. Going to a 50th percentile now—which may have been last year’s 25th percentile and which would have been off the charts just a few years earlier—is not reform.

    Compensation committees cannot afford to ignore the admonitions about surveys in recent governance reports: "The committee should resist an over-reliance on surveys and other statistical analyses in determining compensation levels." (Business Roundtable) "So-called market-based compensation…is a root cause of the current problem." (NACD Blue Ribbon Commission) "Percentile benchmarking should not ever be practiced except to provide broad market reference points." (Breeden Report).

    And here is what the Council of Institutional Investors included in its recently updated policy on executive compensation: "Benchmarking at median or higher levels is a primary contributor to escalating executive compensation.  Although benchmarking can be a constructive tool for formulating executive compensation packages, it should not be relied on exclusively.  If benchmarking is used, compensation committees should commit to annual disclosure of the companies in peer groups used for benchmarking and/or other comparisons.  If the peer group used for compensation purposes is different from that used to compare overall performance, such as the five-year stock return graph required in the annual proxy materials, the compensation committee should describe the differences between the groups and the rationale for choosing between them.  In addition to disclosing names of companies used for benchmarking and comparisons, the compensation committee should disclose targets for each compensation element relative to the peer/benchmarking group and year-to-year changes in companies composing peer/benchmark groups."  

    For more information, see this practice pointer by Fred Cook regarding "The Problem with Surveys."  

  2. Types of Bias in Surveys

    Fred Cook, Chair of Frederic W. Cook & Co., illustrates how surveys can be biased in a dozen ways, such as:

    1. User Bias. Companies that sponsor surveys often do so with an implicit (albeit unstated) objective: to show the company as paying either competitively or somewhat below the market so as to justify positive corrective action. Those who are contracted to conduct the survey and interpret its results on the company's behalf subconsciously take on these same objectives. This user bias does not by itself cause salary inflation, but it does create the climate in which upward bias occurs and is accepted.
       
    2. Sample Bias. Companies like to compare themselves against well-regarded, high-paying, and high-performing companies. Those firms that participate in surveys drawing data from a large number of other organizations often have the ability, through computer technology, to create a subset of participants with whom they wish to compare themselves. This typically results in a higher competitive pay line than the general survey, thereby showing the user company in a more favorable (i.e., less competitive) light in terms of pay rates.

      Alternatively, the survey may be custom designed, in which case the sponsoring company will tend to select a high-paying, high-performing group of companies against which to compare itself. The survey designers will exclude low-paying (and less well-regarded) companies. It is perfectly natural to want to compare oneself against the star performers in an industry; this reflects the company's goal to become more like the leaders.

      This would be fine if surveys compare relative performance as well as relative pay. Most do not. Actually, paying less than others may be quite appropriate if the company's performance is low relative to the survey sample. Likewise, high pay may be justified by high performance.

      What happens, however, is that the low-paying company will downplay its relative performance and use the survey to justify pay increases, while the high-paying company justifiably maintains its high position based on its relative performance. The net effect, over time, is upward movement in competitive pay levels.  

    3. Survey Selectivity. Most companies have access to several surveys covering the same population. In cases where different surveys show the company in different competitive positions (some more favorable than others), compensation professionals tend to disregard, challenge, or downplay those surveys that do not show the company in the desired competitive position.
       

    4. Scope Bias. Survey professionals accept the idea that the relative size of the organization should influence its pay, particularly at upper management levels. Yet "size" can be measured in any number of ways: revenues, equity, assets, market capitalization, net income, etc. Sponsoring organizations tend to select size variables that let them compare themselves favorably to the survey companies. Thus, if revenues are high but market capitalization is low, they will select revenues as the variable.

      If every company in the survey has the ability to select the size or performance variable that favors itself, then it is technically feasible for all companies to show themselves as paying below the market. When this happens, what is the real market?
       

    5. Compensation Selectivity. A total compensation package is composed of many elements, whereas most surveys tend to focus on a few, such as salary and bonuses. It is natural for companies that have a competitive total compensation package to be light on some pay elements and heavy on others. Companies with very generous benefit packages, supplemental executive retirement plans (SERPs), or large equity grants tend to disregard or downplay those pay elements in conducting surveys. If a company surveys only those areas where it is light (cash compensation, for example), it should not interpret or use those findings in isolation. If it does, it will be raising its total compensation levels above the market.
       

    6. Benchmark Bias. In submitting survey data and interpreting the results, companies must match their positions against positions in the survey. In doing so, they tend to match their positions against those that have higher responsibilities and hence higher compensation. One company's positions may not have the full scope of responsibilities typical of that position in other companies. But the interpreter will tend to disregard this when comparing pay levels. Conversely, if a company's position shows up as highly paid relative to the survey, the interpreter will explain that away by saying that the position has more responsibilities, a different reporting relationship, or other factors that justify higher pay. Since very few benchmark positions are perfect matches, it may be quite appropriate to adjust survey data for differences in position responsibility. But, if this is done to explain and justify a higher-paid position, then equal efforts should be made to explain and justify those that are paid below the market.
       

    7. Statistical Bias. Survey professionals use a variety of statistical techniques to interpret the survey results and determine where a particular company stands against the survey population. Single and multiple regression techniques are often used to supplement straight statistical averages, medians, and quartiles. A sponsor motivated by a desire to have the company appear in a good light can select, from among the statistical techniques available, those that will support this purpose. This problem will grow worse as sponsoring companies become more sophisticated in using option-pricing models to compare the values of equity-based, long-term incentive grants. By manipulating the variables and assumptions in ways favorable to the company, the interpreter may be able to make an above-competitive grant practice appear less generous and the grant practices of others appear more generous.
       
    8. Converting Actual Bonuses into Target Bonuses. Many companies use surveys to assess and reset their target bonus practices as a percent of salary. However, most surveys collect data on actual bonuses, not target bonuses. In times of strong economic expansion and performance, it is common for actual bonuses to be above the target level. Companies that rely on high actual bonuses reported in the survey to justify raising their target bonuses lead the way in escalating total pay levels over time.
       

    9. Results Bias. Having a strategy of paying competitively based on performance means a company is obligated to offer a competitive pay opportunity, not guarantee a competitive result. For example, a position may have a target bonus payout of 30% of salary, set so that the base salary plus the target bonus would result in competitive total annual pay when performance standards are met. However, there is no guarantee that the target bonus will be paid or that total pay will be competitive.

      Companies may overlook this obvious truism when they have low or no payouts under annual bonus or long-term incentive plans in comparison with others that have made higher payouts, perhaps because of better performance. Low payouts relative to the competition may be perfectly justifiable and not a cause for corrective action. Surveys that include payouts from variable pay plans should be interpreted with reference to relative performance data. In the absence of such data, surveys should focus on competitive opportunities, not results.
       

    10. Misinterpretation and Normalization of Special Equity Grants. Increasingly, companies make special equity incentive grants to their key people. For example, instead of making smaller stock option grants every year, a company may accelerate option grants into a single, jumbo grant. Or it may give executives a special, one-time equity grant to signal a significant event, such as a new strategy or a merger. Finally, when hiring a senior executive from outside, the employer may use special equity grants to induce the executive to take the job or to make up for benefits lost in the job change.

      These special grants more often than not are included in the survey collection process. Without knowing the reasons

      for the grants, the survey interpreter may assume that they are part of a normal granting practice. Even knowing a grant is a one-time special event, the interpreter may normalize it by assuming it will be repeated every three or five years. The net effect is to raise the survey averages to new levels and to create a situation where special grants by one company become built into normalized practices by the survey population.
       

    11. Value Bias. Some employees are true "value builders" and rewarded as such. Others are "value maintainers," and some are, in fact, "value destroyers" because they are paid more than they are worth. Surveys, however, do not distinguish among these categories. When a survey includes pay data from across the value spectrum, survey averages are pulled up by the justifiably high pay of the value creators. These averages, however, are used to rationalize higher pay levels for value maintainers and destroyers as well in order to be "competitive."
       

    12. No-Decline Bias. Compensation professionals have grown to expect that survey data will show pay rates rising every year. A conceptual framework does not even exist for handling marketplace declines in pay. However, with all the layoffs, downsizings, and early retirements of higher-paid workers, one would think surveys would show declines in market pay rates.

      Even if a survey shows a decline in pay rates, the user will tend either to disregard or reinterpret the data to reflect an increase. Also, survey purveyors know that steady increases in competitive pay levels are good for business, and declines are not. This, combined with user bias, may explain why surveys continue to show increasing pay levels.

  3.  How to Correct Survey Bias

    In addition to some of the practice pointers below, Fred Cook, Chair of Frederic W. Cook & Co., provides some tips on how to correct survey bias. Fred notes: "If there were no compensation surveys, there probably would be wider disparities among company pay practices than exist today. These disparities would tend to be more economically justifiable, both on the high and the low side. Natural economic forces would operate to create a market in which people would be paid according to their relative worth.

    Surveys tend to narrow the disparities, particularly on the low side, because low-paying companies use them to justify pay increases that otherwise would not be justified. They tend to have less effect on the high side because high-performing companies can justify high pay and will do so rather than using the survey to depress their pay levels. By pulling up the bottom, surveys raise the average for everyone, thereby causing an upward escalation in overall pay levels.

    How can you avoid the negative effect of bias? I propose three solutions. First, acknowledge that upward bias exists in surveys and act to correct it where possible. Second, downplay the importance of surveys use them as a check on where you are, not as a rationale for doing something different. And third, use relative size and performance comparisons in a justifiable and consistent manner in interpreting any survey results."
     

  4. Key Peer Group Questions that Compensation Committees Should Be Asking

    • Value of, and need for, periodic review and reassessment of the composition of the peer group(s) being used, taking into account, among other things:

      - industry, size (based on market cap, revenues, etc.), business orientation, organizational style and other similarities and differences between company and various potential comparators;
      - relative financial and non-financial performance of the company (and its senior executives) vs. comparators (and their senior executives) currently and over last 2-3 years; and
      - any major changes in, and developments regarding, the Company’s business focus and its longer-term strategic goals. Is the Committee looking at a full set of peer group data points? For example. . .
       
    • Does the peer group data on annual bonuses look at target award opportunities as well as actual payouts?
       
    • Has the data on comparator company bonuses, option grants, and restricted stock and RSU awards been weighted or otherwise adjusted to take into account relative annual and longer-term financial and strategic performance and any special circumstances at those companies?
       
    • Have the recent Form 10-Q, Form 8-K and Form 4 filings for comparators being fully reviewed to identify what current-year data is available regarding new and amended employment agreements, new equity-based grants, stock option exercises, restricted stock vesting, stock sales, etc., in addition to the data already shown in the comparator companies’ most recent proxy statements?
       
    • Note impact of new Form 8K disclosure requirements scheduled to take effect in August 2004. 5. How good an understanding does the Committee have regarding the primary interactions between annual salary, bonus and other compensation decisions and the affected executives’ severance, deferred compensation and retirement benefit rights?
       
    • Has the Committee been provided with sufficient information regarding the impact of any recent or proposed large cash (and, if applicable, non-cash) compensation actions (e.g., large salary increase or large bonus) on the executive’s severance rights and supplemental retirement benefits?
       
  5. A Practical Tool for Boards to Implement: Internal Pay Equity
    1. Practice Pointers
    2. Companies that Use Internal Pay Equity
  6. Practice Pointers
  7. Firm Memos
  8. Media Articles
  9. Benchmarking Disclosures Practice Area

 
 

For more information, contact info@compensationstandards.com or call 925.685.5111.
© 2006, Executive Press, Inc.
Terms & Conditions, Disclaimer, Register/Subscribe, Contact Us