Home > Industry Analysis > Research & Analysis > FDIC Banking Review
FDIC Banking Review
* The authors are all on the staff of the Federal Deposit Insurance Corporation (FDIC): Charles Collier and Sean Forbush are in the Division of Supervision and Consumer Protection (Collier as chief of the Information Management Section, Forbush as a senior financial analyst), and Daniel Nuxoll and John O'Keefe are in the Division of Insurance and Research (Nuxoll as a senior economist, O'Keefe as chief of the Financial Risk Measurement Section). The multiyear development of SCOR involved many of the authors' colleagues, a number of whom have contributed to this article.
2 CAMELS is an acronym for Capital, Asset quality, Management, Earnings, Liquidity, and market Sensitivity. Examiners have rated sensitivity only since 1998. Strictly speaking, examination ratings before that year are CAMEL ratings, but we ignore this distinction and use “CAMELS” throughout, except in Appendix 1.
4 Actually, the relevant Type I and Type II errors are not those discussed in the text. For the FDIC’s purposes, the critical question is whether the regional office is aware that a bank might present a supervisory concern, but because that awareness cannot be established retrospectively, all backtesting uses examination ratings. Because case managers have information besides examination ratings, the regional office is often aware of potential downgrades before they occur, but the backtests assume that the regional office is not aware of problems until an examination has begun. Thus, the backtests overstate the model’s ability to identify banks that present a concern.
5 This expert system was designed by a group of experienced examiners, who decided which ratios were the best precursors of future problems. Updating the system would involve convoking another group of experienced examiners to deliberate about the model. For more information on CAEL, see FDIC (1997), 507ff.
6 Actually, the mapping is not quite that simple because CAEL was built with a bias toward downgrading institutions. Without any bias, an institution receiving a CAEL rating of 2.5 would be as likely to receive a 2 at the next examination as a 3. The bias, however, means that an institution with a rating of 2.5 will in fact be more likely to receive a 2 than a 3. Because of the bias, CAEL identifies more banks as possible problems, thus increasing the Type II errors while decreasing the Type I errors.
7 The SCOR model is very similar to the SEER rating model, originally called FIMS, developed by the Federal Reserve System. Both SEER and SCOR draw on a long history of models of bank failure and distress. DemirgüH-Kunt (1989) reviews pre-FIMS developments, and Gilbert, Meyer, and Vaughn (1999) explain the rationale behind such models. For a discussion of the SEER system, see Cole, Cornyn, and Gunther (1995). SEER and SCOR differ in one important respect: SCOR does not use past CAMELS ratings to forecast future ratings. For a discussion of the issue of using past ratings to forecast future ratings, see Appendix 1.
8 With regard to underwriting, at the conclusion of each exam FDIC examiners evaluate separately the quality of the institution’s underwriting practices. The FDIC is currently researching whether these ratings can be used to forecast future examination ratings.
9 In two different experiments, credit card banks and large banks were eliminated from the model. In both cases, the model’s forecasting power was worse. Homogeneity is the enemy of statistical models.
10 The FDIC does not collect data on left-handed tellers, but it does collect a vast amount of data on banks. It would be truly remarkable if some of these data were not correlated with CAMELS ratings. Statisticians are well aware that statistics can demonstrate correlation but not causation.
12 An earlier specification of SCOR used 13 variables. Dividends were included, and net income was used instead of income before taxes. Pretax income is now used because of the increasing number of banks that are sub-chapter S corporations and do not pay corporate income tax, and dividends were dropped because supervisors commonly restrict dividends at troubled institutions. Thus, dividends are necessarily low at an institution after supervisors have identified it as troubled (it is important to remember, however, that low dividends do not necessarily signal that a sound institution is having trouble). Both changes—dropping dividends, and replacing pretax income with net income—demonstrate the importance of using informed judgment when selecting variables. The current version of SCOR is at least as accurate as the older version.
13 However, this method might introduce another distortion. Suppose major portions of the disappearing bank (for example, branches or a credit card portfolio) were sold within 12 months of the merger. SCOR’s method of adjusting for mergers would include income from operations that were not part of the merged entity. Although examples of this sort of distortion can be found, they are relatively uncommon.
14 If the bank had a rating of 3, the probability of a downgrade would equal 5.3 percent (4.9% + 0.4%). If it had a rating of 4, the probability of a downgrade would equal 0.4 percent. By definition, the downgrade probability for 5-rated banks is zero.
15 In practice, most banks have a high probability of receiving one or two specific ratings and almost no probability of receiving the other three ratings. The example shown in table 3 is typical. In these cases, we can closely approximate the downgrade probability by dropping the integer part of the SCOR rating. For the hypothetical bank in table 3, the approximate probability of being downgraded to a 3 is 42 percent. This approximation is exact if three of the five probabilities are zero.
16 When SCOR was adopted in 1999, a 30 percent downgrade probability was used to flag banks for review. By 2001, when the weakening economy undoubtedly affected the financials at banks and caused more poor SCOR ratings and more reviews, that flag was resulting in too many reviews. Accordingly, the higher probability was adopted.
17 Another reason for focusing on the four- to six-month horizon is that after an examination, the Call Reports filed immediately before the examination almost always (but not absolutely always) are revised. Revisions to the Call Report that immediately precede the exam will bias the backtest because the SCOR model will have access to the corrected data instead of the data that were actually available to supervisors before the examination. This bias is minimized by the use of forecasts based on the Call Report filed four to six months before the examination—a Call Report less likely to be revised.
18 Banks that have problems are more likely to be examined, and the reported results include only the first examination after the Call Report that provided the data for the SCOR rating. Consequently, the results for a 16- to 18-month horizon include only the strongest banks, and only 2.6 percent of these were downgraded. In contrast, 18.8 percent of the banks identified by SCOR are downgraded 16 to 18 months later.
19 The reviews done by case managers almost inevitably indicate that banks flagged by SCOR have noticeable weaknesses, even though the weaknesses might not warrant an examination or closer supervision.
21 The FDIC’s Division of Supervision and Consumer Protection rightly insists that bank examiners are not a substitute for adequate internal and external auditing. However, it was examinations that uncovered the fraud at both BestBank and First National Bank of Keystone. See Berger and Davies (1998) for a discussion of the auditing function of examinations.
25 Some other aspects of the forecasted management rating are worth noticing. First, the accuracy of the forecasted management rating deteriorates less over time, so at long horizons it is among the more accurate of the component forecasts. Second, the forecasted management rating does help signal downgrades in the composite rating.
28 The median-2 bank is used as the basis of comparison instead of the “mean 2” because outliers tend to increase mean financial ratios. A simple example illustrates the point. If 99 banks have capital-asset ratios of 9 percent and 1 has a capital-asset ratio of 90 percent, the mean capital-asset ratio is 9.81 percent. The median is 9 percent, which is more representative.
29 There are other possible reasons that a stepwise procedure might eliminate a variable. For example, if two of the explanatory variables were highly correlated, the stepwise procedure would choose the one most closely related to CAMELS ratings and would ignore the other.
31 As discussed above, SCOR is reestimated each quarter, so the coefficients change slightly over time. As also discussed above, changes in coefficients would occur if, for example, examiners found that underwriting standards had changed. However, by using the 1998 coefficients to rate banks in 2001 and using 2001 coefficients to rate banks in 1998, one can determine whether the change in SCOR ratings is driven by the underlying financial ratios or by the change in coefficients. This exercise indicates that the change in coefficients accounts for approximately half the trend between 1998 and 2001, while changes in the ratios account for the other half. The change in the model could be interpreted as reflecting examiners’ growing concern about aspects of bank operation (for example, underwriting) that are not measured by the ratios.
32 The distribution of SCOR ratings also resembles the distribution of the average of the six component ratings. It should be noted that this average is not meaningful because examiners would almost certainly assign higher weights to some components than to others. Moreover, SCOR ratings are distributed much like CAEL ratings, though CAEL ratings tend to be lower. (As explained above, CAEL was intended to be “biased” toward downgrading banks, and SCOR is not biased.)
|Last Updated 9/11/2003||Questions, Suggestions & Requests|