Home > Industry Analysis > Research & Analysis > FDIC Banking Review
FDIC Banking Review
Do Local Economic Data Improve Off-Site Bank-Monitoring Models?
Researchers at U.S. bank regulatory agencies have developed several types of statistical models to monitor potential problems at individual banks off-site (that is, without having to visit bank premises). These off-site monitoring models tend to be “unconditional” forecasting models that use available data on a bank’s current and past condition to predict its future condition; they do not require the user to “condition” the forecast on assumptions about the future values of any of the variables in the model. Generally the models attempt to predict one of two phenomena: either that a bank will fail or that its condition has deteriorated enough that it will receive a downgrade in its supervisory rating (composite safety-and-soundness rating) during the next on-site examination. Although most models use fairly standard measures of banking conditions, variables describing conditions in the broader economy in which banks operate have not been important features of the models.1 And whereas historical episodes of regional recessions and banking-sector difficulties have been studied, the contribution of economic data in forecasting future bank distress has received relatively little attention in empirical banking research.2
Improving off-site monitoring capabilities would enable regulatory agencies to allocate supervisory resources more efficiently and intervene more promptly and would reduce the costs associated with bank failures. For these reasons we investigate the extent to which state-level economic data could be used to improve the performance of standard types of statistical models that forecast a bank’s condition off-site. Specifically, we focus on the linkages between economic conditions and problems of bank performance between the mid-1980s and the early 1990s—a period characterized by significant regional disparities in both banking-sector and broader economic conditions. The national economic expansion that followed the recession of the early 1980s was an uneven one: agricultural and oil-producing states alike experienced local economic problems and serious banking-sector difficulties. In addition, the national recession of the early 1990s was largely concentrated along the coasts and was linked to bank failures in New England and California. Since the early 1990s the U.S. banking industry has consolidated into larger, more geographically diverse institutions, so one might argue that the industry is now less vulnerable to local economic conditions of the type experienced in the 1980s and early 1990s. Nonetheless, for thousands of small U.S. banks, linkages between local economic conditions and bank performance are likely to remain significant.
Our empirical strategy is to take variables measuring economic conditions in the state where a bank is located and add them to statistical models that attempt to identify institutions likely to experience financial difficulties. We study the contribution of state-level economic variables in three types of forecasting models—specifically, those that forecast bank failures, those that forecast changes in the quality of bank assets, and those that forecast risky bank growth (as indicated by supervisory rating downgrades). The sole criterion for success is whether these variables improve the accuracy of forecasts.
By way of preview, the addition of state-level economic variables generally does not improve upon the forecasts generated by models using only data on a bank’s condition. Indeed, the models forecasting bank failures and changes in the quality of bank assets perform about the same or worse when state-level economic variables are included. The models predicting risky bank growth, however, show a more consistent, albeit modest, improvement. These findings do not imply that economic conditions are unimportant for a bank’s performance. Rather, as we discuss in the conclusion, it is possible that factors not considered in our models contribute to this finding of no, or little, predictive improvement.
The next section discusses the conceptual link between state-level economic data and bank performance. The subsequent three sections present the results of incorporating state-level economic data into models forecasting the three aspects of bank performance that we focus on (failures, changes in asset quality, and risky growth). The final section presents our conclusions and discusses the implications of our findings for future research on bank off-site monitoring.
Conceptual Link between Local Economic Conditions and Bank Performance
Because the purpose of our study is to investigate whether local economic variables can improve the ability of statistical models to forecast which banks will experience difficulties, we judge the success of each model in terms of the accuracy of its forecasts relative to the forecasts of an otherwise equivalent model that does not include the economic variables. Before we turn to the models we develop, however, it will be helpful to discuss the conceptual link between local economic conditions and bank performance.
Some theories posit that the main comparative advantage of banks relative to other financial firms lies in banks’ information about and expertise in lending locally. This advantage is viewed as particularly important for smaller, more-localized banking institutions. In making its lending decisions, bank management must address the risk that local economic conditions will affect the profitability of local borrowers and the subsequent performance of loans granted to those borrowers. Bank lending tends to move procyclically as borrowers seek to fund profitable business opportunities in economic expansions and to retrench during economic downturns. Once loans are issued, a bank’s profitability and credit quality will depend to some extent on the economic fortunes of its borrowers. Indeed, when economic conditions change dramatically, we expect to find a correlation between these conditions and the likelihood that a bank will fail.3 Thus, when local economic conditions vary substantially, we expect to find some relationship between these variations and the performance of local banks. And because profitability and asset quality are key factors affecting bank supervisory ratings, we also expect to see a link between local economic conditions and the on-site examination ratings received by institutions—all other things being equal.
But all other things may not be equal. The relationship between local economic conditions and a bank’s performance also is affected by the management of the bank. Differences in credit cultures, lending strategies, underwriting standards, and asset-and-liability management will lead to differences in the exposure of institutions to local economic developments. We expect that “better-managed” banks will be able to weather local economic downturns better than poorly managed banks. Because management is so important to a bank’s success, it receives particular attention during on-site safety-and-soundness examinations. The summary, or composite, safety-and-soundness rating (CAMELS rating) reflects not only the bank’s current profitability, asset quality, and capital adequacy but also the soundness of the bank’s current management.4 The linkages among the local economic conditions a bank faces, its management policies, its profitability and asset quality, its on-site composite safety-and-soundness examination rating, and its survival are depicted in figure 1.
Despite the multiplicity of factors at play, banks operating in poorly performing economies are nonetheless more likely to perform worse than banks in healthier environments. This suggests that local economic data have the potential to improve the performance of the statistical models used for identifying banks that are likely to experience problems. Whether these data do improve the models’ performance is ultimately an empirical question. But the fairly dramatic regional differences in U.S. economic conditions and bank performance during the 1980s and early 1990s present a good opportunity to study this question (especially given the regulatory structure of the industry at the time, and in particular the interstate banking restrictions that to a large degree delineated banking activities along state lines).
A number of considerations influenced our decision to investigate the usefulness of state-level economic data in off-site monitoring models. First, we wanted to use economic variables that were consistently reported for all regions during the study period. Second, we wanted to use variables that would have been available in a timely fashion for inclusion in off-site monitoring models. Third, we wanted to use economic data measured for the type of geographic region that reasonably could be expected to reflect the conditions faced by many banks. Various data series are available for counties (or parishes), states, or Census-level divisions, but given our selection criteria, state-level data seemed the best choice.5 A fair number of data series are available for all states within a reasonable time frame.6 In addition, interstate banking restrictions and state banking laws delineated banking markets along state lines. Therefore, for the U.S. banking industry of the 1980s and early 1990s, state-level economic data seemed to be reasonable measures of the local economic conditions affecting banks.
Predicting Bank Failures
The first part of our study examines the contribution that state-level economic variables make when added to standard models predicting bank failures.7 Patterns in the state-level data during the 1980s and early 1990s suggest that regional economic conditions were related to the incidence of bank failure. More specifically, states experiencing economic booms followed by busts tended to have high failure rates. Figure 2 shows this by comparing state personal-income growth rates and bank-failure rates for Texas and for Massachusetts. Although there were also regions where weak economic performance was not followed by high bank-failure rates, these tended to be regions where the economic weakness had not been preceded by an economic boom.
Here we look at whether measures of state-level economic conditions would have helped supervisors identify the institutions that ultimately failed during the late 1980s and early 1990s. Taking what have become fairly standard logistic regression models, we use bank financial data at the beginning of a period to predict the likelihood that an institution will fail sometime during a subsequent two-year interval. In these models, the precise relationships used to assign bank-failure probabilities are based on the historical relationships observed for failures during the prior two-year interval. That is, first we estimate statistical relationships about the conditions preceding failures during the previous two years, and then we use these relationships to forecast specific failures during the subsequent two years.
these models generate a failure probability for each bank, one must choose a
critical (or cutoff) probability in order to classify banks as survivors or
failures. For example, a critical
probability of 50 percent indicates that all banks having estimated failure
probabilities greater than 50 percent are classified as “predicted failures.”
Obviously, choosing a lower, more-stringent
Here we report forecast results for two periods. For the first period, we use the relationship between bank and state-level economic conditions as of year-end 1986 and actual failures in the years 1987 and 1988 to predict failures occurring in 1989 and 1990. For the second period, we use the relationship between bank and state-level economic conditions as of year-end 1988 and actual failures in the years 1989 and 1990 to predict failures occurring in 1991 and 1992.
Table 1 lists the variables in the bank failure- prediction models. As indicated in the top panel, the basic “banking” model uses fairly standard bank financial data and supervisory (CAMEL) ratings to predict failure/survival during the subsequent two years. The statistical relationships yielded by the models for the subperiods studied here are generally consistent with those reported by other researchers. All else being equal, banks with less capital, more asset-quality problems, and lower supervisory ratings for management and liquidity are assigned higher projected failure probabilities. We next examine the contribution to the basic banking model made by various proxies measuring state-level economic conditions (see the bottom panel of table 1). Model results are displayed in figure 3.
The solid line in figure 3A illustrates the prediction-error trade-off yielded by the banking model using actual failures in 1987 and 1988 to predict failure in 1989 and 1990. Here the prediction-error trade-off is not as good as that depicted in figure 3B. There is a greater trade-off between minimizing missed survivors and minimizing missed failures. The broken line summarizes the predictive accuracy of the model when measures of state-level personal-income growth are added to the pure banking model: the addition of the economic data materially reduces the accuracy of the bank-failure predictions for 1989 and 1990.
The solid line in figure 3B illustrates the prediction-error trade-off yielded by the banking model using actual failures in 1990 and 1991 to predict failure in 1992 and 1993. The model predicts fairly well, in the sense that one could have chosen a lower critical probability (fewer missed survivors) without dramatically increasing the proportion of missed failures. The broken line summarizes the predictive accuracy of the model when measures of state-level personal-income growth are added to the pure banking model: the economic data do not materially improve our ability at year-end 1991 to predict bank failures.
Although evidence about the contribution of state-level economic data in off-site monitoring models is sparse, our findings are consistent with what has been reported. The most relevant work in this area was conducted by researchers at the Federal Reserve System when they were developing their near-term-prediction Financial Institution Monitoring System (FIMS) in the early 1990s. These researchers’ systemwide effort yielded two models that have been modified and improved over time. The developers of the FIMS model found that including state-level data on unemployment rates, personal income, and housing permits did not significantly improve upon predictions based solely on bank-examination and bank-financial data.8
Predicting Changes in the Credit Quality of Bank Assets
Since the goal of off-site monitoring models is to identify emerging banking problems, accurate forecasts of bank nonperforming-asset ratios are useful, insomuch as declining asset quality generally is a precursor of more serious banking problems. Thus, the second part of our study investigates whether state-level economic variables would improve the performance of reduced-form models that predict changes in bank profitability and asset quality. Here we report results for models that predict changes in nonperforming-asset ratios.9
As with the incidence of bank failure, one can find examples of states where poor economic conditions have been correlated with higher-than-average bank asset-quality problems. Figure 4A illustrates a situation in which the nonperforming-asset ratio of banks in a state is inversely related to the state’s economic health. However, one also can find examples of states where bank asset-quality problems are not clearly related to state-level economic conditions. As figure 4B shows, the nonperforming-asset ratio of California banks was high even when the state’s economy was healthy.10
The nature of bank asset-quality ratios makes them an attractive candidate to study. First, as discussed above, the economic conditions affecting a bank’s borrowers should be directly related to the credit quality of the bank’s loan portfolio. Second, unlike bank failure (which is a discrete event occurring only when a bank’s condition worsens beyond some threshold level), the quality of bank assets is measured in the same continuous fashion as economic variables; hence, it may exhibit a more systematic correlation with economic variables.
One difference, however, between this part of the study and the first part is that bank supervisory staff do not currently use “standard” models that forecast a bank’s profitability or asset quality. Thus, we begin by using bank financial data from prior periods to construct reduced-form linear models that predict the change in a bank’s nonperforming-asset ratios one year forward. We then include a variety of state-level economic data to see whether they improve upon the forecasts yielded by the bank financial data.
We evaluate the forecasts of asset-quality changes by using a standard summary measure of a linear model’s prediction error. The root mean-squared error (RMSE) measures the square root of the average value of a model’s squared forecast errors. Forecast errors are squared before averaging so that negative errors and positive errors count equally, and larger errors are given more weight.
In the models we use here, the RMSE summarizes those differences in asset-quality changes across banks that are not explained by the model. To put the size of the RMSE in perspective, we compare each model’s RMSE with the RMSE we obtain when we use only the historical mean change in nonperforming-asset ratios (no banking or economic data) to predict future changes.11
Because U.S. banks vary greatly in size, we want to account for the possibility that the link between state-level economic variables and nonperforming-asset ratios could vary with a bank’s size. First, very large banks (those with assets of more than $20 billion in 1994 dollars) are excluded from all analyses because they operate in markets that are much larger than the state in which they are headquartered. We divide the remaining institutions into five classes based on asset size in 1994 dollars, and we estimate separate models for each size class. This allows the measured link between state-level data and the quality of bank assets to differ for each class of banks. Table 2 identifies the bank size classifications.
Here we report results for models that measure the link between lagged bank conditions and annual changes in bank nonperforming-asset ratios during two periods: 1986 through 1989 and 1991 through 1994.12 We assess each of these models in terms of the accuracy of its out-of-sample predictions of asset-quality changes in the year following each model’s estimation period—that is, in 1990 and 1995. In modeling changes in asset quality, we include lagged values of the bank’s financial variables that are most likely to be related to the quality of bank assets.13 These measures are identified in the top two panels of table 3. We then include a set of economic variables (identified in the bottom two panels of table 3) in what we refer to as “banking and economic models.”14
Figure 5 illustrates the amount of variation in nonperforming-assets-ratio changes that is not predicted by the models linking past conditions to asset-quality changes during the previous four years. All the results we report here include the same sets of banking and economic variables. Hence differences in results across specifications can be attributed to the inclusion of the economic variables, differences in bank size, and differences in the sample period under scrutiny.
indicated in figure 5A, the reduced-form models using Call Report variables
predict only modest change in bank asset quality during 1990, and the economic
variables do not materially improve upon these forecasts.
In summary, this part of our study indicates that future changes in bank asset quality are hard to predict even with data on recent trends in bank asset-quality measures. And state-level economic data do not generally improve upon these predictions. These results suggest that, at least for the periods we study, a reasonable predictor of a bank’s nonperforming-asset ratios one year forward is the bank’s current nonperforming-asset ratios.
Predicting Risky Bank Growth
The manner in which a bank grows has important implications for its overall safety and soundness. Imprudent or ill-timed growth can lead to risky loan concentrations, funding problems, or other difficulties for bank management.15 Bank regulators are aware of these possibilities and have included appropriate safeguards in the supervisory process. Most relevant to this article is the FDIC’s growth-monitoring system (GMS), which seeks to identify risky bank growth ex ante.16 We propose that economic conditions in a bank’s market might provide a useful context for assessing the potential risks of bank growth and might therefore contribute to bank off-site monitoring models. To see whether our proposal is correct, we next test whether data on state economic conditions added meaningful information to GMS.17
Before we describe those tests, it is useful to look at the past correlation between bank safety and soundness (that is, risky bank growth) and state economic conditions. The U.S. banking experience of the 1980s and early 1990s suggests that deteriorating economic conditions were associated with declines in the condition of banks. Figure 6A illustrates that sharp increases in state unemployment rates in the southwestern United States during the mid-1980s coincided with deteriorating banking conditions, as identified through composite CAMEL ratings of banks.18 (In the figure, positive changes in the average composite CAMEL rating for the region’s banks indicate a widespread decline in banks’ safety and soundness because the rating is an ordinal index that increases in value the poorer a bank’s assessed safety and soundness.) As indicated in figure 6B, the correlation between adverse changes in state unemployment rates and declines in CAMEL ratings was particularly pronounced in the northeastern United States.19
Although informative, these simple comparisons do not tell us whether data on state economic conditions add to off-site growth-monitoring models. To answer this question, we develop and compare two off-site growth-monitoring models designed to rank banks in terms of the relative riskiness of their growth (that is, we designed two risky-growth indexes). The first model (“bank model”) serves as our basis of comparison and uses information on a bank’s portfolio composition, changes in portfolio composition, and supervisory assessments of bank condition to construct a risky-growth index. The bank model excludes measures of state economic activity, however. The second model (“bank and economic model”) includes all the information the bank model contains plus measures of state-level economic activity. The measures of economic activity we test are quarterly changes in both state unemployment rates and state personal-income growth. Because our conclusions are the same for both of these economic activity measures, for brevity we present only the results of tests that use changes in state unemployment rates.
The premise behind the bank model is that all other things being equal, the risks to a bank’s future safety and soundness increase when growth (1) proceeds too quickly, (2) increases the concentration in risky activities, or (3) increases the reliance on volatile sources of funding. In addition, it is presumed that the poorer a bank’s initial condition, the greater the future risks from growth. As shown in table 4 the bank model uses 11 variables to capture the factors that can lead to risky bank growth. More specifically, the bank model uses 5 measures of portfolio change: the annualized rates of growth in total assets, gross loans and leases, the ratio of loans plus securities with maturities of five years or more to assets, the ratio of volatile liabilities20 to assets and the ratio of equity capital to assets. In addition, the bank model uses 4 portfolio ratios: the ratios of loans plus securities with maturities of five years or more to assets, volatile liabilities to assets, equity capital to assets, and a summary measure of portfolio concentration. The summary measure of loan concentration is used to capture potentially risky shifts in business activity and is based on the Herfindahl-Hirschman Index (HHI). To calculate the concentration measure we first compute the shares of total loans held in 15 well-defined categories of loans and leases. Next we square and sum the loan shares.21 Rather than using the raw values of these measures of portfolio change and portfolio ratios, we use a bank’s percentile ranking for each measure, based on either a peer group or a national ranking, as appropriate.22 Finally, the bank model includes 2 supervisory measures: a bank’s composite CAMEL rating as of the quarter-end, and the number of days since the bank’s last on-site safety-and-soundness examination as of the quarter-end.
The final step in computing the bank-model growth index is to combine the 11 variables into a summary growth index. We do this by weighting each variable in terms of its importance in explaining downgrades in composite CAMEL ratings during the prior period and then summing the weighted variables.23 The reason we choose this approach is that a growth index is most useful to bank supervisors if it can be used to anticipate changes in bank safety and soundness (which is measured by composite CAMEL ratings).
In the banking and economic model, we study the contribution of economic data in growth monitoring by including state-level economic variables as additional explanatory variables. This article presents the results of tests based on the quarterly percentage change in state unemployment rates. To construct the bank and economic model growth index, we use the same approach as with the bank model but add percentage changes in state unemployment rates for the current quarter and four prior quarters.
As we stated at the outset, useful risky-growth indexes should anticipate declines in bank safety and soundness. Hence, to assess each index’s usefulness, we rank banks on the basis of their growth indexes and group the ranked banks into “risk” quintiles. Next we measure the proportion of banks receiving CAMEL downgrades (during the subsequent three years) in each of the quintiles.24 For example, we construct bank-model growth indexes as of year-end 1988 and then compare the distribution of CAMEL downgrades between 1989 and 1991 across risk quintiles.
Here we report results for banks that were examined during five three-year periods. For each period, we compute risky-growth indexes (with and without the state economic data) on the basis of the methodology described above. We then compare the downgrade experiences of the risk quintiles generated by the bank model with those of the risk quintiles generated by the bank and economic model. We measure the contribution of the state economic variables by comparing the proportion of downgrades in each risk quintile across the two models. The model that performs “better” will be the one with a higher proportion of downgrades in its highest-risk quintile and a lower proportion of downgrades in its lowest-risk quintile.
Figure 7A shows the percentage of CAMEL downgrades (during the indicated three-year period) occurring in the highest-risk quintile as classified by each model. Except for the 1990 to 1992 period (which coincided with a national recession), the proportion of downgrades occurring in the highest-risk quintile identified by the bank and economic model is somewhat larger than the proportion in the same quintile for the bank model. Figure 7B shows the percentage of downgrades received by banks in the lowest-risk quintile. Here the proportion of future downgrades occurring in the lowest-risk quintile identified by the bank and economic model is generally lower than the proportion in the same quintile for the bank model. These results suggest that state-level economic data might be useful in identifying imprudent bank growth. Although the improvement in the performance of the growth-monitoring model in anticipating future downgrades is somewhat modest, it is fairly consistent over time and is in line with evidence about historical patterns of local economic conditions, portfolio growth, and subsequent bank performance.25
This study investigates the usefulness of state-level economic data in statistical off-site monitoring models. Our results indicate that state-level economic data do not contribute to the models that forecast bank failures and changes in the quality of bank assets. The results for the model predicting risky bank growth are more encouraging, indicating that the inclusion of state-level economic data slightly improves the predictive power of this model.
Although these results run counter to our initial expectations, we can offer possible reasons for the findings; some of the reasons might be addressed by future research. It makes sense to expect that broad measures of economic conditions, such as state unemployment rates and personal-income growth, have varying relevance to individual banks. This variation would be partly due to wide variation not only in the services and products offered by banks but also in the composition of state economies. We are limited in investigating this possibility because banks do not publicly report business activity (for example, loans) by the geographic markets and industry sectors served. Given this limitation, it is difficult to determine which economic variables are likely to be most relevant to a bank’s current condition and future performance. Our hope was that broad measures of economic conditions would have had relevance for most banks and therefore for off-site monitoring models.
We also anticipate that bank management plays a very significant role in determining how economic conditions affect a bank’s performance. Prior research by the FDIC and others has suggested that bank-specific attributes such as the quality of management, loan underwriting, and risk-management practices should have an important influence on a bank’s performance and its susceptibility to adverse economic conditions. Although these characteristics are hard to quantify, bank supervisors do collect data in some of these areas. For example, all federal bank regulators conduct periodic surveys of bank underwriting practices. The FDIC is pursuing research on the contribution that the data in its semiannual underwriting survey might make to off-site monitoring models.
Avery, Robert B., and Michael B. Gordy. 1998. Loan Growth, Economic Activity, and Bank Performance. Working Paper. Board of Governors of the Federal Reserve System.
Cole, Rebel A., Barbara G. Cornyn, and Jeffery W. Gunther. 1995. FIMS: A New Monitoring System for Banking Institutions. Federal Reserve Bulletin (January), 1–15.
Federal Deposit Insurance Corporation (FDIC). 1997. History of the Eighties, Lessons for the Future: An Examination of the Banking Crises of the 1980s and Early 1990s. 2 vols. FDIC.
Gilbert, R. Alton, and Sangkyun Park. 1994. Value of Early Warning Models in Bank Supervision. Unpublished mimeo. Federal Reserve Bank of St. Louis..
Gilbert, R. Alton, Andrew P. Meyer, and Mark D. Vaughan. 1999. The Role of Supervisory Screens and Econometric Models in Off-Site Surveillance. Federal Reserve Bank of St. Louis Review (November/December), 31–56.
Hooks, Linda M. 1995. Bank Asset Risk: Evidence from Early Warning Models. Contemporary Economic Policy 13, no. 4:36–50.
Neely, Michelle Clark, and David Wheelock. 1997. Why does Bank Performance Vary across States? Federal Reserve Bank of St. Louis Review (March/April), 27–40.
Nuxoll, Daniel A. 2003. The Contribution of Economic Data to Bank Failure Models, Working Paper. FDIC.
Reidhill, Jack, and John O’Keefe. 1997. Off-Site Surveillance Systems. In History of the Eighties, Lessons for the Future: An Examination of the Banking Crises of the 1980s and Early 1990s. FDIC.
Samolyk, Katherine A. 1994a. Bank Performance and Regional Economic Growth: Evidence of a Regional Credit Channel. Journal of Monetary Economics 34, 259–278.
———. 1994b. U.S. Banking Sector Trends: Assessing Disparities in Industry Performance. Federal Reserve Bank of Cleveland Economic Review, Quarter 2, 2–17.
Swamy, P.A.V.B., James R. Barth, Ray Y. Chou, and John S. Jahera, Jr. 1995. Determinants of U.S. Commercial Bank Performance: Regulatory and Econometric Issues. Finance and Economics Discussion Paper, no. 95-29. Board of Governors of the Federal Reserve System.
|Last Updated 7/25/2003||Questions, Suggestions & Requests|