XI-11 CRA Data Review Timeframes and Sampling Guidelines

Introduction

This section provides guidelines to assist examiners in determining the timeframes within which to review loan products and other activities for a CRA evaluation, and how such information should be presented in the Performance Evaluation.  Additionally, this section discusses when and how to select a sample of loans for review.

Background

Large institutions are required to collect and report CRA data, whereas small institutions (including intermediate small banks) are not required to collect data for CRA evaluation purposes.  In addition, certain institutions are required to collect and report home mortgage loans for purposes of the Home Mortgage Disclosure Act (HMDA).  Other institutions may choose to provide data regarding their loans, including the census tract locations and borrower incomes or business revenues, similar to the data requirement for large institutions and HMDA reporters.

Examiners use an institution’s data to analyze:

• Loans inside and outside an institution’s assessment area(s);
• Loans in low-, moderate-, middle-, and upper-income geographies in an assessment area;
• Home mortgage or consumer loans to low-, moderate-, middle-, and upper-income borrowers, and/or small business or small farm loans to businesses/farms of different revenue sizes within an assessment area; and
• Community development activities.

Review Timeframes

Lending Data that is Collected/Reported

For institutions required to report HMDA and/or CRA data, and in other cases when the institution collects but is not required to report the data (collected/reported), examiners must validate the accuracy of the data.  For all loan types, the HMDA validation procedures should be followed, including for the scope of validation.  When validating loan data for CRA purposes only, examiners should validate the necessary fields needed for CRA analysis which include loan type, loan amount, location, and income/revenue.  

If loan data is considered accurate, all collected/reported data for each full calendar year since the previous CRA examination should be analyzed.  This would include the year of the last examination.  There is an exception for small institutions, which typically have a 48-72 month evaluation cycle (timeframe from date of previous CRA evaluation to date of current CRA evaluation).  In this instance, examiners should analyze the most recent two full calendar years of HMDA and/or collected CRA data.  In all cases, a separate analysis should be conducted for each year. It would only be appropriate to analyze partial calendar year data when there is a short evaluation cycle, such as a previous adverse CRA rating or a de novo institution.  In these cases, a separate analysis should be conducted for each partial year.

Presentation of Data

At a minimum, examiners should present the latest full calendar year of data for which aggregate data is available for geographic distribution and borrower profile.  Presentation of additional years of data may be necessary to support conclusions and rating(s) and should be considered in cases such as a borderline overall rating, a significant overall ratings change, an unfavorable rating at any level, lending anomalies between years, CRA complaints or comments, or upon the request of bank management.  Examiners should use their judgment when and for which assessment area(s) additional years of data should be presented.  If only one year of data is being presented, the scope comment should make it clear that all applicable years were analyzed.  Examiners should provide the rationale for including only one year, such as consistent performance between years.  

Even though each assessment area must be analyzed separately, in some cases the presentation of multiple assessment areas may be combined within the CRA performance evaluation. For example, in instances where two or more assessment areas consist of non-contiguous portions in the non-metropolitan area of one state and no anomalies are present, these assessment areas may be combined for presentation purposes.  This also applies in cases where there are multiple assessment areas in the same MSA or CSA.  These guidelines apply to all institutions, not just institutions that have collected CRA and/or HMDA data.

Lending Data that is Not Collected/Reported

For institutions where data is not collected for a particular product under review and sampling must be performed, the analysis should cover the lending activity in the previous full calendar year that is prior to the start of the examination. Reasons to analyze and present additional years of data are a borderline overall rating, a significant overall ratings change, an unfavorable rating at any level, or CRA complaints.  In addition, examiners should confirm with bank management that a one-year sample of a loan product would be representative of the entire evaluation period.  In cases where more than one year will be analyzed, a separate analysis should be conducted for each year.

Exceptions may exist to analyzing data by full calendar year, particularly for institutions that purge their loan system of paid off loans.  In situations where the paid off loans are purged and using the previous full calendar year data would not capture the bank’s actual lending performance, a universe comprised of the most recent 12 months may be appropriate. Examiners should understand the nature of the bank’s lending when making this determination.  If analyzing data that encompasses more than one calendar year, examiners should use the most recent available demographic data.  If changes occur between years (for example, the release of American Community Survey Census data every five years), it is important to see how geographies are affected.  In these situations, it would not be appropriate to combine lending between years if there are significant changes to the geographies because geographic and demographic data are different for each year.

Community Development Data

Community development data (loans, investments, and services) should be reviewed for the entire CRA evaluation cycle.  The extent of validation that the activities qualify will depend on the number of activities.  With large volumes, often the process of validation will include reviewing the process the bank goes through to qualify a particular activity and validating a sample following the CRA Sample Size table.  If the number of activities is smaller, often the entire universe should be reviewed to ensure the activities qualify as community development.

Sampling Guidelines

Small institutions (including intermediate small institutions) are not required to collect small business and small farm data for CRA evaluation purposes.  In addition, not all institutions collect and report home mortgage loan data for purposes of HMDA.  In these cases, it may be impractical or impossible to review all loan files when evaluating a particular loan product, especially if there is a large volume of lending.  Examiners should use sampling to draw conclusions from a random subset of a universe for each loan type and apply such conclusions for the population as a whole. Generally, samples are selected on a bankwide basis.

Examiners should also sample at large institutions if data for a specific loan product has not been collected.  For example, if a large institution has chosen not to collect consumer loan information, yet it comprises over 50 percent of the bank’s lending by both dollar and number, an examiner should review consumer loans and would need to sample.

In instances where collected data is found to be invalid, and the bank is unable to correct the data prior to the examination, examiners will need to rely on sampling to evaluate the institution’s CRA performance.  In these cases, examiners should separately sample and analyze the most recent two full calendar years.

Generally, if the data is readily available, examiners should validate and geocode the entire universe of loans for the loan products and timeframe under review.  Examiners should analyze this data for the assessment area concentration, geographic distribution, and borrower profile lending criteria.  When sufficient information is not available, or addresses do not geocode, examiners should use judgment in determining when sampling will have to be conducted.  Using readily available data for analysis can result in a different number of loans analyzed for different criteria.  One example is where borrower income is not available on the electronic data download and the geographic data is available and valid.  In this case, the universe for assessment area concentration would be all loans originated or purchased during the review timeframe; the universe for the geographic distribution would be the number of loans inside the assessment area; and sampling would have to be conducted for borrower profile.  The differences in number of loans analyzed should be adequately explained in the Scope of Examination comments.  In these instances, examiners should use the review timeframes noted for lending data that is not collected/reported for all of the lending criteria for that product.    

Sampling Process

For small institutions (including intermediate small institutions), examiners should determine major product lines from which to select a sample.  Examiners should take into account factors such as total volume of lending, the institution’s business strategy, and its areas of expertise when selecting product lines to sample.  Initially, examiners may select for review from among the same categories of loans that are to be used when reviewing large institutions (home mortgage, small business loans, small farm loans, and consumer loans).

In order to determine the number of loans for the sample (known as the sample size), examiners should determine the total number of loans in the universe for each product category by year.

NOTE: The universe of loans is defined as the total number of loans, both originated and purchased by the institution in a calendar year, for a major product category.  This should include outstanding and paid-off loans, including renewals.  The definition of the loan category should dictate the universe of that lending product (i.e., small business loans are $1 million or less).

Next, examiners should determine the number of loans to be sampled for each year and product category by using the Sample Size Table below.  Generally, sampling should be conducted at the institution level and not for each assessment area.

*or the entire universe if less than the sample size

The table indicates the sample size based on the universe of loans for each product at a 90 percent confidence level and the desired precision level.  Initially, examiners should select samples based on a 10 percent precision level.  This means that there is a 90 percent chance that the results from the sample will be within 10 percentage points of the true proportion for the criterion evaluated. 

For loan products requiring further investigation or greater scrutiny for any reason, a larger sample may be necessary because examiners may need results with a higher degree of reliability.  Examples of circumstances when increased sample sizes should be chosen include: the receipt of CRA comments or complaints or if management states the selected sample is not indicative of its true lending performance.  Examiners should use their judgment to determine which sample size to use based on the initial scoping of the examination and subsequent findings onsite.

In addition, although examiners may draw conclusions based on a statistically valid sample, they may choose to consider a larger sample in cases where the institution has several assessment areas in order to avoid the result that only a few loans are selected from certain assessment areas in the sample.  Increasing the level of precision will result in an expanded sample.  Examiners may increase the sample to include additional loans up to the 5 percent precision level when a larger sample is necessary to ensure a sufficient number of loans per assessment area for geographic and borrower analysis.  Alternatively, a sample could be chosen for each assessment area using a 10 percent precision.    

Regardless of sampling requirements, samples must be large enough to analyze and to draw meaningful conclusions.  Therefore, analyze the entire universe when the number of loans is less than or equal to 30.

Sample Selection from Electronic Data Download or Loan Trial

Once the number of loans in the sample is determined, examiners should select loans from the financial institution’s electronic data download or a loan trial as opposed to making a general request for loans and allowing bank management to select the loans for review.  If an electronic data download is not available, examiners should select the sample using the financial institution’s loan trial.  The Sampling Job Aid explains the procedures for selecting loan samples from the electronic data download or loan trial.

Examiners should follow random selection methods as compared to those based on judgmental sampling.  The use of random selection methods removes the potential for bias in results associated with the loan selection process.

Data to Collect when Sampling

Once the loans for each sample are identified, examiners should record relevant loan information.  Data for each loan should include, at a minimum:

• Institution’s internal loan ID number;
• Loan type;
• Loan dollar amount;
• Location – In cases where the census tract of the loan is not readily available, examiners are expected to geocode the loans;
• For the home mortgage and consumer loans sampled, the borrower income that was used to approve the loan; and
• For the small business and small farm loans sampled, the business’s or farm’s gross annual revenue.  When income information used in the analysis of the applicant is not readily apparent and tax returns are present, this information can usually be obtained from the “gross receipts;” or “gross farm rental income” and “gross income” fields, respectively.

When data is missing, examiners should attempt to obtain this information through discussions with institution personnel.  Obtaining information through these discussions can significantly reduce the number of records in the sample with “missing data” and thereby increase the validity of each sample.