Stage 2: Assessment

In Stage 2, Assessment, plans are made for the data collection. It is important to remember that:

• Selection of methods should be driven by the research question(s);
• Different methods have different strengths and limitations; and
• Multiple methods should be used over a period of time, whenever possible (use of multiple methods to enhance data is called triangulation).

Some research questions will lend themselves more easily to quantitative measures (e.g., surveys, polls, pre-existing databases) and others will be a better fit for qualitative approaches (e.g., in-depth interviews, focus groups, case descriptions or studies).

 

Example:

If an administrator wants to assess faculty awareness regarding their rights and responsibilities in the accommodations process across several academic departments, a survey may be the most expedient method because surveys are useful for when broad information across a large group is needed. In contrast, however, if an administrator wants to obtain information on the perceptions of campus diversity from African-Americans with disabilities, interviews or focus groups might prove more beneficial given that more nuanced questions will require more nuanced answers and that obtaining a large representative sample is probably not a realistic option in this case.

However, these examples should not be construed as either/or propositions. More often than not, climate assessment and research in general benefits from the use of multiple methods. Called triangulation, surveys and existing data sets are often supplemented with focus groups and interviews and vice versa. In addition, an assessment process that begins with a survey may lead to an assessment that utilizes focus groups, which then may lead to an assessment that uses automated retention data, focus groups, and interviews. Any combination of methods is possible and again, linearity should not be assumed; it is, in fact, cautioned against.

The most effective, accurate, and comprehensive assessments of climate will use a variety of measures that are repeated over time to capture trends, whether they are institutional trends, academic program trends, or trends in a small department. To use an analogy, there are important difference between reporting weather and reporting climate:

 

Figure 2: The Nature of Climate

Weather Is a quick snapshot Consists of a few indicators, such as daily temperature and precipitation. Captures the present.

VS.

Climate Indicates seasonal patterns over time. Consists of multiple indicators, such as average inches of rain and snow, average number of sunny days, relative humidity, etc.  Captures trends from past to present to future.

 

 

 

 

 

 

 

 

 

Although perhaps a simplistic metaphor, this analogy underscores the importance of viewing climate assessment as a continual program-improvement process that is, in the best of circumstances, multi-dimensional, allowing for the triangulation of data to create a more informed picture of reality.  A good assessment process will: 1) focus on trends (data that can be collected from year to year) as opposed to snapshots; 2) use multiple methods to collect the data; 3) utilize the data in a practical way; 4) reevaluate progress from utilizing the data; and 5) repeat and reevaluate the climate assessment process itself with ongoing stakeholder feedback and contributions.