6.0 Household Travel and Activity Surveys

Household travel surveys have traditionally provided the most important data inputs into regional and statewide travel models. These surveys have generally been the largest and most complex travel survey efforts, and, therefore, not very many of these efforts have been undertaken on a routine basis. However, pressures being placed on travel demand modeling by the CAAA and ISTEA have led to renewed interest into improving the quality of available survey data in many regions of the country. This has led to an increased interest in household travel and activity surveys.

In the 1960s and 1970s, household travel surveys were generally designed as in-home surveys, where survey fieldworkers would actually go in to sample households and conduct in-person interviews with the household members. Variations on this method are still widely used in other countries, but the high costs of fieldwork labor, the logistical difficulties (including quality control issues and fieldworker security concerns), and the relatively high rate of telephone availability in the U.S. have led travel surveyors to use telephone-based or mail survey techniques.

The terms, “Household Travel Survey” and “Household Activity Survey,” may be used generically to refer to any surveys in which respondents are contacted at their homes and asked travel-related or activity questions, or they may be used more specifically to refer to surveys that record detailed trip information (usually with survey diary methods) and that are used as inputs to travel behavior models. For the most part, this chapter (like the other chapters in this manual) is aimed at describing the implementation of surveys that would be suitable for developing travel demand models. However, many of the survey elements and procedures described in this chapter also apply to the other types of household surveys that would fall under the more generic survey definitions.

 6.1 Organization of this Chapter

As more and more new household travel and activity survey procedures are applied, it has become quite difficult to determine the best procedures for future efforts. This chapter is intended to provide a brief overview of the available options, to discuss advantages and disadvantages of each option, and to make recommendations (where possible) on how to proceed with each option. The chapter covers common household travel/ activity concerns, but each household travel/activity survey effort is somewhat unique, and each is likely to require the resolution of many issues not covered in this manual.

The first five sections of this chapter cover the steps of the survey implementation process described in Chapter 2.0. Sections 6.2 through 6.6 discuss the survey design stage, including:

• Assembling Background Information;

• Survey Design;

• Survey Organization;

• Sampling; and

• Drafting and Constructing Survey Instruments.

Then, field implementation aspects of household travel/activity surveys are addressed in Sections 6.7, 6.8 and 6.9:

• Pretesting the Household Travel/Activity Survey;

• Training and Briefing Survey Fieldworkers; and

• Interviewing and Questionnaire Distribution.

Finally, the survey data preparation tasks are discussed in the remaining three sections, 6.10 through 6.12.

• Coding and Data Entry;

• Editing and Data Cleaning; and

• Programming and Compilation of Survey Results.

The key issues discussed in each section are briefly outlined in Table 6.1.

Each of the sections begins with a figure presenting the key issues related to the specific survey implementation step, and outlining the content of the section. The sections are then organized by these key issues. However, decisions related to issues at one point in the implementation process are likely to affect decisions at several other points in the process. It will soon become obvious to the reader that the survey steps for the household travel/activity survey are greatly interrelated. Even though this manual and other guidance documents tend to organize the survey process into finite steps for the sake of presentation, we recommend that household travel/activity survey designers understand the process, as a whole, and the interrelationships between the steps before beginning survey development.

6.2 Assembling Background Information

 Key Issues in Assembling Background Information

1. What data are available about the characteristics of study area households?

2. What past survey experiences can be used in designing the new household travel/activity survey?

3. What data are available about travel behavior in the study area?

4. What data are available for developing the survey sampling frame?

5. What data are available for geocoding household travel/activity locations?

 Section Summary

Background Data on Study Area Households 6-7

U.S. Census Data 6-7

Other Household Data 6-7

Background Data on Past Survey Efforts 6-8

How information on past surveys can be used 6-8

Use of previous household survey information
in planning the survey 6-8

Use of non-travel related survey information in planning
the survey 6-8

Background Data on Travel Behavior within the Study Area 6-9

Use of transportation planning databases in planning
the survey 6-9

Use of existing travel demand models in planning
the survey 6-9

Background Data for Developing Sampling Frames 6-10

Importance of a high-quality sampling frame 6-10

Different types of sampling frames for household
surveys 6-10

Address-based sampling frame data sources 6-10

Telephone-based sampling frame data sources 6-12

Other lists 6-12

Background Data for Geocoding Household Travel/
Activity Survey data 6-13

Electronic data sources for geocoding 6-13

Low-technology data sources for geocoding 6-13

Output of the Background Data Assembly Task 6-13

 6.2 Assembling Background Information

Prior to getting extensively involved in the household travel/activity survey implementation process, it is highly desirable for a planning agency to assemble as much relevant background data as possible. As noted in Chapter 2.0, available independent data sources are useful in three ways:

• In lieu of some or all of the household travel/activity survey work;

• For developing household survey samples; and

• For validating the household travel/activity survey results.

The section summary describes the key issues that need to be addressed in the assembly of background data. These five key issues are described, in turn, in this section.

Background Data on Study Area Households

Household travel/activity surveys are conducted because the surveyors and modelers have recognized the unique importance of the household in travel. In most regional travel models, households are the basic unit of trip production, and they are usually assumed to be peoples’ unit of travel decision-making, as well.

Therefore, when beginning a household survey effort, the survey designer will want to have as much information about study area households as possible. The best sources of this information are generally U.S. Census data files and local planning agency datasets.

U.S. Census Data

The most comprehensive database containing detailed information about persons and households available in the U.S. is the Decennial Census of Population and Housing. The 1990 Census database contains a wide array of data on the characteristics of U.S. households and individuals. Relevant Census data items are discussed in Appendix B of this Manual.

Other Household Data

While the Census data files are likely to include the most detailed household information available within a region, the data apply to 1990 conditions. Census data are usually significantly outdated before the new data are collected 10 years later. To the extent possible, travel survey planners should use locally-updated household data to improve upon the Census information. The availability and applicability of locally-developed household data and estimates varies throughout the country.

Background Data on Past Survey Efforts

The household travel/activity survey team should attempt to locate the results of all past travel surveys performed in the area. These survey results will help to:

• Determine which survey methods are likely to be the most successful in the household survey;

• Provide more relevant travel-related data for survey respondents than the Census or other household data sources;

• Define the range of expected responses to specific questions;

• Help define expected variation in survey results and, thus, help determine sample sizes and accuracy levels;

• Provide measures of cooperation and response rates, and potential non-response problems;

• Furnish the survey designer with information on how well particular questions worked; and

• Define the specific conditions for coordination between the new survey and old surveys that may also be used in the travel demand model development.

Actual survey experiences are invaluable resources in the development of new surveys. Surveyors should make use of any and all survey techniques from the previous studies. However, the survey designer should critically review the past efforts to avoid propagating the errors of earlier surveys.

The last full-fledged household travel survey effort may have occurred too far in the past to be of much use in deciding the best approaches for the new survey effort. Nevertheless, the survey designer should examine the previous household surveys to determine how each piece of data was used in model development. This is an excellent way to determine the data and question needs of the new survey.

Survey experiences outside the realm of travel surveys may also be valuable in designing household travel/activity surveys. Although an agency or survey team will have only a small amount of travel survey experience to build upon, it is likely that the households in the region have been surveyed extensively on other topics in recent years. These experiences can help travel survey teams predict:

• Response and cooperation rates;

• The effects of survey length on response levels and quality;

• The efficacy of different survey methods;

• The efficacy and cost-effectiveness of different response-enhancing mechanisms, such as incentives; and

• Data collection costs per completed survey.

Assembling this information on past survey efforts usually requires utilizing the experience and knowledge of survey subcontractors and consultants (and paid or unpaid advisors).

Background Data on Travel Behavior within The Study Area

All local planning agencies in the U.S. have developed transportation planning databases that can be used to plan household travel/activity surveys. Most are based, in part, on the Census data and travel survey data discussed above, but other data sources will be reflected as well. Existing trip generation, trip distribution, and mode choice data, and inventories of transportation infrastructure and services, can be used to:

• Identify the study area boundaries (and thus establish the population for the household travel/activity survey);

• Provide estimates of the variances of key survey sampling measures;

• Help determine the amount of information respondents are likely to provide;

• Help determine the range of valid responses for specific survey questions.

Another type of background data that is useful in designing the household survey is the existing travel demand models themselves. The household survey planner should evaluate the models and determine their adequacy for the new generation of travel models. Some models, or specific aspects of models, will still be valid. In these cases, the new survey effort should seek to provide the necessary information to update model parameters to the current year. Modelers will want to replace other aspects of the modeling system with newer or different statistical approaches. In these cases, the household survey will need to collect new types of data or more detailed data than the previous survey effort.

Background Data for Developing Sampling Frames

The survey designer will also need to investigate the various data sources that can be used to define the survey “sampling frame.” The sampling frame is the list of households from which sample households are chosen. Often, the availability or non-availability of high quality sampling frame data will determine the best survey method to be used.¹

Properly drawn samples provide information appropriate for describing the elements of the sampling frame – not the survey population. Therefore, it is imperative to develop a sampling frame that closely approximates the population of interest. Sometimes, surveyors select samples from a given sampling frame, and then erroneously make generalizations about pop-ulations similar to, but not identical to, the population defined by the sample frame.

For samples of households, the most appropriate sampling frames are those that will contain the addresses and/or telephone numbers for the target population of households. Usually, available sampling frames do not truly include all the elements that their names imply. Omissions are almost inevitable. For example, telephone directories do not include new subscribers or householders with unlisted numbers. During the background data assembly task, survey teams should assess the extent of omissions (and the resultant extent of bias) in all potential sampling frames.

Ideally, for a household survey, the address, location, and phone number of all households within the model study area are known or can be known. In practice, survey planners usually do not have the ability to obtain all these data items for all households. The selected survey method will determine whether address-based data sources or telephone-based sources (or both) are needed. Household travel/activity surveys that rely on mail survey methods or in-person methods to recruit respondents will need to rely on address-based data sources for the sampling frame; survey methods with telephone recruitment will need a telephone-based sampling frame. During the background data assembly task, survey planners should evaluate potential sample frame data sources of all types.

Address-based data sources are likely to be more complete than telephone-based data sources, but also usually involve more time and effort to obtain and to prepare for use by the survey team. Potential address-based sampling frame data sources include the following:

• Property tax records;

• Public utility records;

• Official town maps; and

• Field listings from land-use inventories.

While most of these data sources are maintained by public and quasi-public agencies, they are often quite difficult to obtain because of privacy concerns. Survey planners should be prepared to provide reassurances that the data will be protected to the maximum extent possible (including documentation of planned data storage and use procedures).

Address-based data usually requires fairly extensive manipulation by survey teams. Each data source can be problematic. Property owners listed in tax roles are not necessarily the persons who reside at a particular address. Public utility listings may have one address for all units in a multi-unit complex. Maps and inventories can become out-of-date in very short periods of time. Survey teams usually need to combine data from multiple sources to develop address-based sampling frames, and they usually must perform at least some field verification.

Usually the most appropriate sampling frame for telephone survey methods is a list of telephone exchanges (or prefixes) within the study area of interest from which random-digit-dialing (RDD) respondent selection can be conducted. Sometimes, telephone survey samples are developed using telephone directories as the sampling frames. This approach reduces the cost of contacting respondents, because (unlike the random dialing approach) it is known that these numbers are for telephones that are in service, are residences, not businesses, and are within the study area. However, the high (and increasing) percentage of unpublished and unlisted telephone numbers in most U.S. cities usually precludes this approach for household travel surveys. Table 3.5 shows the unlisted rates for the 100 largest metropolitan areas in the U.S. Some agencies have used a combination of listed and unlisted numbers for their household sampling frames.

Assuming that an RDD approach is utilized (at least in part), the survey designer needs to assemble a complete listing of all prefixes for telephone lines within the modeling study area. Since study area boundaries and telephone prefix boundaries generally do not coincide, a number of the prefixes will apply to areas that are both within and outside of the study area. These prefixes should be included in the compiled list (interview screening techniques can be used to determine whether respondent contacts with these telephone exchanges live within the study area). The geographic coverage of telephone exchanges are usually available from local telephone companies.

In cases where the telephone company data are insufficient, survey designers generally obtain the information from reverse directories. Reverse directories present telephone numbers on a geographical basis and a numerical basis, rather than by people’s names as standard telephone directories do. There are several publishers of reverse directories, and nearly every part of the country is covered in one or more of these directories.

In addition to compiling the list of relevant prefixes, it is usually helpful to estimate the number of working lines in each prefix and any numerical ranges of suffixes that are not in service. This information allows the survey team to generate random telephone numbers in the proportion that each prefix exists within the sampling areas, so the final sample will be stratified by prefix. Again, the best source of this information is the local telephone company, with reverse directories as backup sources.

Professional marketing research firms specializing in telephone surveys usually keep current information on telephone exchange geography and the number and ranges of working suffixes. Alternatively, a number of firms sell randomly generated telephone lists for client-specified geographic areas. These lists can be tailored to include specified numbers of listed and unlisted telephone numbers. Unlisted numbers can be “cleaned” to remove numbers in non-working exchanges or ranges of suffixes and commercial numbers. Listed numbers can be provided with name and address information.

In some special cases, it may be desirable to use available lists of telephone numbers or addresses that have (or are likely to have) a characteristic of special interest. For instance, to ensure that transit riders were included in the household survey in sufficient numbers, a few recent household survey efforts relied partially on lists of telephone numbers of people who had claimed to be transit riders in other newly completed surveys. Similarly, there are commercial marketing research firms that develop lists of telephone numbers of people with particular circumstances and characteristics. Though not perfect, these lists can greatly improve the chances of being able to include people of specific types (such as people who ride transit) in the survey effort. If oversampling of specific hard-to-reach groups appears to be desirable, the survey designer should seek out available lists, and look into the possibility of purchasing an enriched sample.

It is important to note that telephone interview data obtained from respondents that were identified from non-random lists need to be expanded carefully. These data cannot be arbitrarily incorporated into an otherwise random sample. The list-generated data needs to be analyzed separately from other data, or they need to be adjusted with differential weights for some or all analyses.

If a telephone-based sampling frame is chosen, the survey designer should determine whether there are any available data sources for households without telephones. Generally, there are no lists of such households. However, there may be reasonably good sources describing the geographic areas where these households are located. The survey designer can use this information to decide whether supplemental sampling of non-telephone owning households can be accomplished in a cost-effective manner.

Background Data for Geocoding Household Travel/Activity Survey Data

As discussed in Chapter 14.0, one of the key elements of processing travel survey data is the association of activity and travel locations to a pre-defined geography, such as latitude/longitude (or another planar coordinate system), traffic analysis zones, or census tracts. To accurately and efficiently geocode household travel/activity survey data, one or more geographically-referenced databases will be needed. Three types of electronic data files are commonly used for geocoding purposes:

• Census “TIGER” files;

• Commercially-available address-matching databases; and

• Emergency response data files.

The files are discussed in detail in Chapter 14.0.

In addition to the electronic data files, a host of low-technology geocoding tools invariably prove to be essential to a household travel/activity survey effort. These include:

• Up-to-date street maps;

• Telephone directories; and

• Visitor guides to the region.

It is generally a good idea for all survey team members to be aware of the need for such resources so that these materials can be collected and assembled.

Output of the Background Data Assembly Task

At the end of the background data assembly task, the survey team should have a strong understanding of the many data sources that can be used in the survey design and implementation. In later tasks, the survey team will use information from these sources to design the survey, develop a sampling strategy, and to code survey responses.

6.3 Survey Design

 Key Issues in Survey Design

1. What are the anticipated data needs from the household travel/activity survey? How do these needs affect the design of the survey?

2. Which survey method should be employed for the household travel/activity survey?

3. Given the selected survey method, what data collection techniques should be employed?

4. What accuracy-enhancing (bias-reducing) measures should be employed?

 Section Summary

The Data Needed from the Survey and the Survey Design
Implications of the required Data Analyses 6-17

Statement of Goals for the Survey 6-17

The Need to Consider Anticipated Analyses in Survey Design 6-18

Examples of how Anticipated Analyses Affect Survey Design 6-18

The Household Versus the Individual as the
Basic Unit of Analysis 6-18

Cross Sectional Versus Longitudinal Analyses 6-20

Trip Versus Activity Analyses 6-21

The Comprehensiveness of the Travel/Activity Data 6-23

Stated Response Analyses 6-24

Seasonal Analyses 6-25

Analysis Time Periods 6-26

6.3 Survey Design

(continued)

Selection of the Survey Method 6-28

Components of a Survey 6-28

Commonly Used Methods 6-29

Telephone Survey 6-29

Mail Survey 6-29

Telephone-Mail-Telephone Survey 6-29

Telephone-Mailout-Mailback Survey 6-29

In-Home Surveys 6-29

Selection of Data Collection Techniques 6-29

Qualitative Survey Techniques 6-42

Centralized Telephone Interviewing Facilities 6-43

Computer Assisted Interviewing Techniques 6-44

Procedures to Enhance Survey Accuracy 6-47

Improving the Identification of the Survey Population 6-48

Improving the Sampling Frame 6-48

Reducing Non-Response 6-49

Prenotification 6-49

Survey Follow-up 6-50

Incentives 6-57

Response Facilitators 6-59

Output of the Survey Design Task 6-61

 6.3 Survey Design

The survey design task requires planners of household travel/activity surveys to address a series of successively more detailed survey design issues, beginning with the determination of the survey’s role in the sponsoring agency’s long-term planning processes, and including the selection of the best survey methods and data collection techniques. Each survey design decision needs to be guided by the agency’s time and budget constraints and by the practical realities facing transportation agencies today.

The section summary shown above reviews the key issues that the survey team needs to address during the survey design phase of the project. These issues frame the lengthy discussion of design issues that follows.

The Data Needed from the Household Travel/Activity Survey and the Survey Design Implications of the Required Data Analyses

As is the case for any survey effort, the design of a household travel/activity survey needs to be informed by the foreseeable uses of the collected data. At the beginning of the design task, the sponsoring agency should define the goals of the data collection effort. Most household surveys are used as inputs into broadly-defined planning applications, such as the development or refinement of regional travel demand models. Other survey efforts are designed for more narrowly focused analyses, such as infrastructure project analysis. For instance, the California Department of Transportation (Caltrans) sponsored a recent analysis of the feasibility of high-speed rail service in California which utilized a household survey technique. Some household travel/activity surveys are designed as part of a larger data collection effort. Others are essentially stand-alone analyses.

It is recommended that the sponsoring agency develop a Statement of Goals for the survey effort that can be used as a guide in survey design. This statement should describe:

• The data needs that have led the agency to conclude that survey work is necessary;

• The expected analyses and uses of the survey data; and

• The guiding principles of the data collection.

The Statement of Goals should be as detailed as possible in defining potential analyses that will rely on the household survey data, because many types of analyses will not only determine survey question content, but will also have implications on the choice of overall survey strategies and survey methods.

The Statement of Goals is a valuable document for ensuring that the data collection effort provides the necessary information to the sponsoring agency. It can be used throughout the survey planning process to help make survey design decisions, and to provide staff not directly involved with the survey and other agencies and firms with information on the survey project.

A wide range of analysis issues affect the overall design of household travel/activity surveys. Prior to the household travel/activity survey, the survey team should define specific analyses to the maximum extent possible. Some analysis issues that affect the overall design of household/travel activity surveys which survey teams have recently considered include the following:

• Are the needed data related to entire households or to individuals within households?

• Are the needed data cross-sectional in nature (data representing a single point in time), or are the likely analyses going to rely on longitudinal analyses?

• Do the likely uses of the data involve traditional trip-based analyses or activity-based modeling approaches?

• How complete do the travel/activity data need to be?

• Do the likely analyses require only revealed travel behavior data, or is there a need to obtain hypothetical choice and attitudinal information, as well?

• Are the needed data specific to certain seasons of the year?

• Are the needed data specific to certain time periods?

Survey teams should consider analysis issues such as these very early in the survey design process because the selection of survey methods and techniques will be influenced by them. The implications of each of the analysis issues listed above are described below.

The Household versus the Individual as the Basic Unit of Analysis

Household travel/activity surveys are usually complex surveys. Provided that special care is applied in the organization and expansion of the data for analysis, the survey data can typically be analyzed using several different units of analysis, including households, individuals, vehicles, and trips or activities.

However, before any data are collected or any analyses are performed, the survey team needs to define what the basic unit of analysis will be, the household or the individual.

For analyses that treat study area households as the tripmaking unit and the travel decision-making unit, it is necessary to collect survey data about entire households. For instance, trip generation models are generally developed at the household level, and thus need survey information on all trips made by household members over some period of time. On the other hand, analyses that are based on individuals’ travel behavior require the survey team to collect data on only a representative sample of study area residents, so data on only one household member are needed. Stated-response household surveys often will seek out a single individual within a household.

The distinction between these two types of analyses is of critical importance in survey design. Household travel/activity surveys used to obtain information on entire households are usually longer, much more complicated, and more burdensome for respondents than surveys that obtain similar information for only a single household member. Among the issues that need to be addressed for the household-based data collection are:

• Procedures for identifying individuals within the household, and for distinguishing between them throughout the survey data collection;

• Procedures for communicating with each household member or having household members communicate through a designated spokesperson; and

• The potential need for proxies, in which one member of a household answers the survey questions for a member who cannot, either because he or she is too young or because he or she is unavailable.

Person-based survey efforts are far less complicated. The key issue for person-based surveys is how to select the proper household member for the survey. It is widely acknowledged that asking the household members who answer the door, open the mail, or answer the telephone to participate in a survey leads to a non-representative sample of a study area, and so travel surveyors have used a number of techniques for randomly selecting a household member. For instance, a common approach is to select the household member who is the next to have a birthday.

The survey team should determine whether the survey analyses will be based on household-based analyses or person-based analyses. The survey effort is significantly reduced if the latter is true, but more importantly, the analysis needs for household travel/activity surveys typically require that all household members be included in the survey effort.

Cross-Sectional versus Longitudinal Analyses

Traditional travel models rely almost exclusively on cross-sectional data, so household travel/activity surveys which are designed to capture people’s behaviors and attitudes at a single point in time have always been the most appropriate data collection tool. However, in recent years, researchers have recognized that many of the behaviors that travel models attempt to forecast are actually related to people’s decisions over time.^1,2 The renewed interest in how people’s behaviors change over time has led to the use of longitudinal survey designs, such as panel studies, cohort studies, trend studies, and before-after studies.

From analyses conducted thus far, it appears that longitudinal data collection efforts, in general, and panel studies, in particular, hold a great deal of promise for travel demand modeling. Chapter 13.0 discusses survey-related issues related to the emerging and promising use of longitudinal analyses and surveys, but as that section discusses, if longitudinal analyses of household travel/activity survey data are anticipated, the survey team needs to be prepared to make a continuing commitment to high quality data collection, and to expending significantly more resources to address:

• Additional complexity of the survey recruitment;

• Sample maintenance and replacement;

• Wider scope of survey questions;

• Use of responses in past waves to frame questions;

• Attrition (for panel surveys);

• Weighting of longitudinal data; and

• Additional reporting requirements.

Decisions about how to incorporate these issues into the survey design will certainly affect the cost and time estimates for the survey, and may also help determine which survey methods and techniques to use.

Trip versus Activity Analyses This section needs a discussion of how ABM utilize travel/activity survey data and how this informs the survey design KM

For many years, transportation planners have recognized the fact that travel is a derived demand – the demand for travel is related to the activities from which and to which people travel. In the early 1970s, a number of researchers proposed the development of a new set of models that would predict the activities in which households would take part, and then determine the household’s future travel patterns.³

In the past few years, there has been growing interest in looking at activity-based modeling again. One region is currently developing a prototype activity-based model system, and a great deal of research is underway to improve the state-of-the-art in this field.⁴ The initialActivity-based model systems differs from conventional modeling approaches in that it they predicts the numbers and types of activities households will perform, then relies rely on either a set of behavioral rules or a series of econometric equations to forecast how household members will schedule and travel to and from activities. The models rely on stochastic microsimulation techniques to forecast activity and travel patterns. ADD KM

The choice between trip-based analyses and activity-based analyses has a basic effect on the design of household travel/activity surveys. The survey team needs to determine whether to collect detailed information on people’s trips directly, or to collect information on people’s activities and their travel to and from the activities (assuming the respondents need to travel to the activity).

Household surveys can be divided into three types in this regard:

• Trip-based surveys that directly gather information on people’s trips over some period, using either diary methods or recall methods;

• Activity-based surveys that gather information on activities to which respondents need to travel during a set time period; and

• Time Use-based surveys that gather information on all activities in which respondents participate during a set time period.

The primary advantages of trip-based surveys is that they use the most efficient data collection approach, in terms of survey time and respondent burden. Respondents are asked directly about the subject of interest, their travel. The primary disadvantage of the approach is that typically the only information gathered on why the respondent is traveling is a non-detailed trip purpose. These surveys do not typically provide the information to examine the activities that people perform which produce their travel.

Activity-based surveys were developed as a means to improve upon traditional trip-based surveys. Surveyors have found that people do a better job remembering and recording trip information when they are asked about what they did rather than simply about where they went.⁵ Of course, the survey also needs to query respondents about their travel to and from activities, so these questionnaires require more information than the trip-based approach. This translates into more work for respondents and longer data retrieval questionnaires, which in turn is likely to translate into higher non-response rates and more complaints about the survey effort. Although surveys of this type are commonly referred to as activity surveys, they are generally not suitable for activity-based modeling, because they do not provide the full set of activities for respondents.

The final type of household survey, the time use-based survey, asks respondents to record all of their activities over some period of time. These include activities that take place within people’s homes as well as those to which respondents need to travel. The surveys also collect the travel data for any trips between activities. These surveys provide a basis from which either traditional trip-based modeling approaches or activity-based modeling approaches may be developed, and because respondents are asked to record all of their activities over the time period, the number of trips that are accidentally left out is likely to be smaller than either of the other types of surveys.

On the other hand, the time use-based surveys are necessarily much longer than the other types and the respondents are asked to supply a great deal of personal information. These surveys have been found to be too invasive by a number of potential respondents in the regions where they have been fielded. A recent household activity survey in New Hampshire attempted to record both in-home and out-of-home activities, but respondent complaints to the Department of Transportation led the survey team to revise the data collection and analysis approach so only out-of-home activities were collected.

New generation travel demand models, like TRANSIMS, will likely require very detailed time use-based survey data.

The Comprehensiveness of the Travel/Activity Data

Many analyses require that all travel within a time period for a household (or for a person within a household) be reported in the survey data. For instance, to measure daily household trip generation rates, the survey team would want a full accounting of the trips made by each sample household in a 24-hour period. On the other hand, some analyses focus on a few specific trips made by household members, such as work trips or trips by certain modes.

The distinction between these two types of analyses has a very important impact on the household travel/activity survey because it is the primary determinant of whether formal travel or activity diary procedures are needed, or whether the use of respondent recall will be sufficient.

If the survey team is seeking a complete listing of travel and activities for a household, as is often the case, recent household travel/activity survey experience would suggest that the team use either travel or activity diaries, and not rely on respondent recall questions. As Richardson, Ampt, and Meyburg suggest, one needs only to try to remember in detail what they did, and where they went yesterday to realize that it is extremely difficult to obtain reliable and complete information using recall survey questions.⁶ By the early 1980s, the use of travel diaries supplanted recall surveys for collecting travel model input data. for the simple reason that they are more effective at capturing people’s trips. Diary methods consistently outperform trip recall questions in capturing:

• Short trips;

• Off-peak trips; and

• Non-work trips.

Travel and activity diaries are thought to be better than recall methods in these instances, because:

• Respondents are asked to complete diaries for a pre-specified future time period, so they are probably more cognizant of their travel during the particular time period than they would otherwise be; and

• Respondents are asked to record travel and activities as they occur, so the likelihood of forgetting an activity or trip is reduced.

The many diary types are discussed in Section 6.6. The use of the less-expensive and more simple recall method may be a better approach when:

• The survey team is interested only in certain types of travel and activities;

• The survey data are not being used to develop travel volume estimates for the person or household; or

• The survey team is willing to weight trip rates according to known (or estimated) volumes.

The recall method may also serve as a backup approach to try to get households who have refused to participate in a more detailed diary survey.

If limited travel or activity information is needed from a household, then a recall technique might be successful. With careful questioning, and perhaps interviewer probing, respondents can generally be induced to remember specific trips in the recent past, particularly if the trips can be defined specifically (e.g., a trip between home and work) or are somehow noteworthy or unique for the respondent.

The need (or lack of need) for diaries has an important effect on the selection of the survey method. If diaries are required, then the chosen survey method will be required to have a mail or in-person component to physically get the diaries to the household. If only recall methods are required, the household travel/activity survey can be accomplished with a single-contact interview (in-person or telephone) or a simple mail survey.

Stated Response Analyses

The emphasis of household travel/activity surveys has traditionally been to collect people’s actual travel behavior (their revealed travel preferences), but as the analysis demands on the survey data are being increased, travel surveyors have begun to experiment with collecting hypothetical choice information from household travel/activity surveys.

Stated response survey questions can provide the survey team with information such as:

• How people are likely to react to changes in transportation services and infrastructure;

• How people are likely to react to potential new government policies; and

• Confirmation (or rejection) of revealed preference modeling results.

This emerging use of stated response techniques and the survey design issues related to them are described in Chapter 13.0 The key design issues include:

• The added costs and complexity of designing survey questions;

• The potential need for delivering stated-response survey materials to respondents;

• The added burden on respondents of figuring out and answering the stated-response questions; and

• The different analysis requirements of such survey data.

The inclusion of stated response questions on a household travel/activity survey will affect the decision of the best survey method since these questions work best when an interviewer is available to answer respondent questions and to explain the sometimes complex instructions. In addition, including these exercises will affect the sample selection and the need for advanced fieldworker training.

Seasonal Analyses

In general, household travel/activity surveys capture travel conditions over a small period of time during a year. Usually, the data and analyses are extended to look at other times of the year by factoring trips using travel volume data, but because it is generally acknowledged that people’s travel patterns vary between seasons along many dimensions, including trip purpose, duration, frequency, and destination choice, it is highly likely that the household travel or activity data represent the time period for which they were collected to a much higher degree than other periods.

Traditionally, household travel/activity surveys are conducted in either the Spring or Fall. These seasons coincide with the most common traffic data collection periods. In addition, they represent time periods when schools are in session, and when potential respondents are least likely to be away from their homes on vacation.

However, the survey team should consider the analyses that will need to be performed before scheduling the household/activity survey. For the past five years, a primary driving force behind travel demand modeling has been the need to better measure and track air quality. Most regions concerned with air quality issues are most interested in Summer conditions (due to increased ozone levels) and Winter conditions (due to cold start emissions). Nevertheless, most surveys and models continue to be for the Spring or Fall, because they seek to capture specific “average” or “peak” conditions. Agencies whose primary concerns are air quality-related should determine which season is the most important to have accurate travel data, and schedule the survey accordingly.

Some recent household travel/activity survey efforts have collected data from respondents in more than one season in a year. The survey data are allowing the sponsoring agencies to compare travel between seasons, and the resulting analyses of the data are likely to describe “average” travel conditions better than a single season survey would. Unfortunately, this approach is not cost-free. First, many agencies do not have the luxury to add six or nine months on to the survey development schedule to spread out the data collection. Second, the cost per completed survey is likely to be higher since there are economies-of-scale related to many survey cost components. For most common survey methods, it is less expensive to conduct one large household survey than several smaller ones. Third, to perform seasonal comparisons, the total sample size is likely to have to be higher, further increasing costs.

To summarize, the selection of the survey season (or seasons) should be based on the following considerations:

• Are the expected analyses of the survey data seasonal in nature, like air quality analyses?

• Are travel patterns in a particular season predictable based on another season’s travel patterns and available interseasonal travel volume information?

• How do respondent contact and cooperation rates vary by season for the different survey methods? What effects do these variations have on survey cost?

• Do time and budget constraints preclude the possibility of collecting the household travel/activity survey data over two or more seasons?

Analysis Time Periods

Just as some analyses are related to particular seasons, many transportation modeling analyses are related to particular days of the week and hours of the day. Based on the anticipated analyses, the survey team has three important survey design decisions to make with regard to analysis time periods:

• How much travel or activity data are needed from each respondent or respondent household?

• For which weekday time periods are data needed?

• Are data for weekends also needed?

Based on the analysis needs for the survey data, the survey team must determine the days and hours for which travel or activity information will be sought. In the U.S., household surveys have traditionally asked that respondents record travel or activities over a 24-hour period. Some smaller survey efforts, including surveys in Keene, NH (1991) and Southeastern New Hampshire/Southern Maine (1992) and a survey on Staten Island (1990), have asked for this information only for peak travel periods, but most survey efforts collect the full day information, even when only peak-hour analyses are conducted. In Europe, some travel diary periods are as long as two weeks, but European respondents are generally much more tolerant of survey efforts than North American respondents. Diary periods of this length are not likely to be successful in the U.S.

A few recent major household travel/activity survey efforts in the U.S. have asked for the data for 48-hour periods. These surveys have sought to describe day-to-day variation in activities and travel behavior. Although some of the second day trip information is duplicative of the first day information, the surveyors have found that the multi-day survey data better explains day-to-day variation in household and personal trip generation rates, and provides more mode choice data. In addition, the multi-day diaries can provide the survey team with insights about travel behavior that one-day diaries cannot.

The primary reservations that surveyors express about multi-day diaries is that the increased respondent burden of the multi-day diaries will lead to higher fatigue levels and higher non-response rates. In addition, many surveyors worry that because of the fatigue factor, respondents would be more likely to under-report trips and activities on the later days of the multi-day diary. Research on the subject confirms that this is a problem for longer travel periods (seven days or more), but the evidence on two and three day diary periods is less conclusive. Lawton and Pas have found that two-day diary periods are not subject to declining trip reporting.⁷ On the other hand, the recent Dallas-Fort Worth pretest data showed that the second day of the 48-hour diary had significantly fewer reported weekday trips than the first day of the 48-hour diary. The pretest also recorded slightly fewer reported trips in the first 24 hours of the two-day diary than in the 24-hour diary.

The collection of multi-day diary data complicates the development of 24 hour travel models because the daily trip patterns within individual diaries are not independent. Analysts need to account for the depend-encies in developing 24-hour travel models, or estimate models for the multi-day period that corresponds to the diary length. Because the recent multi-day diaries have only just been completed, it has not yet been shown whether and how the additional data improve travel models.

In addition to deciding the duration of the diary period, survey teams must also consider for which days of the week to seek the travel and activity data. Most transportation planning analyses have traditionally sought to describe an average weekday’s conditions. This has led most surveys teams to seek travel information for Tuesdays, Wednesdays, or Thursdays of non-Holiday weeks. In recent years, a number of agencies have identified the need for analyses based on special conditions, such as weekends or Friday afternoon peak periods. Household travel/activity surveys need to reflect these analysis requirements, and travel survey teams need to consider the effect that these special investigations have on required sample sizes, respondent requirements, and survey cost.

Selection of the Survey Method

Once the effects of the likely analyses of the survey results are well-understood, the most basic survey design issue for the household travel/activity survey is the selection of the survey method to be used. This decision needs to be based on the strengths and weaknesses of the different survey methods and the overall goals of the survey team. Most survey implementation issues that will be encountered (and are discussed in this chapter) will relate back to the basic selection of the survey method, and conversely the selection of the survey method should be guided by the survey team’s preliminary evaluation of later key issues.

The Components of A Household Travel/Activity Survey

To define the universe of available survey method options for household travel/activity surveys, it is useful to consider the fieldwork components of a household survey separately. Household surveys consist of the following key components:

• Screening and Recruitment – Enlisting the cooperation of potential respondents and ensuring that a contact meets the geographic and demographic requirements of the study (as needed by the travel demand models);

• Distribution of Materials – Delivery of survey forms and related documents to respondents; and

• Collection or Retrieval of Survey Responses – Obtaining the survey responses from the respondents.

The different survey methods can be defined by how they accomplish each of these component tasks. Some methods combine the basic components. Others do not include one of the components. However, decisions about the three main components, screening and recruitment, distribution of materials, and collection of survey responses, define the available survey methods.

Commonly Used Household Survey Methods

Based on the strengths and weaknesses of the data collection procedures for each survey component, surveyors have applied many combinations of recruitment, materials distribution, and data retrieval in their survey designs. As Figure 6.1 shows, combining all the different methods for each component of the household travel/activity survey yields more than a dozen feasible survey methods.

The household travel/activity survey team may want to consider the strengths and weaknesses of each feasible method for their particular survey effort. However, because most of these methods have not been proven to be efficient for household travel/activity surveys, this Manual focuses on only a few of the methods listed above.

Tables 6.2 through 6.7 summarize the most relevant household travel/
activity survey methods. Table 6.2 discusses the simple single contact telephone survey, and Table 6.3 describes the basic mail survey. Table 6.4 and Table 6.5 describe the two most common combinations of mail and telephone survey methods for household travel/activity surveys, the telephone-mailout-mailback survey and the telephone-mail-telephone survey. These four survey methods are the primary focus of the remaining discussion of household travel/activity surveys.

Table 6.6 describes the simple in-home survey that was commonly used in the 1960s household travel surveys. Table 6.7 summarizes the two stage in-home survey method that was developed as an extension to the traditional in-home survey when the need for travel diaries was recognized. As Tables 6.6 and 6.7 indicate, in-home methods for household travel/ activity surveys are probably relevant only in very specialized situations. The use of these two methods is not recommended for most new travel surveys.

Selection of Data Collection Techniques for Household Travel/Activity Surveys

Once the survey team has selected one or more methods for further survey design, the next survey design task is to determine the best data collection techniques for each method.

As we discuss below, the quality of data collection using mailback surveys can be enhanced by a number of design factors, but the data collection techniques for these types of surveys are essentially the same. The respondent is expected to complete the survey materials as instructed and to send the completed forms back to the survey team.

Figure 6.1

Table 6.2 (1 of 2)

Table 6.2 (2 of 2)

Table 6.3 (1 of 2)

Table 6.3 (2 of 2)

Table 6.4 (1 of 2)

Table 6.4 (2 of 2)

Table 6.5 (1 of 2)

Table 6.5 (2 of 2)

Table 6.6

Table 6.7 (1 of 2)

Table 6.7 (2 of 2)

Collecting data by interviewing respondents, either by phone or in person, can be accomplished in more than one way. If the survey team is considering one of the interview techniques for the household travel/activity survey, they will need to make the following decisions about the data collection techniques:

• Is the use of qualitative survey techniques viable or desirable for the survey effort?

• For telephone surveys, should centralized interviewing facilities be used?

• Should computer-assisted interviewing techniques be employed?

Qualitative Survey Techniques

Typical travel surveys are designed to be highly structured. Whenever possible, respondents are asked to answer closed-ended questions that have predetermined response categories. Sometimes a few open-ended questions are included in the surveys, but usually only when absolutely necessary. For almost all travel modeling applications, a highly structured survey instrument is necessary or at least highly desirable.

However, some planners outside of the U.S. have found that removing the tight structure of the interview is an effective way to obtain information about how respondents actually think and believe about certain issues.⁸ These planners have developed and applied household travel surveys that rely on unstructured (or semi-structured) interactions between respondents and interviewers.

Qualitative surveys (or interactive surveys) are in-depth interviews where respondents’ answers are used to guide the format and topics of the interview. They are similar to focus group discussions, except they are conducted on a one-on-one basis, either in-person or by telephone. Interviewers probe and ask supplementary questions about the most interesting topics raised in the interview, and in some cases the interviewer will ask purposely biased questions to challenge the strength of a response or to clarify the respondent’s opinions. Typically, interviewers work from discussion guides, rather than questionnaires, and the interviews are tape recorded so that responses can be analyzed in detail at a later date.⁹

Unfortunately, the costs related to qualitative surveys and the special skills needed to perform them often make these surveys infeasible. For interactive surveys to be successful, the interviewers have to be highly skilled and must understand the survey topic and the issues facing the sponsoring agency. Therefore, the number of interviewers that are able to perform these types of surveys is small. In addition, the analyses of the tape recorded interviews requires special talents and a significant amount of time.

In general, travel demand models are designed to use structured data, so transportation planners typically do not see any reason to perform qualitative surveys. However, special household travel/activity surveys that are seeking to obtain large amounts of opinion and attitude data could benefit from an interactive approach. Some of the next generation travel models, like TRANSIMS, will likely benefit from the data available from qualitative surveys.

The Use of Centralized Telephone Interviewing Facilities

A telephone interviewer can complete his or her task from virtually any telephone. Many early telephone surveys were conducted from interviewer homes or offices. In these cases, each telephone interviewer was given a subset of the telephone numbers in the sample. The interviewer would contact as many of the households as possible, and then after a pre-specified time they would deliver the completed survey instruments to survey managers.

A very serious problem with this approach is that the ability to supervise interviewers as they conduct the surveys is lost. The survey team needs to rely on the skill and professionalism of the interviewers to conduct the surveys correctly and without biasing results.

For this reason, it is recommended that all travel telephone surveys be conducted from centralized locations with supervisors. Supervisors are able to observe interviewers while they work to ensure that they are following procedures correctly, and if problems are identified, they can be rectified immediately. Interviewers are able to ask questions if needed, and if a respondent wishes to speak to a supervisor to verify the authenticity of the survey or to complain, they can be easily transferred. In addition, interviewers at centralized locations are able to learn from each other as they conduct the interviews.

Using a centralized telephone interviewing facility also allows the survey team to establish regulations on telephone interviewing hours. A common complaint that telephone survey respondents (and non-respondents) have is that they were contacted too late at night or at an inconvenient time. Professional marketing research firms usually have guidelines with regard to calling times. For instance, many firms avoid making calls after 9:00 p.m. Different limits on calling times are likely to be appropriate for different survey populations, so survey teams need to establish the regulations individually for each study.

The central telephone survey location can be either a professional marketing research interviewing facility or a temporary facility fashioned out of an agency’s or firm’s office. Since most telephone interviewing is conducted in the evenings and on weekends, it is possible to transform an office into a primitive telephone interviewing facility during off-hours and then switch back to an office in time for regular business hours. Open plan offices with individual phone lines, which are currently quite common, are especially easy to turn into telephone interviewing facilities.

The facilities are not ideal, however, because monitoring calls and providing general supervision are somewhat difficult. In addition, interviews that require toll calls are best handled from professional facilities with WATS lines and other more sophisticated telephone equipment.

The Use of Computer-Assisted Interviewing Techniques

In the past, the most common technique used to record the results of personal interviews and telephone interviews was the pencil-and-paper interview (PAPI), in which:

1. Interviewers record answers on survey instruments or on interview schedules;

2. Trained coders translate the answers into codes, and record them on coding sheets; and

3. Data entry specialists enter the codes into a computer data file.

However, the widespread availability of desktop and notebook computers has led to the development and wide acceptance of computer-assisted telephone interviewing (CATI) and computer-assisted personal interviewing (CAPI) software. Most household travel/activity surveys in the last few years that have involved telephone interviewing have been performed using CATI techniques.

CATI Advantages

CATI reduces the three step data collection-coding-data entry process into one automated, on-line procedure. A computer screen prompts an interviewer to ask a question, then the interviewer records the response, and the computer codes it and saves it to a data file.

CATI techniques have the following interviewing advantages:^10,11

1. They can be designed to permit the entry of only legal codes in any particular field (preventing many data entry errors);

2. They can be used to check entries to make sure that they are consistent with other previously entered data (preventing data inconsistencies);

3. They automatically route interviewers through the interview (ensuring that respondents are asked all the relevant questions and are not asked ones that should be skipped);

4. They can use information from previous questions or previous interviews to make interview questions or the sequencing of questions specific to a particular respondent; and

5. They can be used to help combine the survey’s data collection and management functions; for example, once a telephone interviewer has finished with one respondent, the CATI system can check whether she or he has arranged to return a call to another number, or search the non-contacted numbers for instances where the current time has not yet been tried.

Appendix C shows an example of a recent household travel survey that illustrates the advantages of using a CATI approach.

In addition to improving interviewing capabilities and reducing editing and coding requirements, CATI systems have a number of other advantages, including:¹²

• Sample Management – The CATI system maintains the sample status of each case and links input data to the interview and output record.

• On-Line Call Scheduling and Case Management – The CATI system sets the priority, sequence, and timing of calls.

• On-Line Monitoring – The CATI system is able to reproduce any interviewer’s screen at a supervisor’s terminal where audio monitoring may occur as well.

• Automatic Recordkeeping – The CATI system stores information on on-line calls, their outcomes, response rates, and interviewer productivity, and makes the information accessible to managers in on-line and printed reports.

CATI Disadvantages

As noted above, CATI systems are now commonly used for household travel/activity surveys. However, despite the advantages of the computer-assisted techniques discussed above, there are also negative aspects of the computer-assisted technologies.

First, CATI surveys require a great deal of lead time so that they can be programmed to produce the desired range-checking, question sequencing, and calculations. The CATI programs need to be perfect before the survey is fielded, because interviewers will not generally be able to fix them as they go along. Testing and debugging complex CATI programs could take several weeks and require well over a person-month to complete.

Second, even though the systems can be taught to accept only answers that fall within an acceptable range, they cannot control the quality of data entry. When a CATI interview is completed, the only record of the interview is the data file. There are no source records like in pencil and paper interviews to verify that the survey data was entered into the computer accurately. In addition, it is often difficult for interviewers to include special notes or extra information.

Third, if the CATI program is not carefully designed so that interviewers can avoid collecting duplicative information and can insert missing information from previous responses, the CATI interview can take longer than a pencil-and-paper interview. The paper-and-pencil phone retrieval of Portland’s two-day diary survey took about the same respondent time as the CATI retrieval of a one-day diary pretest in Dallas/Forth Worth. The diary format for the Dallas survey was subsequently modified to allow the CATI data collection to run much more smoothly.

Finally, CATI systems are highly specialized software routines. Most agencies do not have the resources to develop software of this nature in house, so by selecting to use computer-assisted methods, a survey team is probably also ensuring that they will need to enlist marketing research contractors for the survey effort. Most CATI household travel/activity survey efforts have used the commercial packages that the marketing research contractors purchased or licensed and have customized for collecting data.

It is possible to combine CATI and PAPI techniques within the same survey effort. Some recent household travel/activity surveys have used CATI techniques for recruitment, but PAPI techniques for data retrieval. It is also possible to combine the techniques within the same survey, such as by using CATI to retrieve household and person record information and PAPI to collect trip and activity diary information.

Procedures to Enhance the Accuracy of Household Travel/Activity Surveys

Along with determining the survey methods and data collection techniques to be used for the household travel/activity survey, the survey team needs to consider the different available procedures for managing survey bias and inaccuracy. Chapters 4.0 and 5.0 identified the following sources of survey bias:

• Misidentification of the survey population;

• Imperfect sampling frames and sampling loss;

• Non-response;

• Poor questions and survey instruments;

• Fieldworker and interviewer errors; and

• Coding, data entry, and data processing errors.

Procedures to minimize the last three items in household travel/activity surveys are discussed in detail later in the chapter, but if the survey team is to effectively reduce the biases associated with the first three sources of bias, it will be necessary to address them while the survey is first being designed. Procedures to improve household travel/activity survey accuracy should be viewed as integral to the survey design, rather than “extras,” and the costs associated with these procedures should be considered before the final selection of survey methods and techniques are made.

Procedures for Improving the Identification of the Survey Population

The survey population for a household travel/activity survey is either the collection of all households within a study area or some collection of the people who live within those households. In designing the household travel/activity survey, the survey team must:

• Ensure that the anticipated analyses can be accomplished with household-based data, as opposed to data based on other sampling units like trips within a particular analysis corridor; and

• Define the boundaries of the study area for which analyses will be required.

In most cases, these concerns will have been addressed prior to the detailed household travel/activity survey design. Presumably, the anticipated analyses have led to the need for household-based data because otherwise, different (and usually less expensive) types of surveys would be considered. In addition, the study area for the survey is usually set independently of the survey design effort based on particular analyses needs and political boundaries. Definition of the study area boundary is discussed in a greater detail in Chapter 7.0.

Most regional agencies define the geographical extent of their survey by county (political) boundaries. Often this is expedient since Metropolitan areas are defined by county boundaries, and it is easy for executive boards to understand. In cases where a county may extend very far beyond the urbanized area boundary, a cordon line may be used to determine areas for inclusion or exclusion in the survey.

Procedures for Improving the Sampling Frame and For Reducing Sampling Loss

It is likely that the household travel/activity survey team will be faced with an imperfect sampling frame. Because all the most common address-based and telephone-based sampling frames are designed for other purposes, it is not surprising to find that they often need to be cleaned, edited, and augmented for the survey effort. The most common procedures for improving the sampling frame for a household travel/activity survey include:

• Field validation of address-based data sources;

• Combination and cross-checking of two or more sampling frame databases; and

• Special efforts to include identifiable underrepresented groups.

Procedures for Reducing Non-Response

Survey non-response is commonly categorized into unit non-response, referring to the failure of potential respondents to reply to the survey as a whole, and item non-response, referring to respondents’ failure to respond to particular items on the survey. Methods to reduce item non-response are discussed later in the description of questionnaire design. Methods to reduce unit non-response are described in this section.

Four general approaches are commonly used for reducing unit non-response in household travel surveys:

• Pre-notification of the survey effort;

• Survey follow-up techniques;

• Offering potential respondents tangible incentives to complete the survey; and

• Response facilitators (elements of the mail or telephone surveys that decrease the likelihood that potential respondents will refuse to participate).

These approaches can all have a large effect on the overall design and cost of the household travel/activity survey effort.

Pre-Notification as a Method of Improving Survey Response

Pre-notification of the household travel/activity survey consists of contacting potential respondents by telephone or mail prior to soliciting participation in the survey. The pre-notification contact is used by surveyors to build respondent interest in the survey effort, and to help allay respondent doubts about the validity of the survey. There is evidence that pre-notification improves survey response rates, response speeds, and response quality.¹³ Another potential benefit of pre-notification is that it can provide an early measure of likely response rates and non-response trends.

Pre-notification can be used for household surveys with mail, telephone, or in-person methods, or any combination. In theory, the pre-notification contact can be accomplished in any of the three common ways:

1. Telephone pre-contact;

2. Pre-contact with a letter, brochure, or postcard; and

3. Face-to-face personal pre-contact.

However, in general, the cost of in-person surveying precludes this approach as a pre-notification procedure. In addition, it is not common for telephone pre-notification to be used prior to household surveys with telephone recruitment. In this situation, mail pre-notification or no pre-notification at all are more commonly used. The short recruitment call probably achieves many of the same goals of the telephone pre-notification.

Pre-notification of some type is usually always warranted in the case of mail surveys and surveys with in-home recruitment. Since the sampling frames for these surveys are usually address-based, respondent phone numbers are generally not known. Therefore, the most common approach is to send a postcard or letter of introduction.

In a sense, pre-notification is a sales technique to convince potential respondents to participate in the survey effort. Consequently, the most successful pre-notification efforts tend to employ sales techniques.

A few recent household travel/activity surveys have used formal pre-notification techniques, and those that have seem to have benefited from it. For instance, prior to conducting recruitment calls for their travel survey, the Metropolitan Washington Council of Governments (MWCOG) sent out an introductory letter signed by the directors of the Departments of Transportation in the region. The letter simply provided an overview of the survey and the study, and asked for the recipients’ participation in the upcoming survey. MWCOG estimates that the pre-notification letter increased survey participation by between five and ten percent.¹⁴

Survey Follow-up Techniques for Improving Survey Response

One of the most effective ways to reduce survey non-response is to follow-up with respondents who do not complete the survey. Survey follow-up procedures are generally used with mailback surveys, but the concept can be applied to telephone and in-home surveys, as well.

Survey Follow-Up for Mail and Telephone-Mailout-Mailback Surveys

Mail survey follow-up techniques are used for two reasons:

• To clarify responses on returned questionnaires (corrects item non-response); and

• To convert refusals and other non-responses into completed usable responses (corrects overall non-response).

Follow-Up for Item Non-response and for Clarification of Responses
in Mail and Telephone-Mailout-Mailback Surveys

Clarification of responses is generally done by phone to expedite the process and to ensure that the corrected/edited responses are adequate. In telephone-mailout-mailback designs, the respondent has been recruited by phone, so it is relatively easy to recontact him or her to ask about specific responses (provided that the responses with the problems do not require the respondent to have any survey materials on hand).

Many recent travel survey efforts have used this technique to clarify and correct spurious, suspicious, or out-of-range answers. In general, the surveyors found that the number of clarification calls needed was small, and that because most problems were quickly corrected or clarified, most follow-up calls were short.

To clarify or correct the responses on simple mailout-mailback surveys by telephone, it may be necessary to request telephone contact information from respondents. Ironically, asking for this information to correct item non-response may actually increase the overall non-response rate because of people’s confidentiality concerns. Surveyors may be able to determine some respondents’ telephone numbers from telephone directories and/or reverse directories, but if this approach is adopted, the surveyor must understand that she or he could end up with different quality data for those with listed numbers and those without listed numbers. Since the problems that need to be clarified will probably be minor, the potential bias is generally ignored.

Follow-Up for Overall Non-response in Mail and Telephone-Mailout-Mailback Surveys

The second type of follow-up survey seeks to increase the overall response of the survey by reminding non-respondents that they have not yet completed the survey. Because the overall response rate for mailback surveys is generally fairly low, follow-up techniques are often used to increase the response. Fowler claims that:

“While attractive presentation of the study and good questionnaire design will help, there is no question that the most important difference between good mail surveys and poor mail surveys is the extent to which researchers make repeated contact with non-respondents.”¹⁵

There are several different follow-up approaches for mailout-mailback surveys, including:

• Follow-Up Postcards – respondents are sent a reminder postcard stressing the importance of their responses to the survey;

• Follow-Up Letters – respondents are sent a brief letter (usually from an elected official, such as the one who signs the cover letter for the initial mailing) restating the goals of the survey and its importance;

• New Survey Materials – respondents are sent a new set of survey materials under the assumption that they have misplaced the original set;

• Telephone Reminders – respondents are called, reminded about the survey and are usually asked if they need a new set of survey materials;

• Telephone Retrieval – respondents are called and asked to provide the survey information by telephone; and

• Combinations of any or all of the above.

The best follow-up method will depend on the available budget, available time, the initial response rate, and the surveyor’s level of concern about non-response. Experts differ on the best approach.

The following mail survey sequence is recommended:^16,17,18

1. Send pre-notification letter one week prior to the initial survey mailing or recruitment call;

2. Recruitment call (if chosen method requires it);

3. Initial survey mailing;

4. Send postcard reminder or make telephone reminder call one week after initial mailing;

5. Send letter and new materials three weeks after initial mailing;

6. Send letter reminder four weeks after initial mailing; and

7. If response rate is still unsatisfactory, after six weeks send letter and new materials, or make telephone reminder calls for respondents with listed numbers.

Peterson, Albaum, and Kerin recently compared 27 alternative pre-notification and follow-up strategies for mailout-mailback surveys.¹⁹ Their results are particularly interesting because they used a survey instrument that was designed to generate relatively low response rates, similar to mailed household travel and activity surveys. They compared the contact strategies based on response rates and cost per completed response. Figure 6.2 summarizes some of their findings.

The simple mail survey without pre-notification or follow-up yielded a 10 percent net response rate at a cost of $6 per completed response. Introducing pre-notification increased the response rate to 11 percent (for postcard notification) and 14 percent (for letter notification), and increased survey costs to $8 per completed response. Introducing a single follow-up contact without pre-notification produced a 13 percent return at a cost of $9 per response for postcard follow-up, and an 18 percent return at a cost of $8 per response for a follow-up letter with a copy of the questionnaire. Combining pre-notification and a single follow-up contact produced response rates between 15 and 20 percent at costs between $8 and $10 per response.

As the figure shows, the most successful strategies (in terms of response rate) involved pre-notification and two follow-up contacts. The cost per completed response for these strategies are slightly higher than the simple survey effort, but the response rates were more than double the simple effort. At these low response levels, the higher response rates almost certainly would outweigh the slightly higher costs.

Richardson, Ampt, and Meyburg also conclude that pre-notification and follow-up are cost effective investments for household travel/activity mail surveys.²⁰ Table 6.8 shows a cost comparison based on a 1993 Australian

Figure 6.2

Table 6.8

household mail survey. The top of the table shows the costs of a non-follow-up survey design which yields a total of 6,000 returns. The bottom half of the table shows the costs for the survey design the authors recommend that also yields 6,000 returns. The survey with the extensive follow-up is estimated to actually cost less. The survey with follow-up also has a higher response rate, perhaps reducing the amount of bias.

Survey Follow-Up for Telephone and Telephone-Mail-Telephone Surveys

For telephone and in-home surveys, overall non-response occurs because of the surveyors inability to contact potential respondents, or because potential respondents refuse to participate in the survey. Therefore, non-response-reducing strategies have been designed primarily for the recruitment stage of the survey, rather than for the retrieval follow-up stage.

While item non-response is as much or more of a problem with interview surveys as it is for mailback surveys, for the most part it is dealt with during the actual interview. If a respondent is unable or unwilling to answer a specific question, an interviewer probes for an answer or further explains the question. Follow-up contacts are not likely to improve the quality of the responses that the initial interviewer is able to get. This is particularly true if the interviewer is well-trained and the CAPI or CATI software is designed well to trap inconsistencies, illogical answers, and errors.

Still, it is relatively easy and common to recontact respondents by telephone to correct problems discovered after the interview. Since the respondent has already invested a great deal of his or her time into the interview, clarifying a few questions is generally not a problem. On the other hand, respondents are likely to get tired of re-answering questions so follow-up contacts need to be short. If a response has so many questions or problems that it would require more than a few follow-up questions, the surveyor should probably classify the response as unusable.

Survey Follow-up Considerations for Diary Surveys

Travel and activity diaries usually ask respondents to record their travel or activities over a pre-specified period. If a respondent fails to complete the diary during that period or immediately following the period, she or he is more likely to forget about certain travel or travel details. Consequently, in follow-up contracts, most survey teams ask respondents to consider a different upcoming day (or days) when completing the diary. While this method is probably preferable to asking respondents to remember travel and activities a day or a week or even more in the past, it is still not optimal. In expanding the data, the survey team may need to consider the differences in travel conditions between the desired and actual diary periods.

In addition, when reassigning diary periods for respondents, the survey team should consider potential inconsistencies between the original and new periods. For instance, if schools are in session during the original diary period, they should also be in session on the new date. Usually, the follow-up contact asks respondents to use the same day or days of the week as the original period as soon as possible after the original period.

The Use of Survey Follow-Up to Measure and Correct for Non-response Bias

The primary goal of using survey follow-up techniques in household travel/activity surveys is to reduce the level of non-response in the survey effort. Another possible advantage of conducting the follow-up is that it provides the survey team with a means to infer the characteristics of non-respondents and perhaps to even make corrections. Methods for performing these procedures are discussed in Section 6.12.

Incentives for Survey Methods

Surveyors often provide respondents with incentives of one type or another to motivate them to participate in their survey efforts. The most common incentives that are employed are:

• Prepaid Cash – some denomination sent to the potential respondents with the survey materials;

• Promised Cash – an offer in which a specified amount of money would be provided upon completion of the survey;

• Provided Gifts – a gift, such as a pen, key ring, or refrigerator magnet, enclosed with the survey materials;

• Promised Gifts – an offer to provide the potential respondent with a specified gift upon completion of the survey;

• Lottery – the inclusion of the potential respondent in a lottery drawing;

• Study Results – respondent is promised survey results upon completion of the study;

• Charitable Contribution – prepaid or promised donation of a specified dollar amount to a charity in the name of the potential respondent.

Travel survey specialists, like their general marketing research colleagues, have mixed views on the cost-effectiveness of incentives. Their usefulness is probably related to the population of the region under study, so broad generalizations about their effectiveness are difficult to make. However, it is apparent that incentives do improve response rates and speeds in many cases. The remaining question for the survey designer is whether the benefits of incentives outweigh the investment in providing them and the potential biases that they may cause.

Based on the recent literature, the prepaid cash incentive is the most consistent incentive method for improving response rates. It is also considered the least biasing of available incentives as well as easiest to use. This conclusion is supported by evidence from household travel/activity surveys, such as the Puget Sound Transportation Panel Survey. In this effort, three incentive approaches were used; 1) no incentive, 2) $1.00 per household member prepaid, and 3) $10.00 per household promised incentive. The two groups that received incentives each had diary rates of slightly more than 60 percent, compared to a return rate of 49 percent for the group not receiving the incentive.^{21 22 23 24}

Experience with monetary incentives has revealed that incentives need not be substantial. The incentive should be a small token of appreciation for the respondents’ efforts. Ideally, it should build rapport between surveyors and respondents, and it should motivate respondents to try to please the survey sponsors. Larger incentives, especially in the promised form, take on the feeling of payment for one’s time, and for complex household travel/activity surveys, even relatively high payments are not likely to be adequate compensation for many respondents.

Despite the advantages of the prepaid monetary incentive, there are conditions when another incentive type is more reasonable. Agencies may be able to provide other types of incentives more cost-effectively, or may have reasons for not wanting to provide the pre-paid incentive. Sometimes agencies can obtain suitable gifts, such as pens, maps, or refrigerator magnets, at no cost or reduced cost. Gift incentives would probably be more cost-effective in these cases. A recent household activity survey in the Boston region (an area with a high rate of state lottery participation) offered vouchers for a $1.00 state lottery ticket, in part because it did not require the agency sponsoring the survey to send cash incentives to people at a time of state government cutbacks.

Although incentives of all types are used to increase survey response, evidence suggests that incentives do not have the same appeal for all respondents. Biases can be created when incentives are used. No known studies relate incentive conditions to survey measures of respondent travel, but incentives are known to have different appeals based on the respondent’s sex, marital status, employment status, property ownership, and religion.²⁵ Therefore it is reasonable to assume that trip generation estimates could be affected by the use of incentives, as well. Some travel survey experts do not recommend the use of incentives because they feel the risk of bias outweighs the potential improvement in response.

Response Facilitators

Although the use of incentives is the most well-known mechanism for increasing survey response, it is likely that other survey considerations will have as large or larger effects on survey response and quality. Based on their experiences and intuitions, survey researchers have developed a number of survey response facilitators that they believe increase the likelihood of survey participation. It is not clear how much these facilitators affect response rates, because researchers have difficulty isolating them from other aspects of the survey. However, most survey designers stand by one or more of them.

During the survey design, the household survey team should decide which facilitators are most likely to be important for their survey population, and they should estimate the costs of providing them.

As Dillman points out:²⁶

Non-response is a serious problem under any circumstances. Thus each element that might help prevent it – no matter how trivial – is worthy of design considerations.

Response facilitators include the following:

Mail Survey Response Facilitators

• Include a cover letter signed by a high-ranking and popular elected official.

• Personalize the survey materials for each respondent, where possible.

• Use postage stamps on any packages sent to respondents, rather than prepaid or machine stamped mailings, so the mailing stands out from direct mail.

• Send materials in distinctive envelopes.

• Provide a toll-free telephone number for respondents to call in case they have questions or complaints.

• Have the return address(es) be within the region under study.

• Have the return address(es) be for the agency or another public organization, rather than for a private firm.

• Provide the respondent with a deadline for replying to the survey.

• Provide brief reassurances of anonymity on the survey materials.

• Provide descriptions on the survey materials of the importance of the survey and of the specific respondent’s role in the survey.

Telephone Survey Response Facilitators

• Make sure interviewers have local accents or are relatively accent-free.

• Provide reassurances of anonymity at the beginning of the call.

• Provide descriptions of the importance of the survey and of the specific respondent’s role in the survey.

• Provide a toll-free telephone number for respondents to call in case they have questions or complaints.

In-Person Survey Response Facilitators

• Select interviewers that are of the same age groups, races, ethnic backgrounds, and social classes of potential respondents.

• Provide reassurances of anonymity at the beginning of the interview.

• Provide descriptions of the importance of the survey and of the specific respondent’s role in the survey.

• Provide a toll-free telephone number for respondents to call in case they have questions or complaints.

These mechanisms are all likely to help improve response rates marginally, but the survey team needs to consider the facilitators as a package. Simply selecting a few facilitators to improve response will not be as effective as developing an integrated strategy, using pre-notification, follow-up, incentives and facilitators that work well together and complement one another.

Output of the Survey Design Task

By the time the survey team completes the survey design task, they will have analyzed the output data needs from the household travel/activity survey, and made decisions about the survey method, data collection techniques, and the inclusion of different design elements to improve the quality of the survey results. The survey team will have a clear idea of the approach (or approaches) that will need to be pretested.

The survey design task outputs will feed directly into the sampling, survey organization, and survey materials development tasks, but, in reality, the survey design task will guide all the work conducted on the rest of the tasks.

It can be helpful at this state of the survey implementation process to prepare a detailed plan for the household travel/activity survey. The survey team will be in a position to define detailed survey procedures and to layout more accurate schedules and budgets. The detailed survey plan is a useful document for involving outside agencies and/or technical advisory committees in the development of the household travel/activity survey. In addition, the plan organizes the survey team’s tasks, and can be an effective tool for allocating responsibilities.

By the time the survey design task is winding down, it is likely that the survey team will already have gotten underway on the organization and sampling tasks, which are discussed next.

6.4 Organizing the Household Travel/Activity Survey

 Key Issues in Organizing the Household Travel/Activity Survey

1. What are the staffing needs (numbers of people, required skills) of the household travel/activity survey?

2. How should contractors be selected and used in the survey effort?

3. How should the survey be coordinated with other transportation planning activities and other agencies’ ongoing work?

4. What citizens’ participation and advance publicity efforts should be undertaken?

 Section Summary

Management of the Survey Effort 6-65

Staffing Needs for the Survey 6-65

Hiring Temporary Professionals 6-66

Hiring Contractors 6-66

Agency Coordination 6-70

Advance Publicity 6-70

THIS PAGE INTENTIONALLY LEFT BLANK

 6.4 Organizing the Household Travel/Activity Survey

Closely related to the design of the survey is the need to organize and manage the effort. The section summary page shows the key issues associated with the organization of the household travel/activity survey. The four issues are described in this section.

Management of the Survey Effort

The management structure of travel survey development efforts is discussed briefly in Section 4.3 of this manual. For household travel and activity surveys, the survey team managers must provide:

• Overall management and leadership of the effort;

• Day-to-day management of survey fieldwork; and

• Continuing assessment of the effects of different decisions on the final analyses to be performed with the survey data.

In almost all cases, the overall leadership role is provided by the sponsoring agency’s project manager. Increasingly, the day-to-day management of survey fieldwork is being left to survey subcontractors who are able to provide trained fieldwork staff and specialized facilities. The final management function is often provided by a combination of in-house staff and travel demand consultants.

The use of a peer review panel, as described in Section 4.3, is highly recommended for household travel and activity surveys. If nothing else, these panels provide an extra set of eyes to catch problems before they happen, and they are likely to provide much more, including expertise and experience with most of the challenging issues facing an agency.

Staffing Needs for the Household Travel/Activity Survey

In the early stages of the survey design process, the survey team should scope out the most likely approach to the household survey, and then make a preliminary estimate of the labor and skill requirements of the study. When staff members’ pre-existing schedules are considered, almost all agencies that perform household surveys find the need to temporarily increase staffing. This is generally done in one of two ways:

• By bringing relatively low level temporary professionals or students on to the agency staff for key points of the survey; and/or

• Hiring survey research contractors and other consultants to provide specialty services that agency staff members would have to learn to do.

Hiring Temporary Professionals for a Household Travel/Activity Survey

In most regions, temporary agencies can provide the necessary additional office support people for the survey effort. In addition, many regions have one or more universities whose students could be recruited for temporary work.

These sources may also be able to supply survey fieldworkers for conducting telephone and in-home interviews. However, these people will need to be carefully screened, trained, and briefed on survey interview techniques prior to conducting any interviews. This means that the temporary employee fieldworkers will need to be lined up well in advance of the survey effort, probably three to four months at a minimum.

Once the temporary staff have been hired, it is essential that they receive as much on-the-job-training as possible. Household travel/activity survey workers’ strengths and weaknesses should be well-understood by survey managers prior to the beginning of the survey effort.

Hiring Survey Contractors for a Household Travel/Activity Survey

An easier but sometimes more costly approach to organizing the work force for a household survey is to hire a consultant to perform the survey. Usually, the consultant would be a survey research firm, or a team including such a firm. It may also be advantageous to include a transportation modeling consultant as part of the consultant team, or to have such a consultant available to the agency through a separate contract.

In most cases, the sponsoring agency will not have access to great numbers of trained fieldworkers or to special facilities for centralized telephone interviewing. Survey research firms usually have trained interviewers on their staffs and may maintain telephone interviewing facilities that provide toll-free calling throughout the survey area, CATI capability, and the opportunity for monitoring (either by in-house supervisors or from outside phones that can be used by agency personnel). Because of the need for high quality data for travel modeling purposes and for high response rates to minimize costs, it can be highly efficient to use a survey research firm.

Survey designers can identify potential survey research firms through directories maintained by a number of organizations.¹ Two such directories are:

• GreenBook International Directory of Marketing Research Companies and Services. (American Marketing Association/New York Chapter, Inc., New York, NY). Annual. A listing of market research companies arranged alphabetically, with brief paragraphs that describe the companies’ services. Additional sections list the companies by type of service offered, by market/industry specialty, by computer programs used, by company trademarks/service marks, by geographical area, and by principal personnel.

• MRA Blue Book Research Services Directory. (Marketing Research Association, Inc., Rocky Hill, CT). Annual. More limited in scope than the GreenBook, this guide focuses on services and facilities of data collection companies, research companies and suppliers of related services (data processing, questionnaire coding, field management, etc.) who are members of MRA. Company listings are alphabetical within each geographic area, cross-referenced by the type of service or facility available.

These directories are usually available at business school libraries. In addition, the survey designer can contact other planning agencies that have recently completed similar household travel/activity survey efforts for lists of potential contractors.

The survey sponsoring agency should consider the following factors in selecting a survey research contractor:

• Marketing research experience and qualifications of key staff members;

• Transportation research experience and qualifications of key staff members;

• Household travel survey experience and qualifications of key staff members;

• Range of services offered, including capabilities in research design, sampling statistics, data collection, and statistical analysis;

• Size and quality of interview and other fieldwork staff;

• Interviewer experience levels and pay;

• Interviewer training standards;

• Available facilities, including telephone survey centers and mail processing centers;

• Use of in-house facilities versus contract interviewing facilities (many survey firms contract to other firms to perform telephone and/or in-person interviews);

• CATI and CAPI capabilities and equipment;

• Foreign language interviewing capabilities;

• Coding, editing, and geocoding procedures and capabilities; and

• Quality control procedures and client communications procedures.

Consulting firms with expertise in travel modeling can also provide valuable insights in the survey development process. Such consultants are able to provide an understanding of the data needs and problems associated with model development. In many cases, a consulting team with both market research and transportation firms will be hired to conduct a household survey. In others, an agency may have transportation modeling consultants available through separate arrangements. Some agencies may have sufficient transportation modeling expertise in-house, but unless they have very experienced modelers, there is no way to guarantee that the survey will be appropriate for use in developing model datasets. This has been a substantial problem in several recent surveys.

There are advantages to using qualified local consultants if they exist in the survey area. Surveyors who know the local geography will make fewer errors in recording and spelling local place names. Survey times can be shorter if location information is known to the interviewer, and respondents would be less likely to be exasperated by having to give what to them is obvious information about well known locations. In some cases, respondents may feel more comfortable speaking to interviewers with local accents and knowledge.

The main problem with using local firms is that in many areas, especially small and mid-size areas, there are few if any local firms with sufficient transportation survey knowledge and experience. If a non-local firm is performing the survey, the question then becomes how to provide the necessary local knowledge. In some cases this local knowledge can be provided by agency personnel; in others, it may have to be provided through other consultants. There have been many recent successful survey efforts conducted by non-local firms.

Survey teams have hired survey subcontractors at several different point in the household travel/activity survey implementation process. Sometimes, the contractors are brought into the process early in the survey design phase, so that the survey team can benefit from the market research experience of the survey contractor’s key staff during the evaluation and selection of the survey method, survey techniques, and quality-enhancing procedures. Other times, when the survey sponsor is comfortable with making the design decisions alone, or with the help of independent consultants, the survey subcontracting firm is not brought into the project until the final phases of questionnaire design, just before pretesting.

In either case, it is important that the sponsoring agency recognize the need to carefully delineate the responsibilities of any contractors in the Request for Proposal (RFP) and in the services contract. In preparing the RFP, agency staff should remember that any responsibilities and tasks not explicitly assigned to the contractor will most likely need to be completed by themselves. Therefore, spending extra effort on the RFP is usually worthwhile.

Because RFPs need to be tailored to individual conditions, little specific guidance can be offered on their development. In general, in developing RFPs it is helpful to review recent similar RFPs from other agencies. A list of agencies recently completing household travel/activity surveys is available in the forthcoming “FHWA Scan of Recent Travel Surveys.” The scopes-of-work from recent household travel/activity survey RFPs are shown in Appendix D of this manual.

Because of the nature of survey work, survey firms are not accustomed to establishing a final fixed contract price for a pre-selected number of “complete” households. Estimating contact rates, response rates, interview times, and even the number of surveys needed for specific analyses are usually very difficult prior to the completion of a high-quality pretest. Setting a fixed price prior to that point, while beneficial from a resources planning perspective and an agency procurement perspective, can lead to problems later in the survey. For instance, pretests tend to become pro-forma tasks, rather than opportunities for careful review of procedures and for trying innovative procedures, because if the survey cost is fixed, there is no incentive to look very hard for potential problems.

To avoid these potential problems, it is recommended that agencies consider one or more of the following approaches:

• Select survey contractors primarily on the basis of qualifications and experience, rather than cost.

• Provide detailed surveying parameters with which contractor prices can be compared – if detailed assumptions are not provided in the RFP, proposers may offer cost proposals that are not directly comparable.

Agency Coordination

The need for coordinating travel survey and demand modeling efforts with other local agencies is described in Chapter 4.0. Because the household survey is likely to be the most important survey effort performed in a region, and because household surveys are not (or at least have not been) done on a regular basis, it is essential that agencies work cooperatively on the design and implementation issues.

As soon as possible in the household travel/activity survey development process, the sponsoring agency should contact:

• All affected state agencies;

• Local and regional planning officials;

• Local and regional elected officials;

• Local and state police;

• Federal agencies that may be involved;

• Local transit providers;

• Active public interest groups; and

• Chambers of commerce/business groups (for workplace/establishment surveys).

These agencies should be briefed on the survey plans, and should be provided with the Statement of Goals for the survey. Representatives of these agencies should be invited to participate in the survey development process, and to identify ways in which the survey data could help their organizations’ planning efforts. Many household travel/activity surveys can be easily adapted to provide useful data to many different agencies. However, it is essential that potential data coordination activities be identified early in the survey design effort to minimize the disruption and amount of necessary re-design.

Advance Publicity

The survey designer needs to decide whether and how to publicize the household survey. Generally, telephone-based survey methods are helped by advance publicity. Potential respondents are more likely to believe that a telephone interviewer is legitimate if they have heard that the study would be going on. In addition, respondents are likely to attach a higher level of importance to a survey effort that has been publicized, and therefore consider participating to be more important. A few recent telephone-mail-telephone household survey efforts ran into some criticism in part because the efforts were not well-publicized before they began.

If, for some reason, a survey team is using an in-home interview survey method, they may not want to consider publicizing the effort. The 1973 Travel Survey Manual counsels:²

“Especially in large urban areas where there is the problem of individuals posing as interviewers to gain entrance into households for other purposes, it is often best not to notify the public at large.”

In fact, the potential for this type of abuse is one good reason to avoid in-home methods.

Despite the fact that there are documented reports of thieves posing as household travel/activity survey telephone and in-person interviewers, most agencies sponsoring recent household travel/activity surveys have chosen to use some advance publicity. If advance publicity is determined to be necessary or appropriate, the following efforts could be included:

• An agency press conference explaining how the survey data will be used to improve regional planning or an agency’s planning efforts;

• Press releases for each major survey design milestone;

• Informational meetings with local citizens’ groups and public service organizations, such as the Lions’ Club or the Rotary;

• A project specific newsletter or prominent display within an agency’s regular newsletter; and

• An informational telephone number that respondents can call to contact the agency if they have questions about the survey effort or are concerned about the veracity of the survey effort.

6.5 Sampling for Household Travel and Activity KM Surveys

 Key Issues

1. What variables are of the greatest interest in designing future analyses?

2. How are the study population, sampling frame, and sampling unit defined?

3. Which sampling method should be used to meet the precision requirements?

4. What sample size is required to satisfy these precision requirements?

 Section Summary

Trip Generation 6-75

Trip Distribution 6-78

Mode Choice 6-79

THIS PAGE INTENTIONALLY LEFT BLANK

 6.5 Sampling for Household Travel and Activity KM Surveys

The statistical computations needed to determine sample sizes for travel surveys are described in Chapter 5.0. For household travel surveys used to develop travel demand models such as those maintained by metropolitan planning organizations, the study population is usually known. The sampling unit is the household, and the sampling frame a list of households by telephone number or address.

Urban travel demand model systems include a number of components to be estimated for which the survey data are needed. These include:

• Trip generation;

• Trip distribution; and

• Mode choice.

The variables of interest are different for each of theses models. The sampling requirements for each are discussed below.

Trip Generation

The main variable of interest for trip generation models is the number of trips generated by households for each trip purpose. The household variables generally used in trip production models include number of persons, income, auto ownership, number of workers (for work trips), and number of students (for school trips). Most variables of this type have distributions that can be obtained from census data, providing a good basis for computing sample size requirements. The census, however, provides no information on the number of trips generated (and no information at all on non-work trips).

If information on the mean and variance (or coefficient of variation) of the number of trips generated per household were available from another source – say a previous survey – the required sample size could be computed from Equation 5.7:

where:

 = represents the standard deviation of the population; and

SE(m) = the standard error for the mean for a given confidence level and precision level.

Smith used values from some older (1960s) household surveys to determine the coefficient of variation and computed a typical sample size requirement of about 900-1,200 households at the 90 percent confidence level and a precision level of +/-5 percent.¹ The higher number resulted from an assumed cross-classification in the trip production model by income and auto ownership.

This analysis, however, did not take into consideration different trip purposes; ideally one would compute the required sample size for each purpose in the model and use the largest. With smaller means for the number of trips by purpose, the standard error may be smaller, resulting in larger required sample sizes.

In planning for the 1990 Bay Area household survey effort, the MTC estimated necessary sample sizes by trip purpose using data from their 1981 survey effort. Table 6.9 shows the conversion of trip rate information into sample size estimates for this effort.

Some recent household survey efforts intended for use in developing trip production models have used smaller sample sizes of around 500 households, including surveys in the Portland, Maine and Pittsburgh areas. While information on statistical levels of accuracy and precision have not been reported, it can be assumed that lower levels of one or both were found to be acceptable in these areas.

It is common practice to use a stratified sampling plan for collection of trip generation data. Since trip production models are often cross-classification models, the survey sample can be stratified according to variables in the model such as those described above. Information on existing distributions of the variables is usually available from census data, so required sample sizes can be computed. This is a good strategy for ensuring sufficient sampling of relatively small but important markets such as households without autos.

The main difficulty with such a procedure is that it is impossible to tell which stratum a household is part of until it is recruited. This can be addressed by collecting a larger sample than necessary to account for the expected number of responses in the critical cell or by screening households prior to having them complete the entire survey – basically creating cell quotas.

Table 6.9

Trip Distribution

It is now generally recognized that household travel surveys are not appropriate means of generating acceptable estimates of zone-to-zone trips.² While household surveys taken in the 1960s were generally used for this purpose, they usually had much larger sample sizes, and models had fewer zones. Presently, travel demand modelers use household survey data to estimate parameters of trip distribution models rather than attempt to develop zone-to-zone trip tables directly from the survey data.

Most trip distribution models in U.S. urban areas are gravity models based on travel times. Some areas use generalized cost instead of travel time, but generally gravity models are based on one variable. With that in mind, the variable of interest is the trip length frequency distribution. Again, Equation 5.7 can be used to estimate the required sample size if the coefficient of variation and mean are known. Pearson reported in 1974 coefficients of variation of 0.53 for home-based work trips, 0.58 for home-based non-work trips, and 0.63 for non-home-based trips.³ Using these numbers, samples sizes of about 600-700 trips per purpose would be required at the 90 percent confidence level for the +/-5 percent error level. Since households make several trips per day on average, only a few hundred households would be required to obtain a statistically significant estimate of the mean trip length. Even if travel time estimates are desired for different times of day, as long as there are not a large number of different time periods (most models use one or two, some three or four), there should be enough trips to estimate travel time distributions.

The above discussion leads to the conclusion that any survey which is sufficient for the development of trip generation models is likely also sufficient for the estimation of gravity model parameters.

Some agencies have been developing destination choice (or more accurately attraction choice) models for the purposes of estimating trip tables. These are generally logit models which are similar in function, and often variables, to gravity models. If travel time is the only parameter of these models, then the same analysis as described above for gravity models holds. However, if other variables are used in the model, the problem becomes similar to that of mode choice models, as discussed below.

Mode Choice

In most urban areas, the use of household travel surveys for the estimation of mode choice models is problematic for the following reasons:

• In many areas, there are simply too few transit trips to get an accurate estimate of the distribution of important variables among transit users.

• Unless households are recruited at transit stops or on transit vehicles, it is difficult to determine in advance whether or not there are transit trips made by the household. Therefore, a stratified sampling approach with respect to mode would be difficult to implement.

• It is unlikely to have information about the means, standard deviations, or coefficients of variation of most of the variables in mode choice models unless another survey had collected them. These variables include fares, parking and other auto-related costs, and wait and access times. In addition, data on other variables such as demographic or area type measures are unlikely to be available weighted by trip (as opposed to by household).

• The logit model formulation does not lend itself to simple derivation of statistical computation of sample size.

Given these problems, it is rare that a household travel survey sample size would be based on mode choice model requirements. However, in some large cities where mode choice models can be developed from survey data, it is likely that the sample size would have to be much larger than what would be required for trip generation.

It is possible to develop simple estimates of the required sample size to get a statistically significant sample of users for each mode using Equation 5.7. This is done using the sample variance s² in the equation based on the estimated mode share. For work trips, this is generally available from census data; for non-work trips, a conservative (high) estimate of transit (or other rarely used mode) share can be used to develop a conservative estimate of the sample size requirement. Of course, if there are a large number of modes to be examined, the computation must be repeated for each one.

Stratification of the sample can be an efficient means of increasing the accuracy of the survey data for mode choice purposes. Obviously, if one could identify transit users before recruitment, better information about critical variables for transit users could be obtained. Even though this pre-selection would be very difficult, it is possible to target specific markets that are easier to define. For example, in Portland, Oregon⁴ the household survey was stratified to include areas near transit lines and with favorable land use characteristics for non-auto modes. Such geographic stratification is not difficult to determine using readily available data such as census data for planners familiar with transportation in the local area. In addition, the Portland survey employed choice-based sampling for one stratum, recruiting park-and-ride users at parking lots.

6.6 Drafting and Constructing Household Travel/Activity Surveys

1. What data elements are needed from the Household Travel/Activity Survey and what limitations are there in obtaining the data?

2. What survey instruments are needed for the survey? How should they be designed?

3. How can the required data elements be developed into questions and response categories?

 Section Summary

Data Elements 6-83

Household Data Elements 6-84

Person Data Elements 6-87

Vehicle Data Elements 6-87

Travel and Activity Data Elements 6-91

Attitudinal, Opinion, Knowledge, and Stated Response
Data Elements 6-94

Translating Data Elements Into Questions and
Response Categories 6-96

Question Content and Form 6-97

The Wording of Questions and Response Categories 6-100

Wording Problems 6-101

Special Wording Issues 6-106

Wording Income Questions 6-106

Wording Questions about People’s Property 6-109

Diaries 6-109

Diary Types and Examples 6-110

Selecting Diary Type 6-120

Level of Detail for Diary Questions 6-120

Additional Diary Design Issues 6-122

Sequence of Survey Questions 6-122

6.6 Drafting and Constructing Household
Travel/Activity Surveys

(continued)

Survey Instruments and Materials 6-125

Materials for Any Survey Method 6-129

Pre-Notification Letter, Brochure, Postcard
or Interview Script 6-129

Thank You Card 6-130

Follow-up Letters and Postcards 6-130

Mail Survey Materials 6-131

Pre-Notification Letter, Brochure, or Postcard 6-128

Envelope 6-131

Cover Letter 6-132

Fact Sheet 6-135

Questionnaire 6-135

Household and Vehicle Forms 6-141

Diaries 6-141

Memory Joggers 6-144

Reminder Cards 6-145

Forms for Special Questions 6-145

Cover Letter for Follow-up Survey 6-148

Follow-Up Survey Materials 6-148

Telephone Surveys 6-148

 6.6 Drafting and Constructing Household Travel/Activity Surveys

The household survey is probably the best travel survey for obtaining the most detailed data on respondents and their travel patterns. As discussed in Chapter 3.0, the household survey can include almost any type of survey question. In addition, the survey may include either interviews or self-administered questionnaires, or both. Household survey interviews can either be computer-assisted or manual.

The key issues related to drafting household survey questions and constructing the survey instruments for a household travel/activity survey are listed on the section summary page. The first challenge for the survey team is to determine what data elements are needed from the household travel/activity survey. This determination must be based on the anticipated analyses and the survey goals, but the data elements of other recent household travel/activity surveys will help the survey team narrow its selection. Once the data elements are identified, they need to be developed into survey questions and response categories. At the same time, the survey team needs to identify the different survey instruments and materials that will be needed for the survey effort. The final product of this task will be the survey materials, which will be a combination of the products of the three steps.

The three key steps of drafting and constructing household travel/activity surveys are described below.

Data Elements For Household Travel/Activity Surveys

In most household survey designs, the survey team is in the position to obtain information on a great number of relevant topics. The analysis and travel demand modeling plans for the survey results and the survey’s overall goals will dictate what specific data elements need to be included, which data elements should be included, and which data elements could be included, if possible.

Unfortunately, in general, the surveyor does not have the luxury of including as many data items as possible in a household travel/activity survey because the length of the survey will affect the quality of responses and the level of non-response. The tradeoff between survey length and response quality is discussed below in the section describing the development of survey questions, but because it is almost always true that users of survey data would like more questions than can be asked, the survey team needs to determine the data elements of the most interest for the survey effort, and prioritize their inclusion in the final surveys.

The remainder of this data elements section describes the most common information sought in household travel/activity surveys. Each survey team will have different data needs from their surveys, but most household travel/activity survey efforts have common concerns. Therefore, reviewing data elements that are frequently collected is a productive exercise.

Two recent research papers provide excellent taxonomies of the recent content of household travel/activity surveys. Axhausen provides a de-tailed catalog of many recent household travel/activity surveys, including many examples of North American, European, and Australian surveys.¹ Stecher, Bricka, and Goldenberg provide a breakdown of household survey data categories and elements from recent North American efforts.²

Applying these taxonomies, household travel and activity survey data elements can be categorized into the following five categories:

• Household Data – Information on the characteristics of the household and on the actual physical property in which the household resides.

• Person Data – Demographic, socioeconomic, and employment information for one or more members of the household.

• Vehicle Data – Information on the type, ownership, and usage of private vehicles available to household members.

• Travel and Activity Data – Diary or recall information about the travel and activities of one or more household members.

• Attitudinal, Opinion, Knowledge and Stated Preference Data – Information from respondents that provide surveyors and modelers with the respondents’ views, tastes, and concerns.

Household Data Elements

Table 6.10 lists common household data items for household travel/ activity surveys. These data elements are used to classify respondent households as independent variables in travel demand models, and to compare the sampled households with actual study area households for

Table 6.10

which Census data are available. Three household data items are particularly important for most travel survey efforts: household size, household location, and household income.

The number of people in the household is a key consideration in estimating household level trip generation rates. The primary challenge for the household travel/activity survey with regard to this data element is to define to the respondent what is meant by the term household. In general, travel survey teams define a household as the total group of people who usually reside at the sample address, regardless of whether they are related to each other. Travel survey teams need to consider how they will account for college students and others whose legal address is at the sample household, but who live elsewhere. In addition, the travel survey team must decide how to handle visitors to sample households, particularly for survey efforts with travel or activity diaries.

The geographic location of the household is usually an essential data element. Any survey effort that will rely on geographic analyses of some type needs to accurately and precisely define the household location. If the location cannot be coded in sufficient detail, the data record usually cannot be used. Hence, the travel survey team needs to determine the level of geographic detail necessary for future analyses (are neighborhood definitions sufficient? are exact addresses needed, or can nearest intersections and landmarks be used?). In addition, if interview methods are employed, the travel survey team should determine ways to test the sufficiency of the geographic data as soon as it is collected, and before continuing on with other parts of the survey. If the collected data are unusable, then the interviewer can attempt to clarify the response.

Household income information is commonly used in travel demand models and other survey-related analyses, but the collection of household income data is among the more challenging aspects of household travel/ activity survey instrument design. Household income questions almost always have significant levels of item non-response and refusals. In addition, many travel surveyors have questioned the validity of the self-reported income information, based on comparisons with Census income data and other sources.

The question design issues for this data element are discussed below, but a more basic decision related to household income is how the survey data, which is likely to have problems even with the best designs, may be used for analyses. The non-response workshop of the recent TRB Household Travel Survey Conference recommended that travel modelers recognize the inherent limitations of this data element, and that, at a minimum, they consider combining response categories into a few large categories in an attempt to improve the data reliability.

Person Data Elements

Common person-based data elements are shown in Table 6.11. The person-based data elements for household travel/activity surveys are often used as explanatory variables in trip distribution and mode choice models. In addition, these data are commonly used to compare the survey respondents to the U.S. Census population for the study area. The data items for person-based data are generally straightforward, and often include questions about jobs and workplaces.

Vehicle Data Elements

The recent focus on the interaction between travel demand models and air quality models has led a number of travel surveyors to ask for detailed vehicle information in household travel/activity surveys. It remains to be seen which of the very detailed data elements will prove to have significance in travel demand models, so recent surveys have sought many different elements. Table 6.12 shows some of the data elements that have recently been sought. Because a number of the listed data elements require respondents to record information from the vehicles, not all of the data elements are appropriate for all survey methods. For instance, a simple telephone survey could not be used to obtain these data elements.

It is often a challenge to define for respondents the vehicles of interest for the survey. In general, survey teams are interested only in vehicles that are registered and operable. The survey team needs to determine whether only vehicles kept by household members should be reported, or whether all vehicles that respondents could use should be reported. Since informal car-sharing and borrowing are common in the U.S., it is important that survey teams define before the survey effort what they will need for their analyses.

Often, vehicle availability data are used in conjunction with U.S. Census data to expand the survey sample. If this is the case, it is imperative that the survey question be consistent with the Census vehicle availability question:

“How many automobiles, vans, and trucks of one-ton capacity or less are kept at home for use by your household?”

An easier, but usually more costly approach to organizing the workforce for travel surveys is to hire a consultant to conduct the survey. Consulting firms who may be qualified to conduct travel surveys include transportation consultants, survey research firms, and engineering firms. Often a team combining two or more of these types of firms will be selected.

Table 6.11 (1 of 2)

Table 6.11 (2 of 2)

Table 6.12

Travel and Activity Data Elements

Household travel/activity surveys commonly collect a great deal of highly detailed data on people’s activities and trips. Table 6.13 summarizes many of the most common data elements. Usually, these detailed data are collected by means of diaries that either record all respondent trips over a pre-specified time period or record all the activities that respondents engage in over a pre-specified period. It is likely that all activity-based modeling systems will require the use of activity diaries, but conventional travel demand models can utilize either activity diary data or travel diary data. As discussed above, the trend among recent household travel/ activity surveys has been toward the use of activity diaries.

If the survey team chooses to use an activity diary to record respondent activities, a fundamental question that needs to be addressed is whether and how to categorize activities. As discussed previously in Section 6.3, the first choice for the survey team is whether to include only activities that are performed outside the home, or to include both in-home and out-of-home activities. Recent household surveys have used both techniques.

The next question for the survey team is how to record people’s activities. Because of the wide range of potential responses, the most accurate approach for obtaining the information probably involves the use of open-ended questions, with interviewer probing as necessary. However, this approach increases the burden both on respondents and interviewers. Therefore, survey teams have generally defined activity classification schemes, and have asked respondents to categorize their activities on the basis of those schemes. Table 6.14 shows some of the activity categories that have been recently used in household surveys. More classifications of people’s activities are obtainable from the many time-use surveys conducted since the 1970s. These studies tend to have extremely detailed classification schemes (some with more than 100 categories) for how people spend their time.³

Because of the large number of activities that people perform in a typical travel diary period, many household activity survey teams have asked respondents to record only activities that last for more than 30 minutes or activities that require travel (regardless of how long the activity takes). This decision rule limits the reportable activities to a manageable number, but may also dilute the usefulness of the time-use data because many activities may not be reported. Certain types of activities – like meals – that often take less than 30 minutes and do not involve travel are likely to be under-reported.

Table 6.13

Table 6.14

The level of under-reporting can be limited by having interviewers probe for any at-home activities that could be substituted with activities that take place outside the home, such as eating out/catalog shopping. High quality interviewers are essential for an approach like this, because the probing could greatly increase the data retrieval time if it is not performed efficiently. Another circumstance that time-use and activity-based survey designs must consider is when activities occur simultaneously or when an activity (like eating, reading, or paperwork) is completed in the course of travel.

When trip-based methods are employed, rather than activity-based methods, a key issue in the design of diaries is the definition of a trip. Respondents are likely to define the word, ‘trip’ differently than the survey team members. Therefore, the interviewer or the survey instruments need to explain the term to respondents so that they will answer the question as accurately as possible. In most studies, trips are defined as one-way travel between an origin and a destination other than the origin. This issue is discussed in more detail later in the section on question wording.

Attitudinal, Opinion, Knowledge, and Stated Preference Data Elements

The final data elements that are commonly collected in household travel/activity surveys are the attitude, opinion, knowledge, and stated preference data, including the data elements shown in Table 6.15. These data are not discussed extensively in either the Stecher, Bricka, and Goldenberg paper or the Axhausen paper, because of their focus on household surveys that are used for regional travel model development. Because of their length and complexity, household travel and activity surveys that are conducted for purposes of model building should be limited to the collection of data that will be (or could be) used as model inputs. Survey teams should resist the temptation to collect “nice-to-know” information without first understanding exactly how it will be used.

Other types of household travel/activity surveys, such as those performed for transit agencies, are more likely to employ the types of data elements listed in the table.

Stated response data have been collected in many different types of household surveys, including those that are used for regional model development. The issues related to these data elements are discussed in Chapter 13.0.

Table 6.15

Translating Required Data Elements Into Questions and Response Categories

Once the survey team and travel demand modeling staff have established the essential and optional data items to be included in the household travel/activity survey, survey questions need to be developed that will produce the data elements accurately and reliably.

A survey question should be included in a household travel/activity survey interview or questionnaire if the following are true:⁴

• The information obtained from the question is relevant to the models being developed or refined, or to other anticipated analytical efforts.

• The question and response categories are expected to be valid measures of the modeling variables.

• The responses can be coded meaningfully for modeling analyses.

• Analysts, interviewers (if any), and respondents agree unambiguously on the meaning of the question and response categories.

• The question and response categories have no wording problems.

• The wording of questions and responses is the same or equivalent to any measure from other surveys that will be used in the modeling work.

• Response categories exhaust all meaningful answers that can be anticipated.

• Response categories are meaningful and understandable to respondents.

• (For interviews) the questions and response categories are easily learned by interviewers.

In addition, each survey question should be reviewed in terms of its effect on the overall survey quality. The following should also be true:

• The benefits in the survey analysis from the question outweigh its costs in terms of survey length, respondent burden, and increased non-response.

• The information gained from the question is more useful than the information that would be gained from other questions that will not be on the survey.

• The question does not provoke respondents to be hostile to the survey effort or to question the goals of the surveying agency.

In short, a question and its response categories should provide unambiguous, accurate, reliable, and usable information without affecting the overall validity of the survey effort.

To meet these requirements, the survey team must successfully perform three tasks in translating data elements into actual survey questions:

1. Determine exact question content and the forms of the questions;

2. Develop the wording for questions and response categories; and

3. Determine how questions should be sequenced.

These tasks are described below.

Determining the Form and Content of the Questions

The survey team needs to operationalize the survey’s required data elements by defining in more detail what data are needed and by determining the appropriate question forms to obtain the data.

The list of needed data elements that the survey team assembles while analyzing the proposed analyses based on the survey data will include a number of items that will lead directly and logically to the development of one or more survey questions without much effort. For instance, if one of the identified needed data elements is whether a person has a valid drivers’ license, the data could be collected by simply asking her or him.

On the other hand, for some data elements there will not be a clear set of survey questions. These data elements require that the survey team improve its data definitions and more carefully consider what survey information is required.

For instance, one data element that is often sought in household travel/activity surveys is whether an individual had a vehicle available for a particular trip from home to work. The most straightforward and commonly employed approach to obtaining this data element is to ask, “did you have a vehicle available for this trip?,” or some variation. However, as Axhausen points out, this simple question is riddled with ambiguity.⁵ Does answering yes to the question mean that there was a vehicle at home that the person could have driven, or does it mean that, if necessary and with advance planning, the person could have arranged to take a vehicle, perhaps by changing how other household members travel, or does it mean something else?

Given this problem, the survey team might want to consider the development of a series of questions to determine how the household allocates vehicle usage and how mode choice decisions are made, or they might want to consider asking questions that would allow them to trace the usage of all household vehicles throughout the travel period in question. Before deciding how best to proceed the survey team should re-examine the expected analyses that will rely on these data elements to find out the best approach.

As the survey team evaluates the household travel/activity survey’s needed data elements, they should also consider the types of survey questions with which the data elements are best matched with. In Chapter 3.0, different kinds of survey questions, including factual, behavioral, test-of-knowledge, attitudinal, opinion, and stated response questions, were defined. However, in developing the survey questions it is useful to classify questions differently, according to the question form. Figure 6.3 shows a classification scheme for the forms of survey questions. As the top of the figure shows, survey questions can be either open-ended, which allow respondents to reply to the questions freely, or closed-ended, which provide respondents with preset response categories. Open-ended questions are sometimes useful, particularly when:

• The survey team is uncertain about the possible range of responses to a question;

• The question has so many potential responses that providing categories would be infeasible;

• The survey team needs to have precise information on how respondents think about something;

• The survey team would like to provide respondents with the opportunity to sound off about the survey topic(s); or

• The survey team is seeking verbatim remarks to complement the statistical analyses.

Figure 6.3

Geographic questions are almost always open-ended, because locations that individual respondents refer to cannot be foreseen by the survey team. In addition, questions that ask for respondents’ opinions are generally open-ended.

However, almost all household travel/activity surveys rely to a large degree on closed-ended questions for most data elements, because both data collection and survey processing are greatly facilitated by forcing responses into a small number of categories. Closed-ended questions allow respondents to simply pick a reply, rather than to form one from scratch. In addition, the possibility of respondents providing inappropriate answers or of misinterpreting questions is limited by providing preset responses. Finally, closed-ended questions are much easier to code than verbatim respondent replies.

Because of these advantages, many interview questions that appear to be open-ended are actually made closed-ended by having interviewers categorize respondents’ remarks into preset divisions.

Closed-ended questions are generally of one of four types, nominal, ordinal, interval, or ratio. This classification is useful in question design, because it is usually easy to determine which of the four categories best addresses particular needed data elements. Because there are relatively few ways to phrase survey questions in each of the four categories, the question construction of other survey questions in the category can often be used as a guide in converting the data element to a useful survey question.

The Wording of Questions and Response Categories

Before developing the wording for any questions, the survey team should understand how the survey process and specific questionnaire wording can bring inaccuracies and biases into the survey results. Two basic survey question problems can harm the survey effort:

• Item non-response; and

• Inaccurate replies (response errors).

To minimize the effects of these problems, it is first important to understand what possible motivations respondents might have to be less than forthcoming or to mislead the surveyor. There are four general reasons why respondents can provide incomplete or inaccurate information in surveys:

1. The respondent does not know the answer to a question or questions;

2. The respondent cannot remember the answer to a question or questions;

3. The respondent misunderstands the question; and

4. The respondent has some motivation not to be totally forthcoming.

While it is not possible for survey teams to eliminate these issues, it is clear from recent household travel/activity survey efforts that good question design can certainly reduce the number of response errors significantly.

The remainder of this section discusses common survey question wording problems, and then discusses question wording issues for particular household travel/activity survey questions, including diaries.

Survey Wording Problems

Each question should be tested by the designer for the potential problems listed above. To minimize the chances that a respondent will not know, remember, understand, or be willing to answer survey questions, the survey team should seek to avoid questions that fall into three broad categories:

• Confusing questions;

• Ambiguous questions; and

• Loaded questions.

Confusing questions are questions that mix up respondents in some way. Ambiguous questions are questions which not everyone would agree mean the same thing. Loaded questions are questions that suggest to respondents that certain responses are preferable to others. Loaded questions are usually of the most concern on attitude, opinion, and stated response questions. Since household/activity questions use these types of questions infrequently, they are probably less of an issue than misperceived or ambiguous questions.

Tables 6.16, 6.17, and 6.18 provide examples of survey questions that have these problems. Although these examples are contrived, they illustrate the many ways question wording can lead to response errors. It is fairly easy to identify questions that could be potential problems in many recent household travel/activity survey materials. In most of these cases, the survey teams probably considered alternative wording but found that the alternatives introduced wording problems of other types. Ultimately, the household travel/activity survey team needs to use its judgment, experience, and the results of carefully-designed pretests to make final decisions about survey wording.

Table 6.16

Table 6.17

Table 6.18

As a general rule, the wording of questions and response categories on surveys should be aimed at the respondent audience. For household travel/activity surveys, this means that the wording needs to be designed for a broad audience with a wide-range of reading and comprehension abilities, and with differing levels of interest in transportation. The challenge for the survey team is to word the survey as simply as possible without boring more advanced respondents.

The following question-writing principles are suggested to achieve this goal:

• The household travel/activity survey should be understandable by the average fourth grader. Once a preliminary set of survey questions is developed, the survey team may want to try administering the questions to some children to see whether any questions are confusing for them.

• The household travel/activity survey should hold the interest of intelligent adults who are not involved in transportation planning or market research. The travel survey team may also want to test this principle by administering the survey to non-technical friends and acquaintances. These people can indicate where the survey is tedious and whether certain questions feel condescending.

• Almost none of the household travel/activity survey respondents will have a background in transportation planning or surveys, and many will have little or no interest in the subjects. Survey team members need to be extremely careful about projecting their level of knowledge and interest onto potential respondents. In particular, the transportation planning field is full of jargon and expressions that are not obvious to non-planners. The survey team needs to be very careful with the use of many words, including:

• “Trip”

• “Journey”

• “Travel”

• “Activity”

• “Origin”

• “Destination”

• “Mode”

• “Trip purpose”

• “Bus”

• “Shuttle”

• “Transit”

• “Transfer”

This is not to say that such terms should not be used. Rather, the terms are often central to the information that is being sought, and can be absolutely necessary in many cases. The survey team needs to be aware, however, that these and other terms do not always mean the same thing to everyone, and for some people, they will mean nothing. The survey team needs to make sure that the terms are either defined for respondents, or that the context in which they are being used does not allow for ambiguity in their meaning.

• In addition, marketing research and survey analysis terms and jargon should be avoided in surveys, because some respondents will resent being part of an experiment. Among the words to watch for in this regard are:

• “Questionnaire”

• “Research”

• “Data”

Question Wording Issues for the Household Travel/Activity Survey

In household travel/activity surveys, a few question types are known to be problematic. Three particular question types are discussed here:

• Questions about household income;

• Questions about personal property; and

• Activity and travel diary questions.

Questions About Household Income

Household income questions usually have the highest levels of non-response of all the survey questions. Recent survey efforts have reported item non-response rates for household income questions of more than 10 percent. Unfortunately, the people who refuse to answer income questions are usually not representative of the whole population. Research indicates that those who refuse to answer income questions are more likely than the population as a whole to have higher income levels.⁶ In addition, some recent U.S. survey efforts, such as a recent statewide effort in New Hampshire, have found that households in the lowest income categories are less likely to complete these surveys. This may be due to either a higher income non-response rate for this group or to an overall higher unit non-response. Therefore, analyses of income data with significant non-response are likely to be biased.

Many surveyors believe that the income question has a high response error level, as well. Many respondents who are unwilling to provide accurate income information sometimes make up an answer, rather than simply refusing to answer it. Also, some respondents will not know their total household income or will be confused about which types of income to include in their estimates. It is difficult to determine the magnitude of the response error, because it is impossible to tell valid responses from invalid ones.

Because of the perception of response error, some surveyors believe that there is a practical maximum number of income categories beyond which the data are probably too inaccurate and the question either takes too long in an interview or takes up too much space on the mail survey form. These practitioners believe the survey team should limit the number of income categories to between 8 and 12.

It is generally accepted by travel surveyors and market researchers that the household income question should, if possible, be the last question on the survey. If the question is asked earlier in the survey, the likelihood of the respondent not completing the survey is increased. In addition, once a respondent refuses to answer a question, the likelihood that he or she will refuse to answer others, as well, increases. For some survey efforts, such as those where more than one household member are interviewed, asking income as the very last question is not always feasible. In these cases, survey teams should attempt to sequence the income question after other demographic questions and after the collection of any other descriptive information that may be used in survey expansion.

A number of wording and questionnaire design techniques have been tried to reduce the level of non-response and improve the quality of data from income questions on household travel/activity surveys. Unfortunately, many of the techniques contradict each other, and their success may be specific to certain survey populations. Since careful comparisons are not usually made, it is impossible to say how effective each is.

If the survey team identifies the need for a survey question about household income, they should consider evaluating alternative wording and questionnaire designs as part of the survey pretest to identify the best solution for their area. The form of the income question can be varied, so that pretest respondents receive different questions. The preliminary pretest results may indicate that one question form is superior to others. Alternatively, if a survey team is using one or more focus groups for developing the questionnaire, the focus group participants can be asked to assess the relative invasiveness of different question forms.

Some recent household travel/activity survey teams using telephone data retrieval have reported some success with asking for household income information in a series of choices, rather than by listing all the categories.

The question is structured, as follows:

For 1992, will your household’s total income from all sources, before taxes and any other deductions from pay be less than $35,000, or $35,000 or more?

Ask all with household income less than $35,000: Will it be under $20,000, or $20,000 to 35,000?

Ask all with household income $35,000 or more: Will it be more than $35,000 and less than $60,000, or $60,000 or more?

The simple choice questions continue until the desired level of categorization is achieved. In this type of question structure, the first query ($35,000 in the example) is usually set near the median household income, or a little lower.

This approach can be useful, because in some cases, partial data can be collected from people who would not have responded to the usual income question. On the other hand, the question lengthens the interview at a point where most respondents really would like the interview to be over.

Travel surveyors have tried different approaches of leading into the income question. Some surveys have re-stressed confidentiality and the study goals before asking the question. The household income question on the 1994 Boise survey is worded:

Now a question just for statistical and travel forecasting purposes, we need to know your total household income before taxes. I will read several ranges to you. Please stop me when we reach the right one (interviewer then reads categories in ascending order).

Other travel surveyors believe the best approach to asking income is to include the question at the end of a series of short factual demographic questions. The income question is asked in the same quick way as the other demographic questions. The hope is that the respondent will sense that the income question is just another question that will be used to differentiate groups of people, and that the respondent will simply fall into the rhythm of answering questions.

In telephone-mail-telephone surveys, some surveyors believe that difficult questions, like income, should always be deferred until the retrieval call. On the other hand, some telephone-mail-telephone surveys have asked the income question at the end of both the recruitment interview and the data retrieval interview. Surveyors have found that some people who refuse to supply the information in the recruitment (which is a “cold call”) will answer the question during the data retrieval (presumably, they have been convinced of the survey’s legitimacy).

Similarly, some household travel surveys have asked all adult members of the household to answer the question, in the hopes that if one household member is reluctant to give out the information, others may not be. Of course, methods that seek the information more than once may lead to consistency questions if the survey team ends up with more than one household income estimate for a household.

Questions about People’s Property

Usually the only questions on the household travel/activity survey that have significant non-response problems are those that ask about household income. However, as household travel/activity surveys become more detailed, a few other questions need to be carefully worded and presented. In particular, questions about people’s property that might interest enterprising thieves are likely to be a problem for an increasing number of respondents.

It has become fairly common to collect detailed household vehicle information on household travel/activity surveys. While most respondents will recognize a question about the number of vehicles available to the household as having valid transportation planning use, most will not be familiar enough with air quality analyses to understand the need to know the make, model, and year of the vehicle. Untrusting respondents, who question the legitimacy of the survey effort, may feel that they are contributing to some car thief’s shopping list. Some recent survey efforts have asked respondents to record the vehicle identification number (VIN) of all their vehicles. Because the most widely-known use of this serial number is to track stolen vehicles, this question may raise the suspicions of respondents even more.

As issues involving telecommuting become more important in transportation planning, it is likely that more and more household travel/activity surveys will also seek information about people’s ownership of computers, fax machines, and other equipment. Combined with travel and activity diary data and detailed addresses, this dataset would be ideal for thieves, and it is likely that respondents will recognize this fact.

To limit the non-response on questions of these types, survey teams should:

• Put the questions near the end of survey instruments;

• Explain to respondents what the data are to be used for; and

• If possible, ask only in the second or third contact with the respondents, so that they are more comfortable with the survey’s legitimacy.

Activity and Travel Diaries

Diary design is an extremely important element of questionnaire development because response errors in the form of unreported trips are common, and are almost always a serious problem for those who analyze the survey data.

Types of Diaries

Over the past 20 years, household travel/activity surveys have used several types of diaries for which Axhausen has developed the following typology:

• Stage-Based Diaries – Treat the travel on a single mode (and the associated wait time) as a building block to construct the whole trip (data are gathered on the basis of each trip segment);

• Trip-Based Diaries – Establish the trip as a whole and then disaggregate them into stages, if necessary (data are gathered on the basis of the whole trip);

• Activity-Based Diaries – Focus on activities and then collect trip details to and from the activities (data are gathered on the basis of trip-end activities); and

• Half Tour-Based Diaries – Collect information for travel between home and the farthest point of a trip chain, and then fill in information on individual trips in the chain (data are gathered on the basis of the key stop in a trip chain).

Stage-based diaries have recently been used in a 1991 mail survey in Chicago, in a 1990 telephone-mail-telephone household survey in the Bay Area, and in a 1993 telephone-mail-telephone survey in Tucson. Figure 3.2 shows the example page from the Chicago survey. In this diary, information on each trip stage is recorded in one column. The Chicago survey is stage-based, because one of the valid answers for the question, “Why did you go to this destination?” is to “change type of transportation.” Several columns of information might be needed to describe a single trip.

Figure 6.4 shows the Bay Area’s stage-based diary design. Actually, the diary instrument used for this survey was a simple memory jogger. Respondents recorded a minimum amount of information about the stages of the trips they made in the memory jogger, and then the telephone retrieval call was used to obtain details about the trip stages. Again, the respondent is asked to record information about each stage of his or her trips.

Figure 6.5 shows the Tucson survey diary’s example, which illustrates yet another diary format for stage-based reporting. In this diary, the trip stages are recorded in the numbered rows, labeled with “Then I went to.”

The stage-based design is most useful for survey efforts where the survey team has identified the need for path choice and sub-mode choice information. These diaries readily provide information both on the number of

Figure 6.4

Figure 6.5

modal transfers and their locations. Because respondents record each stage of a trip, the stage-based designs tend to require more from respondents, and therefore, more space on the questionnaires than the other designs.

Figure 6.6 shows a trip-based diary used in a 1990/1991 British survey, the Sainsbury’s Swindon Survey. Trip-based diaries, like this one, rely on respondents to characterize their main mode of travel or provide respondents with more exhaustive mode lists which incorporate sub-modes.

A recent U.S. example of a trip-based diary is the ongoing panel mail survey being conducted by New York MTA. The instruction page for this diary survey is shown in Figure 6.7. In this diary, respondents are asked to provide the origin and final destination of trips and to record the travel modes that they used in order of usage. This panel survey effort is being used primarily to track people’s travel choices with respect to MTA services and to measure the MTA’s market share in different markets, not for the development of regional travel models. Therefore, stage-based information are not considered to be necessary.

In the past few years, the activity-based diary has become the predominant form of diary in the United States. Recent telephone-mail-telephone household travel/activity surveys in Portland, Detroit, and New Hampshire (to name a few) have used activity-based diaries. Figure 6.8 shows a portion of the activity diary for the 1994 Research Triangle Activity and Travel Survey.

Figure 6.9 shows the diary for a 1994 Detroit survey, and Figure 6.10 shows the activity diary for the 1994 Portland survey. Respondents in the Detroit survey were instructed to treat travel mode changes as destination activities. This is the activity-based counterpart to stage-based diary design. All the trip stage information is collected for detailed modeling analyses, but at the cost of asking respondents to record more details and of needing much thicker diary booklets. The Portland diary was used to record all activities, both at-home and outside-the-home. The question “Where did your activity take place?” is needed to identify multiple activities at the same location. Note that the activity-based diaries collect as much, or more, travel data than the other “Travel diaries.” The survey team is not limited to activity-based analyses by selecting to use activity-based diaries.

The final type of diary, the half-tour based approach has not been widely applied in the U.S., but might be useful for certain special types of analyses. This diary approach seeks detailed information on the primary trip within a trip chain, and then asks for less-detailed information about the other stops on the trip chain (usually limited to the number of stops). This type of diary might be particularly useful in the analysis of intercity and long-distance trips. As an example of this diary approach, Axhausen provides the diary from a recent Canadian fuel-usage survey, shown in Figure 6.11.

Figure 6.6

Figure 6.7

Figure 6.8

Figure 6.9

Figure 6.10

Figure 6.11

Selecting the Best Type of Diary

There are good reasons to use each type of diary, so survey teams need to decide which approach to follow based on their data needs. An increasing number of survey teams are choosing to ask people to record activities, rather than trips, primarily because there is strong evidence to suggest that diaries that focus on activities, rather than trips, measure travel more completely than travel diaries. Based on analysis of the recent Boston Household Survey, Stopher concludes that: 1) activity diaries appear able to capture non-home-based trips better than travel diaries; and 2) overall trip rates per person and per household from the activity diary are significantly higher than most travel diaries measure.⁷ Jones found that asking about activities, rather than trips on either diary surveys or recall surveys results in improved trip-rate estimates, compared to trip-based approaches.⁸

Researchers and surveyors also seem to be selecting stage-based designs more frequently, whether the diary is travel-based or activity-based. This is probably a function of two things:

• Data analyses are increasingly requiring more detailed information; and

• Asking respondents to remember each part of their trips can sometimes help them remember brief stops that they would have otherwise forgotten.

Level of Detail for Diary Questions

As the example forms show, most household travel and activity surveys seek detailed information about people’s travel. This means that surveys that are to be mailed back must be carefully designed to obtain all the necessary information. However, if the diary data are to be retrieved by telephone, the survey team has two options:

• Provide complete diary forms, similar to those used in mailback surveys, from which respondents can simply read their answers; or

• Provide simplified diary forms with which respondents record key information about trips or activities, and then are asked to provide more detailed facts during the retrieval call.

With the first option, respondents are not surprised in the data retrieval call by questions that they were not expecting. In the second option, they are, to some extent. This is important because of the extreme length of diary retrieval call. If respondents are already dreading having to supply a large amount of trip or activity information, suddenly being asked “additional” questions may cause them to refuse to complete the survey. For instance, respondents to a recent NPTS pretest objected to the additional questions.

Another advantage of providing the complete diary forms is that they are in a form that can be used if they are mailed back. If a respondent is unwilling to complete the data retrieval telephone call, the survey team may want to ask them to mail the completed diary instead. If a shortened, or abridged, diary form is used, this is not an option. In addition, with a complete diary, the quality of information collected by proxy (where one household member provides another household member’s diary information) is improved.

On the other hand, complete diary forms are more likely to appear complex or confusing, and may discourage respondents before they even begin to complete them. A complete diary form will usually have questions that will not be answered by a particular respondent. For example, questions about transit trips would not be answered by someone who never used transit. In addition, with the complete diaries, it is less likely that respondents will be willing to complete the diaries as they perform the activities and make the trips throughout the period, as requested. Instead, they may try to complete the form at one or a few times only, raising the likelihood that trips will be forgotten.

To alleviate this problem, some survey teams have provided memory joggers, similar to the Bay Area form shown in Figure 6.4, as well as diaries, so that the respondents could record less detailed information with the memory joggers throughout the diary period, and then fill-in the more detailed diaries from them. The memory joggers are much easier for respondents to carry with them, and so it is believed that respondents are more likely to record trips as they occur, but respondents have found the duplication of effort required with this method to be burdensome. Focus group participants in the recent Research Triangle study said that the two forms were unnecessarily redundant. To address this, the simplified “checkbook-style” diary form shown in Figure 6.8 was developed so that respondents would complete the form throughout the diary period, but would not have to complete more than one form.

Both the complete diary form approach and the simplified diary approach are currently being used in the U.S. The relative importance of the different advantages and disadvantages cited above do not seem to support the recommendation of one approach over the other. The survey team can select the best approach for the particular survey population under-study through pretesting or through focus groups with potential respondent groups.

Additional Diary Question Design Issues

Often, one or more household member is unavailable or unwilling to complete the survey diaries, so survey teams must develop a set of procedures for accepting or not accepting proxy reports. Usually, proxy reports for children under age 14 are deemed acceptable. However, surveyors have differed on how to address the potential need for proxies for older household members. Most surveyors try to avoid proxy reports for adults. Survey teams have specified that interviewers attempt to reach these respondents as many as four times. If these individuals are still not reached or persuaded to provide the information, some surveyors have sought proxies from within the household. Others have deemed the household contact as incomplete, because proxies consistently report lower numbers of trips.

One final diary design consideration is that some travel surveyors have found that CATI data retrieval can enhance the completeness and accuracy of the diary data. CATI systems can identify missing links in people’s trip patterns (indicating a mis-recorded or forgotten trip or activity) and check for data anomalies, such as a person not ending up at home by the end of the period. A common diary completion error is to not record the final trip to home on a day. The CATI system can look for such curiosities, and instruct the interviewer to confirm that the data are correct.

Sequence of Survey Questions

In conjunction with developing the wording for the survey questions, the survey team needs to determine where and when each question will be asked. The survey team must decide:

• In which survey instrument each question will be asked;

• The order of the questions; and

• The length of the survey.

For the simple mail and simple telephone survey, the first choice is easy. There is only one place the survey question can be – in the mailed survey, or in the telephone interview. However, for the more complicated survey methods, the survey team has the choice of placing questions in the recruitment call, or in the mailed survey, or (for the telephone – mail – telephone) the data retrieval call, or in some combination of the instruments.

Usually, the recruitment call is designated to be short. The call needs to achieve the following:

1. Screen potential respondents to ensure they are in the survey population of interest;

2. Gain the cooperation of potential respondents;

3. Obtain information necessary for the next phases of the survey (mailing address, number of diary instruments needed, best time for data retrieval call, etc.); and

4. Obtain respondent information, so that the survey team can weight the survey results to account for people who were recruited, but who do not complete the survey.

For the recruitment interview, the general order of the questions should be close to this sequence. The recruitment call should pique the interest of respondents, and make them look forward to the mailing, so if time permits, interesting attitudinal and opinion questions could be included. For the most part, the recruitment call should avoid questions that will make respondents uncomfortable. As noted above, it is generally not a good idea to ask about people’s property in these calls. Asking household income questions on the recruitment call has both positive and negative points. By asking the question, the survey team can gain an important variable for non-response weighting, but they do so at the risk of alienating respondents before they have completed their tasks.

For survey methods with mailed data retrieval, the mailed survey instrument usually contains most (or all) of the key survey questions, plus the diary (if one is being used). As discussed below, the travel/activity diary and associated materials are generally separate documents from the rest of the survey questions, and usually do not need to be sequenced with the other questions. The other questions should be ordered according to the following principles:⁹

• Order the questions in descending order of how important respondents are likely to perceive them. Begin with questions that are clearly related to the survey topic (recent travel information) and then move on to questions which are only tangentially related (demographics).

• On printed survey materials, organize the questions into related groupings, perhaps separated with titles, “About your travel choices,” or “about your household.”

• Within each group, sort the questions by the tasks they ask of respondents. For instance, all “yes” or “no” questions in one sequence and all the scale questions in another.

• Build ties between the question groupings, so that one flows into the next. If a mail survey has several question groupings which do not seem to relate to one another, the respondents are more likely to feel burdened, and will more easily become confused.

• Put questions that are possibly objectionable to respondents at the end of each grouping of questions.

• Always put demographic questions last, with household income questions at the very end.

If the survey team has special concerns about how two different questions will interact with each other, it makes the most sense to remove one from the mail survey and ask it in the recruitment or data retrieval interview. There is no way to prevent respondents in a mail survey from using (or misusing) information from one question in another, regardless of whether the two questions are near each other, or in which order they appear.

The sequence of the data retrieval call will depend on how much data are to be retrieved directly from mailed materials, and how much will be from new questions to respondents. Most data retrieval calls are designed simply to obtain the information that the respondent has recorded. These calls will follow the mail survey and diary questions precisely, perhaps asking a question or two along the way to clarify or probe responses. It is usually a good idea for the survey team to ask for a few details about the information collected from the mailing, simply to keep respondents actively involved in the interview.

Since both survey cost and non-response levels are related to the length of the interviews and the size of the mail survey, questionnaire length is very important. In general, recruitment calls are brief, between eight and 15 minutes. Recruitment call times for recent household travel/activity surveys have been:

Portland, 1995	8-9 minutes
Beaumont, TX, 1993	7 minutes
North Carolina Research Triangle, 1994	10 minutes
Baltimore, 1993	9 minutes
NPTS, Pretest 1994	12 minutes

Interviewers and respondents exchange information during this call, so it is more likely to be interesting for respondents.

Data retrieval calls are substantially longer. Retrieval calls for one-day diary surveys usually average 30 to 50 minutes per household. Two-day diary survey retrieval average times range from 40 to 75 minutes per household. Since these figures represent averages, larger households have had to have been on the phone for very long periods of time. In some recent surveys, data retrieval has been spread over two or three calls. Data retrieval calls are also less interesting for respondents than recruitment calls since they usually are simply being asked to read their written response. The effect of survey length on non-response is highly variable, depending on the level of interest the respondent has in the survey topic, but most household travel/activity surveys are well into the range of being too long for a substantial number of people. Beside the obvious approach of cutting survey questions out, two procedures have been used to shorten data retrieval calls:

• Move questions from the retrieval interview into the recruitment interview; and

• Split the sample so that respondents are only asked a subset of questions. Different respondents are asked different questions so all the questions are asked of at least some of the sample.

Both of these approaches have clear limitations, however. Recent survey efforts that have tried to ask too much of respondents in the recruitment interview had lower respondent cooperation rates, and one effort ran into public relations problems because respondents were uncomfortable with the level of detail they were being asked to provide in the initial call. Splitting the sample is usually very difficult for household travel activity surveys, because almost all the retrieval time is spent on the diary information, which cannot be split practically.

The survey team must ensure that every effort has been made to keep the respondents interested in the survey, and to provide an organized interview as soon as possible.

Survey Instruments and Materials

Because household travel/activity surveys are generally used to collect a wide range of detailed data, the survey team is likely to need to develop a number of different survey instruments and materials. Table 6.19 summarizes the most common survey materials. As the table indicates, the key questions in designing any of these materials are:

• Simply put, what is the purpose of the survey material or instrument?

• Who are the “users” of the survey material or instrument?

It is important to remember that the answer to the latter question usually includes more than one group, who sometimes require very different design decisions. Designing survey materials with only respondents in mind can lead to interviewer errors and coding, editing, and cleaning problems. These problems will almost certainly show up in the survey cost and scheduling.

The following sections describe the development of the survey materials used in the most common household travel/activity survey methods.

Table 6.19 (1 of 2)

Table 6.19 (2 of 2)

Materials for Any Survey Method

Pre-Notification Letter, Brochure, Postcard, or Interview Script

The primary purpose of pre-notification is to inform potential respondents of the upcoming survey and to persuade them to participate. Pre-notification is in effect a sales effort. The survey team wants to convince a potential respondent that the household travel/activity survey effort is:

• Legitimate;

• Important;

• Worth looking for in their mail; and

• Worth their time and effort to complete.

To convince respondents of these facts, the following design guidelines are suggested:

• The pre-notification phone call should be made, or the letter, brochure, or postcard should be sent so that it arrives three or four days before the survey materials.

• If pre-notification is by mail, the document needs to be short and to the point. It should convey its message to the reader in less than a minute.

• The document should look official. If possible, it should be printed on letterhead or have the name of the sponsoring agency or agencies prominently displayed.

• The document can either be from the agency sponsor or another interested agency that might be more recognizable to respondents.

• If possible, the document should be signed by a recognizable public figure or an elected official.

• The document should explain the reasons for the survey in plain language, and stress the confidentiality of the survey effort.

• The document should provide the name and telephone number of someone who can answer questions about the survey and confirm its validity. In most cases, almost no one will call the number, but if potential respondents see that it is offered, it will help them believe in the validity of the survey.

• For those people who do call the number, it is best if the number is for someone at the sponsoring agency, rather than at a private market research firm. Some survey contractors can provide a phone number that can be covered by an answering machine with the name of the agency on the message. Representatives of either the survey firm or the sponsoring agency can return the few calls that will come in. It is useful to draft a list of the most frequently asked questions with the response for use by the sponsoring agency and/or the contractor. Whoever receives the call will answer respondent questions in a consistent manner.

A secondary use of the pre-notification material is to use it to identify bad addresses for which mail is undeliverable. To do this, the pre-notification letter must be sent by first-class mail with a return address (many surveyors believe all materials should be sent by first-class mail to separate the materials from junk mail). Mailing the pre-notification material by this method helps the survey team to identify the percentage of bad addresses in the sampling frame, and allows the team to save some postage costs by sending the survey materials only to valid addresses.

Figure 6.12 shows an example pre-notification letter from the 1996 Oregon (statewide) Travel Behavior Survey.

Thank You Card

From a myopic data collection point-of-view, once the survey team has been able to retrieve complete and seemingly valid data from a respondent, further contact with that respondent is superfluous. However, in some instances, it is politically advantageous for an agency to send a postcard thanking the respondent households for their efforts. These cards build a sense of good will between the respondents and the agency, which might be valuable in future planning efforts that are increasingly relying on citizen participation.

The cards also help to confirm for respondents that the survey effort was legitimate and that their information was valuable. A survey team that is considering the possibility of using the household travel/activity survey as part of an ongoing panel design should certainly consider the use of thank you cards.

Follow-Up Letters and Postcards

Usually, if a potential respondent household fails to respond to the initial mail survey, they are sent one or more reminder letters or postcards. These materials are designed to:

• Remind the respondent households who have completed the survey, but not yet returned it to mail it back;

• Re-stress the importance of the survey effort and the importance of the particular household receiving the letter or card; and

• Provide potential respondents with another opportunity to complete the survey by assigning new diary periods, and/or by inviting them to contact the survey team by phone.

The first set of follow-up letters or cards stress the reminder message. Often, a simple postcard is sent that states that the respondent household’s survey has not yet been received, and asks the household please make sure that they do not forget to send it.

With each successive follow-up round, the content of the letters and/or cards shifts away from the reminder message toward a more persuasive message. Later follow-up mailings tend to be either letters stressing the importance of the household’s participation or letters combined with a new set of survey materials.

Richardson, Ampt, and Meyburg suggest that each successive follow-up mailing be sent in different color and style of envelope so that they are less easily dismissed by potential respondents.

Figure 6.13 shows an example follow-up letter from the 1991 CATS household travel mail survey.

Materials for Surveys with a Mail Component

Envelope for the Survey Mailing

The first challenge in getting households to respond to the survey is to get one of the household members to open and read the survey package. A very high percentage of direct mail is thrown away without ever having anyone open it. Unfortunately, mailed household travel/activity surveys generally look very much like direct mail when they show up in people’s mailboxes.

The travel survey team needs to take steps to get their envelope opened. One simple step for survey efforts with pre-notification or where the mailing is taking place after the respondent has been contacted by phone is to describe the envelope to the respondent before it is mailed to them, “You will be receiving the mail survey in a large blue envelope with our return address on it.”

In addition, Dillman suggests the following steps to separate the mailing from junk mail:¹⁰

• Use as small an envelope as possible to limit postage costs and to avoid the “bulk” image;

• Type the name (use specific people’s names, rather than only the family name whenever possible) and address directly on the envelope, rather than on address labels;

• Do not embellish the envelope with messages, like “dated materials inside” or “immediate reply requested”; and

• Use actual stamps, rather than metered or bulk rate mail, even though postage costs will be higher (not all surveyors see this as a justified expense, particularly if respondents have been pre-notified of the effort).

The design of return envelopes for mailback questionnaires is less important than the survey package envelope, because the respondent would use it only if she or he has decided to complete the survey. The important design considerations for these letters include the following:

• The envelopes should be provided;

• The envelopes should be prepaid through the use of business reply mail; and

• The return address should be for the sponsoring agency, if possible. If the replies are going to be sent to a survey contractor, respondents should already be aware of the name of the firm from the cover letter and other survey materials or interview contacts. Unless it is absolutely necessary, envelopes should not be sent to addresses outside the study region.

Cover Letter for the Survey Mailing

The cover letter for the survey mailing has been shown to be an important survey element. It must perform the same functions as the pre-notification letter, plus introduce the attached survey materials. The cover letter should be no more than one page, and should be both motivational and informative.

Figure 6.12

Cover letters should have the following general outline:¹¹

• Top of Page – Official letterhead of the sponsoring agency;

• Date – Exact (and correct) date of transmittal;

• Address – Name and address, similar to any business letter;

• Salutation – Specific salutation (e.g., Dear Ms. Thompson:), if possible (sometimes not possible because only the family name is known or because the person’s sex is not obvious from his or her name), otherwise no salutation at all;

• Paragraph 1 – 1) topic of the study; and 2) social usefulness of the study;

• Paragraph 2 – 1) why recipient is important to the study; and 2) who in the household should participate;

• Paragraph 3 – 1) promise of confidentiality; and 2) explanation of privacy procedures and serial numbers on survey forms;

• Paragraph 4 – 1) usefulness of the study results; and 2) explanation of incentive (if one is being employed);

• Paragraph 5 – What to do if questions arise;

• Closure – Thank you;

• Signature – Business letter-type signature block, with typed name and title under the signature, and with an original handwritten signature in blue ink.¹²

If possible, the signature should be from a well-known elected official. Recent household surveys have had cover letters signed by Governors and U.S. Senators. Of course, when higher profile people are used for the signature, the ability to provide an actual handwritten signature is eliminated.

Figure 6.14 provides an example of a recent household travel/activity survey cover letter. While this letter does not follow the suggested format with precision, it covers the main points that need to be included, and appears to be an effective communication tool.

With address-based sampling, it is possible to individualize the cover letters so that the study’s benefits can be defined specifically for the individual receiving the letter. In an ongoing Bay Area survey, this individualization is being done as follows:

“Dear <<respondent name>>:

Do traffic conditions on <<main corridor near respondent’s home>> sometimes concern you? What about the number of parking spaces at the <<nearest transit station>> station?”

Individualization can also take place later in the cover letter.

“You are important to the success of this study – no matter how much or how little you travel. You may be the only household on <<street>> that has been randomly selected for participation in the study.”

Survey Fact Sheet

The design of the cover letter involves a tradeoff between supplying more information and the need to keep the letter short and punchy so that it gets read, instead of just skimmed. Sometimes, when survey teams feel it is necessary to provide additional information about the survey, they include a one-page fact sheet or pamphlet. Figure 6.15 shows the survey fact sheet from a recent household activity survey.

Survey Questionnaire

The development of mail survey questionnaires and the mail component of telephone-mailout-mailback surveys requires the survey team to consider both the extremely difficult wording issues discussed above and issues of survey layout, as well. The quality of the layout of self-administered survey will affect:

• Overall non-response rates;

• Item non-response rates;

• Response quality;

• Survey coding quality; and

• Data entry and editing efficiency.

Figure 6.13

The importance of questionnaire layout on self-administered survey materials is illustrated by the results of an actual household products panel survey effort in 1980.¹³ Figure 6.16 shows two mail survey questions that were asked of the same 4,000 respondents about the same products within a year of each other. The only difference between the surveys in which the two questions were offered was the questionnaire layout.

The question on the left produced results which were consistent with the expectations of the study sponsor. The question on the right produced the dramatically different results shown in the figure. These results were found to be invalid in clarification telephone calls. After discovering the problem, the survey team theorized that the line on which the other brand was to be entered was too close to the box around the question, and that respondents counted from the bottom to check their brand.

It is fortunate (or perhaps unfortunate) for the survey team that the study sponsor happened to be the manufacturer of one of the two products in question. The survey team had a strong sense of the relative market shares of the two products from previous waves of the panel, so the problem was discovered and rectified. For travel surveys, there are generally no recent survey data to test the validity of questions, and so survey teams need to be as careful as possible in designing the layout of questionnaires.

Fowler provides the following guiding principles for developing self-administered questionnaires:¹⁴

1. A self-administered questionnaire should be self-explanatory. Reading instructions should not be necessary, because they will not be read consistently.

2. Self-administered questionnaires should be restricted to closed-end answers whenever possible. Checking a box or circling a number should be the only task required. When respondents are asked to answer in their own words, the answers are usually incomplete, vague, or difficult to code, and therefore are of only limited value as measurements.

Figure 6.14

Figure 6.15

3. The question forms in a self-administered questionnaire should be few in number. The more the questionnaire can be set up so that the respondent has the same kinds of tasks and questions to answer, the less likely it is that respondents will become confused; also, the easier the task will be for the respondents.

4. A questionnaire should be typed and laid out in a way that seems clear and uncluttered. Photoreduction, or other strategies for putting many questions on a page, actually reduces the response rate compared with when the same number of questions are spaced more attractively over more pages.

5. Skip patterns should be kept to a minimum. If some respondents must skip some questions, arrows and boxes that communicate skips without verbal instructions are best.

6. Provide redundant information to respondents. If people can be confused about what they are supposed to do, they will be.

Since household travel/activity surveys often collect very detailed information from all household members, they can become quite repetitive and boring for respondents. Survey designers have sought ways to use graphical and tabular formatting to both improve the presentation of the survey materials and to ease the burden on respondents.

The tabular and graphical formats are used both on mailed survey instruments and on interviewer scripts. The formats improve the appearance and user-friendliness of the survey forms for respondents and interviewers, because they help organize the information logically and unambiguously. On the other hand, the formats tend to require shortened forms of questions and the informal wording of questions. The survey team needs to be especially careful when using tables or graphics that they do not introduce any of the wording problems discussed above.

Still, if the tables and graphics are carefully-designed, their advantages far outweigh their potential disadvantages. Survey teams should seek opportunities to present the survey questions in clear, logical, and visually interesting formats. The widespread availability of desktop-publishing and graphics software greatly enhances the ability of survey teams to use special formats.

Survey teams should consider the use of colored paper and different ink colors to enhance the attractiveness of the survey materials, to improve the organization of the different survey materials in the survey packet, and to aid in data entry. Using different colored paper for each survey form makes it easier for respondents to keep them straight, and provides a useful mechanism for simplifying written instructions or data retrieval calls. For instance, an interviewer could say, “Now I would like you to read me the answers you filled out on the pink form.”

Using ink colors other than black has a few advantages, as well. First, the forms can be made more visually interesting for respondents. In addition, since many respondents will fill out the forms in black ink, or pencil using some other color for questions will make it easier for data entry specialists to pick up the answer, or for the respondents, themselves, who could be asked to read what they wrote. The recent Research Triangle survey used green ink to improve the data entry.

Household and Vehicle Forms

One common approach for improving the layout of survey questionnaires in household travel/activity surveys is to separate out groups of related questions into stand-alone survey forms. Travel diary data are almost always recorded in separate diary booklets, and, increasingly, survey teams are using household and/or vehicle forms, that seek the same type of information for all members of the household and all the vehicles available to the household.

Figures 6.17 and 6.18 show examples of different formats used to collect household and vehicle data from respondents. Each form is designed to be a separate one- or two-page instrument. In surveys that use such forms, respondents are generally asked to complete several different forms. For instance, a household survey might include a household form, a vehicle form, a travel diary, and a stated response exercise, which are all separate forms. This design is appealing to respondents, because it logically categorizes the questions, it improves the visual layout of the questions, and it allows them to feel a sense of accomplishment as they finish off individual forms.

Travel or Activity Diary

Virtually all recent household travel/activity surveys seeking diary information have presented the diary questions in an easy-to-complete tabular format, rather than as a series of unformatted questions. Diaries are usually organized in booklets, one per household member, that record trips or activities, one (or a few) per page.

The several diary pages shown in Figure 3.2 and in Figures 6.4 through 6.11 illustrate the many different diary layouts. The Chicago (Figure 3.2), and Ontario (Figure 6.11) surveys collect information about each activity or trip in a single column. This columnar format was first developed for a 1973 West German trip-based diary survey. The design has come to be known as the “KONTIV” format, and has been used in many surveys since.

Figure 6.16

Figure 6.17

The Bay Area (Figure 6.4), Tucson (Figure 6.5), and New York MTA (Figure 6.7) diaries collect trip information in rows. This format is easy for respondents to follow, but it is limited in the amount of data it can collect. The Bay Area format is actually a memory jogger, in which only the key trip details are collected. The telephone data retrieval call collected the more detailed information.

The row-wise format uses space efficiently. The MTA diary questionnaire was produced in a small booklet measuring only 8.5 inches by 3.5 inches. This convenient size makes it easier for respondents to carry the questionnaire with them as they travel.

The other diary pages shown, the British diary (Figure 6.6), the Research Triangle diary (Figure 6-8), the Detroit diary (Figure 6.9), and the Portland diary (Figure 6.10) use a diary format for which each activity or trip is recorded on a single page of the diary booklet. This format is the most graphical and readable of the three. It is especially well-suited for surveys relying on mailback data retrieval. The British survey was a self-completion instrument and, therefore, required the added clarity of the format. In addition, although the Detroit survey used telephone data retrieval, the form used for this survey was based on an earlier activity-based diary that appeared in a telephone-mailout-mailback survey in Boston.

Memory Jogger

Because of the level of detail to be reported, household travel/activity survey diaries usually need to be several pages. Although survey designers have produced the diaries in attractive booklets, it is not generally convenient for respondents to carry the booklets with them during the diary period. Survey designers suspect that a high proportion of respondents complete their diaries at the end of the diary period, rather than during it as they asked to do. This can lead to recall problems and inaccurate trip or activity reports.

To reduce this problem, past survey teams sometimes included a memory jogger, a one page trip or activity record where only the most important details of travel are recorded. The memory joggers were designed to be more convenient for respondents to keep with them as they travel around, so it was believed that respondents would be more likely to use them.

For most of these studies, at the end of the diary period, the respondent was asked to copy the information from the memory jogger into the diary and to supply the missing details that the diary required. Unfortunately, it has been found that in many cases, respondents complete either the memory jogger or the diary, but not both. Therefore, it is recommended that memory joggers not be combined with complete diaries.

As discussed previously, for some studies with telephone data retrieval, a simplified diary is the only travel information the respondent is asked to complete. Details of the trips and activities on the simplified diaries are obtained directly through the telephone interview. In these designs, past examples of memory joggers are likely to be more useful in terms of instrument design than complete diaries.

Figure 6.19 shows an example memory jogger from the recent Boston survey.

Forms for Special Questions and Exercises

Some special survey questions, such as stated response questions, often require respondents to examine certain information, to sort printed cards, or to perform some other similar task. This requires the survey team to include even more materials in the survey mailing. Frequently, the materials for these questions are purposely made to be visually interesting for respondents, which enhances respondent willingness to perform the exercises. But, when respondents first receive the mailing, these materials can be distracting.

To limit the level of confusion and distraction that respondents will feel if they are bombarded with too many loose materials, it is recommended that such materials be placed in smaller envelopes within the main mailing. The envelopes can be labeled with a simple message, such as “Survey Materials for Question 13. Please see survey instructions.”

Reminder Card

Survey teams that ask respondents to complete diaries or to record certain information on particular days often include a card reminding respondents of their day-of-record. The respondent is encouraged to hang the card in some central location, such as on the refrigerator, to remind themselves (and other household members if data are being sought from entire households) that they need to perform their survey duties that day.

Figure 6.20 shows a reminder card from a recent survey effort.

Figure 6.18

Figure 6.19

Cover Letter for Follow-Up Survey

In general, the cover letter for a follow-up mailing of a household travel/activity survey needs to cover the same points as the cover letter for the original mailing. However, this letter should stress the motivational points made in the earlier letter, rather than the informational points. The key message of this letter is that the sponsoring agency needs this household’s travel information.

Follow-Up Survey Materials

In general, the survey materials sent in follow-up mailings are the same as for the main mailing. However, the survey team needs to ensure that any dated materials in the original mailing are changed to reflect the follow-up mailing dates. Of particular importance in this regard are any dates associated with travel or activity diary periods. Usually, survey teams establish new diary periods for follow-up survey respondents. If this is the case, the survey team needs to ensure that none of the materials sent in the follow-up reference the old date.

Materials for Surveys with an Interview Component

Even though respondents never see any of the survey materials related to the telephone survey, the design of these materials can have a dramatic impact on the quality of the overall survey effort. Household travel/ activity surveys often involve very long and involved telephone inter-views. It is essential that these interviews go as smoothly as possible, and for this to happen, the interview forms need to be designed to relieve the interviewer of as much burden as possible.¹⁵

If a survey contractor is used on the survey, the manager will be able to convert the survey team’s questionnaire into a usable interview script for either a PAPI or CATI technique. If no survey firm is involved, the survey team should work closely with the telephone interviewers prior to any

Figure 6.20

data collection to establish the most useful presentations of questions and responses. In almost all cases, surveys that do not use survey contractors will use PAPI techniques, so issues that the survey team should decide on with the interviewers are:¹⁶

• Standard questionnaire typeset conventions (e.g., questions are typed in capital and lower case letters, instructions are in capital bold letters, etc.);

• Techniques for recording answers;

• Rules for recording open-ended items;

• Standard skip pattern conventions (e.g., color-coded responses and questions); and

• The use of tables and matrices to record data items.

6.7 Pretesting Household Travel/Activity Surveys

1. What components of the survey design need to be tested?

2. What pretests are needed to adequately test the household travel/activity survey?

3. How should pretest results be evaluated?

 Section Summary

Survey Design Components to be tested 6-153

Pretesting Procedures 6-154

Office Test 6-154

Questionnaire Test 6-154

Pilot Survey 6-154

Evaluation of Pretest Results 6-155

Survey Procedures and Logistics 6-155

Survey Questions and Questionnaires 6-156

Survey Responses 6-157

Adequacy of Survey Population Definition and
Sampling Frame 6-157

Sample Size Estimates 6-158

Response Rates 6-159

Survey Completion Time and Cost 6-161

Alternative Approaches and Methods 6-161

Pilot Survey Schedule 6-162

THIS PAGE INTENTIONALLY LEFT BLANK

 6.7 Pretesting

The complexity and size of household travel/activity surveys makes careful and thorough pretesting an essential element of the survey implementation process. The pretest offers the survey team the opportunity to analyze the household travel/activity survey design elements before it is too late to change them. However, survey teams often fail to take full advantage of the pretests by failing to conduct them in earnest or by conducting them too late in the survey implementation process.

The section summary lists the three overriding questions with regard to pretesting. Essentially, the questions are: 1) what does one pretest?; 2) how does one conduct the pretests?; and 3) how does one analyze the results?.

Survey Design Components Analyzed in the Pretest

As discussed in Chapter 2.0 of this manual and reiterated below, one pretests as many aspects of the survey as possible by performing the exact same steps as the actual survey. Household travel/activity survey pretests provide the survey team with the opportunity to:

• Refine fieldworker, interviewer and office worker procedures and logistics;

• Test and revise question wording, sequencing, and formatting;

• Determine the range of potential responses to questions (such as how many trip or activity spaces are needed on diaries) and identify unexpected responses and respondent behavior;

• Identify problems with the sample population and the sampling frame;

• Develop preliminary estimates of the variance in key variables to help establish final sample sizes;

• Estimate response rates;

• Estimate the survey completion time and cost; and

• Compare alternative approaches to gathering certain data items.

The Pretesting Process

Despite the potential advantages of careful and exhaustive pretesting for household travel/activity surveys, all too often the pretest is the first part of the survey process that is cut back when time and cost pressures build.

The ideal approach for conducting pretests is outlined in Chapter 2.0. The process has three steps: the office pretest, the questionnaire pretest, and the survey dry-run.

In the office pretest, survey team members ask 10 to 12 colleagues and other experts who are not directly involved with the household travel/activity survey to review proposed procedures, and more importantly, to examine the survey questions and the questionnaire. The office pretest can be conducted formally or informally, and usually is a very effective way to identify problems with the questionnaire and with specific questions.

Based on the advice of this expert group, the survey team should revise the questionnaire content, wording, sequencing, length, and format. If the revisions are serious, the revised questionnaire should be brought back to the experts for another round of corrections.

Once the expert group is fairly comfortable with the questionnaire, the survey team should perform a questionnaire pretest. Household travel/ activity survey questionnaires, and particularly heavily-formatted question types like diaries, should be tested on non-experts because they are often confusing to people without knowledge of transportation planning or people who are unfamiliar with survey questionnaires, and because the surveys often rely upon respondents’ understanding of technical (and sometimes ambiguous) terms and expressions. During the questionnaire pretest, respondents are administered the survey and are asked to describe any problems or areas of confusion that they encountered.

Often, these pretests are personally-administered even when the ultimate survey will not be. It is becoming increasingly popular to conduct this portion of the pretest as part of a formal or informal focus group of 10 to 15 participants. This allows the survey team to observe first-hand how respondents react to the survey; something which is usually impossible for mail surveys and only partially possible for telephone surveys.

Based on what the non-experts say about the questionnaire, the survey team may decide to revise the questionnaire, and to re-start the pretesting process. More likely, the revisions will be minor, and the survey team will be ready to take the survey instrument to the third phase of the pretest, the household travel/activity survey dry-run, or pilot survey.

In this part of the pretest, the survey team actually completes the survey on a small number of respondents, following all planned survey procedures as closely as possible. Survey teams for large household travel/ activity surveys usually conduct pilot surveys with 100 to 300 res-pondents, often testing slight variations in design. Smaller survey efforts and most market research efforts have pretests with 30 to 50 respondents.

In the pilot survey, or dry-run pretest, all the field implementation and data processing tasks should be performed in an identical manner to the full survey effort. This effort should be “a cradle-to-grave” testing of the entire survey study, from drawing the sample, to conducting the survey, to geocoding responses, to analysis of responses (including perhaps trip linking, sample weighing and expansion, and imputation of missing data). The pretest will be unable to test issues that arise when the survey is applied to the high volume of final respondents, but ideally, the pretest can be used to ensure that all aspects of the survey effort are in place and are operating as expected prior to the beginning of any data collection. Note that if a full dry-run is performed, almost all of the final survey procedures and programs will be developed, improving the efficiency of analysis of the full survey.

Evaluation of the Pilot Test Results

The analysis of pilot study results is described in detail in two recent publications by E.S. Ampt et al.^{1 2} This section briefly describes the analysis of the eight survey design components listed above, but survey teams can expand upon these analyses whenever the pilot survey data permit.

Analysis of Survey Procedures and Logistics

Once the pilot survey is completed, the survey team should compare the actual amount of time and level of effort required to complete individual components of the survey effort to the levels that were predicted prior to the study. Those components that cost more or took longer than expected should be evaluated in detail. The survey team should determine whether the planned procedures had problems and/or whether the individuals responsible for completing the procedures did not complete them correctly.

The survey team should also consider whether inefficiencies in the survey implementation process are likely to be magnified by the much larger final survey effort. For instance, if preparing labels for a mail survey based on a recruitment call is a little slow with a pilot test of a few hundred responses, it may be a major problem with a survey with thousands of responses. Some aspects of the pilot survey process will necessarily be different from the main survey effort, but many procedural and logistical survey components can be evaluated.

If problems are identified, the survey team will likely want to change training procedures or to re-design staff responsibilities. It is important that the actual survey staff work on the pretest. If interviewers are being used, the ones that are scheduled to conduct the survey should also conduct the pretest. If only experienced interviewers are used in the survey pretest, the pretest will not accurately reflect actual conditions.

Analysis of Survey Questions and Questionnaires

Richardson, Ampt, and Meyburg recommend the following analysis of the questionnaire:³

1. Do instructions appear to have been read and followed?

2. Are definitions clear? Are there any consistent misinterpretations on the part of either the respondent or the interviewers?

3. Are questions clear and unambiguous? Are there signs that respondents or interviewers have misunderstood the intent of the question?

4. Do the cover letters appear to have been read and understood? Was the survey information telephone number used by any respondents?

5. Do the answers to individual questions indicate any problems? Are special techniques such as attitude rating scales producing valid answers? Too much bunching of answers may indicate a leading question or badly chosen categories. Too many “Don’t know” responses might indicate a vague question, a confusing question, or an unimportant question. Too many refusals to a question may indicate that it should be asked more delicately, the order of questions should change, or the question should be omitted.

6. Is there any evidence that the questionnaire is too long? Too many unanswered questions or hurried answers towards the end of the questionnaire indicate that perhaps the questionnaire is too long for the amount of interest shown in the subject matter.

7. Is the sequencing of questions clear? Are interviewers asking and/or respondents answering questions that do not pertain to them? Are more branching and skipping instructions needed? Is it clear what question should be answered next after a branching question?

Questionnaire problems should be handled in the same way as for the earlier pretests. If major questionnaire revisions are needed, the questionnaire should be redesigned and the survey team should consider restarting the pretest task. Otherwise, minor questionnaire problems should simply be addressed.

Analysis of Responses

The survey team should evaluate completed questionnaires for layout problems. All responses should be easily read by coders and data entry specialists. The survey team should ensure that enough space is provided for potential responses, and that the questionnaires do not require interviewers or respondents to try to fit written responses in extremely small areas. A common problem on factual questions is that an “other” response requests a more detailed description, but the layout of the form makes it difficult or impossible for the respondent to comply. If a diary form is used, the survey team should ensure that the number of potential activities and trips on the diary is adequate for the diary period without being excessive (causing the survey to appear even more challenging).

The survey team should also determine whether any questions or parts of the survey forms have been systematically overlooked by respondents, and conversely whether sections of the questionnaire that should be skipped are being completed.

Analysis of the Adequacy of the Survey Population Definition and the Sampling Frame

Because the output of the pilot survey, like the output of the actual survey, will usually be the input datafiles for one or more travel models, the survey team can evaluate the pilot survey returns to determine whether the survey population and sampling frame have been defined properly for the likely analyses. The survey team should determine whether individuals outside of the scope of potential analyses are being asked to complete the survey. If they are, the survey team should consider ways to redefine the survey population, to improve the sampling frame, or perform better screening procedures. There is no sense in expending resources on individuals or households that will not factor into the final analyses.

Similarly, the survey team should attempt to identify segments of the survey population (geographic areas, income levels, household types, etc.) that did not seem to be getting contacted in the pilot survey. In most cases, missing population segments are caused by the small pilot survey sample sizes, but the survey team should ensure that these segments are, in fact, represented in the sampling frame. Usually, Census data sources can be used to identify missing segments of the population of interest.

Analysis of Sample Size Estimates

The pilot survey provides preliminary information on how precisely the actual survey sample sizes will measure certain parameters. As discussed in Chapter 5.0 and Section 6.5, the actual precision of survey-derived parameters can only be known after the final survey is complete and the data collection resources have been depleted. To estimate the necessary sample size that will provide adequate precision before the survey, the survey team must estimate the variance in key variables. The pilot survey represents one of the best sources of information on the likely final variances.

The survey team should calculate the variances and coefficients-of-variation for the key pretest results, and estimate the likely final survey precision levels. The survey team may determine the need for either more or less completed surveys based on this information. The necessary sample size to produce a desired confidence level and a maximum sampling error is given by Equation 5.8a:

(Eq. 5 8a)

where:

CV = Coefficient of Variation

Z = Z-Statistic

d = Relative precision level

² = Variance of the random variable in the population

m = Mean of the random variable in the population

The pilot test provides the survey team with reasonably accurate estimates of what the variance, ², and the coefficient of variation, CV, will be in the final survey. By plugging in the pilot test derived coefficients of variation and the desired precision and confidence levels into the above equation, the survey team can determine the minimum final sample size. Based on this calculation, the survey team can increase or decrease the survey sample size target or adjust desired levels of precision and confidence.

Shiffler and Adams have found that for small sample pilot surveys, the variance and coefficient of variation estimates tend to systematically understate the final survey’s actual variance and coefficient of variation.⁴ The authors suggest applying the correction factors shown in Table 6.20 to calculated sample size estimates that are based on small sample pilot surveys. For a pilot survey with ten responses, Shiffler and Adams suggest that it is necessary to increase the sample size calculated with Equation 5.8a and the pilot survey’s coefficient of variation by 7.1 percent.

It is recommended that survey teams perform pilot surveys with more responses than are shown in the table, but because household travel and activity survey samples are usually stratified, it is likely that individual strata will have pilot survey responses in the ranges shown.

Of course, whenever possible, the survey team should use more than the minimum calculated sample size because, even if the above correction factors are applied, there is no way to know if the final variance and coefficient of variation will be higher than for the pilot study. The above corrections simply equalize the probability that the calculated sample size will be sufficient.

Analysis of Response Rates

The survey teams should carefully track the levels and types of non-response encountered in the pilot survey. The biggest question in this regard is whether response levels are different than the survey team has anticipated. Depending on the survey method, variations in response rates can have a significant effect on the survey cost and completion schedule. In addition, higher than expected non-response levels may severely weaken the confidence one can have in survey-derived parameters.

With 30 to 50 completed questionnaires, the survey team can begin to get a picture of what the actual response rates and survey completion times are likely to be. This will allow the team to get a much more accurate estimate of the survey fieldwork costs and the amount of time the survey will take. If these actual measures are significantly different than had been estimated, the survey team will want to consider drastically changing the survey method and techniques. Of course, this will entail re-performing the survey design, sampling, and organization tasks, to some extent.

In addition to measuring non-response levels, the survey team should, to the extent possible, surmise the causes of the non-response. If any identifying information is available for both the pretest respondents and non-respondents (such as the location of their homes or type of housing unit),

Table 6.20

the survey team may be able to determine whether non-response bias is present. Some practitioners have used follow-up interviews with pilot survey non-respondents to develop strategies in the final survey design that address the potential biases.

Analysis of Survey Completion Time and Cost

If the pilot survey is carefully monitored, the survey team will be able to obtain fairly accurate estimates of the times and costs of each element of the survey fieldwork. Of particular interest, in this regard, is the time of any interviews that are conducted. Interview time is an important design parameter because interviewers are generally paid by the hour, and because as interview times increase, so do the rates of incompletions and non-responses.

If the final survey times and costs are estimated to be longer and/or higher than the sponsoring agency has available, the survey team can:

• Change survey procedures;

• Omit some questions to shorten the survey;

• Lower sample size targets;

• Increase the data collection budget; or

• Cancel or postpone the final survey work.

Note that if contractors and consultants are being used, the sponsoring agency should include in the contract an allowance for this decision to be made. Often, agreements are formulated in which it is to neither party’s advantage to consider the full range of choices. Under these conditions, the usefulness of the survey pretest is diminished.

Analysis of Alternative Approaches and Methods

During the survey design effort, it is common for survey team members and/or other experts to disagree on the best approach to be used for certain elements of the survey design, or for the survey team to simply not be sure what is the best approach. The pilot survey is the ideal place to select between alternatives because slightly different procedures or instruments can be given to different groups of pretest respondents, and then can be compared somewhat objectively.

The recent NCTCOG household activity survey pretest involved comparing:

• Two different diary recording periods (24-hour versus 48-hour);

• Two different diary formats (complete diary versus simplified diary);

• Two different data retrieval methods (CATI versus mailback); and

• Three different types of incentives.

To perform these types of controlled experiments, the pilot survey sample size may need to be increased, especially if the different elements being compared are likely to interact with each other.

Pilot Survey Schedule

Usually, the pilot survey results do not imply the need for drastic changes in the survey design prior to the final survey, but because revisions of some type are to be expected, it is essential that the survey team complete the pretest well in advance of the planned fieldwork period. In addition, the survey team needs to allocate sufficient time and money resources to pretesting, so that they are prepared to make changes in the questionnaire and procedures based on the outcome of the pilot survey.

6.8 Training and Briefing for Household Travel/Activity Surveys

1. What topics need to be covered in fieldworker training and briefing sessions?

2. How should fieldworker training and briefing be conducted?

 Section Summary

Training and Briefing Topics 6-165

Training for Telephone Interviews 6-165

Briefing for Telephone Interviews 6-166

Training and Briefing for Mail Survey Office Workers 6-167

Conducting the Training and Briefing 6-167

Training and Briefing Sessions for Telephone Interviewers 6-167

Training and Briefing Sessions for Mail Survey
Office Workers 6-168

THIS PAGE INTENTIONALLY LEFT BLANK

 6.8 Training and Briefing

Regardless of the level of effort that the survey team puts into the household travel/activity survey design, the success of the project is ultimately up to the interviewers and survey office staff who actually implement the effort. If survey workers are unprepared for the work, the survey will suffer in terms of schedule, budget, and quality. Following the convention established in Chapter 2.0, we define two tasks for the preparation task, training and briefing. Training involves teaching workers their jobs or refreshing their memories about the basic aspects of their jobs. Briefing involves teaching workers about the nature of the particular household travel/activity survey, and going over specific issues related to this job.

The two broad issues related to training and briefing workers to conduct household travel/activity surveys are listed in the section summary. This section briefly describes these issues.

Training and Briefing Topics

Since household travel/activity surveys generally involve mail surveys, telephone surveys, or most likely both, the survey workers who will need to be trained and briefed include telephone interviewers and survey office workers.

Training Topics for Telephone Interviews

The goal of the telephone interviewer training sessions are to bring the basic skill levels of interviewers up to the point where they can consistently perform their jobs well. For professional interviewers, the topics covered will be ones they have heard before. However, given the interviewers’ importance in the survey process, it is most certainly worth reviewing good practices, and preventing the development of bad habits.

The following topics should be addressed:

• Basic Survey Research Principles – Why interviewing people is a useful and accurate way to obtain information;

• Importance of Quality Interviewing – How interviewers can contribute to survey bias, and the difficulty (impossibility) of correcting these problems;

• Respondent Motives – How and why respondents evade questions or fail to provide the truth, and how the interviewer can help to reduce these problems (and how interviewers can inadvertently make them worse);

• The Importance of Consistency Between Interviewers – How to be a “standardized” interviewer, not too detached, not too enthusiastic;

• Ways for Handling Refusals – How to be, and the importance of being both polite and persuasive;

• Productivity Expectations – Quality and quantity of interview data expected from them, and their refusal rate;

• Supervision – How they will be monitored, and where they can find help;

• Interview Scripts – Conventions used on the written or computerized questionnaire;

• Recording Data – Where and how to enter data;

• CATI Procedures – How to run a computerized system, and what to do if problems are encountered (CATI surveys);

• Progress Forms – Procedures for completing tally sheets (PAPI surveys); and

• Employment Considerations – Interviewer pay and benefits, as well as other administrative procedures, like timesheets.

Briefing Topics for Telephone Interviewers

The briefing session for telephone interviewers should address each of the topics listed in Chapter 2.0, as well as more specific telephone interviewing issues. The briefing should include:

• Purpose of the Survey – Including sponsorship, analysis goals, expected uses of the survey data;

• Description of Sampling Approach – Respondents sometimes ask about how they were selected;

• Use of the Call Sheet or CATI Calling Procedures – How interviewers will record which numbers have been tried, and which need to be tried;

• Introductory Language and Screening Questions – How interviewers begin the call;

• Details About Specific Questions – Why the questions are being asked, and why they are worded as they are;

• “Fallback” Statements – Standard replies for likely respondent questions;

• Assurance of Confidentiality – Specific steps being taken to assure respondent confidentiality; and

• Editing Completed Interviews – What to check and verify on the survey forms to ensure completeness.

Training and Briefing Topics for Mail Survey Office Workers

None of the tasks related to implementing mail surveys are likely to be very difficult for people with clerical or administrative experience, but the nature of the survey work requires that staff be particularly good planners. The mail survey office worker needs to consistently know what he or she needs to do next, and what to do after that, because deadlines of various types creep up quickly once the survey process is underway.

Therefore, the key training and briefing topic for mail survey office workers is to develop within each worker a detailed understanding of all the mail survey steps, and the amount of time and effort required in each task.

Conducting the Training and Briefing

Training and Briefing Session for Telephone Interviewers

Typically, for previously trained telephone interviewers, the training and briefing sessions can be combined. The sessions generally consist of four elements:

• Training lecture by survey manager;

• Review of the household travel/activity survey by a representative of the survey-sponsoring agency;

• Formal review of the questionnaire; and

• Role-playing and practice interviews.

Most training procedures are presented to interviewers by the survey manager. Next, the interviewers are told about the household travel/ activity survey. It is usually very effective to have a survey team member or a staff member from the sponsoring agency lead this discussion because it demonstrates to interviewers how important the survey is to the sponsor, and the personal contact motivates the interviewer.

The third element of the training/briefing session is a detailed walk-through of the questionnaire. To perform this, many recent survey teams have prepared interviewer manuals which can be used as references by interviewers once the training/briefing session is completed. These recent documents are likely to be extremely useful to new survey teams. An example manual from a recent Tucson telephone-mail-telephone survey is presented in Appendix E of this manual. Survey teams should consider contacting the sponsors of previous household surveys that employed similar methods to the ones the team envisions for copies of their interviewer manuals.

The final part of the session is used to have interviewers practice with each other and/or with supervisors. All interviewers need to go through the entire survey two or three times before actually beginning the interviews. One way of helping interviewers is to record their “role play” interviews and to ask them to identify ways in whch they could improve.

Training and Briefing Session for Mail Survey Office Workers

The briefing session for office staff involves presenting them with detailed descriptions and schedules of tasks. Office workers should be shown the likely “deadline crunches,” and the importance of the many identified deadlines.

6.9 Interviewing and Questionnaire Distribution for Household Travel/Activity Surveys

 Key Issues

1. What are the processes for conducting the household travel/activity survey?

2. What techniques should be used to monitor and maintain data collection quality?

 Section Summary

Monitoring and Maintaining Data Collection Quality in
Telephone Surveys 6-171

Supervisor Functions 6-171

Validation of Interviews 6-171

Monitoring Phone Calls 6-171

Monitoring Mail Survey Data Quality 6-172

THIS PAGE INTENTIONALLY LEFT BLANK

 6.9 Interviewing and Questionnaire Distribution

Once the household travel/activity survey data collection is underway, the survey team’s primary role is to monitor progress and quality, and to be prepared to correct problems as they (inevitably) arise.

Monitoring and Maintaining Data Collection Quality in Telephone Surveys

One of the major advantages of centralized telephone interviewing facilities is that they provide the opportunity for monitoring the telephone calls as they take place. Usually, survey contractors have trained supervisors to perform the following functions:

• Monitor interviews by listening in on the telephone calls, and, if CATI is being used, follow along with the data entry.

• Validate completed questionnaires in PAPI surveys. Once an interviewer finishes an interview, he or she checks the completed form to make sure it is complete and legible, and then turns the completed form into a supervisor, who validates that the form is in fact complete and legible.

• Ensure call sheets and tally sheets are being completed and processed correctly by respondents.

• Provide assistance to interviewers who have unexpected problems.

Sometimes, supervisors or survey team managers randomly call respondents for whom interviewers claim to have completed their interviews to verify: 1) that the interview did, in fact, take place; and 2) confirm the answers to a few key questions.

Many survey contractors are able to allow survey managers and clients to monitor survey calls remotely. Survey team members can listen from their own phones to the telephone interviews without being heard by interviewers or respondents. By monitoring pretest and actual telephone surveys, survey team members may be able to head off potential problems before they have too large of an effect on survey results.

Monitoring Mail Survey Data Quality

During the fielding of mail surveys and telephone-mailout-mailback surveys, the burden is primarily on the respondents. The survey team can do very little to maintain the survey’s quality once it reaches potential respondents. However, the survey team is likely to remain quite busy throughout the fieldwork period collecting and evaluating bits of information and early survey results in an effort to monitor the survey’s quality and to determine if any procedures should be revised in follow-up mailings.

The survey team will receive its first piece of information on survey data quality soon after the first mailing of either a pre-notification letter or of the mail survey materials, with the return of undeliverable mailings. There are a number of reasons that the mailings might be returned, including a wrong address or an outdated address for someone who has moved, and even in the telephone-mailout-mailback survey where people are asked to provide address information, some percentage of returned mailings is to be expected. The survey team should determine whether the number that are returned is in line with prior expectations. If it appears too high, then it is likely that there is either something wrong of a clerical nature or that there is something wrong with the sampling frame.

Shortly after surveys are mailed, some completed forms will trickle in to the survey office. Survey team members should analyze these early returns fairly carefully to determine if any identifiable trends in item non-response or response errors can be detected. In addition, the survey team should evaluate the physical condition of the returned surveys, and whether the survey instruments appear to be sufficient in terms of space and layout to obtain readable responses. If problems are detected with early returns, some modifications and clarifications might be able to be made for follow-up letters and questionnaires. Finally, the survey team will need to respond to inquiries and complaints from respondents and potential respondents as they occur. It is usually very difficult to plan for staff time for this period, because of the variability of potential tasks that could pop up.

6.10 Coding and Data Entry for Household Travel/Activity Surveys

 Key Issues

1. What needs to be done to translate survey responses into raw data?

2. What techniques are available for coding complex responses?

 Section Summary

Translating Survey Responses Into Usable Data 6-175

Coding Complex Responses 6-175

Data Entry 6-176

Once the household travel/activity survey data are collected, the survey team needs to begin the process of transforming raw responses into usable data. Coding survey responses and entering the coded data into a database are the first steps of data processing for the household travel/activity survey. These tasks have become substantially easier due to improved computing capabilities – in fact, for CATI surveys, the tasks have been completely folded into the data collection task –but for mail surveys and PAPI telephone surveys, the tasks remain sources of potential serious errors, and for that reason should be carefully planned and carried out.

Translating Survey Responses Into Usable Data

Once the survey instruments are finalized, the survey team can begin drafting the survey’s “code-book.” This manual will serve as a guide for staff assigned to the coding task, and eventually as an important piece of survey documentation. The first draft code-book will set the basic coding conventions for each survey question’s potential responses, and begin to outline any special coding considerations and procedures, such as the issues related to geocoding. This draft code-book will be revised after the survey pretest to incorporate any unexpected replies, and to include any new or revised questions. Finally, the code-book may be revised again as the survey results begin to filter in.

Using the code-book, survey coders go through the survey instruments, and either mark codes in the margins of the instruments or on separate coding sheets. Data entry specialists then enter the coded responses into a database file or an ASCII flat file. All codes should be clear and unambiguous, so that they can be easily seen and understood by those who enter the data.

In some instances, where all response categories are closed-ended, and the survey responses have been pre-coded, the coding and data entry task may be performed simultaneously. However, even if pre-codes are present, if a questionnaire has any complex questions or open-ended responses, it is usually better to formally code the responses before data entry.

Coding Complex Responses

The most difficult and time-consuming coding task is the coding of questions without response categories. Coding open-ended responses is a challenging intellectual task, requiring the coder to have a feel for subtleties of language, subject matter nuances, and the study area.³⁹

Coders need to be consistent over time, and they must also be consistent with each other. Most household travel/activity surveys will have sample sizes that will require coding teams, rather than having one individual set all the codes. It is essential that each coder on the team performs his or her tasks consistently with the rest of the team. Usually, all members of the coding team are asked to code some of the same surveys, so that comparisons can be made between coders.

Among the most difficult questions to code are ones that ask for geographic locations. Chapter 14.0 discusses the many aspects of this task.

Data Entry

Once all responses have been coded, the data from all self-completion forms and completed PAPI telephone survey forms should be entered into data files. Survey data are most commonly entered in ASCII flat files, with specified character columns for each coded response. Alternatively, the survey data can be entered directly into database or spreadsheet software packages. These packages can be manipulated to provide user friendly data entry displays and reasonableness checks on entered data.

It is generally cost-effective to validate all data entry by having different data entry specialists enter the same data. Professional data entry personnel make typographical errors very rarely, and the probability that two people will make the same error is very small. By comparing the two files, almost all data entry errors can be detected. The small additional cost of entering all data twice is almost always preferable to the cost and time spent sorting out errors during the editing task.

6.11 Editing and Cleaning Household Travel/Activity Survey Data

 Key Issues

1. What data cleaning tasks should be completed?

2. What can be done to validate survey responses?

3. What analytical techniques can be used to correct for non-response?

 Section Summary

Data Cleaning Tasks 6-179

Incomplete Records 6-179

Invalid Data Entry 6-179

Inconsistent Data Entry and Consistency Checks 6-180

Validation of Survey Responses 6-181

Verification calls 6-181

Aggregate Validation 6-181

Corrections for Item Non-Response and Response Errors 6-181

Strategies for Dealing with Item Non-Response 6-181

Eliminating data records 6-181

Modeling Item Non-response 6-181

Imputation of Missing Data 6-182

Strategies for Addressing Response Errors 6-183

THIS PAGE INTENTIONALLY LEFT BLANK

 6.11 Editing and Cleaning Data

The section summary page identifies three editing and cleaning tasks that can be completed for any survey effort, including a household travel/ activity survey. Once coded data are entered into the preliminary survey database, the survey team will need to spend a great deal of effort verifying the validity of the entered data, and editing the data, as necessary. Richardson, Ampt, and Meyburg summarize the data editing phase of the household travel/activity survey process, as follows:³⁹

The editing phase of the survey process is perhaps the most boring, but it is also one of the most important tasks. Most survey designers would admit that more time and effort goes into the editing task than almost any of the other tasks; and such effort is worthwhile. It is useless to proceed straight into analysis hoping that the data are free from error; there will always be errors in the data as initially coded.

Data Cleaning Tasks

Three simple types of errors are likely to be present in the raw survey dataset:

• Incomplete records;

• Invalid field entries; and

• Inconsistent field entries.

Simple visual inspection of the data file will reveal if records are of variable lengths. If a record is found to have the wrong length, then it has either been miscoded or has been entered incorrectly. The survey team should locate the original data collection instrument, and use it to correct the record.

Next, the survey team should compare all data field entries with the range of acceptable codes for the field, taken from the final code-book. The data collection instruments for records with any fields that are out-of-range should be located, and the records should be adjusted.

Finally, the raw data should be examined by comparing related fields to ensure that they consistent with each other. Richardson, Ampt, and Meyburg recommend the following consistency checks:⁴⁰

• If the last diary trip of the day is not to home, check the data collection instrument;

• If a trip’s origin and destination are the same place, check the data collection instrument;

• Compare place types at origin-and-destination to origin-and-destination activities;

• Calculate implied trip speeds, and check records with very high or very low speeds;

• Check records with unusually long trip times;

• Check records with unusually long walk trips;

• Compare age and possession of a driver’s license;

• All drivers should have licenses;

• Compare age and employment status, school status, retired status;

• Check vehicle, make, model, and body type consistency;

• Check to make sure that the number of diaries matches the number of household members; and

• Check to make sure that vehicle information has been obtained for all vehicles.

If the travel survey team has used a CATI system that has been programmed well, the CATI survey data should not have any of these data cleaning problems listed above. However, if the system failed to prevent an illogical response, the record may need to be deleted, or the respondent household may have to be re-contacted for clarification.

On the other hand, mail survey and telephone PAPI instruments are liable to have a number of data records that need to be verified and edited. Fortunately, the original data collection instruments are likely to be available for review.

Validation of Survey Responses

Survey teams should seek ways to test the validity of the collected survey data. For interviews, the most simple validation technique is to re-contact a small number of respondents to verify that they completed the survey, and that they provided certain answers in the database. In addition to allowing the survey team to check-up on interviewers, this approach is used to test the reliability of questions (whether they receive the same answers from the same respondent over time). This type of validation can also be used to a limited extent with mail surveys, provided that respondents have listed telephone numbers, or that the mail survey asks respondents for their phone numbers.

Both mail and telephone surveys can be validated with aggregate third-party data from other survey efforts, the Census, or the agency’s databases. For instance, demographic distributions should be compared to Census distributions. In addition, preliminary trip-rate information could be compared with existing model estimates.

Corrections for Non-Response and Response Errors

Strategies for Dealing with Item Non-Response

In every household travel/activity survey, some respondents will be unwilling or unable to answer all the questions they are asked. In particular, respondents often refuse to answer income questions. If the survey team intends to use income data in later analyses, they need to do one of the following:

• Ignore the non-response;

• Use a modeling variable to describe the non-response; or

• Apply statistical procedures to impute missing or incorrect data items.

In the first strategy, the survey team develops travel models and other analyses with the data records that are complete. These records with missing data are not used in the analyses. This strategy is the most straightforward approach for getting into the analysis, and for obtaining results, but the resulting analyses are hampered by higher levels of imprecision (due to smaller numbers of input data records) and potential non-response bias. If the incomplete records are treated as non-responses, and the survey team has high-quality socioeconomic demographic data, (such as Census data) the bias may be reduced somewhat through the application of correction weights. This correction is described in the next section.

A second approach for dealing with missing data fields is to develop models and analyses using one or more parameters that indicate whether a record is missing certain data items. These parameters are used to explain the non-random nature of item non-response and limit the effect of the bias, but they can be problematic when the estimated models are applied.

Suppose a multiple regression trip generation model were developed from a household travel/activity survey, and that a dummy variable for missing income information (equals one if income data are missing; zero if income data are present) is included in the model estimation. If the coefficient on the summary variable is positive, and significantly different from zero, the model indicates that income non-respondents travel more than income respondents (all other things being equal). The model parameter identifies and quantifies the relationship between income non-response and travel levels. The problem is that when the regression equation is applied to the population, it is necessary to predict how many and which members of the population would have answered the income question if they had been asked it.

Two methods can be used to apply such models. First, the parameter can simply be ignored during application, and be used only in estimation. Alternatively, a separate regression, discrete choice, or simulation model could be constructed to determine whether a household is likely to respond to the particular data item.

Note that the first two strategies for dealing with item non-response do not require the survey team to change the survey database in any way, except for adding a few simple dummy variables. In the third approach, missing data are actually estimated, and input into the database.

To limit the effects of item non-response, European and Australian travel survey teams commonly develop statistical relationships between different survey variables, which can then be used to predict the value of a particular missing item. These methods are also beginning to gain acceptance to some degree in the U.S.⁴¹

In its most simple form, data imputation involves using completed survey records to develop mathematical models that try to predict the missing components of the other survey records. For instance, if a household travel survey consists of 1,200 records with valid responses for questions about household income and all other household variables, and 100 records with valid responses for all the variables except income, the survey team could use the 1,200 records to estimate a regression, cross-classification, or discrete choice model that relates income to the other variables, and then apply the estimated model to the 100 other records to predict their income levels. The survey records for these 100 records could then be edited to include the imputed estimates.

The problem with this simple imputation approach is that it fails to recognize that the factors that lead respondents to fail to report data items may be the same factors that are being used as explanatory variables in the imputation model. To address this, Bhat suggests estimating two simultaneous models – one model seeks to explain the reporting/non-responding phenomenon and the other develops the relationship between the variable of interest and other variables – using maximum likelihood estimation techniques.

Some travel surveyors have raised concerns that imputing data can increase bias in some cases. To some analysts, imputing survey data is the same as simply making up data. However, the recent non-response workshop at the TRB Conference On Household Travel Surveys: New Concepts and Research Needs, endorsed the use of imputation techniques, provided that they were done with care and were well-documented. All changes and corrections to the original survey data need to be noted, and where possible, the effects of these changes should be evaluated.

Strategies for Addressing Response Errors

In general, it is quite difficult for survey teams to identify response errors. If a respondent answers a question incorrectly for some reason, the survey team will almost never be able to determine it. However, for com-plicated multi-part questions, such as travel and activity diaries, response errors (primarily in the form of incomplete information) are sometimes discernible and correctable.

Travel and activity diaries, like those shown in Figures 6.4 through 6.11, often request a great deal of information on single activities or trips. It is easy (and quite common) for respondents to miss certain questions under these conditions. Fortunately, it is often possible for survey editors to determine what the respondent should have filled-in. For instance, probably the biggest single response error on travel diaries is for respondents to forget to mark whether a given time was a.m. or p.m.

Clever editing staff can often determine trip purpose, activities and modes (among other things) in incomplete diary entries by examining other trips or activities of the respondent or the respondent’s fellow household members. Although this detective work is generally slow, and tedious, the rewards, in terms of cleaner data for analyses, are significant. It is recommended that survey teams that field diary surveys budget significant resources for such efforts.

If a diary cleaning effort is undertaken, it is imperative that editors:

• Correct only those data items that can be unambiguously determined; and

• Carefully document any changes made to the survey database.

A special type of diary response error is the unreported trip (or activity). It is very important that the survey team seek to minimize these errors, because one of the most common survey analyses is the calculation of average household trip rates, and because there is evidence that unreported trips are not representative of the population of total trips.

Without other information about respondents’ trips, or activities it is usually very difficult to impute an entire trip, along with its details.

However, there are ways to detect missing trips, including:

• Are the trip ends or activity locations listed in the diaries linked properly? Does trip number N start where trip number N-1 ended?

• Do other household members report being with the respondent at a time or place not reported by the respondent?

• Do all trips back to a respondent’s home seem to be reported? Return home trips are the most commonly omitted trips in diaries.

• Does the respondent make it back home by the end of the period? Not all respondents finish the diary period at their homes, but the vast majority do.

• Does the respondent report the expected amount of time and number of instances of particular activities, such as meals, sleep, etc.?

If the data retrieval is completed with CATI techniques, these or other tests can prompt interviewers to probe for further details. When mail surveys or PAPI retrieval calls are used, the survey team should consider validation/verification contacts, and perhaps adjustments to trip rate calculation procedures.

6.12 Programming and Compiling Data for Household Travel and Activity Surveys

 Key Issues

1. Are the responses adequate for analyses?

2. How should the household travel/activity database be structured?

3. How should the survey data be expanded for future analyses?

4. What survey data summaries should be compiled?

 Section Summary

Determining the Adequacy of Responses 6-187

Unusable Data Records 6-187

Incomplete Household Data 6-188

Database Structure 6-188

Household File 6-188

Person File 6-189

Trip/Activity File 6-189

Expansion of Survey Results 6-189

Using Census Data for Expansion 6-190

Household Characteristics Used in Survey Expansion 6-190

Data Expansion with a Single Control Variable 6-191

Data Expansion with Multiple Control Variables 6-192

Data Expansion for Non-Reported Trips 6-196

Missing Household Members 6-196

Nonrespondents 6-196

Data Expansion with Choice Based Samples 6-198

Summarizing Survey Results 6-199

Survey Data Tabulations 6-199

Response Rate Report 6-201

THIS PAGE INTENTIONALLY LEFT BLANK

 6.12 Programming and Compiling Data

The key issues related to programming and compiling data for household travel/activity surveys are briefly discussed in this section. The programming and data compiling activities are directly related both to the objectives of the survey analysis and to the design of the survey which is also guided in part by the ultimate survey analysis objectives.

Determining the Adequacy of Responses

Once individual records have been cleaned and edited to the maximum extent possible, they should be evaluated for completeness and usability for analysis. The survey team should examine the database, and drop any data records that:

• Have been developed from out-of-scope individuals or households that may have been surveyed by mistake;

• Exhibit serious basic flaws in consistency and logic; or

• Are too incomplete to be useful in the anticipated analysis.

Sometimes, despite efforts to the contrary, survey responses are obtained from people who live outside the study area or for people who should have been screened out of the survey process. Before conducting any analysis, the records for these individuals should be dropped from the database. These records should not be considered in the calculation of response rates, nor should they be included in any tabulations.

The second type of data record that should be removed from the database are those that contain numerous inconsistencies and illogical information. Despite efforts to edit and clean the data, some records will still contain basic flaws that will make them unusable. If attempts to verify or correct response problems are unsuccessful, and the problems are felt to cover several data fields or particularly important pieces of information, the corresponding data record should be marked unusable and deleted from the analysis dataset. In this case, the record would be counted in the same way as a refusal in the calculation of the final survey response rate.

In household travel/activity surveys in which all household members provide information, it is common to be unable to get the requested information from one or a few individuals. Usually, repeated efforts are made to track down these elusive respondents with follow-up calls and mailings, but there will almost always be a few people who cannot be contacted or who refuse to participate. In these cases, it is important that the survey team have a clear description of what will be considered a “completed household”. For some transportation planning analyses, it is essential that data be collected for all household members. For other analyses, planners can work around having some incomplete data by weighting or by estimating key parameters for the missing individuals.

As discussed in Section 6.3, the survey team should determine the definition of a completed record prior to any field work, because the definition will affect the survey follow-up strategy. It is quite easy to spend hundreds of dollars trying to complete surveys on households with elusive members. The survey team should consider at what point it is more cost-effective to give up on a particular household, and find a different one, so the definition of an acceptable response is important.

If a household data record is unacceptably incomplete, according to whatever standard definition the survey team sets, then the record should be marked unusable and dropped from the dataset. The record would be counted the same way as a refusal in the calculation of the final survey response rate.

Database Structure

Household travel/activity surveys can differ considerably with respect to the level of detailed travel information that is collected in the survey, the focus of the survey on the travel behavior of the household as a whole versus its individual members, or the broader analysis framework that may require an activity-based rather than the more traditional trip-specific approach.

The database for a household travel survey could therefore be developed to accommodate a hierarchical structure which can include up to four layers nested within each other. For the most detailed household travel/activity survey, the corresponding units of analysis at each layer would thus be:

• The household treated as a whole;

• Each surveyed member of the household treated individually;

• Each trip made by the household members that were surveyed; and

• Each activity carried out as part of a specific trip.

The original raw database for the household travel/activity survey would include all the household-related information and would thus consist of as many records as there are usable responses. However, such a database could also be used as a basis to create three additional databases that could be used to support the analysis of individual travelers’ trip making, the analysis of specific trips, or the analysis of activities undertaken by the respondents. As a result, each of the four datasets that would be created would have all the information necessary for transportation analysis and modeling.

The contents of each dataset can be distinguished between information that is unique to each layer of analysis and information that is common to two or more hierarchical layers. Information that uniquely characterizes each household such as income and household size, location of residence, automobile ownership, and household trips rates would be linked to or included in each person, trip or activity data record. Similarly, information that uniquely characterizes individual members of the household such as age, sex, occupation, education, location of workplace, and auto availability would be linked to or be repeated in each trip and activity record generated by the respondent. Finally, if activity-based design is being used, trip-specific information on mode choice, total cost, travel time, and distance traveled would be linked to or be repeated in the record of each activity that was part of the same trip.

Although such a database structure results in a set of four internally-consistent datasets from the household survey, it does not always represent the most efficient way of storing information. This is particularly true in the case of extensive surveys with large sample sizes. In such cases, a relational database structure could be used instead of the four-file structure to reduce the amount of overlap and the repetitiveness of information across the different layers.

Expansion of Survey Results

The objective of data expansion is to make it possible to reach valid conclusions about the entire study population based on the survey results. Data expansion for simple random samples is straightforward. Suppose a sample of 100 households is drawn from a population of 65,000 households, and that 12 households of the 100 are found not to have any automobiles available to them. We can expand the survey results to say that there are 65,000 x 12/100=7,800 households in the study area that do not have an available auto. Unfortunately, the expansion of household travel and activity survey data is complicated by two factors:

• Household survey sampling is generally performed using stratified random sampling procedures, rather than simple random sample procedures; and

• Invariably, because of random sampling error and various survey biases, such as nonresponse, interviewer error, etc., the actual survey sample will not be totally representative of the survey population in terms of the variables that explain travel behavior.

Expanding the household survey data set, given these limitations, is discussed below.

Using the U.S. Census Data for Survey Expansion

Fortunately, in most cases, the survey team has an excellent source of information on study area respondents from which the household survey data may be expanded – The U.S. Census data. The Census summary tape files provide the survey team with detailed socioeconomic and demographic summary information for small geographic areas. In addition, detailed cross-tabulations of key travel-related data are available from the Census Transportation Planning Package.

These and other Census products are described in Appendix B of this manual.

If the household travel/activity survey is performed more than a few years after the Census, the travel survey team may need to consider ways to update the Census data to more accurately expand the household survey. Some potential strategies for using the Census data for expansion purposes in non-census years include:

• Accept the last Census as the most accurate information source and use the data without adjustment;

• Make ad-hoc adjustments to the Census data based on available Census, state, and local estimates and, perhaps, on econometric models developed from Census PUMS data; and

• Use one of the first two options temporarily, and then re-expand the data with the next Census data or with interpolated estimates based on the two Censuses.

Household Characteristics Used in Survey Expansion

The survey team should expand the household survey data so that the demographic characteristics that best describe variations in key travel behaviors are made to match those of the study area population. The survey team should select the characteristics based on the expected uses of the survey results and on the availability of data on the survey population.

For most household travel and activity surveys, a key analysis will be the determination of household trip rates and trip generation. Key expansion variables that are commonly used in this situation include:

• Geographic location (such as super district);

• Household size;

• Number of vehicles available per household;

• Number of workers per household;

• Type of housing unit; and

• Household income.

The first three variables are probably the most common in the U.S., but for highly specific analyses, the survey team should consider other appropriate variables. Expansion variables are typically categorized into a small number of categories for expansion. The variables used for expansion can be limited to those used for stratifying the sample (assuring a stratified sample) or include other variables, as well.

Data Expansion with a Single Control Variable

When a survey team is seeking to use a single variable for data expansion, the process is not complicated. The expansion factors are just the actual number of people in each data category for the population, divided by the number derived from the survey. Suppose a household travel survey for a region obtains the geographic distribution of households shown in Table 6.21.

Table 6.21 Geographic Distribution of Household Survey Responses for an Example Survey

Subregion	CBD	NE	E	SE	S	SW	W	NW	N	Total
Responses	115	114	111	97	108	113	106	120	116	1,000

The relatively equal distribution across subregions is consistent with a survey effort that was stratified on the basis of geography. Table 6.22 shows the actual number of households in each subregion based on Census data.

Table 6.22 Actual Distribution of Households for the Example

Subregion	CBD	NE	E	SE	S	SW	W	NW	N	Total
Responses	1,200	3,800	4,400	9,000	8,500	5,200	11,400	10,300	10,400	64,200

The expansion factors for the household survey are shown in Table 6.23.

Table 6.23 Calculated Expansion Factors for the Example

Subregion	CBD	NE	E	SE	S	SW	W	NW	N
Responses	10.4	33.3	39.6	92.8	78.7	46.0	107.5	85.8	89.7

This means that each of the 115 survey records for households in the CBD is equivalent to 10.4 actual households (1,200/115=10.4). Each of the 116 survey records from the North Subregion represents 89.7 households (10,400/116).

This simple expansion process may be applied on two or more variables, if the expansion data provide cross-tabulations. Suppose the survey team wants to expand the survey data based on both household size and number of vehicles available per household. As long as this cross-tabulation is available for the population, from the Census or another source, the process is the same as shown above. Tables 6.24, 6.25 and 6.26 illustrate an example of this expansion.

Note that the calculations are the same as for the previous example, except that the two categories representing one-person households with more than one vehicles are combined. It is generally a good idea to avoid expanding variable categories with either a very small number of responses or with a small actual population because very high or very low expansion factors may result in skewed analyses.

Data Expansion with Multiple Control Variables

Often the survey team faces a situation where it would be advantageous to expand the survey data using two or more variables for which there are no cross-tabulations for the population. This may occur when the Census Bureau does not publish such a cross-tabulation or when the Census cross-tabulation is not yet published. Census Summary Tape File data have become available two to three years before the Census Transportation Planning Package cross-tabulations.

In this situation, survey teams generally rely on the marginal variable totals for controls, and use an iterative method to develop the specific expansion factors. As an example, suppose the survey team wants to develop control totals based on both single variable controls, shown above, but for some reason cross-tabulations are not available. Table 6.27 shows the household survey cross-tabulation of the variables of interest.

Table 6.28 shows the information available for use as control totals. In this situation, the survey team develops the expansion factors iteratively by

Tables 6.24, 6.25 and 6.26 (3 tables on one page)

Table 6.27

Table 6.28

first developing expansion factors based on the row control totals, then by adjusting these preliminary factors to match column control totals, and then repeating the process until the expansion factors produce a reasonably accurate representation of both the row and column control totals. This iterative procedure, commonly referred to as iterative proportional fitting or the Furness Method, is the same process that is used in the Fratar trip distribution model.

Data Expansion Procedures for Non-reported Trips and Trips by Nonrespondents

Because the calculation of trip generation rates is often an important use of household survey data, many survey teams perform data expansion exercises to incorporate into the survey database the likely number of trips that were missed in diaries that were not returned. To include trips by household members that failed to complete the diary, survey teams have used completed diary data to develop person-based trip generation models that relate socioeconomic and household relationship variables to the number of trips being made. These trip generation models are then applied to the individuals who failed to complete the diary so that the household’s total trip generation may be approximated.

Another potential survey data expansion option for some household travel and activity surveys is to use survey follow-up data to estimate household trip rates for non-respondent households.

Limited research has shown that the travel and socioeconomic characteristics of respondents that reply to each wave of follow-ups are successively closer to the characteristics of those who never respond.^{42 43 44} By tracking the characteristics of respondents at each follow-up stage, and extrapolating, the survey team can estimate the characteristics of non-respondents. Figure 6.21 illustrates the extrapolation process that Wermuth, Richardson, Ampt, and Meyburg advocate based on empirical survey data from Germany and Australia. The estimates for non-respondents can be used to adjust the survey estimates or to simply determine whether non-response bias is likely to be present for a survey effort.

Figure 6.21

Data Expansion with Choice-Based Samples

It is becoming a common practice to supplement the household survey’s random sample or stratified random sample with a smaller choice-based sample to increase survey representation of certain types of households, people, or trips. The most common form of choice-based sampling in household travel and activity surveys is to recruit a certain number of households who are known to have transit riders. This is usually done to improve the mode choice model estimation process.

Survey teams need to remember that samples of this type need to be expanded separately and differently from the random sample or stratified random sample. Although the trip or activity records from these households may be used in conjunction with the trip or activity records of the other households for disaggregate analyses, such as multi-nominal logit mode choice models, they usually cannot be expanded to the general survey population because: 1) their sampling frame includes an undefined portion of the study area population, and in some cases, 2) the samples are not probability-based. The choice-based records can only be used for population expansion if:

1. The choice-based or targeted sample has been drawn from a sampling frame that describes a defined population using a probability based sampling method;

2. The defined population is able to act as a separate sample stratum for the larger survey effort, and the members of this stratum can be identified in the larger sampling frame; and

3. Information is available for defining the total size of the defined population and for setting control totals.

These conditions are rarely met in practice, but it is important to note that choice-based data are often collected for disaggregate mode choice model development. Some of the most common analytical procedures to develop these models, such as multinomial logit modeling, are better performed on data that are unexpanded.

A number of excellent discussions of the practical aspects of household travel and activity, survey data expansion can be found in the transportation literature. ^45,46,47

Summarizing Survey Results

Although this manual does not discuss the presentation or analysis of survey results, three procedures for summarizing household survey results provide important diagnostic information about the survey effort that can be used in later analyses. It is recommended that all household travel and activity survey teams:

• Perform a detailed set of survey data tabulations; and

• Calculate actual precision levels for key survey variables; and

• Formally calculate and report the survey response rate.

These procedures are discussed below.

Survey Data Tabulations

Tabulation of the survey data is a very important, although sometimes overlooked, aspect of the survey analysis. Its value is even more pronounced given the small amount of effort that is required to specify the analyses and produce the summary statistics for a preliminary analysis. A careful review and interpretation of the preliminary one-way and cross-tabulations could be instrumental in:

• Finding and correcting errors in respondents’ answers, as well as errors due to coding and programming;

• Making reasonableness checks for variables included in the survey;

• Identifying survey responses with extreme values and determining whether the response to a particular question should be treated as an outliner or whether the data are unreliable, in which case the data record should be dropped from the analysis;

• Obtaining a fairly accurate picture of the distributions for different variables of interest and identifying differences due to geographic, socioeconomic, or choice behavior factors; and

• Uncovering response patterns that may be useful for subsequent more detailed analyses.

To accomplish these objectives, a set of preliminary survey data tabulations need to be specified. These would include both frequency analyses of the survey variables (also referred to as one-way tables) and cross-tabulations (also referred to as two-way tables) that relate the frequency of a particular variable to other variables of interest. For example, a frequency analysis would provide us with the total number of households with no, one, or two or more vehicles, while a cross-tabulation would further provide the same information on automobile ownership in the study area broken down by county.

For each continuous variable, the distribution of variable values in the sample is obtained by measures of the mean, median, and standard deviation. For categorical (discrete) variables and for continuous variables that can be easily grouped into different categories (e.g., 10-minute categories of travel time), the distribution in the sample can be assessed by examining the frequency of values in each variable category both in absolute and percentage terms. For example, the mean household trip rate in the study area, the distribution of automobile ownership, and the share of transit could provide some useful preliminary information on existing travel patterns in the area.

A preliminary assessment of relationships among variables of interest can be obtained by cross-tabulating each variable of interest to a variety of geographical, socioeconomic, or choice-based variables. These tables can offer a means of checking the reasonableness of existing differences by market segment and can also provide the rationale for further more detailed types of analysis. For example, differences in the mean household trip rate, automobile ownership, and transit share by county or by household income can help validate the existing data and can provide insights into the factors affecting trip making and mode choice behavior.

Calculation of Precision Levels for Key Survey Variables

Prior to conducting the survey, the survey team will have made an estimate of the required sample size to achieve some pre-determined levels of precision and confidence for important survey variables. Once the data collection is complete, the survey team can use the relationships presented in Chapter 5.0 to determine the actual precision of the survey estimates.

The degree of precision for any variable is dependent on its variance and mean a pre-determined desired statistical confidence level, and the sample size and population size. If the actual values of these parameters are different than the pre-survey estimates, the precision level for the variable will also be different than expected. The survey team should report the degree of precision on all key survey variables and survey-derived parameters.

Response Rate Report

As discussed, a household survey’s response rate is a basic measure of the quality of the data collection process. An unusually low response rate is an indication to users of the data that any analyses they conduct could be biased to a greater extent than they are accustomed. A strong response rate is an indication that the input data to their analyses are more likely to be accurate. However, before any conclusions can be drawn from response rate information, it is important to understand exactly how the rate has been calculated. Unfortunately, despite the fact that the term “response rate”, has a very specific technical definition, it is frequently used carelessly and incorrectly. “Response rate” has come to mean many things to many people.

This problem is not restricted to household surveys or travel surveys. Misuse of the term has been, and continues to be, common in many fields. In the early 1980s, the Council of American Survey Research Organizations (CASRO) commissioned a blue-ribbon committee to establish standardized definitions of survey response rate and completion rates.

The basic definition of response rate is: ⁴⁸

Response Rate =	Number of Completed Interviews with Reporting Units
	Number of Eligible Reporting Units in the Sample

There is an interpretation of this basic definition for each type of household survey effort. However, it is likely that the number of eligible units may not be known exactly, and that the response rate can only be approximated. For example, in a telephone survey the eligibility of phone numbers that are never connected (always busy, unanswered, or routed to an answering machine) cannot be determined. The best strategy is to make repeated attempts until contact is made.

One useful way to provide response information is to develop a process chart, similar to those shown in Chapter 3.0 (Figures 3.8, 3.10, and 3.11) manual, for the specific survey effort, and then to record the disposition of all survey field work on the chart. With this information, analysts can calculate each completion rate of the survey effort. This type of reporting will be especially useful for future household survey teams trying to plan fieldwork resources.

Table 6.29 presents an example of response rate information from a recent household travel/activity survey in New Hampshire. This table provides information on the number of contacts, the contact percentage, eligibility percentage, reasons for ineligibility, participation rate, and overall response rate. The response rate that corresponds to the CASRO definition is 19%; however, some analysts also consider the percentage of those recruited (53% in this case). It is recommended that complete summaries such as Table 6.29 be prepared for all household surveys to provide the variety of information needed by different analysts.

In the New Hampshire survey, the eligibility of 28,125, or 97%, of the 29,036 different phone numbers was eventually determined. The response rate of 19% was computed assuming that none of the phone numbers not reached was eligible. This is obviously an oversimplification; the true percentage of eligible respondents among the 911 numbers that were never connected to cannot be known. Given that survey recruitment was done in the evening, it is likely that the eligibility percentage of the non-connects was lower than for those that were reached (since many businesses are not open at night). However, even in the unlikely event that all of those numbers were eligible, the response rate would not have been much different (18%). The true rate, of course, is somewhere in between. The fact that the range can be computed so narrowly illustrates the advantage of continuing to attempt to contact potential respondents who are not reached on the first attempt.