INTRODUCTION:Why Surveys?

The International Institute of Business Analysis (IIBA®) defines surveying as “a means of eliciting information from many people, anonymously, in a relatively short time” (IIBA® 2006, 177). Surveys are also called questionnaires. In the IIBA® Business Analysis Body of Knowledge® (BABOK®), the terms questionnaire and survey are used interchangeably. We will do the same throughout this book.

In the nineteenth century, Sir Francis Galton invented the questionnaire for use by psychologists (Rathus 2004, 23). Surveying, therefore, is not a new technique, but surveys are used less often in projects than are other elicitation techniques. A recent study conducted by the IIBA® indicated that 28 percent of all participants do not use surveying as an elicitation technique, 50 percent occasionally use the technique, and only 22 percent of participants regularly use surveying to elicit requirements (Figure I-1).

When business analysts use surveying to elicit requirements, they examine and interpret the results, which become a source of requirements. Surveying can be used as the sole elicitation technique, but more often it is used in addition to other elicitation techniques. A survey may focus on the participant’s opinions or on factual information, depending on its objective. For business analysts, surveys can be used:

To understand users’ reactions to an existing product

To determine the relative priority of product features

FIGURE I-1: Use of Surveying as an Elicitation Technique

To identify an existing solution’s strengths and weaknesses

To identify potential scope items

As input to other requirements elicitation activities

To identify changes users would like made to an existing system

To gather feedback from a requirements workshop that can be used to improve future workshops

To gather feedback from potential vendors of commercial off-the-shelf (COTS) packages as part of a request for information (RFI) or request for proposal (RFP) process.

Surveying can be used in addition to other elicitation techniques. For example:

Interview results on a specific topic could be used as input to survey questions.

Results from a survey question might be used to elicit greater detail in interviews.

A small subset of requirements could be explored using a survey, while other sets of requirements may be elicited through workshops or interviews.

The advantages of using surveys include:

Flexibility to ask almost anything.

The ability to gather information from a large group of people.

The possibility of anonymity for participants.

Gaining access to users who are not able to participate in interviews. Some users might not be able to participate in interviews because they lack the time to commit to an interview or live in a different time zone. There could also be too many people to get input from, which would require you to interview some and survey the rest.

Simplicity of administration.

Simplicity of analysis and compilation of results.

Uniformity. All participants are asked the same questions, ensuring the consistency of data collection.

The ability to sample a population instead of requiring universal participation. The business analyst can make reasonable extrapolations from the sample to the entire population.

Using surveys also has several disadvantages:

Surveying is not always a substitute for interviewing. Qualitative data is hard to obtain in surveys.

Surveys do not provide an opportunity to ask participants follow-up questions about their responses. This is especially problematic if participants’ responses are unclear.

Participants may misinterpret questions because they cannot ask for clarification.

Surveys garner lower response rates than do other methods of gathering information, such as interviews.

Consider these advantages and disadvantages carefully before deciding whether it is appropriate to include a survey in the elicitation plan for your project.

Types of Surveys

There are three types of surveys: self-administered, group-administered, and mail-out.

Self-administered: The business analyst conducting the survey meets with each participant to explain the purpose of the survey, and then the participant completes the survey independently. The business analyst may or may not be present while the participant answers the survey. This is often a matter of preference that can be discussed with the participant.

Group-administered: A group of participants completes the survey at the same time under supervision so that discussion among participants is controlled. The business analyst answers participants’ questions. Group-administered surveys can be more effective to administer. This type of survey allows participants to ask for clarification on questions. You can be fairly confident that you will obtain a higher response rate.

Mail-out: The survey is sent to the participant, and the participant and the business analyst have no contact. Mail-out surveys can be sent as hard copies or in email, or they can be web-based. They are very useful if you have a limited budget, your questions are very straightforward, or you need to gather input from a large group of participants. Mail-out surveys do not allow the business analyst to ask participants for additional information, so strong, well-written questions are the key to ensuring valuable results.

Surveys and Structured Interviews

Surveys and structured interviews are very similar; both techniques require that questions be defined ahead of time. The advantage of interviews is that they allow the business analyst to ask participants additional questions about their responses.

When conducting an interview, either in person or over the phone, ensure that participants do not feel threatened by how the information might be used. Whenever possible, be honest with participants about how the outcome may affect an individual or group. Before beginning an interview, decide what you will say if a participant asks about potential fallout or consequences. You may have to contact a manager, a project sponsor, or a member of a communications team for advice.

If we expand the definition of survey to include interviews, there are three more methods of administering surveys: by telephone, one on one, and in a focus group.

Telephone: A telephone survey should be no more than 15 minutes in length. Each participant must be presented with exactly the same questions in the same order because participants may answer questions differently depending on the preceding questions. The survey can include any combination of open-ended or closed-ended questions (see Chapter 3 for more on question types).

One-on-one: These surveys can be longer than telephone surveys, often requiring an hour or more to complete. The business analyst meets with the participant and asks the participant each question on the survey. As in a telephone survey, each participant must be presented with exactly the same well-defined questions in the same order. The business analyst may deviate from the plan as needed to further explore a participant’s responses.

Focus groups: Focus groups are composed of prequalified individuals who share their opinions, needs, or impressions in an interactive group setting. A business analyst facilitates the discussion. Focus groups are widely used in marketing to elicit requirements or feedback. They can be used in the surveying process to elicit or verify usability requirements or to pilot the survey.

When management is planning to make important business decisions based on the data gleaned from a survey, the business analyst should follow up with interviews to verify the survey data. For instance, usability studies generally start with a survey and are followed by focus groups.

The Survey Process

Conducting surveys is an eight-step process:

1. 1.Defining the objective

2. 2.Identifying the group to be surveyed

3. 3.Writing questions

4. 4.Ordering and laying out the survey

5. 5.Piloting the survey

6. 6.Sending out the survey and following up with participants

7. 7.Analyzing and interpreting the results

8. 8.Reporting on the results.

Subsequent chapters will discuss each step in detail.

When planning a survey, consider:

How much time and what resources will be necessary to complete all survey development activities

The deadline for completing the analysis

The processes needed to collect data, contact participants, produce the survey, conduct the analysis, and produce the results report

Whether external specialists will be required throughout the survey process

How you will gain the approval required to conduct the survey and to contact members of the target audience

What data analysis method will yield results that facilitate decision-making.