Delphi Technique

Researchers and evaluators are often faced with a situation in which outside parties have conflicting views on a topic under study. Opinions can differ over various aspects such as the appropriate goals of a program, the types of outcomes that should be investigated, or the merit and worth of particular activities.  The Delphi Technique is a useful decision-making tool that can be used to build consensus in situations like these, as well as in situations without marked disagreement.  Through a process characterized by anonymity of responses and iterative and controlled feedback, a group can arrive at conclusions on issues such as priorities, goals, or courses of action. The Public Health Informatics Research Laboratory has made use of the Internet to facilitate and expediate the Delphi Process using state-of-the art online technology.

Uses of the Delphi Technique

The Delphi Technique was originally developed by the Rand Corporation to gather opinions from a group of experts (Patton 1986).  Its use has spread to many areas and is widely recognized as a consensus-building tool.  For example, the Delphi Technique might be used to develop standards by which to evaluate a controversial new educational program where various parties cannot agree on what issues are important.  The process can also be used to forecast outcomes of new policies or situations.  It is currently being applied in a variety of fields including administration, medicine, technical forecasting, and program evaluation.  The Public Health Informatics Research Laboratory has used online Delphi techniques to understand complex issues from community health initiatives to funding priorities for statewide agencies to the design process for software applications.

Issues

The anonymous and iterative nature of this process allows participants to state their opinions, discover other viewpoints, raise issues and potential problems, and come to a decision, all without meeting in person.  This process retains the positive aspects of group feedback and collaborative work and yet eliminates the group dynamics that may impede decision making.  While some people are uncomfortable about making decisions anonymously because they may perceive little accountability to their opinions, a greater understanding of the issues can result through the candor of participant responses.

Steps in Conducting a Delphi Study Online

The first step is to form a panel of experts and involved parties in the area under research.  A survey on various aspects of the issues is posted on the Internet on a password protected site that the panel members can access. This survey can vary depending on the research needs, but it often consists of open-ended questions presented in an unstructured format to allow for creativity, exploration, and expression in participants' responses.  Participants document their opinions and supporting facts, then submit the surveys via the World Wide Web to the researchers.  The researchers synthesize the information into a single document detailing patterns of responses and unique answers.  Care is taken that no comment or opinion is traceable to its author.  Round Two begins when this synthesized information is posted on the Internet so that participants can review and respond to it.  At this stage, participants may change their minds, continue to support their original position, or raise further issues.  These responses are submitted to the researchers, who again compile all responses into a document.  The third and successive rounds proceed in a like fashion.  Similar rounds continue either for a preset number of iterations or until the researchers judge that the comments have ceased to change substantively on successive rounds.

A similar process can be used in face-to-face meetings, where a moderator compiles the answers and presents them immediately, while ensuring confidentiality.  There are several computer programs designed to assist in this process.  While some anonymity and time for reflection are lost, using this process in a face-to-face meeting can provide quicker results.

References

1. McKillip, J. 1987. Need analysis: Tools for the human services and education. Newbury Park, CA: Sage Publications.
2. Patton, M.Q. 1986. Utilization-focused evaluation. (2nd ed.). Newbury Park, CA: Sage Publications.