Success of Homeland Security ad campaign won’t be easy to measure

The government's new media campaign urging Americans to prepare for possible terrorist attacks is well-funded and has already received considerable public attention. But gauging the effectiveness of such an ad campaign is often a complex and difficult task, as other federal efforts to modify social behavior-including the anti-drug campaign-have illustrated.

The Homeland Security Department launched its citizen preparedness public service ad campaign last week, encouraging Americans to be ready for a possible terrorist attack by creating emergency supply kits, devising a family communication plan and posting emergency numbers near the phone. The "Ready Campaign" media blitz includes a series of television, radio, print and Internet ads aimed at "educating people that there are things they can do to prepare for an attack and to motivate them to take some of those actions," said Peggy Conlon, president and CEO of the Advertising Council, one of the major co-sponsors of the initiative.

But similar public service media campaigns, including those aimed at preventing drug use among young adults and encouraging Americans to eat more fruits and vegetables, have demonstrated that evaluating the success of a program aimed at influencing people's behavior is a complicated and often multi-layered process, said Stephanie Shipman, assistant director of the Center for Evaluation Methods and Issues at the General Accounting Office.

Shipman said that in most such ad campaigns, "evaluation is an afterthought, which can create problems because typically…you need a baseline" to assess the net impact a program has had on the public. A baseline for proper evaluation would have to include data collected before and after a particular campaign was launched, so that analysts could rule out separate influences that could be affecting the desired change in social behavior, Shipman said.

For example, in a hypothetical survey conducted by the Homeland Security Department on the effectiveness of its citizen preparedness campaign, interviewers could ask respondents very specific questions that relate to whether the campaign was successful or not. Such questions could include: "Did you create an emergency plan with your family?" or "Did you assemble a first aid kit?" If people answered "yes" to these types of questions, it may very well be a result of the department's ad campaign, but it could also be a consequence of long-term efforts by the American Red Cross or the American Automobile Association to encourage people to prepare for an emergency, Shipman said. Analysts must ensure that program evaluations account for those kinds of variables.

In October, GAO released a report (GAO-02-923) concluding that the results of federal media campaigns, including the anti-drug initiative run by the White House Office of National Drug Control Policy, were difficult to measure because many agencies lacked sufficient date to accurately study changes in social behavior.

Officials with the anti-drug media campaign developed an extensive survey, administered by the National Institute on Drug Abuse, to gauge the effectiveness of the ads. The survey, designed to track how children in three different age groups view the campaign, is ongoing, but early installments showed the campaign had mixed results.

Shipman said that the National Cancer Institute's "Five a Day for Better Health" program offers a good model for the Homeland Security Department to follow when it comes to evaluating the success of its initiative. The purpose of the program-to increase the consumption of fruits and vegetables among Americans to five servings daily and to educate people about the benefits of healthy eating-is straightforward and provides specific guidance for the public to follow, much like Homeland Security's "Ready Campaign." Another similarity is that the health initiative has several co-sponsors in the public and private sectors to help expand outreach efforts. The National Cancer Institute funds periodic program evaluations to determine whether its media strategies are actually helping to increase fruit and vegetable consumption among Americans.

"Public health people have done more research than most in how to get behavioral change," Shipman said. "You need to not only convince and educate [the public], but also make the environment conducive to and supportive of that change," she said. For example, part of motivating people to eat more fruits and vegetables is convincing cafeterias to stock a more appealing salad bar with these foods, Shipman said.

The Federal Emergency Management Agency, which is part of the Homeland Security Department, is overseeing the citizen preparedness ad campaign, according to department spokesman Brian Roehrkasse. He said the department is not currently using specific measures to evaluate the effectiveness of the campaign, but that based on various polls "we've already seen people responding to this type of thing." The success will be measured "ultimately in how people respond and how informed and aware they are," he said.

The Ad Council's Conlon said her organization will routinely evaluate the success of the "Ready Campaign" by conducting telephone surveys in the various markets in which ads are run to gauge public response. In the first few days after the campaign was launched, its Web site-www.ready.gov-received 2.5 million hits and its toll-free number-1-800-BE-READY-received more than 100,000 calls, Conlon said. The television ads are playing in every media market nationwide, Conlon said.

Conlon said the Ad Council has made a three-year minimum commitment to the preparedness campaign, and said early indications of its impact were verypositive. "I have never gone into a campaign with more media donated than this one, which is really a good barometer for the kind of attention I think this campaign will receive."

The initiative's other co-sponsor, the Alfred P. Sloan Foundation, a New York-based nonprofit organization that focuses on science and technology issues, paid $1.2 million to help cover the campaign's start-up production and distribution costs. According to Conlon, the Sloan Foundation developed the ad content, which was then vetted and approved by the Homeland Security Department.

A host of other public and private partners, including the Martin Agency and Ruder Finn, both advertising and public relations firms; the Yellow Pages Integrated Media Association; the Postal Service; the Salvation Army; the American Red Cross; the National Association of Broadcasters and the Outdoor Advertising Association of America are also involved in the campaign. For example, the Yellow Pages will provide information about what to do in an emergency in each of its 550 million directories next year, while the Postal Service will distribute preparedness brochures to consumers via their 35,000 post offices nationwide.

The Yellow Pages and the Outdoor Advertising Association of America alone plan to allocate about $48 million in free advertising over the life of the campaign, Roehrkasse said. "No tax dollars are being used," he said.