NMCI officials refuse to release polling questions
Several experts say the poll would have more validity with an outside auditor.
The Navy Marine Corps Intranet project is refusing to release customer satisfaction survey questions that have produced controversial results, program officials said last week.
Several national polling experts criticized the decision to withhold the questions and encouraged the use of an outside auditor to validate the survey results.
Texas-based defense contractor EDS is developing the system to connect all Navy and Marine Corps uniformed and civilian personnel. NMCI, the largest intranet in the world, could eventually cost the Navy more than $8 billion.
During an NMCI press conference last week, Navy officials said releasing the questions would compromise the program's ability to conduct future surveys because they want to measure user opinion on the same issues over time. NMCI officials released a statement Thursday saying they would "strongly oppose" publishing the survey questions.
"It is a bad practice to publish surveys in advance when you want people's current opinions on things," Edward Schmitz, the senior NMCI customer satisfaction official, said in the statement. "The current set of questions [is] much easier to answer than previous surveys and provide useful high level diagnostic information on service quality."
EDS conducts quarterly surveys for the NMCI office, which recently reported almost 80 percent user satisfaction. Disgruntled users have complained, however, that the new network is plagued by poor, unreliable connections and a slow delivery schedule. Critics have also expressed skepticism over the results of the customer satisfaction survey.
Some users are particularly concerned that EDS stands to reap financial rewards when NMCI customer service levels reach certain benchmarks, beginning at 85 percent customer satisfaction. As customer satisfaction levels rise, EDS has the potential to earn as much as $100 per NMCI computer for each financial quarter.
Navy officials have expressed complete confidence in the survey results and in the integrity of EDS. The NMCI program office is now conducting its first audit of the EDS survey.
Outside pollsters said, however, that the NMCI survey arrangement is not in line with industry standards and could present a conflict of interest for EDS.
"Usually there is a separation between the implementation and the evaluation process," said Cliff Zukin, a polling expert at the Eagleton Institute of Politics at Rutgers University and vice president of the American Association for Public Opinion Research. "If you are genuinely concerned with satisfaction, you would want an outside independent auditor, specifically if it involves incentives."
Eric Nielsen, senior director for media strategies at the Gallup polling organization, agreed that an objective third party could give the NMCI customer satisfaction poll more credibility.
"When there are financial incentives tied to achievement … I would want to look doubly hard," Nielsen said. "The way [NMCI] is structured, I would think that it would be good to have somebody else review it."
Both polling experts, as well as others who asked not to be identified, said the poll would also carry more weight if the questions were released.
"It's impossible to assess the results of the survey without knowing the questions," Zukin said.
Nielsen said there is leeway for opinion polling within organizations, but he also suggested it would be better to release the questions.
"It is standard practice for us, with public opinion polls, to release not only the results but the questions," he said. "This being government, what is private is public."