During a speech at the Eisenhower Presidential Library and Museum here, Gates said the Pentagon, whose base budget has nearly doubled in size since 2001, must rein in its spending, focus its more limited dollars on priorities, and restructure its sprawling bureaucracy to run in a more cost-effective and efficient manner.
"Given America's difficult economic circumstances and parlous fiscal condition, military spending on things large and small can and should expect closer, harsher scrutiny," he said. "The gusher has been turned off, and will stay off for a good period of time."
As the military begins work on the 2012 budget, Gates said he is directing the armed services, joint staff, major functional and regional commands and the civilian side of the Pentagon to take a "hard, unsparing look at how they operate - in substance and style."
Gates told reporters before the speech that he needs to cut roughly $10 billion in overhead cost -- which accounts for 40 percent of the defense budget -- to get about 3 percent real growth in modernization and other accounts.
That growth, he said, is necessary to sustain the military's combat power during war while also making investments in weapons systems for the future.
The savings, he said during his speech, must be "sustained and added to over time."
Gates signaled that his review will look at everything, from eliminating unnecessary or duplicative commands to reducing the number of general officers in the military's ranks - moves that could rile both military leadership and Capitol Hill.
"The Defense Department must take a hard look at every aspect of how it is organized, staffed and operated - indeed, every aspect of how it does business," Gates said.
"In each instance we must ask: first, is this respectful of the American taxpayer at a time of economic and fiscal duress? And second, is this activity or arrangement the best use of limited dollars, given the pressing needs to take care of our people, win the wars we are in, and invest in the capabilities necessary to deal with the most likely and lethal future threats?"
As an incentive, Gates said the services would be able to keep within their own budgets whatever money they saved from cutting overhead.
Gates said the private sector has flattened and streamlined its middle and upper echelons, but the Defense Department "continues to maintain a top-heavy hierarchy that more reflects 20th century headquarters superstructure than 21st century realities."
Speaking on the 65th anniversary of V-E Day when Germany's unconditional surrender ended World War II in Europe, Gates pointed to the fact that, two decades after the end of the Cold War and resulting cuts in U.S. forces in Europe, the U.S. military still has more than 40 generals, admirals or civilian equivalents on the continent.
"Yet we scold our allies over the bloat in NATO headquarters," he said.
Gate said he plans to review which headquarters and secretariats are primarily in the business of reporting to or supervising other headquarters or secretariats, as opposed to overseeing "activity related to real-world needs and missions."
He also will review whether executive or flag-officer billets could be converted to a lower grade to create "a flatter, more effective and less costly organization."
And he said he will look at how many commands or organizations are conducting repetitive or overlapping functions and could be combined or eliminated altogether.
But Gates ruled out the possibility of another base realignment and closure round, saying it would likely be too difficult politically.
"It may be in the 'too hard' column," he told reporters. "I think being able to further consolidate facilities is always a good idea but there are just huge political challenges associated with it. That's not an important element of what I'm trying to do."
Meanwhile, Gates warned that the Defense Department's approach to how it builds requirements for new programs must change. Criteria, he said, should be based on a real-world, threat-based context.
"We're not just going to roll over to preserve programs that we think we don't need regardless of where the pressure is coming from," Gates said.
Gates, who shocked Congress last year with a fiscal 2010 budget request that cut many prized weapons programs, said he has "just sort of come to this in the last couple of weeks." He added that he plans to play an integral role in this process.
During his speech, Gates warned that the Pentagon's problems do not need to be solved with more study or legislation.
"It is not a great mystery what needs to change," he said. "What it takes is the political will and willingness, as Eisenhower possessed, to make hard choices -- choices that will displease powerful people both inside the Pentagon and out."