There's something of a debate in the supercomputing universe about this very issue. On the one side, you've got people saying that you want a smaller number of fancy big/fast computers, since communication overhead tends to limit scalability. On the other hand, you've got people who worry about power consumption and say that large numbers of slow nodes, despite having lower utilization, might still win.

Here's a researcher looking at this from the low-power perspective. I'd say that if you want maximum performance without power constraint, then you'd go for traditional CPUs. Otherwise, maybe little ones.