[don't know how to quote things here, so I will just use the plain old method]

I think so, too: as soon as you give more resource X to spend, there will be a need (generated) for using X up completely.
You are also right, CPU efficiency (power) is getting better and better. But then as soon as we pack more cores in box, we are still getting a higher and higher power and compute density. To actually make use of all this (assumed we can, which we cannot currently), we have to have to-be-written software that will make use of the incredible 500k IOP/s PCIe SSDs.
My understanding is, the more we will increase the core count, the more we will see we simply cannot make use of it anyways — and that we need a fundamental change in our approach.

There was a famous paper recently (about the Barrelfish OS) that basically stated: with a 16+ core system, shared memory will not scale any more. Modelling the hardware like being distinct CPU entities that communicate over a shared bus, needs less CPU cycles for intra-core communication than using the shared memory at all. This is quite and important and also shocking result, as it shows our scale-up-by-increasing-core-counts approach is doomed.
Basically, we can either choose to waste more and more cycles when having more and more cores (this is somewhat like the inversion of Moore’s law!) with newer generations of CPUs, or we start solving the problem by scaling our systems down and distribute load to more effective low-power entities.

Regarding the “wimpy nodes approach”: correct, not with all of these systems that are currently being tested you will be able to run say Windows 7. But there is some hope like ARM said to be building an ATOM equivalent with HW virtualization instruction set – paired with at least double performance and some <5W power draw for a 2 core system. There are rumours of Dell shipping these systems to some of their large-scale hosting customers (guess you read this at El Reg, too) who run some standard LAMP VMs on them.
This might be a viable approach for the future. These systems are basically an intelligent harddrive from a space and power-consumption perspective, powerful enough for task, that's it. This is quite a neat approach and quite a fundamental one.

To sum it up: I think we have software and software-design crisis, less a hardware crisis. We should seriously start thinking about scaling-down our per-node performance and scale-up our networking gear likewise.

let me start by thanking you for the reply and a different perspective on all of this.

I agree that power draw and density are major problems (and let’s not forget cooling). On the other hand we have seen a big dip in power consumption on a per core basis. Newer cpu’s have features such as Cool’n'Quiet, but with consolidation and virtualization initiatives became inherently useless since we are overprovisioning until our CPU’s are near the 80% use limit (sometimes even over).

Basic problem: As soon as we provide more power we use it. Technologies like virtual machines or hardware/software partitioning (zones and such) only make the problem worse.

And yes, wimpy nodes are known to me, and they can work great. If you check out the example of the US army buying PS3′s for their cell architecture and implementing special software for a custom purpose, I can only state this is a great idea and a great implementation, but it’s not generic enough for most implementations. Custom development would drive costs up in a major way.

And I again agree when you say that there are fundamental changes we need to make, but this is a general performance/utilization problem where the technology provided just does not grow as fast.

Also looking forward to your response,
Bas

This post was mentioned on Twitter by BasRaayman: [Blog] The RAM per CPU wall: http://is.gd/5Fux7…

actually, I don’t think servers will get larger and larger. Power draw and density is already yet a problem in many datacenters. With increasing core count, optimizations at the gate level will not scale as fast as core-# scales.
More importantly, newer systems have a lower “getting-things-done per Watt”-ratio than specialized, “wimpy nodes” (e.g. project FAWN). With +~8 times more output with the same power draw, there is a cost argument for running such lower-powered more-effective systems (again, datacenter as a problem, etc.).
James Hamilton has released several highly recommend blog posts with having all this in mind: http://perspectives.mvdirona.com/2009/11/30/2010TheYearOfMicroSliceServers.aspx

If all this is true, this has a lot of interesting implications. Our software landscape has to change fundamentally as the networking layer has to change. Software needs to be more distributed, concurrent and parallelized, software has to handle failures as being the norm, not being a special case, etc.
[I will stop here].

Looking forward for your replies,
Martin