Fabian Giesen

you do a perfect job and make A take zero time and impart no additional overhead on B in communicating with A, the factor by which you can ever speed up the program is limited by B. If B is half the original program's runtime, the program can only ever get 2x faster.

If it's 1/3 the original program's runtime, the program can only ever get 3x faster. The Wikipedia link here has some good graphs to help you visualize. I like Amdahl's Law because it's so obvious you might just think "duh, why are you telling me this?", https://en.wikipedia.org/wiki/Amdahl%27s_law …

but it's easy to make mistakes if you don't take this seriously. One such mistake happened during the time of the Cell processor and the PS3 / Xbox 360 consoles. (Both consoles made different versions of the same mistake).

Today we're in a situation where you can buy a 32-core Intel CPU for not that much money ... but you can't buy a single-core Intel CPU that has 32x as many transistors. The reason why is because we just don't physically know how to do it. The single-core 32x monster would not be

very much faster than one of the cores on the 32-core CPU. So when Intel wants to deliver more CPU power to customers, the main way they know how to do it is to add cores. Many people can't use most of those cores effectively, but some people can, so there is still a reason

to do this. But, back in the early 2000s, the situation was different. We still knew how to make single processors faster by making their pipelines deeper and making them more out-of-order and speculative. But that was starting to take a lot of transistors, so there was this

idea, hey, instead of using all those transistors to do this out-of-order stuff, we could just have (in the case of PS3) a bunch of blazing-fast SPU coprocessors that are very simple and linear, or (in the case of X360) three in-order main processors instead.

Unlike today, where we don't have a tradeoff to make ... we just know how to add more cores, we can't make one core faster ... back in that time a tradeoff was made: we can have more potential compute power, *as long as we make the slow part of our system slower*.