Definition: If it's hard to make computers run faster (at least reliably), one way to solve big computing problems is to parallelize them and run different parts on different computers. The same principle is used on multi-core computers which have effectively 2 or 4 CPUs sharing the same memory.
Linking computers together in this way is known as clustering and both Linux and Microsoft support the clustering of computers. An important aspect of this is how the different computers talk to each other and of course how you write programs to make the best use of parallism. Then you get problems of synchronizing and how data is moved between the different boxes. Also you get new bugs such as race conditions where some kind of locking mechanism has to be used to prevent data being corrupted.
