Multi-processing

The New York Times has an article on multi-core processors. It's focus is primarily on the fact that Microsoft is taking the problem seriously and putting lots of money into it. The problem, if course, is that software developers don't know how to develop software for parallel computation. The article quotes David Patterson of Berkeley as saying

Industry has basically thrown a Hail Mary. The whole industry is betting on parallel computing. They’ve thrown it, but the big problem is catching it.

The "Hail Mary" reference is to the fact that the hardware industry has run into problems that makes it very difficult to make individual chips run faster. The alternative to faster chips is more chips. That also increases the raw computing power. The problem is that we don't know how to use it.

Parallelizing software is very difficult. We don't know how to do it. It's unlikely that we will solve that problem soon. So what's to do? Two things.

• We can and should focus on more loosely coupled processes. Parallelization is the attempt to break a single process down into subprocesses that can perform the same overall computation but with some of them done in parallel. Loose coupling is an attempt to think about computation in terms of multiple computations that interact less often. It's coordinating the interactions that make parallelization so difficult. If there were fewer interactions or if the interactions were less tightly coupled, the result would be easier to produce. Of course that's easier said than done. But agent-based computing has been around for quite a while, and it has produced some useful results.
  
  On the discouraging side is the fact that process algebra has also been around for quite a time and has not produced much in the way of practical results. We just don't know how to coordinate even loosely coupled processes except through very generic mechanisms. The fact that most of what we experience in life consists of loosely coupled processes should be encouraging. But we have a long way to go before we understand how it works.


• The second approach is to work on developing systems that work at many levels of processing simultaneously. Apparently our brains do that. At the same time as we are doing something we are aware of the fact that we are doing it, and we can regulate how we do it. Aspect-Oriented Programming (AOP) is a move in the right direction. It is an attempt to develop programming languages that allow the programmer to write software that both operates at the process level while at the same time operates at a meta-level that observes the operational level. AOP as currently structured attempts to map all of these processes onto a single process programming language. That, of course, is not only not necessary, it's the wrong way to proceed. In the future we will have many processors. We should be working on putting them all to work at different observational levels.