In practice, as someone who has spent the last 18 months writing MR code at Quantcast, the difficult parts of writing map reduce programs all are how does one take communicative/global knowledge algorithms and either (1) change the algorithm to require only a handful of passes each with partial knowledge, or (2) create a probabilistic algorithm to solve an approximation of your problem that can be computed with only partial knowledge. No one says otherwise.
But Map Reduce does, IMO, do one thing that MPI doesn't: the easy things are easy. Hard stuff is still hard, but easy things are easy. If you have a low communication algorithm that requires only local knowledge and fits into the MR paradigm well, hadoop + hdfs makes it really simple to write and reliably execute, including automatically handling things like (i) worker nodes dying, (ii) sorters dying, (iii) storage nodes dropping out, (iv) whole steps disappearing and having to be rerun.
But Map Reduce does, IMO, do one thing that MPI doesn't: the easy things are easy. Hard stuff is still hard, but easy things are easy. If you have a low communication algorithm that requires only local knowledge and fits into the MR paradigm well, hadoop + hdfs makes it really simple to write and reliably execute, including automatically handling things like (i) worker nodes dying, (ii) sorters dying, (iii) storage nodes dropping out, (iv) whole steps disappearing and having to be rerun.