Previous | Next --- Slide 20 of 44
Back to Lecture Thumbnails
haboric

Upon worker node failure, completed map tasks are re-executed because their output is stored on the local disk(s) of the failed machine, which is inaccessible by other nodes. On the contrary, there is no need to re-execute the completed reduce tasks because their output is stored in global file system, which can be accessed by other nodes. More information on map-reduce fault tolerance can be found in section 3.3 of mapreduce paper: http://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf.

Lawliet

From what I understand, any number of worker nodes can fail and it would still be able to work correctly (after restarting tasks as necessary). Or is there some critical point at which the computation fails because too many worker nodes failed?