Previous | Next --- Slide 17 of 43
Back to Lecture Thumbnails
lilli

Hadoop is not great for iterative algorithms (a lot of them are used in ML, like in neural nets or any kind of gradient descent), because every iteration loads from disk.

ask

In some sense, the problem of writes to disk between every iteration in hadoop is analogous to the example in slide 3. Once some intermediate data has been generated, it could be more beneficial to consume it and further process it when it is still in memory.