Again this is the tradeoff when deciding at which level of abstraction to operate. The higher the abstraction the more expressive the code is. This increases programmers' productivity at the cost of losing control over the actual implementation. In the ideal world, code will be perfectly compiled into the optimal implementation, regardless of the language the program is actually written. Nowadays compilers are good but not "that" good yet, however the balance is tipping more towards high abstraction languages (Kesden's 213 assembly programming story). In fact there are a lot of research in creating better high level languages for modern computers. Google High Productivity Language Systems for some interesting readings.
The take home lesson is: use languages closer to implementation to squeeze the last drop from your hardware, and use high abstraction languages to produce complex, maintainable, and readable programs.
This comment was marked helpful 1 times.
Amanda
I liked what Kayvon had to say about how, with the data parallel programming model, one must "think functionally", despite not programming in a functional way
This comment was marked helpful 0 times.
GG
I think that Google's MapReduce Programming Model is like streaming programming to some extent. The foo is the mapper and the bar is the reducer. Instead of a sophisticated compiler, the MapReduce leverages a sophisticated infrastructure to handle the data stream, failure recovery etc. But the basic thought remains the same. This is also another example of the tradeoff that @Xiao says.
This comment was marked helpful 0 times.
kayvonf
@GG: correct, hence the name "MapReduce" ;-)
Notice that even MapReduce is a very hybrid system. It's functional in terms of mapping a kernel onto a stream of chunks. But the kernel itself is written in an imperative fashion. Thus, the framework applies constraints on the overall structure of the part of the computation that spans multiple machines (where challenges such as parallelism, fault-tolerance, etc. are very hard for an average programmer to deal with efficiently and correctly) but it does not constrain the computation inside the kernel (where its believed to be difficult for a programmer to shoot themselves in the foot, since a kernel is meant to run within a single thread of a single machine).
The design strikes a nice balance and the usefulness of the design is reflected in the popularity of the system.
Again this is the tradeoff when deciding at which level of abstraction to operate. The higher the abstraction the more expressive the code is. This increases programmers' productivity at the cost of losing control over the actual implementation. In the ideal world, code will be perfectly compiled into the optimal implementation, regardless of the language the program is actually written. Nowadays compilers are good but not "that" good yet, however the balance is tipping more towards high abstraction languages (Kesden's 213 assembly programming story). In fact there are a lot of research in creating better high level languages for modern computers. Google High Productivity Language Systems for some interesting readings.
The take home lesson is: use languages closer to implementation to squeeze the last drop from your hardware, and use high abstraction languages to produce complex, maintainable, and readable programs.
This comment was marked helpful 1 times.
I liked what Kayvon had to say about how, with the data parallel programming model, one must "think functionally", despite not programming in a functional way
This comment was marked helpful 0 times.
I think that Google's MapReduce Programming Model is like streaming programming to some extent. The
foois the mapper and thebaris the reducer. Instead of a sophisticated compiler, the MapReduce leverages a sophisticated infrastructure to handle the data stream, failure recovery etc. But the basic thought remains the same. This is also another example of the tradeoff that @Xiao says.This comment was marked helpful 0 times.
@GG: correct, hence the name "MapReduce" ;-)
Notice that even MapReduce is a very hybrid system. It's functional in terms of mapping a kernel onto a stream of chunks. But the kernel itself is written in an imperative fashion. Thus, the framework applies constraints on the overall structure of the part of the computation that spans multiple machines (where challenges such as parallelism, fault-tolerance, etc. are very hard for an average programmer to deal with efficiently and correctly) but it does not constrain the computation inside the kernel (where its believed to be difficult for a programmer to shoot themselves in the foot, since a kernel is meant to run within a single thread of a single machine).
The design strikes a nice balance and the usefulness of the design is reflected in the popularity of the system.
This comment was marked helpful 0 times.