Professor Kayvon made a comment in class today about how no compilers today can automatically parallelize code, or at least not to the extent that humans can. This question is probably more for the TAs/professor, but will compilers one day (soon?) be able to achieve automatic parallelization of code, or in other words, is there an expiration date on parallel programmers?
TypicalChazz
This seems unlikely. A lot of the parallelization done by the compiler and hardware is Instruction level parallelism(ILP) as discussed in class. The source code written by software developers is translated to assembly code by the compilers, and the machine that executes this assembly code attempts to leverage it's architecture to execute as many instructions as possible(taking into account data dependencies of course). This is achieved via mechanisms such as branch prediction etc.
Since the compiled assembly code is 'contractually obligated' to obey the data dependencies set forth by the software developer's code, a compiler that could 'automatically parallelize code' would have to be smart enough to understand what the source code was trying to do, and also be smart enough to come up with a complementary parallelized version of that code. It would practically have to be a program that could write itself.
cacay
Automatic parallelization is an active research topic and there are some simple implementations. For example, gcc will sometimes rewrite loops to use vector extensions (SIMD). Full automatic parallelization is extremely hard in theory and practice. You need to prove that your translation does not change the meaning of the program. Program equivalence is undecidable in general. Also, just because a program is more parallel in theory does not mean it will run faster due to the reasons we talked about in class.
lament
If such a compiler is created in the next couple of decades, it seems reasonable that in SOME cases it will be better at "parallelizing" code than humans - while in other cases it is worse than humans. That seems to be the story with most successful cases of making machines more "intelligent" - they have strengths and weaknesses compared to human abilities.
himmelnetz
With regards to compilers achieving parallelism, I like to think of the goal as something similar to 210 where we write code that theoretically is parallel but then don't actually care that much about what exactly happens under the hood (and as most know, the par function is just sequential). Programmers write using things like map to modify arrays and then the compiler writes the best code for that situation. That might be parallel code, or if for some reason the compiler knew the size of the array code that doesn't even use parallelism because the cost is too great. Compilers can be pretty smart (see lab 5 of 15-411) and freakishly aware of their target platform and its strengths. But as stated above, there is lots of complications about parallelism which makes it non-trivial.
But hey, memory management is non-trivial, but yet we have some pretty good garbage collectors today. Manual management in C is faster, but for most applications the perf gains of garbage collecting is quite good.
Professor Kayvon made a comment in class today about how no compilers today can automatically parallelize code, or at least not to the extent that humans can. This question is probably more for the TAs/professor, but will compilers one day (soon?) be able to achieve automatic parallelization of code, or in other words, is there an expiration date on parallel programmers?
This seems unlikely. A lot of the parallelization done by the compiler and hardware is Instruction level parallelism(ILP) as discussed in class. The source code written by software developers is translated to assembly code by the compilers, and the machine that executes this assembly code attempts to leverage it's architecture to execute as many instructions as possible(taking into account data dependencies of course). This is achieved via mechanisms such as branch prediction etc.
Since the compiled assembly code is 'contractually obligated' to obey the data dependencies set forth by the software developer's code, a compiler that could 'automatically parallelize code' would have to be smart enough to understand what the source code was trying to do, and also be smart enough to come up with a complementary parallelized version of that code. It would practically have to be a program that could write itself.
Automatic parallelization is an active research topic and there are some simple implementations. For example, gcc will sometimes rewrite loops to use vector extensions (SIMD). Full automatic parallelization is extremely hard in theory and practice. You need to prove that your translation does not change the meaning of the program. Program equivalence is undecidable in general. Also, just because a program is more parallel in theory does not mean it will run faster due to the reasons we talked about in class.
If such a compiler is created in the next couple of decades, it seems reasonable that in SOME cases it will be better at "parallelizing" code than humans - while in other cases it is worse than humans. That seems to be the story with most successful cases of making machines more "intelligent" - they have strengths and weaknesses compared to human abilities.
With regards to compilers achieving parallelism, I like to think of the goal as something similar to 210 where we write code that theoretically is parallel but then don't actually care that much about what exactly happens under the hood (and as most know, the par function is just sequential). Programmers write using things like map to modify arrays and then the compiler writes the best code for that situation. That might be parallel code, or if for some reason the compiler knew the size of the array code that doesn't even use parallelism because the cost is too great. Compilers can be pretty smart (see lab 5 of 15-411) and freakishly aware of their target platform and its strengths. But as stated above, there is lots of complications about parallelism which makes it non-trivial.
But hey, memory management is non-trivial, but yet we have some pretty good garbage collectors today. Manual management in C is faster, but for most applications the perf gains of garbage collecting is quite good.