Although this will probably not fix the true problem, I think that we could say that mathematics fits in the middle.
Productivity
The algorithmic description of a computation can be done rather efficiently by an algorithm designer and requires only communicative language. (i.e. pseudo code, english, etc)
While it does take effort to invent an algorithm, once invented it is rather straight forward to communicate an algorithm to a peer, at least with an amount of effort proportional to the amount of complexity and difficulty in the algorithm.
High Performance
If Professor Kayvon writes a research paper describing a procedure for implementing a highly efficient algorithm, then I would claim that the description is performant, at least in a theoretical sense.
Completeness
I think that math is at least as complete as any Turing complete programming language.
The Challenge
I do not think the challenge is in creating a language for implementing all three points on the triangle. We as human beings already know how to imagine programs that fulfill all three points, but the real challenge is in the design of ways in which our mathematics can be communicated and interpreted correctly by an actual computer.
meatie
Theano (a Python library for deep learning) http://deeplearning.net/software/theano is another example of domain-specific programming framework. It also has the advantage of "transparent use of a GPU".
ChandlerBing
OptiML developed at Stanford is an example of an implicitly parallel DSL for Machine Learning.
rojo
cuDNN (CUDA accelerated Deep Neural Networks) - https://developer.nvidia.com/cuDNN is another example of domain-specific programming framework. It can be integrated with many higher-level machine learning frameworks such as Caffe, Theano and Torch for higher performance.
kayvonf
@rojo. I'm really glad you brought this one up. The difference between a library, and a framework, and a programming system is completely fuzzy, but in my opinion cuDNN falls short of my bar for a "programming system". What is provides is more like a math library: a highly optimized implementation of a neural network layers for the GPU.
Of course you can ask me where we drawn the line between a library and a language/programming system... and I won't have a good answer to that question. It's a bit religious.
rojo
@kayvonf: Thanks for the information. I would not have figured that out.
My interpretation was that the machine learning code is converted to a format that can best use the Nvidia CUDA core for the complex math operation, similar to what a Matlab does on CPU. Hence I interpreted cuDNN as framework.
Although this will probably not fix the true problem, I think that we could say that mathematics fits in the middle.
Productivity
The algorithmic description of a computation can be done rather efficiently by an algorithm designer and requires only communicative language. (i.e. pseudo code, english, etc)
While it does take effort to invent an algorithm, once invented it is rather straight forward to communicate an algorithm to a peer, at least with an amount of effort proportional to the amount of complexity and difficulty in the algorithm.
High Performance
If Professor Kayvon writes a research paper describing a procedure for implementing a highly efficient algorithm, then I would claim that the description is performant, at least in a theoretical sense.
Completeness
I think that math is at least as complete as any Turing complete programming language.
The Challenge
I do not think the challenge is in creating a language for implementing all three points on the triangle. We as human beings already know how to imagine programs that fulfill all three points, but the real challenge is in the design of ways in which our mathematics can be communicated and interpreted correctly by an actual computer.
Theano (a Python library for deep learning) http://deeplearning.net/software/theano is another example of domain-specific programming framework. It also has the advantage of "transparent use of a GPU".
OptiML developed at Stanford is an example of an implicitly parallel DSL for Machine Learning.
cuDNN (CUDA accelerated Deep Neural Networks) - https://developer.nvidia.com/cuDNN is another example of domain-specific programming framework. It can be integrated with many higher-level machine learning frameworks such as Caffe, Theano and Torch for higher performance.
@rojo. I'm really glad you brought this one up. The difference between a library, and a framework, and a programming system is completely fuzzy, but in my opinion cuDNN falls short of my bar for a "programming system". What is provides is more like a math library: a highly optimized implementation of a neural network layers for the GPU.
Of course you can ask me where we drawn the line between a library and a language/programming system... and I won't have a good answer to that question. It's a bit religious.
@kayvonf: Thanks for the information. I would not have figured that out. My interpretation was that the machine learning code is converted to a format that can best use the Nvidia CUDA core for the complex math operation, similar to what a Matlab does on CPU. Hence I interpreted cuDNN as framework.