Question: In 15-213's web proxy assignment you gained experience writing concurrent programs using pthreads. Think about your motivation for programming with threads in that assignment. How was it different from the motivation to create multi-threaded programs in this class? (e.g., consider Assignment 1, Program 1)
Hint: What is the difference between concurrent execution and parallel execution?
rofer
In 15-213 our goal was to handle several concurrent events at once. In this class our goal is going to be to speed up a single task by performing many calculations in parallel.
martin31hao
I think concurrency is more about the operating system giving user the illusion that different programs can run simultaneously on single CPU core, while parallelism gives the idea of speeding up a single task by breaking it into independent pieces on different CPUs (to ensure correctness of the program).
mingf
In my understanding, concurrent execution is that multiple tasks execute interleavedly. It is an implementation detail whether they actually execute on different cores or on a single core.
Whereas parallel execution means that multiple tasks execute in different independent cores.
ESINNG
To some extent, I agree with mingf. Concurrent execution means that you start several tasks and they will run interleavedly. If the machine it runs on has several cores or only a single core but it supports hyper-threading, and the OS supports running multiple tasks at the same time, it can be executed in parallel. Besides, when you run the tasks in multiple machines or heterogeneous machines at the same time it's also executed in parallel.
So concurrent just guarantee that all tasks will have some time slot to run, and some methods will be used to schedule them. Parallelism means that these tasks can run at the same time.
I'm not sure, but I think when tasks run in parallel, they runs concurrently. And the reverse is not right.
Olorin
I agree with rofer and would like to add on.
In 213's proxy lab, we wanted to be able to write clean code to handle multiple connections at the same time, devoting some time to each user so that one didn't block another. In this class, on the other hand, such an approach wouldn't (necessarily) be useful since it doesn't (or might not) give any speed gains. With parallelism, we want to improve the speed of computations by doing as much work as possible at one time.
doodooloo
I agree with mingf and I think "interleavedly" is a right word to use for concurrent execution. An intuitive definition from CSAPP (p713), "flows X and Y are concurrent with respect to each other if and only if X begins after Y begins and before Y finishes, or Y begins after X begins and before X finishes".
As for parallel execution, I see it as a proper subset of concurrent execution. The only difference is parallel execution requires multiple processors.
For the motivations, I think concurrent programs are more concerned about not letting the processor been idle for too long, while parallel programs are more concerned about speedups.
I agree with the ideas. Just want to clarify the definition of multi-core - "SINGLE physical processor contains the core logic of two or more processors". However I'm not sure how "multi-core" is related to those definitions though because apparently core is not a processor. Also, according to what I found, concurrency and parallelism should be distinct concepts. For concurrency, I agree with mingf, it simply represents "interleaved" tasks; on the other hand, parallelism is an idea based on multiple processors and focuses more on correctness and speedup.
Sherry
In a web server we use pthread to create more threads or practically use a threading pool to server user requests, and we would do that even on a one core server. In this way CPU is shared among users, making them all feel the website is responding. Also, when one thread is blocked (maybe waiting for the response of DB or file system), another thread can get executed. However, in parallel computation, we create more threads just to occupy all the CPU cores, trying to get all the ALUs busy to maximize the throughput. We would NOT create more threads on a single core machine.
flyne
The difference between concurrent execution and parallel execution is that in concurrent execution, the tasks need not necessarily be processed at the same time but they must have overlapping execution time, that is the time when each of the task starts to when it completes execution where as in parallel execution the tasks must be processed simultaneously on different processors.
landuo
In my opinion, the difference between concurrent execution and parallel execution is that concurrent execution is about doing several things at once logically and parallel execution is about doing several things physically. Concurrent execution focuses on reducing latency. For example, the web proxy handles multiple service requests and I/O requests by allocating certain period of run time for each tasks. While for parallel execution, the goal is to maximize utilization of processor computing power in order to increase throughput.
Elias
In principle, I agree with the collective opinion. Summarized, it seems to be:
1) Concurrency: interleaved execution, defined by processes starting before others finish
2) Parallel: simultaneous execution, defined by multiple hardware threads of execution
In my paraphrase, I mentioned a "hardware" thread of execution. When Kayvon asked this question in class, I was somewhat surprised that neither a student answer, nor his own made any mention of the underlying hardware which enables parallelism [that I recall]. You may have a single execution unit running concurrent processes, but it is inherently limited to concurrent execution, and by nature cannot run tasks in parallel, precisely because there is only a single set of control paths - they can only have one voltage at any given point in time!
To me, this suggests that the defining feature of parallelism is that there are separate physical control paths, enabling hardware level parallelism. In a course on digital system design (such as 18-240), hardware threads are discussed in great detail.
A caveat: I do not believe, as some stated above, that concurrency or parallelism have inherent intent (i.e. concurrency aims to reduce latency, while parallelism aims to increase throughput). These are simply definitions (or guidelines), which clarify what is, or is not concurrent or parallel. A programmer may utilize them to reduce latency or speed up code, but that is not guaranteed, just because the code written is concurrent, or the system supports parallel execution.
nan
I think in concurrency, even though we are doing multiple tasks at the same time, we are using the single machine.... but in parallel programming our task is to use multiple machines at the same time efficiently
grose
Another part of the motivation for proxylab being concurrent is not wasting time waiting. For example, if A requests sony.com, and Sony is being DDOS'd (when does that ever happen), so that it takes 10 seconds before it even starts to respond; while B then asks for cnn.com, which is fast and fully functional, there's no reason we can't start helping B now.
That's a pretty contrived example to tie it into proxylab, but it is similar to some real-world issues. For example, if you need to request data from a variety of databases, you may as well run the code that requests and processes that data concurrently, since often you can shoot requests off fast, and then each database might take a few seconds to respond.
crispyhexagons
I agree with much that has been previously mentioned and what I gather from the previous comments, assignments, and lectures on these two topics is that concurrency focuses more on doing many rather related tasks simultaneously while parallelism breaks a larger task into smaller tasks to be carried out at the same time.
But what I appreciate is that these two both focus on speed up. Efficiency is so important these days (I liked the example mentioned in class about traders in particular) (Also I remember back when loading a single webpage seemed to take forever) and it's really cool that we are creating more and more complicated systems and tasks but are still able to make them even faster than they used to be.
MJ
All the (many!) comments so far discuss parallelism and concurrency as slightly different properties of how a program is executed. I'm interested if anyone has run across the idea before that concurrency is a property of how the program is written, not how it's executed. My friend Lindsey, who's an Intel research scientist working on Parallel JavaScript (RiverTrail), espouses that in this blog post.
qzhuang
In my thought, concurrency means more than one threads take turns to use the CPU and execute itself. However, parallel means that we manage to make some calculations in one single task happen at the same time, by arranging different CPU instructions in a smart way to reduce the idle time spent on waiting.
afa4
I mostly agree with the above comments. In my opinion, concurrent programs ensure speedup by allowing creation of multiple code paths which could be run simultaneously. Simultaneously here means they could be running interleaved on a single execution unit or in parallel on multiple execution units.
Parallel programs ensure speedup by performing several computations in parallel. This can be done by using instructions that can be executed on multiple data points parallely or executing different instructions on multiple cores/processors parallely. In any case, parallel programs need multiple execution units.
carlosdp
Concurrency is when tasks are broken up into isolated units of work. These units are able to be executed at the same time as each other, but do not necessarily need to be (as in proxy lab). Parallelism is when we take these broken up tasks and execute multiple tasks at a time. So we can create concurrent threads that each process an HTTP request (and are executed overlapping), but they don't actually happen at the same process time unless they are run in parallel.
rbandlam
I agree with above comments that concurrent execution is when tasks are interleaved, which gives end user the impression that his tasks are always running. Parallel execution is when two tasks are running in parallel on different cores. I feel hyperthreading on single core is considered as concurrent execution because at any point in time, only one thread will be running. Hyperthreading will keep cpu core busy with other thread's work when first thread's instruction is fetching or storing data (which takes multiple cycles due to cache misses). Please correct if my interpretation of hyperthreading as concurrent execution is wrong.
rojo
I agree with rofer. In proxy lab of 15213, we handle all concurrent incoming request using threads. These threads then perform the task independently and respond to the request. In assignment 1, program 1 we speed up the given task by handing same computation in parallel across different threads.
rohitban
Proxy aimed at dividing multiple requests among threads, but in assg1 we sped up a single
computation by dividing it among multiple threads
sanchuah
In 213 proxy homework, our goal was to execute the same task multiple times simultaneously for concurrent requests. This class wants to split a task to be parallelizable to finish it faster.
Question: In 15-213's web proxy assignment you gained experience writing concurrent programs using pthreads. Think about your motivation for programming with threads in that assignment. How was it different from the motivation to create multi-threaded programs in this class? (e.g., consider Assignment 1, Program 1)
Hint: What is the difference between concurrent execution and parallel execution?
In 15-213 our goal was to handle several concurrent events at once. In this class our goal is going to be to speed up a single task by performing many calculations in parallel.
I think concurrency is more about the operating system giving user the illusion that different programs can run simultaneously on single CPU core, while parallelism gives the idea of speeding up a single task by breaking it into independent pieces on different CPUs (to ensure correctness of the program).
In my understanding, concurrent execution is that multiple tasks execute interleavedly. It is an implementation detail whether they actually execute on different cores or on a single core. Whereas parallel execution means that multiple tasks execute in different independent cores.
To some extent, I agree with mingf. Concurrent execution means that you start several tasks and they will run interleavedly. If the machine it runs on has several cores or only a single core but it supports hyper-threading, and the OS supports running multiple tasks at the same time, it can be executed in parallel. Besides, when you run the tasks in multiple machines or heterogeneous machines at the same time it's also executed in parallel. So concurrent just guarantee that all tasks will have some time slot to run, and some methods will be used to schedule them. Parallelism means that these tasks can run at the same time. I'm not sure, but I think when tasks run in parallel, they runs concurrently. And the reverse is not right.
I agree with rofer and would like to add on.
In 213's proxy lab, we wanted to be able to write clean code to handle multiple connections at the same time, devoting some time to each user so that one didn't block another. In this class, on the other hand, such an approach wouldn't (necessarily) be useful since it doesn't (or might not) give any speed gains. With parallelism, we want to improve the speed of computations by doing as much work as possible at one time.
I agree with mingf and I think "interleavedly" is a right word to use for concurrent execution. An intuitive definition from CSAPP (p713), "flows X and Y are concurrent with respect to each other if and only if X begins after Y begins and before Y finishes, or Y begins after X begins and before X finishes".
As for parallel execution, I see it as a proper subset of concurrent execution. The only difference is parallel execution requires multiple processors.
For the motivations, I think concurrent programs are more concerned about not letting the processor been idle for too long, while parallel programs are more concerned about speedups.
Rob Pike (one of the Go creators) has a great explanation in this video: http://vimeo.com/49718712
I agree with the ideas. Just want to clarify the definition of multi-core - "SINGLE physical processor contains the core logic of two or more processors". However I'm not sure how "multi-core" is related to those definitions though because apparently core is not a processor. Also, according to what I found, concurrency and parallelism should be distinct concepts. For concurrency, I agree with mingf, it simply represents "interleaved" tasks; on the other hand, parallelism is an idea based on multiple processors and focuses more on correctness and speedup.
In a web server we use pthread to create more threads or practically use a threading pool to server user requests, and we would do that even on a one core server. In this way CPU is shared among users, making them all feel the website is responding. Also, when one thread is blocked (maybe waiting for the response of DB or file system), another thread can get executed. However, in parallel computation, we create more threads just to occupy all the CPU cores, trying to get all the ALUs busy to maximize the throughput. We would NOT create more threads on a single core machine.
The difference between concurrent execution and parallel execution is that in concurrent execution, the tasks need not necessarily be processed at the same time but they must have overlapping execution time, that is the time when each of the task starts to when it completes execution where as in parallel execution the tasks must be processed simultaneously on different processors.
In my opinion, the difference between concurrent execution and parallel execution is that concurrent execution is about doing several things at once logically and parallel execution is about doing several things physically. Concurrent execution focuses on reducing latency. For example, the web proxy handles multiple service requests and I/O requests by allocating certain period of run time for each tasks. While for parallel execution, the goal is to maximize utilization of processor computing power in order to increase throughput.
In principle, I agree with the collective opinion. Summarized, it seems to be:
1) Concurrency: interleaved execution, defined by processes starting before others finish
2) Parallel: simultaneous execution, defined by multiple hardware threads of execution
In my paraphrase, I mentioned a "hardware" thread of execution. When Kayvon asked this question in class, I was somewhat surprised that neither a student answer, nor his own made any mention of the underlying hardware which enables parallelism [that I recall]. You may have a single execution unit running concurrent processes, but it is inherently limited to concurrent execution, and by nature cannot run tasks in parallel, precisely because there is only a single set of control paths - they can only have one voltage at any given point in time!
To me, this suggests that the defining feature of parallelism is that there are separate physical control paths, enabling hardware level parallelism. In a course on digital system design (such as 18-240), hardware threads are discussed in great detail.
A caveat: I do not believe, as some stated above, that concurrency or parallelism have inherent intent (i.e. concurrency aims to reduce latency, while parallelism aims to increase throughput). These are simply definitions (or guidelines), which clarify what is, or is not concurrent or parallel. A programmer may utilize them to reduce latency or speed up code, but that is not guaranteed, just because the code written is concurrent, or the system supports parallel execution.
I think in concurrency, even though we are doing multiple tasks at the same time, we are using the single machine.... but in parallel programming our task is to use multiple machines at the same time efficiently
Another part of the motivation for proxylab being concurrent is not wasting time waiting. For example, if A requests sony.com, and Sony is being DDOS'd (when does that ever happen), so that it takes 10 seconds before it even starts to respond; while B then asks for cnn.com, which is fast and fully functional, there's no reason we can't start helping B now.
That's a pretty contrived example to tie it into proxylab, but it is similar to some real-world issues. For example, if you need to request data from a variety of databases, you may as well run the code that requests and processes that data concurrently, since often you can shoot requests off fast, and then each database might take a few seconds to respond.
I agree with much that has been previously mentioned and what I gather from the previous comments, assignments, and lectures on these two topics is that concurrency focuses more on doing many rather related tasks simultaneously while parallelism breaks a larger task into smaller tasks to be carried out at the same time.
But what I appreciate is that these two both focus on speed up. Efficiency is so important these days (I liked the example mentioned in class about traders in particular) (Also I remember back when loading a single webpage seemed to take forever) and it's really cool that we are creating more and more complicated systems and tasks but are still able to make them even faster than they used to be.
All the (many!) comments so far discuss parallelism and concurrency as slightly different properties of how a program is executed. I'm interested if anyone has run across the idea before that concurrency is a property of how the program is written, not how it's executed. My friend Lindsey, who's an Intel research scientist working on Parallel JavaScript (RiverTrail), espouses that in this blog post.
In my thought, concurrency means more than one threads take turns to use the CPU and execute itself. However, parallel means that we manage to make some calculations in one single task happen at the same time, by arranging different CPU instructions in a smart way to reduce the idle time spent on waiting.
I mostly agree with the above comments. In my opinion, concurrent programs ensure speedup by allowing creation of multiple code paths which could be run simultaneously. Simultaneously here means they could be running interleaved on a single execution unit or in parallel on multiple execution units.
Parallel programs ensure speedup by performing several computations in parallel. This can be done by using instructions that can be executed on multiple data points parallely or executing different instructions on multiple cores/processors parallely. In any case, parallel programs need multiple execution units.
Concurrency is when tasks are broken up into isolated units of work. These units are able to be executed at the same time as each other, but do not necessarily need to be (as in proxy lab). Parallelism is when we take these broken up tasks and execute multiple tasks at a time. So we can create concurrent threads that each process an HTTP request (and are executed overlapping), but they don't actually happen at the same process time unless they are run in parallel.
I agree with above comments that concurrent execution is when tasks are interleaved, which gives end user the impression that his tasks are always running. Parallel execution is when two tasks are running in parallel on different cores. I feel hyperthreading on single core is considered as concurrent execution because at any point in time, only one thread will be running. Hyperthreading will keep cpu core busy with other thread's work when first thread's instruction is fetching or storing data (which takes multiple cycles due to cache misses). Please correct if my interpretation of hyperthreading as concurrent execution is wrong.
I agree with rofer. In proxy lab of 15213, we handle all concurrent incoming request using threads. These threads then perform the task independently and respond to the request. In assignment 1, program 1 we speed up the given task by handing same computation in parallel across different threads.
Proxy aimed at dividing multiple requests among threads, but in assg1 we sped up a single computation by dividing it among multiple threads
In 213 proxy homework, our goal was to execute the same task multiple times simultaneously for concurrent requests. This class wants to split a task to be parallelizable to finish it faster.