When we use 1 processor, does it have to be identical to one of the P processors? Or can we use a better processor to match the cost of P processors? The former seems to be what this slide is implying, but the later seems to be a fairer comparison to me.
coffee7
@williamx I might have misunderstood your second interpretation that you think is more fair, but I think it only makes sense to measure speedup if we keep the processors constant (i.e. the 1 processor is the same kind of processor as any of the P processors). This is because we want to see the exact difference adding more processors makes, especially since the difference is not necessarily linear (e.g. doubling the number of processors does not mean a 2x speedup). It would hard to interpret what is going on if we compare 1 processor of type A to P processors of type B.
themj
@coffee7 How about when the speed of the processors must be decreased to accommodate them all in multi-core processors? Shouldn't that be taken into account? And, therefore, the processors wouldn't be constant (i.e. the 1 processor is not the same kind of processor as any of the P processors) but would be a more realistic model of reality?
ask
@themj I think taking into account factors such as the one you mentioned might be effective in giving a more accurate view of speedup achieved in practical multi-core processors. However, I think standardizing the speedup computation if those factors are to be included, might be very hard and not achievable in practice.
For instance, how would you benchmark a quad-core processor from manufacturer A against a competing processor from manufacturer B without any knowledge of the tradeoffs the manufacturers made while designing the processors? In my opinion, not including these factors helps in benchmarking and standardization.
paramecinm
@themj I think the speedup is used to theoretically measure the performance of an algorithm, namely if our parallel algorithm runs much faster on multi-processors than single processor, it is a good algorithm. You are to some extent considering the actual running time on some particular hardwares. When you measuring a regular algorithm, you will also calculate the time complexity rather than its actual running time on some given computers.
shhhh
@williamx I'm not understanding why the later would be a fair comparison. I think the point of the speedup measurement is to test how much faster a single type of processor will be in parallel. If you wish to compare two different types of processors, there is no need to compare processor A with multiple processor B's, it would be more useful to do a single processor A vs a single processor B.
chandana
As explained in the following slides, speedup is limited due to
1. communication overhead between processors
2. imbalance in work load between multiple processors
3. for massively parallel execution, if the amount of communication is greater than computation.
When we use 1 processor, does it have to be identical to one of the P processors? Or can we use a better processor to match the cost of P processors? The former seems to be what this slide is implying, but the later seems to be a fairer comparison to me.
@williamx I might have misunderstood your second interpretation that you think is more fair, but I think it only makes sense to measure speedup if we keep the processors constant (i.e. the 1 processor is the same kind of processor as any of the P processors). This is because we want to see the exact difference adding more processors makes, especially since the difference is not necessarily linear (e.g. doubling the number of processors does not mean a 2x speedup). It would hard to interpret what is going on if we compare 1 processor of type A to P processors of type B.
@coffee7 How about when the speed of the processors must be decreased to accommodate them all in multi-core processors? Shouldn't that be taken into account? And, therefore, the processors wouldn't be constant (i.e. the 1 processor is not the same kind of processor as any of the P processors) but would be a more realistic model of reality?
@themj I think taking into account factors such as the one you mentioned might be effective in giving a more accurate view of speedup achieved in practical multi-core processors. However, I think standardizing the speedup computation if those factors are to be included, might be very hard and not achievable in practice.
For instance, how would you benchmark a quad-core processor from manufacturer A against a competing processor from manufacturer B without any knowledge of the tradeoffs the manufacturers made while designing the processors? In my opinion, not including these factors helps in benchmarking and standardization.
@themj I think the speedup is used to theoretically measure the performance of an algorithm, namely if our parallel algorithm runs much faster on multi-processors than single processor, it is a good algorithm. You are to some extent considering the actual running time on some particular hardwares. When you measuring a regular algorithm, you will also calculate the time complexity rather than its actual running time on some given computers.
@williamx I'm not understanding why the later would be a fair comparison. I think the point of the speedup measurement is to test how much faster a single type of processor will be in parallel. If you wish to compare two different types of processors, there is no need to compare processor A with multiple processor B's, it would be more useful to do a single processor A vs a single processor B.
As explained in the following slides, speedup is limited due to 1. communication overhead between processors 2. imbalance in work load between multiple processors 3. for massively parallel execution, if the amount of communication is greater than computation.