X hits on this document

121 views

0 shares

0 downloads

0 comments

33 / 35

33

Overhead of Parallelism

  • Given enough parallel work, this is the biggest barrier to getting desired speedup

  • Parallelism overheads include:

    • cost of starting a thread or process

    • cost of communicating shared data

    • cost of synchronizing

    • extra (redundant) computation

  • Each of these can be in the range of milliseconds (=millions of flops) on some systems

  • Tradeoff: Algorithm needs sufficiently large units of work to run fast in parallel (I.e. large granularity), but not so large that there is not enough parallel work

COMP 422, Spring 2008 (V.Sarkar)

Document info
Document views121
Page views121
Page last viewedTue Jan 17 20:11:45 UTC 2017
Pages35
Paragraphs492
Words2148

Comments