Parallel Computing Theory

Serial Computing

Serial computing - where instructions executed one after another

1. Problem statements are broken down into discrete instructions

2. Each of the instructions are then executed one by one

3. Only one instruction is executed at any moment in time

Parallel Computing

Parallel computing - The idea of using multiple processing elements for simultaneously solving a problem

Advantages:

• Saves time and money

• Can be impractical to solve problems using serial computing

• Can take advantage of nonlocal resources when local resources are finite

Disadvantages:

• Communicating between subtasks is hard

• Algorithms must be managed in such a way that they can be handled in the parallel mechanism

• The program must have low coupling and high cohesion

• It's just bloody hard to do

Types of Parallel Computing

Bit-level Parallelism

The idea of increasing the processes size, thus reducing the number of instructions that need to be executed, i.e. 8-bit to 16-bit processors

I thinks is like CISC

Instruction-level parallelism

Note: a processor can only address one instruction for each clock cycle

Instructions are re-ordered in such a way that allows them to be executed congruently without affecting the results of the program

Task Parallelism

Decomposes each task into sub-tasks then execute each sub-task congruently