HPC
High Peformance Computing
- About distributing storage and processing over many processors.
- Not just hardware, but also software to manage the execution of the tasks on ha rdware
- This is why we need the orchestration softwares.
- e.g. The AlphaGo distributed processing on many different GPUs.
Distributed Machine Learning
- Different models can be trained in parallel using different subsets of the data, and then the results can be merged.
- The operation of a single but large model can be distributed over many processors.
- e.g. A deep NN which have many layers, parts of it can be run on different processors working as a pipeline
Time and Space
- Machine learning is not just a component in a product.
- Time and space sometimes are more important than accuracy
- Time: a self-driving car in a real world only has limited amount of time to decide whether there is pedestrian on the road or not.
- The accuracy of the recognizer becomes irrelevent when it can not decide fast enough.
- Space: IoT devices have limited computing power.
- Such systems may be making decisions on site or may just work as smart processors of the signal.
- Time: a self-driving car in a real world only has limited amount of time to decide whether there is pedestrian on the road or not.
Last modified March 10, 2021