Abstract
本文： PyTorch
Task: detail the implementation and architecture of PyTorch
Github: https://github.com/pytorch/pytorch
特点：
- PyTorch同时关注可用性和速度
- PyTorch provides an imperative and Pythonic programming style
1. Intro
2. Background
3. Design Principles
Be Pythonic, Put researchers first, Provide pragmatic performance, Worse is better
4. Usability centric design
4.1 Deep learning models are just Python programs
4.2 Interoperability and extensibility
4.3 Automatic differentiation
5 Performance focused implementation
5.1 An efficient C++ core
5.2 Separate control and data flow
PyTorch is designed to execute operators asynchronously on GPU by leveraging the CUDA stream mechanism [38] to queue CUDA kernel invocations to the GPUs hardware FIFO. This allows the system to overlap the execution of Python code on CPU with tensor operators on GPU
5.3 Custom caching tensor allocator
5.4 Multiprocessing
5.5 Reference counting
6. Evaluation
6.1 Asynchronous dataflow
6.2 Memory management
6.3 Benchmarks
6.4 Adoption
7. Conclusion and future work
- high-performance performance imperative CDeepFuzz learninghigh-performance performance imperative cdeepfuzz high-performance performance generating learning high-performance high-performance performance networking the high-performance performance storage object comprehensive cdeepfuzz compiler learning combinatorial cdeepfuzz learning reading deepmutation cdeepfuzz mutation learning large-scale tensorflow cdeepfuzz learning metamorphic cdeepfuzz compilers learning