PyTorch is a widely-used, open-source deep learning framework developed by Facebook’s AI Research lab. Known for its flexibility and dynamic computation graph, PyTorch simplifies building and training machine learning models. This cheat sheet serves as a quick reference for developers, covering essential commands, functions, and libraries.
Introduction to PyTorch
PyTorch is a Python-based library that provides tools to:
• Build neural networks.
• Perform tensor computations (similar to NumPy but with GPU support).
• Enable automatic differentiation for optimization.
• Utilize prebuilt datasets and pretrained models for faster development.
Key Imports in PyTorch
General
import torch # Root package
from torch.utils.data import Dataset, DataLoader # Data handling
Neural Network API
import torch.nn as nn # Neural network layers
import torch.nn.functional as F # Activations, losses, and other functions
import torch.optim as optim # Optimizers like SGD, Adam, etc.
import torch.autograd as autograd # Automatic differentiation
from torch import Tensor # Tensor objects
ONNX Integration
ONNX (Open Neural Network Exchange) allows interoperability between PyTorch and other frameworks.
torch.onnx.export(model, dummy_data, “model.onnx”) # Export model to ONNX
model = onnx.load(“model.onnx”) # Load ONNX model
onnx.checker.check_model(model) # Validate ONNX model
Computer Vision
from torchvision import datasets, models, transforms # Datasets, models, and transforms
Distributed Training
import torch.distributed as dist # Distributed communication
from torch.multiprocessing import Process # Multi-process handling
Tensors in PyTorch
Tensors are the fundamental data structure in PyTorch, similar to arrays in NumPy, but with GPU support.
Tensor Creation
x = torch.randn(3, 4) # Tensor with random values (normal distribution)
x = torch.ones(3, 4) # Tensor of ones
x = torch.zeros(3, 4) # Tensor of zeros
x = torch.tensor([1, 2]) # Tensor from a list
Tensor Operations
x.size() # Get tensor dimensions
y = x.view(2, 6) # Reshape tensor
z = x.transpose(0, 1) # Transpose dimensions
Deep Learning Modules
PyTorch’s torch.nn module provides tools for building neural networks.
Building Layers
nn.Linear(in_features, out_features) # Fully connected layer
nn.Conv2d(in_channels, out_channels, kernel_size) # 2D convolutional layer
nn.Dropout(p=0.5) # Dropout for regularization
Loss Functions
loss = nn.CrossEntropyLoss() # Cross-entropy loss for classification
loss = nn.MSELoss() # Mean squared error loss
Optimizers
optimizer = optim.SGD(model.parameters(), lr=0.01) # Stochastic Gradient Descent
optimizer.zero_grad() # Reset gradients
optimizer.step() # Update model weights
GPU Utilization
Harness the power of GPUs to accelerate training.
Checking GPU Availability
torch.cuda.is_available() # Check if CUDA is available
Moving Data Between CPU and GPU
device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)
x = x.to(device) # Move tensor to GPU
model.to(device) # Move model to GPU
Advanced Features
Data Utilities
from torch.utils.data import DataLoader, Dataset
Datasets
• Dataset: Abstract class for creating datasets.
• TensorDataset: Combines features and labels as tensors.
• ConcatDataset: Combines multiple datasets.
DataLoader
Efficiently load batches of data for training.
loader = DataLoader(dataset, batch_size=32, shuffle=True)
Distributed Training
dist.init_process_group(backend=’nccl’) # Initialize distributed training
Key PyTorch Resources
1. PyTorch Official Documentation
2. Deep Learning with PyTorch: A 60-Minute Blitz
This cheat sheet provides a structured reference to PyTorch’s core features. For more details and examples, see PyTorch’s tutorials and official documentation.