Deep Learning

Table of Contents Scaling, Optimizing, and Exporting Transformers with PyTorch Lightning Introduction to Scaling PyTorch Lightning Transformer Training Configuring Your Development Environment Preparing PyTorch Lightning Models for Scalable Multi-GPU Training Revisiting the Code Architecture Enabling Mixed Precision Training with PyTorch…

Table of Contents Training with PyTorch Lightning: Structured MLOps Development Why PyTorch Lightning Improves Reproducible MLOps Training Pipelines Configuring Your Development Environment Project Structure PyTorch LightningDataModule Explained: Building Efficient Data Pipelines PyTorch LightningModule Explained: Building Modular Deep Learning Models Using…

Table of Contents Building and Training a Kimi-K2 Model Using DeepSeek-V3 Components Kimi-K2 vs DeepSeek-V3: Key Architecture Differences in LLM Design Mixture of Experts Scaling in Kimi-K2: Model Size, Sparsity, and Efficiency Attention Head Optimization in Kimi-K2 for Efficient Long-Context…

Table of Contents Autoregressive Model Limits and Multi-Token Prediction in DeepSeek-V3 Why Next-Token Prediction Limits DeepSeek-V3 Multi-Token Prediction in DeepSeek-V3: Predicting Multiple Tokens Ahead DeepSeek-V3 Architecture: Multi-Token Prediction Heads Explained Gradient Insights for Multi-Token Prediction in DeepSeek-V3 DeepSeek-V3 Training vs.…

Table of Contents DeepSeek-V3 from Scratch: Mixture of Experts (MoE) The Scaling Challenge in Neural Networks Mixture of Experts (MoE): Mathematical Foundation and Routing Mechanism SwiGLU Activation in DeepSeek-V3: Improving MoE Non-Linearity Shared Expert in DeepSeek-V3: Universal Processing in MoE…

Table of Contents Build DeepSeek-V3: Multi-Head Latent Attention (MLA) Architecture The KV Cache Memory Problem in DeepSeek-V3 Multi-Head Latent Attention (MLA): KV Cache Compression with Low-Rank Projections Query Compression and Rotary Positional Embeddings (RoPE) Integration Attention Computation with Multi-Head Latent…

Table of Contents Vector Search with FAISS: Approximate Nearest Neighbor (ANN) Explained From Exact to Approximate: Why Indexing Matters The Trouble with Brute-Force Search The Curse of Dimensionality Enter the Approximate Nearest Neighbor (ANN) Accuracy vs. Latency: The Core Trade-Off…

Table of Contents Setting Up LLaVA/BakLLaVA with vLLM: Backend and API Integration Why vLLM for Multimodal Inference The Challenges of Serving Image + Text Prompts at Scale Why Vanilla Approaches Fall Short How vLLM Solves Real-World Production Workloads Configuring Your…

Table of Contents Meet BLIP: The Vision-Language Model Powering Image Captioning What Is Image Captioning and Why Is It Challenging? Why It’s Challenging Why Traditional Vision Tasks Aren’t Enough Configuring Your Development Environment A Brief History of Image Captioning Models…