12-Week Generative AI
Training Program

FlashAttention-3 — memory-efficient attention for long contexts

Rotary Positional Embeddings (RoPE) — why they beat sinusoidal

State Space Models (SSMs) — Mamba architecture deep dive

Why SSMs are replacing standard attention for long-context windows

Pre-training at scale — data curation pipelines & quality filtering

Tokenizer optimization — BPE, SentencePiece, and vocabulary design

Distributed Training — DeepSpeed ZeRO stages & Megatron-LM tensor parallelism

Training stability — gradient clipping, loss spikes, and checkpointing strategies

Beyond Stable Diffusion — Rectified Flow and why it converges faster

ControlNet internals — conditioning mechanisms & adapter architecture

Latent Consistency Models (LCMs) — real-time generation in 4 steps

Flow Matching vs DDPM — mathematical intuition and practical tradeoffs

How models "see" — Vision Transformers (ViT) and patch embeddings

CLIP-style contrastive alignment — training vision-language models

Audio-native LLMs — Whisper architecture and speech-text alignment

VLM architectures — LLaVA, Flamingo, and cross-modal attention

QLoRA — 4-bit quantized LoRA for fine-tuning on consumer GPUs

DoRA (Weight-Decomposed Low-Rank Adaptation) — why it outperforms LoRA

Unsloth — 2x faster fine-tuning with memory optimization tricks

Evaluation — perplexity, ROUGE, and domain-specific benchmarks

RLHF pipeline — reward modeling, PPO, and preference datasets

Direct Preference Optimization (DPO) — why it's simpler than RLHF

KTO (Kahneman-Tversky Optimization) — aligning models for better reasoning

Constitutional AI & self-critique — Anthropic's alignment approach

GraphRAG — using knowledge graphs for structured retrieval (Microsoft GraphRAG)

Agentic RAG — model decides what to search, when, and how many times

Self-RAG — models that critique and re-rank their own retrieved context

HyDE & query rewriting — hypothetical document embeddings for better recall

Quantization techniques — AWQ, GGUF, and FP8/INT4 precision tradeoffs

vLLM — PagedAttention and continuous batching for high-throughput serving

TGI (Text Generation Inference) — production serving with HuggingFace

Edge deployment — running quantized models on-device vs cloud serving

Agentic loops — ReAct, Plan-and-Execute, and AutoGPT-style architectures

LangGraph — building stateful, cyclical agent workflows

Long-term memory — episodic, semantic, and procedural memory for agents

Tool-use & function calling — structured outputs and API orchestration

Multi-agent architectures — Coder, Critic, Manager agent patterns

State management across agents — shared memory and message passing

Agent Communication Protocols — structured handoffs and error recovery

CrewAI & AutoGen — frameworks for multi-agent collaboration

LLM-as-a-judge — building custom evaluators for factuality and quality

NeMo Guardrails — programmable safety rails for production AI apps

Llama Guard — open-source content moderation for jailbreak prevention

Hallucination detection — RAGAs, TruLens, and automated eval pipelines

World Models — how AI learns to simulate environments (JEPA, Dreamer)

Test-Time Compute — scaling inference like OpenAI o1 (chain-of-thought search)

Capstone architecture review — final feedback and polish session

Career pathways — AI Research Engineer, MLOps, AI Product roles