Visão Geral

Este curso explora em profundidade a arquitetura Transformer, considerada a base tecnológica dos modernos Large Language Models (LLMs), sistemas de IA Generativa e modelos multimodais. O participante aprenderá os princípios matemáticos, arquiteturais e computacionais que sustentam os Transformers, incluindo mecanismos de atenção, embeddings, codificação posicional, treinamento distribuído e otimizações avançadas. O curso também aborda a evolução das arquiteturas Transformer e sua aplicação em soluções corporativas de Inteligência Artificial.

Objetivo

Após realizar este curso, você será capaz de:

• Compreender os fundamentos da arquitetura Transformer
• Entender o funcionamento dos mecanismos de atenção utilizados em modelos modernos
• Analisar os componentes internos dos Transformers e seu papel no processamento de linguagem natural
• Avaliar arquiteturas avançadas derivadas dos Transformers
• Compreender desafios de treinamento, inferência e escalabilidade de modelos baseados em Transformers
• Aplicar conceitos arquiteturais em projetos de IA Generativa e modelos corporativos de linguagem

Publico Alvo

• Engenheiros de Machine Learning
• Engenheiros de IA Generativa
• Cientistas de Dados
• Arquitetos de Soluções de IA
• Pesquisadores em Inteligência Artificial
• Desenvolvedores interessados em compreender o funcionamento interno dos LLMs

Pre-Requisitos

• Conhecimentos básicos de Machine Learning e Deep Learning
• Familiaridade com Python
• Noções de álgebra linear, cálculo e estatística
• Conhecimentos equivalentes aos cursos Fundamentos de Machine Learning e LLM Fundamentals são recomendados

Conteúdo Programatico

Module 1: Introduction to Transformer Architecture

1. Evolution of neural network architectures
2. Limitations of RNNs and LSTMs
3. Emergence of the Transformer architecture
4. Overview of modern AI models
5. Applications of Transformers
6. Enterprise use cases

Module 2: Mathematical Foundations

1. Linear algebra fundamentals
2. Matrix operations and vector spaces
3. Probability and statistics concepts
4. Optimization principles
5. Gradient descent overview
6. Mathematical foundations for deep learning

Module 3: Neural Networks and Sequence Modeling

1. Deep neural network fundamentals
2. Sequence processing challenges
3. Recurrent Neural Networks overview
4. Long-term dependency problems
5. Representation learning
6. Evolution toward attention-based models

Module 4: Attention Mechanism Fundamentals

1. Concept of attention
2. Query, Key and Value architecture
3. Attention score computation
4. Scaled dot-product attention
5. Context-aware learning
6. Benefits of attention mechanisms

Module 5: Multi-Head Self-Attention

1. Self-attention architecture
2. Multi-head attention design
3. Parallel attention processing
4. Context representation learning
5. Information aggregation techniques
6. Computational considerations

Module 6: Transformer Encoder Architecture

1. Encoder block components
2. Attention layers
3. Feed-forward neural networks
4. Residual connections
5. Layer normalization
6. Encoder processing workflow

Module 7: Transformer Decoder Architecture

1. Decoder block structure
2. Masked self-attention
3. Cross-attention mechanisms
4. Output generation process
5. Sequence prediction techniques
6. Decoder optimization strategies

Module 8: Embeddings and Positional Encoding

1. Tokenization fundamentals
2. Word and token embeddings
3. Semantic representations
4. Positional encoding techniques
5. Context preservation methods
6. Embedding optimization

Module 9: Training Large Transformer Models

1. Pre-training architectures
2. Self-supervised learning
3. Large-scale dataset preparation
4. Distributed training strategies
5. Hardware acceleration
6. Training optimization techniques

Module 10: Transformer Variants and Modern Architectures

1. BERT architecture
2. GPT architecture
3. Encoder-only models
4. Decoder-only models
5. Encoder-decoder models
6. Modern Transformer innovations

Module 11: Scaling, Optimization and Enterprise Deployment

1. Model scaling laws
2. Efficient Transformer architectures
3. Inference optimization
4. Quantization concepts
5. Enterprise deployment strategies
6. Operational considerations

Module 12: Transformer Architecture Workshop

1. Attention mechanism analysis
2. Transformer component exploration
3. Architecture comparison exercises
4. Model design evaluations
5. Enterprise AI architecture case studies
6. Final Transformer architecture project