Day 02 Optimization

Partial Derivatives and Gradient Descent

Multivariable functions, partial derivatives, the gradient vector, and gradient descent with momentum — the optimization algorithm at the heart of every deep learning framework.

~1 hour Intermediate Hands-on Precision AI Academy

Today's Objective

Multivariable functions, partial derivatives, the gradient vector, and gradient descent with momentum — the optimization algorithm at the heart of every deep learning framework.

01

What You'll Cover Today

Day 2 of Calculus for AI in 5 Days builds directly on Day 1. You're moving from theory into applied practice. The concepts today require the foundation from yesterday, so if anything felt unclear, review it now.

ℹ️
Topics today: chain rule, composite functions, Jacobian. Each section has code you can copy and run immediately.
02

chain rule

Understanding chain rule is the core goal of Day 2. The concept is straightforward once you see it in practice — most confusion comes from skipping the mental model and jumping straight to implementation. Start with the model, then write the code.

chain rule
# chain rule — Working Example
# Study this pattern carefully before writing your own version

class chainruleExample:
    """
    Demonstrates core chain rule concepts.
    Replace placeholder values with your real implementation.
    """
    
    def __init__(self, config: dict):
        self.config = config
        self._validate()
    
    def _validate(self):
        required = ['name', 'type']
        for field in required:
            if field not in self.config:
                raise ValueError(f"Missing required field: {field}")
    
    def process(self) -> dict:
        # Core logic goes here
        result = {
            'status': 'success',
            'topic': 'chain rule',
            'data': self.config
        }
        return result


# Usage
example = chainruleExample({
    'name': 'my-implementation',
    'type': 'chain rule'
})
output = example.process()
print(output)
💡
Key insight: When working with chain rule, always start with the simplest possible case that works end-to-end. Complexity is easier to add than simplicity is to recover.
03

composite functions

composite functions is the practical application of chain rule in real projects. Once you understand the underlying model, composite functions becomes the natural next step.

💡
Pro tip: When working with composite functions, always read the official documentation for the exact version you're using. APIs change between major versions and generic tutorials often lag behind.
04

Jacobian

Jacobian rounds out today's lesson. It connects chain rule and composite functions into a complete picture. You'll use all three concepts together in the exercise below.

05

Common Mistakes on Day 2

📝 Day 2 Exercise
Chain Rule — Hands-On
  1. Set up your environment for today's topic: install required tools and verify the basics work before writing any logic.
  2. Implement a minimal working version of chain rule using the code example in this lesson as your starting point.
  3. Extend your implementation to incorporate composite functions — this is where the two concepts connect.
  4. Test your implementation with both valid and invalid inputs. What happens at the boundaries?
  5. Review your code: is there anything you'd name differently? Any function doing more than one thing? Refactor one thing.

Day 2 Summary

Challenge

Extend today's exercise by adding one feature that wasn't in the instructions. Document what you built in a comment at the top of the file. This habit of going one step further is what separates engineers who grow fast from those who stay stuck.

What's Next

The foundations from today carry directly into Day 3. In the next session the focus shifts to Backpropagation from First Principles — building directly on everything covered here.

Day 2 Checkpoint

Before moving on, verify you can answer these without looking:

  • What is the core concept introduced in this lesson, and why does it matter?
  • What are the two or three most common mistakes practitioners make with this topic?
  • Can you explain the key code pattern from this lesson to a colleague in plain language?
  • What would break first if you skipped the safeguards or best practices described here?
  • How does today's topic connect to what comes in Day 3?

Live Bootcamp

Learn this in person — 2 days, 5 cities

Thu–Fri sessions in Denver, Los Angeles, New York, Chicago, and Dallas. $1,490 per seat. June–October 2026.

Reserve Your Seat →
Continue To Day 3
Day 3: Partial Derivatives & Gradients