Hello, world

Sanity check: math, code, and images all render.

This is a placeholder post verifying the three things a technical blog needs.

Math (KaTeX)

Inline math like \pi(a \mid s;\theta) works, and so do display equations — here’s the policy gradient:

\nabla_\theta J(\theta) = \mathbb{E}_{\tau \sim \pi_\theta}\!\left[\sum_{t=0}^{T} \nabla_\theta \log \pi_\theta(a_t \mid s_t)\, \hat{A}_t\right]

Code

import numpy as np

def gae(rewards, values, gamma=0.99, lam=0.95):
    """Generalized Advantage Estimation."""
    deltas = rewards + gamma * values[1:] - values[:-1]
    advantages = np.zeros_like(deltas)
    acc = 0.0
    for t in reversed(range(len(deltas))):
        acc = deltas[t] + gamma * lam * acc
        advantages[t] = acc
    return advantages

Image

A loss curve, as required by law on every ML blog.