Layer Backpropagation

Gradients Through One Layer

Consider a single neuron with sigmoid activation ($\sigma$ = sigmoid function, not standard deviation) and MSE loss:

$z = \mathbf{w} \cdot \mathbf{x} + b \qquad a = \sigma(z) \qquad \mathcal{L} = (a - y)^2$

To update $\mathbf{w}$ and $b$, we need $\frac{\partial \mathcal{L}}{\partial \mathbf{w}}$ and $\frac{\partial \mathcal{L}}{\partial b}$.

Applying the Chain Rule

$\frac{\partial \mathcal{L}}{\partial w_i} = \frac{\partial \mathcal{L}}{\partial a} \cdot \frac{\partial a}{\partial z} \cdot \frac{\partial z}{\partial w_i}$

Each factor:

$\frac{\partial \mathcal{L}}{\partial a} = 2(a - y)$

$\frac{\partial a}{\partial z} = \sigma'(z) = a(1-a)$

$\frac{\partial z}{\partial w_i} = x_i \qquad \frac{\partial z}{\partial b} = 1$

Define the error signal $\delta = \frac{\partial \mathcal{L}}{\partial a} \cdot \frac{\partial a}{\partial z} = 2(a-y) \cdot a(1-a)$. Then:

$\frac{\partial \mathcal{L}}{\partial w_i} = \delta \cdot x_i \qquad \frac{\partial \mathcal{L}}{\partial b} = \delta$

Your Task

Implement layer_backward(inputs, weights, bias, target) that:

1. Performs the forward pass to compute $a$
2. Computes $\delta = 2(a - y) \cdot a(1-a)$
3. Returns (dw, db) where dw[i] = delta * inputs[i]