Network Forward Pass

Composing Layers

A neural network is a sequence of layers. The forward pass is just function composition:

$\mathbf{a}^{(0)} = \mathbf{x}$

$\mathbf{a}^{(l)} = f^{(l)}(\mathbf{a}^{(l-1)}) \quad \text{for } l = 1, 2, \ldots, L$

Where each $f^{(l)}$ is either a DenseLayer or an ActivationLayer.

Activation Layers

An ActivationLayer simply applies a scalar function element-wise to its input:

$\mathbf{a} = [f(z_1), f(z_2), \ldots, f(z_m)]$

Network Architecture

A typical two-hidden-layer network looks like:

Input → Dense(n, 64) → ReLU → Dense(64, 32) → ReLU → Dense(32, 1) → Sigmoid

Each Dense layer mixes features. Each Activation layer introduces non-linearity.

Your Task

Implement:

• ActivationLayer(activation) with a forward(inputs) method that applies activation element-wise
• Network(layers) with a forward(inputs) method that passes data through each layer in sequence