---
myst:
  html_meta:
    description: Normalization layers in PyTorch C++ — BatchNorm, LayerNorm, GroupNorm, InstanceNorm, and LocalResponseNorm.
    keywords: PyTorch, C++, normalization, BatchNorm, LayerNorm, GroupNorm, InstanceNorm
---

# Normalization Layers

Normalization layers stabilize and accelerate training by normalizing intermediate
activations. They help with gradient flow and allow higher learning rates.

- **BatchNorm**: Normalizes across batch dimension; most common in CNNs
- **InstanceNorm**: Normalizes each sample independently; popular in style transfer
- **LayerNorm**: Normalizes across feature dimension; standard in transformers
- **GroupNorm**: Normalizes within groups of channels; works with small batches
- **LocalResponseNorm**: Lateral inhibition inspired by neuroscience (less common today)

## BatchNorm1d / BatchNorm2d / BatchNorm3d

```{doxygenclass} torch::nn::BatchNorm1d
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::BatchNorm1dImpl
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::BatchNorm2d
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::BatchNorm2dImpl
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::BatchNorm3d
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::BatchNorm3dImpl
:members:
:undoc-members:
```

**Example:**

```cpp
auto bn = torch::nn::BatchNorm2d(
    torch::nn::BatchNorm2dOptions(64)  // num_features
        .eps(1e-5)
        .momentum(0.1)
        .affine(true)
        .track_running_stats(true));
```

## InstanceNorm1d / InstanceNorm2d / InstanceNorm3d

```{doxygenclass} torch::nn::InstanceNorm1d
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::InstanceNorm1dImpl
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::InstanceNorm2d
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::InstanceNorm2dImpl
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::InstanceNorm3d
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::InstanceNorm3dImpl
:members:
:undoc-members:
```

## LayerNorm

```{doxygenclass} torch::nn::LayerNorm
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::LayerNormImpl
:members:
:undoc-members:
```

**Example:**

```cpp
auto ln = torch::nn::LayerNorm(
    torch::nn::LayerNormOptions({768}));  // normalized_shape
```

## GroupNorm

```{doxygenclass} torch::nn::GroupNorm
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::GroupNormImpl
:members:
:undoc-members:
```

**Example:**

```cpp
auto gn = torch::nn::GroupNorm(
    torch::nn::GroupNormOptions(32, 256));  // num_groups, num_channels
```

## LocalResponseNorm

```{doxygenclass} torch::nn::LocalResponseNorm
:members:
:undoc-members:
```

```{doxygenclass} torch::nn::LocalResponseNormImpl
:members:
:undoc-members:
```
