StripeCycle: A CycleGAN You Can Actually Read — From-Scratch PyTorch, Start to Finish

Motivation

Most existing CycleGAN implementations I found fell into one of two camps: research code frozen at publication time, dense with flags nobody explains anymore, or a minimal script that captures the idea but skips everything you'd need to actually train and reason about a real model. I wanted something in between — modular enough that I could read it end-to-end and genuinely understand every piece, not just treat it as a black box I point a config file at. That's the reason this is organized into small, single-purpose modules (cyclegan/models/, cyclegan/losses.py, cyclegan/engine/) rather than one long training script: each piece should be readable, testable, and understandable on its own.

Problem definition

CycleGAN solves unpaired image-to-image translation: learning a mapping between two image domains — horses and zebras, photos and paintings, summer and winter — when no aligned ground-truth pairs exist. There's no photo of the exact same horse standing as a zebra, so a pix2pix-style approach (which needs paired input/output examples to supervise directly) simply doesn't apply here.

Without paired supervision, a plain adversarial loss alone has a well-known failure mode: nothing stops a generator from producing a realistic-looking zebra image that ignores its input horse entirely. CycleGAN's fix is to combine three signals: the standard adversarial loss (make outputs look like real images of the target domain), a cycle-consistency loss (translating there and back should reconstruct the input), and an identity loss (translating an image that's already in the target domain shouldn't change it). Together, these are what make learning from two unrelated image folders — rather than a paired dataset — actually work.

What the repo does

This repository is a from-scratch CycleGAN implementation in PyTorch: a ResNet generator and PatchGAN discriminator built from the ground up (not wrapped around someone else's model), entirely config-driven via YAML rather than a pile of argparse flags, with modern training infrastructure — multi-GPU (DDP), mixed precision (AMP), and full resumable checkpoints (model, optimizer, scheduler, and RNG state) — wired in from the start rather than bolted on later. It's tested with a CPU-only pytest suite that runs against small synthetic images, so the test suite needs no GPU and no dataset download. And it comes with a tutorial notebook and a full theory write-up (docs/architecture_and_theory.md) for anyone who wants to learn the algorithm by reading the code, not just run a training script against it.

How it works

CycleGAN trains four networks jointly. Using domain $AA$ (e.g. horses) and domain $BB$ (e.g. zebras):

• $G_{AB}G\_\{AB\}$ (ResnetGenerator): translates $A\to BA\; \backslash to\; B$
• $G_{BA}G\_\{BA\}$: translates $B\to AB\; \backslash to\; A$
• $D_{A}D\_A$ (PatchGANDiscriminator): tells real $AA$ images apart from images $G_{BA}G\_\{BA\}$ produced
• $D_{B}D\_B$: tells real $BB$ images apart from images $G_{AB}G\_\{AB\}$ produced

Generator: a ResNet encoder–decoder

ResnetGenerator follows the Johnson et al. style-transfer architecture the CycleGAN paper adopts:

c7s1-64, d128, d256, R256 x N, u128, u64, c7s1-3

A wide 7×7 stem that doesn't downsample, two stride-2 convs that downsample while doubling channels (\( 64 \to 128 \to 256 \)), $NN$ residual blocks operating on that downsampled feature map, two stride-2 transposed convs that upsample back to the input resolution, and a final 7×7 conv into $\tanh \backslash tanh$ to keep outputs in $[-1,1][-1,\; 1]$ — matching the normalization applied to real images. Each residual block is two 3×3 convs with a skip connection, $x+\text{block}(x)x\; +\; \backslash text\{block\}(x)$, which is where most of the network's depth and capacity lives. InstanceNorm2d is used throughout rather than BatchNorm2d, since normalizing per-image (not across the batch) suits style-transfer-like tasks and stays well-behaved even at the paper's default batch size of 1. $NN$ is a config value, not a hardcoded constant: the paper uses $N=6N\; =\; 6$ for 128×128 images and $N=9N\; =\; 9$ for 256×256 and above. The residual blocks operate on a feature map downsampled 4x in each spatial dimension, so a 256×256 input becomes a 64×64 feature map at that stage — large enough to need more blocks to build sufficient capacity than a 128×128 input's 32×32 feature map. This is an empirical finding from the paper, not a hard requirement, so cyclegan/config.py only emits a soft warning if n_residual_blocks looks mismatched for image_size.

Discriminator: PatchGAN

PatchGANDiscriminator doesn't output a single real/fake score for the whole image — it outputs a grid, where each value judges a roughly 70×70-pixel patch of the original-resolution input. This is cheaper than a full-image discriminator, works at any resolution, and is empirically sufficient to enforce realistic local texture, which is what matters most for translation tasks like this.

The 70×70 receptive field is a specific consequence of a 5-layer stack:

C64 (no norm) -> C128 -> C256 -> C512 (stride 1) -> C1 (stride 1, output)

where each $C_{k}C\_k$ is a 4×4 conv with $kk$ filters and LeakyReLU(0.2); the first three layers downsample (stride 2), the last two hold resolution fixed (stride 1). The receptive field of a stack of conv layers can be computed backward from the output, one layer at a time:

$$r_{l-1}=s_l\cdot (r_l-1)+k_{l}r\_\{l-1\}\; =\; s\_l\; \backslash cdot\; (r\_l\; -\; 1)\; +\; k\_l$$

where $r_{l}r\_l$ is the receptive field measured at layer $ll$, and $k_{l}k\_l$, $s_{l}s\_l$ are that layer's kernel size and stride. Starting from the 1×1 output and working backward through five 4×4-kernel layers gives exactly 70 pixels — which is why the discriminator needs five conv layers, not four. This is worth calling out because it was a real bug I found while porting an earlier, smaller version of this code: an earlier discriminator was missing the C512 stride-1 layer entirely. Dropping it doesn't crash anything — the network still trains — but it quietly shrinks the effective receptive field, weakening the local-texture signal the discriminator can actually enforce. It's the kind of bug that's easy to miss because the loss curves still look fine.

The three losses

Adversarial loss (LSGAN). Rather than the original GAN's cross-entropy loss, CycleGAN uses the "least-squares GAN" formulation: the discriminator's output is compared against a constant target via MSE instead of a sigmoid + binary cross-entropy, which tends to produce more stable gradients far from convergence. For generator $G:A\to BG:\; A\; \backslash to\; B$ and its discriminator $D_{B}D\_B$:

$${\mathcal{L}}_{GAN}(G,D_B)={\mathbb{E}}_{b\sim p_{data}(b)}[(D_B(b)-1{)}^2]+{\mathbb{E}}_{a\sim p_{data}(a)}[D_B(G(a){)}^2]\backslash mathcal\{L\}\_\{GAN\}(G,\; D\_B)\; =\; \backslash mathbb\{E\}\_\{b\; \backslash sim\; p\_\{data\}(b)\}\backslash big[(D\_B(b)\; -\; 1)^2\backslash big]\; +\; \backslash mathbb\{E\}\_\{a\; \backslash sim\; p\_\{data\}(a)\}\backslash big[D\_B(G(a))^2\backslash big]$$

Cycle-consistency loss. For a real image $aa$ in domain $AA$, $G_{BA}(G_{AB}(a))G\_\{BA\}(G\_\{AB\}(a))$ should reconstruct $aa$; symmetrically for $b\in Bb\; \backslash in\; B$. This is an L1 loss between each reconstruction and the original:

$${\mathcal{L}}_{cyc}(G_{AB},G_{BA})={\mathbb{E}}_a[\parallel G_{BA}(G_{AB}(a))-a{\parallel }_1]+{\mathbb{E}}_b[\parallel G_{AB}(G_{BA}(b))-b{\parallel }_1]\backslash mathcal\{L\}\_\{cyc\}(G\_\{AB\},\; G\_\{BA\})\; =\; \backslash mathbb\{E\}\_\{a\}\backslash big[\backslash lVert\; G\_\{BA\}(G\_\{AB\}(a))\; -\; a\; \backslash rVert\_1\backslash big]\; +\; \backslash mathbb\{E\}\_\{b\}\backslash big[\backslash lVert\; G\_\{AB\}(G\_\{BA\}(b))\; -\; b\; \backslash rVert\_1\backslash big]$$

This is the loss that makes unpaired training possible at all — it ties $G_{AB}G\_\{AB\}$ and $G_{BA}G\_\{BA\}$ together without ever needing a paired ground-truth target image.

Identity loss. Feeding $G_{AB}G\_\{AB\}$ an image already in domain $BB$ should return it mostly unchanged: $G_{AB}(b)\approx bG\_\{AB\}(b)\; \backslash approx\; b$. Not required for the core algorithm, but the paper found it helps preserve color composition:

$${\mathcal{L}}_{identity}(G_{AB},G_{BA})={\mathbb{E}}_b[\parallel G_{AB}(b)-b{\parallel }_1]+{\mathbb{E}}_a[\parallel G_{BA}(a)-a{\parallel }_1]\backslash mathcal\{L\}\_\{identity\}(G\_\{AB\},\; G\_\{BA\})\; =\; \backslash mathbb\{E\}\_\{b\}\backslash big[\backslash lVert\; G\_\{AB\}(b)\; -\; b\; \backslash rVert\_1\backslash big]\; +\; \backslash mathbb\{E\}\_\{a\}\backslash big[\backslash lVert\; G\_\{BA\}(a)\; -\; a\; \backslash rVert\_1\backslash big]$$

Combined objective. All three terms combine into the full generator loss:

$${\mathcal{L}}_G={\mathcal{L}}_{GAN}(G_{AB},D_B)+{\mathcal{L}}_{GAN}(G_{BA},D_A)+{\lambda }_{cyc}{\mathcal{L}}_{cyc}+{\lambda }_{cyc}{\lambda }_{id}{\mathcal{L}}_{identity}\backslash mathcal\{L\}\_G\; =\; \backslash mathcal\{L\}\_\{GAN\}(G\_\{AB\},\; D\_B)\; +\; \backslash mathcal\{L\}\_\{GAN\}(G\_\{BA\},\; D\_A)\; +\; \backslash lambda\_\{cyc\}\backslash ,\; \backslash mathcal\{L\}\_\{cyc\}\; +\; \backslash lambda\_\{cyc\}\backslash ,\; \backslash lambda\_\{id\}\backslash ,\; \backslash mathcal\{L\}\_\{identity\}$$

with ${\lambda }_{cyc}=10.0\backslash lambda\_\{cyc\}\; =\; 10.0$ and ${\lambda }_{id}=0.5\backslash lambda\_\{id\}\; =\; 0.5$ as paper-default config values (training.lambda_cycle, training.lambda_identity), not hardcoded constants — both live in cyclegan/losses.py as small, independently unit-tested functions.

The image replay buffer

A subtlety in adversarial training: if a discriminator only ever sees the generator's most recent output, $GG$ and $DD$ can chase each other into tight oscillations instead of converging. ImagePool (cyclegan/image_pool.py) keeps a buffer (default 50 images per domain) of previously generated fakes; when updating a discriminator, each fake in the current batch has a 50% chance (once the pool fills) of being swapped for an older buffered one instead of used directly. This is a standard stabilization trick from Shrivastava et al. (2017) that the CycleGAN authors reuse — and it's easy to overlook if you're implementing from the paper's loss equations alone, since it doesn't appear in any of them.

The full derivations, architecture diagrams, and design rationale live in docs/architecture_and_theory.md — worth reading end-to-end alongside notebooks/tutorial_cyclegan.ipynb if you want to see every term computed and printed for a single manual training step.

Examples

Output of a training run on horse2zebra using the default hyperparameters in `configs/horse2zebra.yaml`.

These are illustrative outputs from a training run, not a benchmarked result — there's no hosted pretrained checkpoint yet (see the Limitations section below), so treat this as "here's what training with this codebase produces," not a claim about model quality relative to other implementations.

How to use it

git clone https://github.com/turhancan97/train-cycleGAN
cd train-cycleGAN
pip install -r requirements.txt
pip install -e .

# 1. Get the default dataset
python data/download_horse2zebra.py

# 2. Train
python train.py --config configs/horse2zebra.yaml

# 3. Translate images with the trained checkpoint
python inference.py --checkpoint checkpoints/horse2zebra_256/last.pt \
    --config configs/horse2zebra.yaml --direction a2b \
    --input data/horse2zebra/testA --output_dir outputs/

# 4. (Optional) Evaluate translation quality with FID
python evaluate.py --checkpoint checkpoints/horse2zebra_256/last.pt \
    --config configs/horse2zebra.yaml --direction a2b

UnpairedImageDataset is fully generic — nothing in the data pipeline is specific to horse2zebra. Copy configs/horse2zebra.yaml, point data.root_a / data.root_b at any two folders of unpaired images, and the rest of the pipeline (training, checkpointing, inference, evaluation) works unchanged.

Technical detail

The parts that make this usable for real training runs, not just a toy demo:

• Multi-GPU: single-node DistributedDataParallel, launched with torchrun --nproc_per_node=<N> train.py --config configs/horse2zebra.yaml. DDP specifics live in cyclegan/engine/distributed.py and no-op cleanly without torchrun, so the core training step reads like plain single-GPU code.
• Mixed precision (AMP): forward pass and loss computation run inside torch.cuda.amp.autocast, with a single shared GradScaler across the generator and both discriminator optimizers — scaler.update() is called exactly once per iteration, after all three step() calls, to keep its growth tracking correct.
• Cluster training: a generic single-GPU SLURM template under scripts/train.sh, parameterized by environment variables (CONDA_ENV, CONDA_SH, CONFIG_PATH) so it drops into most cluster setups without editing the script itself.
• Full resumable checkpoints: model, optimizer, and scheduler state, the AMP scaler state, and best-effort RNG state (torch/CUDA/NumPy/Python), plus a snapshot of the full config and a schema version. --resume checkpoints/.../last.pt restores all of it, so a killed and restarted run has no discontinuity in its loss curves.
• Config-driven, always-on logging: every hyperparameter lives in a YAML file under configs/; TensorBoard logging is always on, with optional Weights & Biases if you have it installed and configured.

Limitations

There's no hosted pretrained checkpoint yet — download_pretrained.py is wired up to fetch and checksum-verify one by name, but the registry is currently empty. Training your own with train.py is the path for now.

Beyond that, this inherits CycleGAN's own known failure modes, not something this implementation fixes: it's good at texture and color changes but struggles with tasks that require geometric changes (e.g., changing an object's shape or pose), and can introduce visible artifacts, especially outside the resolution and domain pairs it was tuned for. It's a solid general-purpose unpaired translation method, not a universal one.

Future directions

The obvious next step is training and hosting a checkpoint so download_pretrained.py actually has something to fetch, plus wiring up a couple more dataset pairs beyond horse2zebra to make the "bring your own unpaired folders" path easier to try.

Separately — and this is more of an open question than a roadmap item — I'm curious whether (and how well) this kind of translation preserves the underlying spatial structure of a scene as opposed to just its texture and color. Not something I have an answer to yet, but it's the direction I'd want to poke at next.

Links

• Repository: github.com/turhancan97/train-cycleGAN
• Tutorial notebook: notebooks/tutorial_cyclegan.ipynb
• Theory and architecture write-up: docs/architecture_and_theory.md
• Original paper: Zhu et al., 2017 — Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks