CAT VLM V3.5: FP16 BPTT Stability and the vNext Run

The Short Story

Last week the Cat Chat GPU training path was nine out of eleven stages green and a queue of numerical warnings. This week it is eleven out of eleven, ./scripts/test.sh --tools passes 268/268, and the current training run completed 1066 steps with a final running loss of 3.55 and an eval top-5 of 27.09%. The fixes are small, but the explanation matters: FP16 BPTT is a precision contract, and every backprop step is a place for that contract to break.

11/11

GPU stages passing

268/268

tool tests passing

1066

steps in current run

3.55

final running loss

5.10%

eval top-1

27.09%

eval top-5

What Went Wrong in FP16 BPTT

Three issues surfaced as soon as the model left the short benchmark runs and tried to train on real sequences. First, the per-timestep hidden-state gradient grad_h in the GRU backward pass would saturate in FP16, not because the values were large in absolute terms, but because the clamp was being interpreted as already scaled. With loss_scale=1 the value looked reasonable, but the same config with a static loss scale would have silently changed the effective clamp. We moved the multiplication into nn/systems/lm/training/gpu/gpu_trainer_dispatch.cpp, so the user-facing grad_h_clamp is always an unscaled value and the dispatch scales it by loss_scale before it reaches the shader.

Second, the RMSNorm shader computed inv_rms = 1 / sqrt(mean_sq + eps). For a nearly-zero hidden state at the start of training, mean_sq + eps is still small and inv_rms can explode into the tens of thousands, feeding non-finite values straight back into BPTT. We added min(1.0 / rms, 1000.0) and a mean_sq <= 1e-6 identity fallback in assets/shaders/cat_chat/rmsnorm.comp to match the CPU reference and stop the divide-by-near-zero.

Third, Stage 6/9 of the cat_chat --gpu-test ladder compares GPU Tensor Core cooperative-matrix accumulation against the CPU reference. The accumulation order is non-deterministic across subgroups, so an absolute tolerance would fail even when the result was mathematically correct. We switched to a relative tolerance for the cooperative-matrix accumulation-order non-determinism and lowered the gradient-precision test from a higher static scale to loss_scale=128, which still exercises the precision path without masking real errors.

grad_h clamp

Loss-scale-aware dispatch in gpu_trainer_dispatch.cpp keeps grad_h_clamp invariant under different FP16 loss-scale choices.

RMSNorm inv_rms

A clamp at 1000.0 and a near-zero identity guard in the shader prevent 1/√mean_sq blow-up for small hidden states.

Stage 6/9 tolerance

Relative tolerance for cooperative-matrix non-determinism plus loss_scale=128 for the gradient-precision test.

Test coverage

cat_chat --gpu-test now 11/11 and ./scripts/test.sh --tools 268/268.

FP16 is not a smaller number format. It is a contract about when and where you are willing to lose precision.

Training Telemetry: Previous vs. Current

The previous run is the longest available historical telemetry captured before the current fixes; the current run is the one taken after them. The old curve is short because the pipeline was not yet stable enough to run for thousands of steps. The new curve is the first long run under the production config. Direct step-for-step comparison is approximate because vocabulary size, sequence length, and per-step diagnostics differ between the two runs.

Previous run: 17 steps, final running loss 7.79, top-1 3.35%, top-5 14.02%. Current run: 1066 steps, final running loss 3.55, top-1 5.10%, top-5 27.09%. Source: data/cat_chat_gpu/training_telemetry_history_20260802_033754.csv (previous) and data/cat_chat/training_telemetry.csv (current).

What 11/11 Means

The cat_chat --gpu-test ladder is an incremental check of the GPU training pipeline: GEMM, GRU gates, RMSNorm, attention, softmax cross-entropy, gradient reduction, and optimizer. Each stage compares the GPU against the CPU reference on the same batch. Moving from 9/11 to 11/11 does not mean the numerical problems are gone forever; it means every known class of failure has a guard and a test, so the next failure will be a new one instead of a repeat.

./scripts/test.sh --tools passing 268/268 gives the same guarantee at the tool boundary. The CPU and GPU paths are still different machines running different code, but the contracts they agree on are now encoded in tests rather than hope.

Note on throughput: The previous run averaged 29.03 sequences/second with 2.22s batches, while the current run averaged 7.05 seq/s with 9.08s batches. The current run uses a larger vocabulary, longer and more varied sequences, and heavier per-step diagnostics, so raw throughput is not a fair head-to-head metric. The important number is that both are stable and finite.

A stable training pipeline is not one that never fails. It is one that fails at the test bench instead of at hour three of an overnight run.

vNext: Dual-Stream Attention Decoder V3

With the FP16 BPTT pipeline stable, the next run is CAT VLM V3.5, also called the Dual-Stream Attention Decoder V3. It keeps all the fixes above, then extends them:

Larger context

max_seq_len grows from 256 to 512 tokens. The context stream is the whole reason the dual-stream architecture exists; giving it twice the window should let it catch longer-range dependencies.

Longer training run

The current run proves the pipeline can survive 1,000+ steps. The next run will train longer and log per-step eval and gradient-norm telemetry now that the pipeline is trustworthy.

Production config

grad_h_clamp=1000.0 with loss_scale=1.0, dispatched as an unscaled clamp multiplied by loss scale at the shader push constants.

Richer telemetry

Per-step eval accuracy and gradient norms will be written to the telemetry CSV at regular intervals, not only at the final step.

Related Work

These posts are the foundation this one builds on:

Dual-Stream GRU-L3 GPU Training Teaching Vulkan to Multiply FP16 Batch Interleaving