Which LLM actually runs on a 4060 Ti 16 GB?

RTX 4060 Ti 16 GB (288 GB/s) + 48 GB DDR4-3200 + i5-6600K (4 cores), under llama.cpp. Modelled, not measured — see the report for method and error bars.

Model

Configuration

Context

Quantisation

KV cache type

Usable VRAM

CPU throughput per core

Expert offload

auto (least offload that fits)

assume MTP self-speculation works (×1.6) GLM: pessimistic MHA reading of attention

–tok/s generated

weights in VRAM KV cache compute + CUDA ctx free weights in system RAM (off-chart)

Command line

Run bench.sh from the repo folder to check these predictions against your actual machine, then python3 vram_model.py --calibrate bench.csv.

All seven at this context

Each model shown with its own best Q4-class quant at the current context and settings.