TyloQuant MFQ
Put every bit where precision matters most
TyloQuant MFQ co-designs quantization formats, precision allocation, and inference kernels so packed weights can run directly through CUDA kernels and a C++ runtime for high-fidelity local LLM inference.
BPW encoding range
Custom weight encodings span quality tiers from below 1 bit to above 8 bits.
Production encodings
NINT, NVQ, NPQ, NEPQ, and related formats serve different precision budgets.
Direct inference runtime
Packed weights execute without materializing complete FP16 weights.
The precision budget follows the full path from weight structure to execution
Mixed-format encoding
NINT, NVQ, NPQ, and NEPQ create different quality and size tiers instead of forcing one format on every weight.
Precision allocation
Gradient calibration selects precision per compute group; MoE models can also allocate bitrate per expert.
Heterogeneous expert container
NINTM v2 stores heterogeneous formats in one MoE tensor so individual experts can receive different precision.
Direct packed execution
CUDA kernels and the C++ runtime execute compressed weights directly, with an OpenAI-compatible API and local WebUI.
The public implementation covers several recent model architectures
This is an inspectable research prototype, not an unconditional performance promise
Full-model serving currently remains primarily a single-GPU CUDA research prototype. Quality, speed, and memory requirements vary with architecture, precision tier, hardware, and runtime configuration. The Metal HTTP server remains in development.
浙公网安备33011002019019号