Hi3ddlmZddlmZer ddlmZddlmZddlm Z m Z m Z m Z m Z ddlmZe r dd lZdd lmZe j&eZd ZGd d eZy ) ) TYPE_CHECKING) HfQuantizer)PreTrainedModel) Mxfp4Config)is_accelerate_availableis_kernels_availableis_torch_availableis_triton_availablelogging)get_module_from_nameN)WeightConverterceZdZUdZdZded<fdZdZdZdd d e d e fd Z dd Z ddd de fdZ dZdZdZdZed e fdZdZdZxZS)Mxfp4HfQuantizerz/ FP4 quantization using fbgemm kernels Frquantization_configc 4t||fi|d|_yN)super__init__triton_kernels_hub)selfrkwargs __class__s i/var/www/pod-logistic/pod-ai/venv/lib/python3.12/site-packages/transformers/quantizers/quantizer_mxfp4.pyrzMxfp4HfQuantizer.__init__2s ,77"&c|j ddlm}|d|_|jS|jS#t$r tdwxYw)z3Lazy import and initialize kernels only when neededr) get_kernelz(kernels-community/gpt-oss-triton-kernelsz2kernels package is required for MXFP4 quantization)rintegrations.hub_kernelsr ImportError)rrs r_lazy_import_kernelsz%Mxfp4HfQuantizer._lazy_import_kernels6s]  " " * XA*45_*`'&&&t&&& X!"VWW Xs 9Ac>ts td|jjryt s tdt j jxst jd}|jdvrF|jr+tjd|dd|j_ytd|d t jjrd}t!d }t#}nt j$jr9t j$j'}|d k\}t!d }t#}n-|jdk(rd}t!d }t#}nd }d }d }|jr{|s'tjdd|j_y|s'tjdd|j_y|sNtjdd|j_y|s t)d|s t)d|s t)d|js|j+|j-d}|= 7.5 (e.g T4, A100, L4, H100, or B200) or XPUs (e.g Intel® Data Center GPU Max Series). We will default to dequantizing the model to bf16.zMXFP4 quantization requires Triton: CUDA requires Triton >= 3.4.0, XPU/CPU requires Triton >= 3.5.0. Please install triton: `pip install triton`. We will default to dequantizing the model to bf16.zMXFP4 quantization requires the `kernels` package: `pip install kernels>=0.12.0`. We will default to dequantizing the model to bf16.uMXFP4 quantization is only supported on GPUs with compute capability >= 7.5 (e.g T4, A100, L4, H100, or B200) or XPUs (e.g Intel® Data Center GPU Max Series) or CPUzMXFP4 quantization requires Triton: CUDA requires Triton >= 3.4.0, XPU/CPU requires Triton >= 3.5.0. Please install triton: `pip install triton`zPMXFP4 quantization requires the `kernels` package: `pip install kernels>=0.12.0` device_mapdiskzYou are attempting to load an FP4 model with a device_map that contains a disk device.This is not supported when the model is quantized on the fly. Please use a quantized checkpoint or remove the disk device from the device_map.)r r r dequantizer torch acceleratorcurrent_acceleratordevicetype pre_quantizedlogger warning_once RuntimeErrorr% is_availabler r r$get_device_capability ValueErrorr!get isinstancedictvalues) rargsrr.is_device_supported_mxfp4triton_availablekernels_installedcompute_capabilityr(s rvalidate_environmentz%Mxfp4HfQuantizer.validate_environmentAs!#]   # # . . &(YZ Z""668OELL)#^^,=r2rJF#^^,={BPRSF)66>>EETT$1199>>)B#7= dV1TF'236< dV1TF'23 = =28##rcy)NTrs ris_serializablez Mxfp4HfQuantizer.is_serializablesrc.tjdy)NzMXFP4 quantization don't support training, please consider dequantizing the model first by passing quantization_config=Mxfp4Config(dequantize=True) to .from_pretrained()F)r1r2rs r is_trainablezMxfp4HfQuantizer.is_trainables x rcddlm}||S)Nr) Mxfp4Quantize)integrations.mxfp4r)rrs rget_quantize_opsz!Mxfp4HfQuantizer.get_quantize_opss6T""rcddlm}m}|jrE|jj r/t ddgd||gt ddgd g||ggSt ddgd ||gt ddgd||ggS) Nr)Mxfp4DequantizeMxfp4Deserializedown_proj_blocksdown_proj_scalesz down_proj$)source_patternstarget_patterns operationsgate_up_proj_blocksgate_up_proj_scalesz gate_up_proj$)rrrr0rr*r)rrrs rget_weight_conversionsz'Mxfp4HfQuantizer.get_weight_conversionssJ   $":":"E"E%79K$L$1 / 56  %:rs[!0734   H %Q {Q r