U diG@s>ddlmZmZmZmZer&ddlZeeZdddZ dS))is_optimum_quanto_availableis_quanto_availableis_torch_availableloggingNFc sddlm}tr4ddlm}m}m}m} m} m } n0t rdt dddl m}m}m}m} m} m } || | | d} d|| d} |dkrg}|D]"\}}dkrg|tfdd |Ds|t|tjjr,||j|j|jdk |jj| |j| |jd |j|<|j|d d }nJt|tjjrv|jdk rv||j|j |j!|jdk | |jd |j|<d }W5QRXt"t#|$dkrt%||||d\}}&dq||fS)a[ Public method that recursively replaces the Linear layers of the given model with Quanto quantized layers. Returns the converted model and a boolean that indicates if the conversion has been successfull or not. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`AqlmConfig`, defaults to `None`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`list`, *optional*, defaults to `None`): A list of modules to not convert. If a module name is in the list (e.g. `lm_head`), it will not be converted. current_key_name (`list`, *optional*, defaults to `None`): A list that contains the current key name. This is used for recursion and should not be passed by the user. has_been_replaced (`bool`, *optional*, defaults to `None`): A boolean that indicates if the conversion has been successful or not. This is used for recursion and should not be passed by the user. r)init_empty_weights) QLayerNormQLinearqfloat8qint2qint4qint8zuImporting from quanto will be deprecated in v4.47. Please install optimum-quanto instead `pip install optimum-quanto`)float8int8Zint4Zint2N)Nrrc3s|]}|dkVqdS).N)join).0keycurrent_key_nameD/tmp/pip-unpacked-wheel-bm_b0l5e/transformers/integrations/quanto.py Fsz-replace_with_quanto_layers..) in_features out_featuresbiasdtypeweights activationsFT)r)quantization_configmodules_to_not_convertrhas_been_replaced)'Z acceleraterrZoptimum.quantorr r r r r rloggerZ warning_onceZquantoZnamed_childrenappendany isinstancetorchnnZLinearrrrZweightrrrZ_modulesZrequires_grad_Z LayerNormZnormalized_shapeZepsZelementwise_affinelenlistchildrenreplace_with_quanto_layerspop)modelrr rr!rrr r r r r Z w_mappingZ a_mappingnamemodule_rrrr,sb "        r,)NNNF) utilsrrrrr'Z get_logger__name__r#r,rrrrs