U di@s`dZddlmZmZmZer&ddlZeeZddZ ddZ d d Z dd d Z dddZ dS)z2HQQ (Half-Quadratic Quantization) integration file)is_hqq_availableis_torch_availableloggingNcCs|D]\}}||_qdSN) named_modulesname)modelrmoduler A/tmp/pip-unpacked-wheel-bm_b0l5e/transformers/integrations/hqq.pyautoname_modulessr cCsddd|dDS)N.cSs g|]}|dkr|s|qS))r Zlayers) isnumeric).0nr r r !sz&name_to_linear_tag..)joinsplit)rr r r name_to_linear_tag srcCsRtrddlm}t}|D](\}}t|tjj|fr | t |q t |S)Nr) HQQLinear) rZhqq.core.quantizersetr isinstancetorchnnLinearaddrlist)r r linear_tagsrr r r r get_linear_tags%s rc Cs|D]\}}|dkrg}||t|tjjrt|j}||kr||dk r|||j|_ t ||j|_ |j| dd}dD]}t ||dqtt|dkrt|||d\}}|dq||fS)NFT)ZW_qmetar patch_paramshas_been_replaced)Znamed_childrenappendrrrrrrZ_modules quant_configtypeZ source_clsZrequires_grad_setattrlenrchildren_prepare_for_hqq_linearpop) r r"r#Zcurrent_key_namerr Z linear_tagZatt_r r r r+0s,     r+Fcs|dkr gn|}t|t||j}|jttt|t|tfddDr|ddD}|nfddD}t |||d\}}|j |d|j _ |st d |S) a Prepares nn.Linear layers for HQQ quantization. Since each layer type can have separate quantization parameters, we need to do the following: 1- tag each module with its neme via autoname_modules() 2- Extract linear_tags (e.g. ['self_attn.q_proj', ...]) 3- Map quantization parameters as a dictionary linear_tag -> quant_params as HQQLinear exepects it, this is referred to as patch_params Nc3s|]}|kVqdSrr rkey)rr r ksz)prepare_for_hqq_linear..cSsi|] }|dqSrr r.r r r msz*prepare_for_hqq_linear..csi|] }|qSr r )rk)r&r r r1qsr!)r& quant_method skip_moduleszs  %