U di@sddlmZmZmZmZmZddlmZer8ddlm Z ddl m Z m Z m Z mZddlmZe rjddlZeeZGd d d eZdS) ) TYPE_CHECKINGAnyDictListOptional) HfQuantizer)PreTrainedModel)is_accelerate_availableis_eetq_availableis_torch_availablelogging)get_module_from_nameNc seZdZdZdZdZddgZfddZdd Zd d d d d Z dde e e e fdddZ d#dde de e e feee dddZddddZgfdee dddZd$ddZeed d!d"ZZS)%EetqHfQuantizera 8-bit quantization from EETQ quantization method: before loading: converts transformer layers into W8A16Linear during loading: load 16bit weight and pass to the layer object after: quantizes individual weights in Linear8bitLt into 8bit at first .cuda() call TFeetqZ acceleratec stj|f|||_dSN)super__init__quantization_config)selfrkwargs __class__J/tmp/pip-unpacked-wheel-bm_b0l5e/transformers/quantizers/quantizer_eetq.pyr-szEetqHfQuantizer.__init__cOststdtstd|dds4|ddrr?rFrHpropertyboolrI __classcell__rrrrr!s<       r)typingrrrrrbaserZmodeling_utilsr utilsr r r rZquantizers_utilsrr!Z get_loggerrJr#rrrrrs