U diF @s@ddlZddlmZddlmZmZmZmZmZddl Z ddl m Z m Z mZmZmZddlmZmZmZmZmZddlmZmZmZmZmZmZmZerddlZddl m Z erddl!Z!erddl"Z#erddl$m Z%erdd l&m'Z(nerdd l)m'Z(dQe j*ee e+feee e+fe j*d d d Z,de j-dfe j*e.ee e j/eee+e fe j*d ddZ0ddZ1dRee j*ddddfee2ee+eee+e fddddZ3dSe j*ee4ee4e4fee4ee4fe2ee4eee+e fe5dddZ6dTe j*ee4e4fdee4ee e2eee+e fe j*ddd Z7dUe j*ee.ee.fee.ee.fee eee+e fe j*d!d"d#Z8dVe j*ee4e4feee+e feee+e fee2e j*d$d%d&Z9ddd'd(d)Z:e j*e j*d'd*d+Z;ddd'd,d-Ze j*e j*d0d3d4Z?ddd0d5d6Z@eed0d7d8ZAd9d:ZBd;d<ZCGd=d>d>eZDeDjEd?ddfe j*ee4ee4e4feee4e4ffeDee.ee.feee+e feee+e fe j*d@dAdBZFe e dCdDdEZGdWe j*ee eee+e fe j*dFdGdHZHdIdJZIGdKdLdLZJGdMdNdNZKGdOdPdPZLdS)XN)ceil)IterableListOptionalTupleUnion)ChannelDimension ImageInputget_channel_dimension_axisget_image_sizeinfer_channel_dimension_format) ExplicitEnum TensorType is_jax_tensor is_tf_tensoris_torch_tensor)is_flax_availableis_tf_availableis_torch_availableis_torchvision_availableis_torchvision_v2_availableis_vision_availablerequires_backends)PILImageResampling) functional)image channel_diminput_channel_dimreturncCst|tjstdt||dkr.t|}t|}||krB|S|tjkrX|d}n$|tj krn|d}nt d ||S)a) Converts `image` to the channel dimension format specified by `channel_dim`. Args: image (`numpy.ndarray`): The image to have its channel dimension set. channel_dim (`ChannelDimension`): The channel dimension format to use. input_channel_dim (`ChannelDimension`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred from the input image. Returns: `np.ndarray`: The image with the channel dimension set to `channel_dim`. ,Input image must be of type np.ndarray, got N)rr)rr!rz(Unsupported channel dimension format: {}) isinstancenpndarray TypeErrortyper r FIRST transposeLAST ValueErrorformat)rrrZtarget_channel_dimr,A/tmp/pip-unpacked-wheel-bm_b0l5e/transformers/image_transforms.pyto_channel_dimension_format=s     r.)rscale data_formatdtypeinput_data_formatrcCsPt|tjstdt||tj|}|dk rBt|||}||}|S)a Rescales `image` by `scale`. Args: image (`np.ndarray`): The image to rescale. scale (`float`): The scale to use for rescaling the image. data_format (`ChannelDimension`, *optional*): The channel dimension format of the image. If not provided, it will be the same as the input image. dtype (`np.dtype`, *optional*, defaults to `np.float32`): The dtype of the output image. Defaults to `np.float32`. Used for backwards compatibility with feature extractors. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred from the input image. Returns: `np.ndarray`: The rescaled image. r N)r"r#r$r%r&astypeZfloat64r.)rr/r0r1r2Zrescaled_imager,r,r-rescaleds   r4cCs|jtjkrd}nt||trftd|krFt|dkrFd}qtd|d| dn@td|krt|dkrd}ntd |d| d|S) z Detects whether or not the image needs to be rescaled before being converted to a PIL image. The assumption is that if the image is of type `np.float` and all values are between 0 and 1, it needs to be rescaled. FrzZThe image to be converted to a PIL image contains values outside the range [0, 255], got [z, z%] which cannot be converted to uint8.rTzXThe image to be converted to a PIL image contains values outside the range [0, 1], got [) r1r#uint8Zallcloser3intallr*minmax)r do_rescaler,r,r-_rescale_for_pil_conversions r<zPIL.Image.Image torch.Tensorz tf.Tensorz jnp.ndarray)rr; image_moder2rcCsttdgt|tjjr|St|s.t|r8|}n2t|rLt |}nt|t j sjt d t|t|tj|}|jddkrt j|ddn|}|dkrt|n|}|rt|d}|t j}tjj||dS) a Converts `image` to a PIL Image. Optionally rescales it and puts the channel dimension back as the last axis if needed. Args: image (`PIL.Image.Image` or `numpy.ndarray` or `torch.Tensor` or `tf.Tensor`): The image to convert to the `PIL.Image` format. do_rescale (`bool`, *optional*): Whether or not to apply the scaling factor (to make pixel values integers between 0 and 255). Will default to `True` if the image type is a floating type and casting to `int` would result in a loss of precision, and `False` otherwise. image_mode (`str`, *optional*): The mode to use for the PIL image. If unset, will use the default mode for the input image type. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If unset, will use the inferred format from the input. Returns: `PIL.Image.Image`: The converted image. visionz"Input image type not supported: {}rZaxisNr5mode)r to_pil_imager"PILImagerrnumpyrr#arrayr$r*r+r&r.r r)shapeZsqueezer<r4r3r6Z fromarray)rr;r>r2r,r,r-rDs        rDT) input_imagesizedefault_to_squaremax_sizer2rc Cst|ttfr@t|dkr"t|St|dkr8|d}ntd|rL||fSt||\}}||krj||fn||f\}}|} | t| ||} } |dk r|| krtd|d|| |krt|| | |} } ||kr| | fS| | fS)a Find the target (height, width) dimension of the output image after resizing given the input image and the desired size. Args: input_image (`np.ndarray`): The image to resize. size (`int` or `Tuple[int, int]` or List[int] or `Tuple[int]`): The size to use for resizing the image. If `size` is a sequence like (h, w), output size will be matched to this. If `size` is an int and `default_to_square` is `True`, then image will be resized to (size, size). If `size` is an int and `default_to_square` is `False`, then smaller edge of the image will be matched to this number. i.e, if height > width, then image will be rescaled to (size * height / width, size). default_to_square (`bool`, *optional*, defaults to `True`): How to convert `size` when it is a single int. If set to `True`, the `size` will be converted to a square (`size`,`size`). If set to `False`, will replicate [`torchvision.transforms.Resize`](https://pytorch.org/vision/stable/transforms.html#torchvision.transforms.Resize) with support for resizing only the smallest edge and providing an optional `max_size`. max_size (`int`, *optional*): The maximum allowed for the longer edge of the resized image: if the longer edge of the image is greater than `max_size` after being resized according to `size`, then the image is resized again so that the longer edge is equal to `max_size`. As a result, `size` might be overruled, i.e the smaller edge may be shorter than `size`. Only used if `default_to_square` is `False`. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If unset, will use the inferred format from the input. Returns: `tuple`: The target (height, width) dimension of the output image after resizing. r!rrz7size must have 1 or 2 elements if it is a list or tupleNz max_size = zN must be strictly greater than the requested size for the smaller edge size = )r"tuplelistlenr*r r7) rJrKrLrMr2heightwidthshortlongZrequested_new_shortZ new_shortZnew_longr,r,r-get_resize_output_image_sizes(%   rUr)rrKresample reducing_gapr0 return_numpyr2rc Csttdg|dk r|ntj}t|dks2td|dkrBt|}|dkrN|n|}d}t|tj j szt |}t |||d}|\}} |j| |f||d} |rt | } | jdkrt j| dd n| } t| |tjd } |rt| d n| } | S) a Resizes `image` to `(height, width)` specified by `size` using the PIL library. Args: image (`np.ndarray`): The image to resize. size (`Tuple[int, int]`): The size to use for resizing the image. resample (`int`, *optional*, defaults to `PILImageResampling.BILINEAR`): The filter to user for resampling. reducing_gap (`int`, *optional*): Apply optimization by resizing the image in two steps. The bigger `reducing_gap`, the closer the result to the fair resampling. See corresponding Pillow documentation for more details. data_format (`ChannelDimension`, *optional*): The channel dimension format of the output image. If unset, will use the inferred format from the input. return_numpy (`bool`, *optional*, defaults to `True`): Whether or not to return the resized image as a numpy array. If False a `PIL.Image.Image` object is returned. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If unset, will use the inferred format from the input. Returns: `np.ndarray`: The resized image. r?Nr!zsize must have 2 elementsF)r;r2)rVrWr@rArgp?)rresizerZBILINEARrPr*r r"rErFr<rDr#rHndimZ expand_dimsr.r r)r4) rrKrVrWr0rXr2r;rQrRZ resized_imager,r,r-rZs.!   rZ)rmeanstdr0r2rcCs2t|tjstd|dkr$t|}t||d}|j|}t|jtj sV| tj }t|t rt ||krtd|dt |n |g|}tj||jd}t|t rt ||krtd|dt |n |g|}tj||jd}|tjkr|||}n|j||j}|dk r*t|||n|}|S)a Normalizes `image` using the mean and standard deviation specified by `mean` and `std`. image = (image - mean) / std Args: image (`np.ndarray`): The image to normalize. mean (`float` or `Iterable[float]`): The mean to use for normalization. std (`float` or `Iterable[float]`): The standard deviation to use for normalization. data_format (`ChannelDimension`, *optional*): The channel dimension format of the output image. If unset, will use the inferred format from the input. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If unset, will use the inferred format from the input. zimage must be a numpy arrayN)r2zmean must have z$ elements if it is an iterable, got r1zstd must have )r"r#r$r*r r rIZ issubdtyper1Zfloatingr3float32rrPrHr r)Tr.)rr\r]r0r2Z channel_axisZ num_channelsr,r,r- normalizefs.           ra)rrKr0r2rXrcCs(ttdg|dk r tdt|dkr,dn|}t|tjsNtdt |t|t rdt |dkrlt d|dkr|t |}|dk r|n|}t|tj|}t|tj\}}|\}} t|t| }} ||d} | |} || d} | | } | dkr<| |kr<| dkr<| |kr<|d | | | | f}t||tj}|St||}t| |}|jdd ||f}tj||d }t||d}||}t||d}||}||d ||||f<| |7} | |7} | |7} | |7} |d td| t|| td| t|| f}t||tj}|s$t|}|S) a Crops the `image` to the specified `size` using a center crop. Note that if the image is too small to be cropped to the size given, it will be padded (so the returned result will always be of size `size`). Args: image (`np.ndarray`): The image to crop. size (`Tuple[int, int]`): The target size for the cropped image. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. input_data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. return_numpy (`bool`, *optional*): Whether or not to return the cropped image as a numpy array. Used for backwards compatibility with the previous ImageFeatureExtractionMixin method. - Unset: will return the same type as the input image. - `True`: will return a numpy array. - `False`: will return a `PIL.Image.Image` object. Returns: `np.ndarray`: The cropped image. r?Nz8return_numpy is deprecated and will be removed in v.4.33Tr r!zOsize must have 2 elements representing the height and width of the output imager.)rI)r center_cropwarningswarn FutureWarningr"r#r$r%r&rrPr*r r.r r'r r7r:rIZ zeros_likerr9rD)rrKr0r2rXZoutput_data_formatZ orig_heightZ orig_widthZ crop_heightZ crop_widthtopZbottomleftrightZ new_heightZ new_widthZ new_shapeZ new_imageZtop_padZ bottom_padZleft_padZ right_padr,r,r-rcsR#     (  .rc) bboxes_centerrcCsL|d\}}}}tj|d||d||d||d|gdd}|S)Nr@?ZdimZunbindtorchstack)rjcenter_xcenter_yrRrQZ bbox_cornersr,r,r-_center_to_corners_format_torchs *rrcCsH|j\}}}}tj|d||d||d||d|gdd}|S)Nrkr@rAr`r#rorjrprqrRrQbboxes_cornersr,r,r-_center_to_corners_format_numpys *rvcCsPtj|dd\}}}}tj|d||d||d||d|gdd}|S)Nr@rArktfZunstackrortr,r,r-_center_to_corners_format_tfs *rycCsJt|rt|St|tjr$t|St|r4t|Stdt |dS)a| Converts bounding boxes from center format to corners format. center format: contains the coordinate for the center of the box and its width, height dimensions (center_x, center_y, width, height) corners format: contains the coodinates for the top-left and bottom-right corners of the box (top_left_x, top_left_y, bottom_right_x, bottom_right_y) Unsupported input type N) rrrr"r#r$rvrryr*r&)rjr,r,r-center_to_corners_format%s  r{)rurcCsD|d\}}}}||d||d||||g}tj|ddS)Nr@r!rlrm)ru top_left_x top_left_ybottom_right_xbottom_right_ybr,r,r-_corners_to_center_format_torch:s  rcCs@|j\}}}}tj||d||d||||gdd}|S)Nr!r@rArsrur|r}r~rrjr,r,r-_corners_to_center_format_numpyEs   rcCsHtj|dd\}}}}tj||d||d||||gdd}|S)Nr@rAr!rwrr,r,r-_corners_to_center_format_tfSs   rcCsJt|rt|St|tjr$t|St|r4t|Stdt |dS)a Converts bounding boxes from corners format to center format. corners format: contains the coordinates for the top-left and bottom-right corners of the box (top_left_x, top_left_y, bottom_right_x, bottom_right_y) center format: contains the coordinate for the center of the box and its the width, height dimensions (center_x, center_y, width, height) rzN) rrr"r#r$rrrr*r&)rur,r,r-corners_to_center_formatas  rcCst|tjr|t|jdkr||jtjkr2|tj}|dddddfd|dddddfd|dddddfSt |dd|dd|dS)z* Converts RGB color to unique ID. Nrrir!) r"r#r$rPrIr1r6r3Zint32r7)colorr,r,r- rgb_to_idxs   JrcCst|tjrb|}tt|jdg}tj|tjd}t dD]}|d|d|f<|d}q@|Sg}t dD]}| |d|d}qn|S)z* Converts unique ID to RGB color. rr^r.) r"r#r$copyrNrOrIzerosr6rangeappend)Zid_mapZ id_map_copyZ rgb_shapeZrgb_mapir_r,r,r- id_to_rgbs     rc@s eZdZdZdZdZdZdZdS) PaddingModezP Enum class for the different padding modes to use when padding images. constantreflectZ replicate symmetricN)__name__ __module__ __qualname____doc__CONSTANTREFLECT REPLICATE SYMMETRICr,r,r,r-rs rg)rpaddingrCconstant_valuesr0r2rcsdkrtfdd}||}|tjkrL||}tj|d|dnb|tjkrhtj|ddnF|tjkrtj|ddn*|tjkrtj|d dntd ||dk rt |nS) a Pads the `image` with the specified (height, width) `padding` and `mode`. Args: image (`np.ndarray`): The image to pad. padding (`int` or `Tuple[int, int]` or `Iterable[Tuple[int, int]]`): Padding to apply to the edges of the height, width axes. Can be one of three formats: - `((before_height, after_height), (before_width, after_width))` unique pad widths for each axis. - `((before, after),)` yields same before and after pad for height and width. - `(pad,)` or int is a shortcut for before = after = pad width for all axes. mode (`PaddingMode`): The padding mode to use. Can be one of: - `"constant"`: pads with a constant value. - `"reflect"`: pads with the reflection of the vector mirrored on the first and last values of the vector along each axis. - `"replicate"`: pads with the replication of the last value on the edge of the array along each axis. - `"symmetric"`: pads with the reflection of the vector mirrored along the edge of the array. constant_values (`float` or `Iterable[float]`, *optional*): The value to use for the padding if `mode` is `"constant"`. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use same as the input image. input_data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. Returns: `np.ndarray`: The padded image. Ncst|ttfr ||f||ff}nt|trXt|dkrX|d|df|d|dff}nft|trt|dkrt|dtr||f}n8t|trt|dkrt|dtr|}ntd|tjkrd|n|d}jdkrd |n|}|S) za Convert values to be in the format expected by np.pad based on the data format. rrr!zUnsupported format: rr)r)r)r) r"r7floatrNrPr*r r'r[)valuesrr2r,r-_expand_for_data_formats"$ $z$pad.._expand_for_data_formatr)rCrrrBZedgerzInvalid padding mode: ) r rrr#padrrrr*r.)rrrCrr0r2rr,rr-rs +    r)rrcCs:ttdgt|tjjs|S|jdkr,|S|d}|S)z Converts an image to RGB format. Only converts if the image is of type PIL.Image.Image, otherwise returns the image as is. Args: image (Image): The image to convert. r?ZRGB)rconvert_to_rgbr"rErFrCconvertrr,r,r-rs   r)rr0r2rcCsx|dkrt|n|}|tjkr2|ddddf}n,|tjkrP|ddddf}ntd||dk rtt|||d}|S)a Flips the channel order of the image. If the image is in RGB format, it will be converted to BGR and vice versa. Args: image (`np.ndarray`): The image to flip. data_format (`ChannelDimension`, *optional*): The channel dimension format for the output image. Can be one of: - `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use same as the input image. input_data_format (`ChannelDimension`, *optional*): The channel dimension format for the input image. Can be one of: - `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. N.r@zUnsupported channel dimension: rY)r r r)r'r*r.)rr0r2r,r,r-flip_channel_order s  rcCs|r |S|SN)Zis_floating_pointr)xr,r,r-_cast_tensor_to_float2src@s0eZdZdZd eedddZddd d Zd S) FusedRescaleNormalizez< Rescale and normalize the input image in one step. ?F)rescale_factorinplacecCs2t|d||_t|d||_||_dS)Nr)rnZtensorr\r]r)selfr\r]rrr,r,r-__init__=szFusedRescaleNormalize.__init__r=rcCs t|}tj||j|j|jdS)N)r)rFrar\r]rrrr,r,r-__call__BszFusedRescaleNormalize.__call__N)rF)rrrrrboolrrr,r,r,r-r8src@s.eZdZdZd edddZdddd Zd S) RescalezM Rescale the input image by rescale factor: image *= rescale_factor. rrcCs ||_dSrr)rrr,r,r-rLszRescale.__init__r=rcCs||j}|Srrrr,r,r-rOs zRescale.__call__N)r)rrrrrrrr,r,r,r-rGsrc@s eZdZdZejdddZdS) NumpyToTensorz4 Convert a numpy array to a PyTorch tensor. rcCst|dddS)Nr!rr)rnZ from_numpyr( contiguousrr,r,r-rYszNumpyToTensor.__call__N)rrrrr#r$rr,r,r,r-rTsr)N)NNN)TNN)NNNTN)NN)NNN)NN)MrdmathrtypingrrrrrrGr#Z image_utilsr r r r r utilsrrrrrZutils.import_utilsrrrrrrrrErrnZ tensorflowrxZ jax.numpyZjnpZtorchvision.transforms.v2rrZtorchvision.transformsr$strr.r_rr1r4r<rrDr7rNrUrZrarcrrrvryr{rrrrrrrrrrrrrrrr,r,r,r-s $      * & : F  K @  c       W &