U di@sddlZddlmZmZmZmZddlmZmZm Z m Z m Z ddl m Z mZe r^ddlmZer|ddlZddlmZmZe eZeeefZeeZeed d Gd d d e ZdS) N)AnyDictListUnion)add_end_docstringsis_torch_availableis_vision_availableloggingrequires_backends)Pipelinebuild_pipeline_init_args) load_image)(MODEL_FOR_OBJECT_DETECTION_MAPPING_NAMES,MODEL_FOR_TOKEN_CLASSIFICATION_MAPPING_NAMEST)Zhas_image_processorcszeZdZdZfddZddZeeee fdfdd Z dd d Z d d Z dddZ deeefdddZZS)ObjectDetectionPipelinea Object detection pipeline using any `AutoModelForObjectDetection`. This pipeline predicts bounding boxes of objects and their classes. Example: ```python >>> from transformers import pipeline >>> detector = pipeline(model="facebook/detr-resnet-50") >>> detector("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png") [{'score': 0.997, 'label': 'bird', 'box': {'xmin': 69, 'ymin': 171, 'xmax': 396, 'ymax': 507}}, {'score': 0.999, 'label': 'bird', 'box': {'xmin': 398, 'ymin': 105, 'xmax': 767, 'ymax': 507}}] >>> # x, y are expressed relative to the top left hand corner. ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This object detection pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"object-detection"`. See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=object-detection). csTtj|||jdkr*td|jdt|dt}|t | |dS)NtfzThe z is only available in PyTorch.Zvision) super__init__ framework ValueError __class__r rcopyupdaterZcheck_model_type)selfargskwargsmappingrK/tmp/pip-unpacked-wheel-bm_b0l5e/transformers/pipelines/object_detection.pyr5s   z ObjectDetectionPipeline.__init__cKsFi}d|kr$tdt|d|d<i}d|kr<|d|d<|i|fS)NtimeoutzUThe `timeout` argument is deprecated and will be removed in version 5 of Transformers threshold)warningswarn FutureWarning)rrZpreprocess_paramsZpostprocess_kwargsr r r!_sanitize_parameters@s  z,ObjectDetectionPipeline._sanitize_parameters)returncs,d|krd|kr|d|d<tj||S)a Detect objects (bounding boxes & classes) in the image(s) passed as inputs. Args: inputs (`str`, `List[str]`, `PIL.Image` or `List[PIL.Image]`): The pipeline handles three types of images: - A string containing an HTTP(S) link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly The pipeline accepts either a single image or a batch of images. Images in a batch must all be in the same format: all as HTTP(S) links, all as local paths, or all as PIL images. threshold (`float`, *optional*, defaults to 0.5): The probability necessary to make a prediction. Return: A list of dictionaries or a list of list of dictionaries containing the result. If the input is a single image, will return a list of dictionaries, if the input is a list of several images, will return a list of list of dictionaries corresponding to each image. The dictionaries contain the following keys: - **label** (`str`) -- The class label identified by the model. - **score** (`float`) -- The score attributed by the model for that label. - **box** (`List[Dict[str, int]]`) -- The bounding box of detected object in image's original size. imagesinputs)popr__call__)rrrrr r!r,Lsz ObjectDetectionPipeline.__call__NcCstt||d}t|j|jgg}|j|gdd}|jdkrF||j}|j dk rh|j |d|ddd}||d<|S)N)r"pt)r)return_tensorswordsboxes)textr0r. target_size) rtorchZ IntTensorheightwidthimage_processorrtoZ torch_dtype tokenizer)rimager"r2r*r r r! preprocessms    z"ObjectDetectionPipeline.preprocesscCsB|d}|jf|}|d|i|}|jdk r>|d|d<|S)Nr2bbox)r+modelrr8)rZ model_inputsr2outputs model_outputsr r r!_forwardxs     z ObjectDetectionPipeline._forward?c sN|d}jdk r|d\fdd|ddjddjdd\}}fdd |D}fd d |d dD}d d dgfdd t|||D}nj||} | d} | d}| d}| d}|| d<fdd |D| d<fdd |D| d<d d dgfdd t| d| d| dD}|S)Nr2rc sHt|dd|dd|dd|ddgS)Nrir r)_get_bounding_boxr3ZTensor)r;)r4rr5r r! unnormalizesz8ObjectDetectionPipeline.postprocess..unnormalizeZlogits)Zdimcsg|]}jjj|qSr )r<configid2label).0Z predictionrr r! sz7ObjectDetectionPipeline.postprocess..csg|] }|qSr r )rGr;)rCr r!rIsr;Zscorelabelboxcs&g|]}|dkrtt|qS)rdictziprGvals)keysr#r r!rIs scoreslabelsr0csg|]}jjj|qSr )r<rErFitem)rGrJrHr r!rIscsg|]}|qSr )rB)rGrKrHr r!rIscsg|]}tt|qSr rLrO)rQr r!rIs)r8tolistZsqueezeZsoftmaxmaxrNr6Zpost_process_object_detection) rr>r#r2rRclassesrSr0 annotationZraw_annotationsZraw_annotationr )r4rQrr#rCr5r! postprocesss,  " "   z#ObjectDetectionPipeline.postprocessz torch.Tensor)rKr(cCs8|jdkrtd|\}}}}||||d}|S)a% Turns list [xmin, xmax, ymin, ymax] into dict { "xmin": xmin, ... } Args: box (`torch.Tensor`): Tensor containing the coordinates in corners format. Returns: bbox (`Dict[str, int]`): Dict containing the coordinates in corners format. r-z9The ObjectDetectionPipeline is only available in PyTorch.)xminyminxmaxymax)rrintrU)rrKrZr[r\r]r;r r r!rBs z)ObjectDetectionPipeline._get_bounding_box)N)r@)__name__ __module__ __qualname____doc__rr'r Predictionsr Predictionr,r:r?rYrstrr^rB __classcell__r r rr!rs  !  -r)r$typingrrrrutilsrrr r r baser rZ image_utilsrr3Zmodels.auto.modeling_autorrZ get_loggerr_loggerrerdrcrr r r r!s