Fiv"ddlmZddlZddlmZddlmZddlmZddl m Z ddl m Z ddl mZd d lmZGd d eZy) ) annotationsN)Image)SegmentationPredictor) DEFAULT_CFG)box_iou) scale_masks) TORCH_1_10)adjust_bboxes_to_image_bordercJeZdZdZeddfdfd ZfdZd dZdZdZ xZ S) FastSAMPredictoraFastSAMPredictor is specialized for fast SAM (Segment Anything Model) segmentation prediction tasks. This class extends the SegmentationPredictor, customizing the prediction pipeline specifically for fast SAM. It adjusts post-processing steps to incorporate mask prediction and non-maximum suppression while optimizing for single-class segmentation. Attributes: prompts (dict): Dictionary containing prompt information for segmentation (bboxes, points, labels, texts). device (torch.device): Device on which model and tensors are processed. clip (Any, optional): CLIP model used for text-based prompting, loaded on demand. Methods: postprocess: Apply postprocessing to FastSAM predictions and handle prompts. prompt: Perform image segmentation inference based on various prompt types. set_prompts: Set prompts to be used during inference. Nc6t||||i|_y)a;Initialize the FastSAMPredictor with configuration and callbacks. This initializes a predictor specialized for Fast SAM (Segment Anything Model) segmentation tasks. The predictor extends SegmentationPredictor with custom post-processing for mask prediction and non-maximum suppression optimized for single-class segmentation. Args: cfg (dict): Configuration for the predictor. overrides (dict, optional): Configuration overrides. _callbacks (dict, optional): Dictionary of callback functions. N)super__init__prompts)selfcfg overrides _callbacks __class__s d/var/www/pod-logistic/pod-ai/venv/lib/python3.12/site-packages/ultralytics/models/fastsam/predict.pyrzFastSAMPredictor.__init__#s i4 c|jjdd}|jjdd}|jjdd}|jjdd}t | |||}|D]} t j dd| j d| j dg| jjjtj} t| jj| j } t jt| d| d kDj} | j!dk7s| | jj| <|j#||||| S) aApply postprocessing to FastSAM predictions and handle prompts. Args: preds (list[torch.Tensor]): Raw predictions from the model. img (torch.Tensor): Input image tensor that was fed to the model. orig_imgs (list[np.ndarray]): Original images before preprocessing. Returns: (list[Results]): Processed results with prompts applied. bboxesNpointslabelstextsrr )devicedtypeg?)rrrr)rpopr postprocesstorchtensor orig_shapeboxesdatarfloat32r xyxynonzerorflattennumelprompt)rpredsimg orig_imgsrrrrresultsresultfull_boxr%idxrs rr!zFastSAMPredictor.postprocess2sM!!(D1!!(D1!!(D1   $/'%eS)< 2F||Av((+V->->q-AB6< DEMMOCyy{a)1 !!#& 2{{76&W\{]]rc  ||||Sg}t|ts|g}|D])}t|dk(r|j|$|jj }|j dd|jk7r;t|dj|jddkDj}tjt|tj|j} |tj|tj |j}|j"dk(r|dn|}|dddf|dddfz |dddf|dddfz z} tj$|D cgc],} |dd| d| d| d| dfj'd .c} } tj&|d } | dddf| z| z }d | tj(| |z d <|tj|tj |j}|j"dk(r|dn|}|"tj*|j d}tj|tj |j}t|t|k(s Jd t|d t|d |j'dk(r9tj*t|tj|jn8tjt|tj|j}t-||D]9\}}t||tj.|dd|d|dfd d<;| |z} |t|t0r|g}gg}}t3|j4j6j9D]\}} d| D\}}}}t:r||j'n!||j'dj'dkr|j|d|j<||||f||||||dfj?jAz}|jtCjD|dddddddf|jG||}tj(|d }t|rStIt|Dcgc] }||vs| }}tjJ|tM||j}d | |<|j|| ,|Scc} wcc}w)a Perform image segmentation inference based on cues like bounding boxes, points, and text prompts. Args: results (Results | list[Results]): Original inference results from FastSAM models without any prompts. bboxes (np.ndarray | list, optional): Bounding boxes with shape (N, 4), in XYXY format. points (np.ndarray | list, optional): Points indicating object locations with shape (N, 2), in pixels. labels (np.ndarray | list, optional): Labels for point prompts, shape (N, ). 1 = foreground, 0 = background. texts (str | list[str], optional): Textual prompts, a list containing string objects. Returns: (list[Results]): Output results filtered and determined by the provided prompts. Nrr g?)rr)r r6)dimTz?Expected `labels` to have the same length as `points`, but got z and .)as_tuplec32K|]}t|yw)N)int).0xs r z*FastSAMPredictor.prompt..s%8c!f%8sdr)' isinstancelistlenappendmasksr&shaper$rfloatbyter"zerosboolr as_tensorint32ndimstacksumargmaxoneszipr)str enumerater%r(tolistr orig_imgcpunumpyr fromarray_clip_inferenceranger#r;)rr0rrrrprompt_resultsr1rFr3 bbox_areasb mask_areasfull_mask_areasunion point_idxpointlabelcrop_ims filter_idxix1y1x2y2crop similaritytext_idxori_idxss rr,zFastSAMPredictor.promptMs >fnN'4(iG6 /F6{a%%f-LL%%E{{12&"3"33$U4[%6%6%8&:K:KLQORUU[[]++c&kDKKPC!u{{4;;W)/)9v$QTlVAqD\9fQTlVTUWXTX\>YZ "[[fl)mab%1Q4!A$;!qt 0K*L*P*PU[*P*\)mn "'))Ev">"1d7+o= J?CELLe!3;<!u{{4;;W)/)9v>"ZZ Q8Fu{{4;;W6{c&k1UVYZ`VaUbbghklrhsgttuv1 zz|q(JJs6{%**T[[QS[ 4;;W %($7kLE5_cdi_jIemmE!U1XuQx2G,HSWXYZ[\ky  eS)"GE')2*%fll&7&7&>&>&@AGDAq%8a%8NBB*4a %(,,q/:M:M:OTWW"))!, !??2b5"R%<85BrE2b5RVAV;W;[;[;];c;c;eeDOOEOODAtt4D$EF G"11(EB  << ;z?+0V+=Ua*ATUHU$||HS],CDKKXH $H  ! !&+ .m6 /pW*nJ Vs,1U 4 U>Ucddlm}t|ds|d|j|_t j |Dcgc]6}|jj|j|j8c}}|jj|}|jj|jj|}||jzScc}w)aPerform CLIP inference to calculate similarity between images and text prompts. Args: images (list[PIL.Image]): List of source images, each should be PIL.Image with RGB channel order. texts (list[str]): List of prompt texts, each should be a string object. Returns: (torch.Tensor): Similarity matrix between given images and texts with shape (M, N). r)CLIPclipzViT-B/32rA) ultralytics.nn.text_modelrrhasattrrrsr"rOimage_preprocessto encode_image encode_texttokenizeT)rimagesrrrimageimage_features text_featuress rr[z FastSAMPredictor._clip_inferences 3tV$Z rs1# A)--40],]r