Fi%ddlmZddlZddlmZddlmZddlZddl m Z ddl m Z ddl mZmZddlmZdd lmZmZd ej*d <Gd d ZGddZy)) annotationsN)Path)Any)Image) IMG_FORMATS)LOGGER TORCH_VERSION)check_requirements) TORCH_2_4 select_deviceTRUEKMP_DUPLICATE_LIB_OKcBeZdZdZd dZd dZd dZd dZd ddZddZ y)VisualAISearchaA semantic image search system that leverages OpenCLIP for generating high-quality image and text embeddings and FAISS for fast similarity-based retrieval. This class aligns image and text embeddings in a shared semantic space, enabling users to search large collections of images using natural language queries with high accuracy and speed. Attributes: data (str): Directory containing images. device (str): Computation device, e.g., 'cpu' or 'cuda'. faiss_index (str): Path to the FAISS index file. data_path_npy (str): Path to the numpy file storing image paths. data_dir (Path): Path object for the data directory. model: Loaded CLIP model. index: FAISS index for similarity search. image_paths (list[str]): List of image file paths. Methods: extract_image_feature: Extract CLIP embedding from an image. extract_text_feature: Extract CLIP embedding from text. load_or_build_index: Load existing FAISS index or build new one. search: Perform semantic search for similar images. Examples: Initialize and search for images >>> searcher = VisualAISearch(data="path/to/images", device="cuda") >>> results = searcher.search("a cat sitting on a chair", k=10) c LtsJdtdddlm}t dt d|_d|_d|_t|jd d |_ t|jd d |_ |jjsOdd lm}t#j$|jd|dddlm}||dddtd |_ |d|j|_d|_g|_|j1y)zDInitialize the VisualAISearch class with FAISS index and CLIP model.z1VisualAISearch requires torch>=2.4 (found torch==)r)build_text_modelz faiss-cpufaissz faiss.indexz paths.npydataimagesdevicecpu) ASSETS_URLz( not found. Downloading images.zip from z /images.zip) safe_downloadT)urlunzipretryz clip:ViT-B/32)rN)r r ultralytics.nn.text_modelrr __import__r faiss_index data_path_npyrgetdata_dirr rexistsultralytics.utilsrrwarningultralytics.utils.downloadsrmodelindex image_pathsload_or_build_index)selfkwargsrrrs i/var/www/pod-logistic/pod-ai/venv/lib/python3.12/site-packages/ultralytics/solutions/similarity_search.py__init__zVisualAISearch.__init__1s^Mm_\]^^y>;'( ((VZZ9: #FJJx$?@ }}##% 4 NNdmm_,TU_T``kl m A  K8A N NDM%odkkJ     "c|jjtj|j j j S)z7Extract CLIP image embedding from the given image path.)r) encode_imageropendetachrnumpy)r-paths r/extract_image_featurez$VisualAISearch.extract_image_featureNs;zz&&uzz$'78??AEEGMMOOr1c|jj|jj|gjj j S)z6Extract CLIP text embedding from the given text query.)r) encode_texttokenizer5rr6)r-texts r/extract_text_featurez#VisualAISearch.extract_text_featureRsCzz%%djj&9&94&&ABIIKOOQWWYYr1ct|jjrt|jjrdt j d|j j|j|_tj|j|_ yt j dg}|jjD]x}|jjj!dt"vr3 |j%|j'||jj%|j(z|s t/dtj0|j3d}|j j5||j j7|j8d |_|jj;||j j=|j|jtj>|jtj@|jt j d tC|jd y#t*$r0}t j,d|j(d|Yd}~d}~wwxYw) aeLoad existing FAISS index or build a new one from image features. Checks if FAISS index and image paths exist on disk. If found, loads them directly. Otherwise, builds a new index by extracting features from all images in the data directory, normalizes the features, and saves both the index and image paths for future use. zLoading existing FAISS index...Nz#Building FAISS index from images....z Skipping z: z'No image embeddings could be generated.float32zIndexed z images.)"rr!r%r"rinfor read_indexr*nploadr+r$iterdirsuffixlowerlstriprappendr8name Exceptionr' RuntimeErrorvstackastype normalize_L2 IndexFlatIPshapeadd write_indexsavearraylen)r-vectorsfilees r/r,z"VisualAISearch.load_or_build_indexVs   ! ( ( *tD4F4F/G/N/N/P KK9 :..t/?/?@DJ!wwt'9'9:D    9:MM))+ =D{{  "))#.kA =t99$?@  '' 2 =HI I))G$++I6 (ZZ++GMM!,<=  w tzz4+;+;< ""BHHT-=-=$>? hs4#3#345X>? =499+Rs;<< =sAJ J>%J99J>c $|j|jd}|jj||jj ||\}}t |dDcgc]1\}}|d||k\s|j|t|d|f3} }}| jddtjd| D]!\} } tjd| d| d #| D cgc]} | d c} Scc}}wcc} w) alReturn top-k semantically similar images to the given query. Args: query (str): Natural language text query to search for. k (int, optional): Maximum number of results to return. similarity_thresh (float, optional): Minimum similarity threshold for filtering results. Returns: (list[str]): List of image filenames ranked by similarity score. Examples: Search for images matching a query >>> searcher = VisualAISearch(data="images") >>> results = searcher.search("red car", k=5, similarity_thresh=0.2) r@rc |dS)NrA)xs r/z'VisualAISearch.search..s 1Q4r1T)keyreversez Ranked Results:z - z | Similarity: z.4f) r=rOrrPr*search enumerater+floatsortrrB) r-queryksimilarity_thresh text_featDr*idxiresultsrKscorers r/rbzVisualAISearch.searchs --e4;;IF   *::$$Y25BKERSHBU 8>QYZ[\Y]^aYbfwYwT  a %!S "2 3     6 '(" AKD% KK$tfOE#;? @ A&&!&& 's,D"D8 D c$|j|S)z.Direct call interface for the search function.)rb)r-rfs r/__call__zVisualAISearch.__call__s{{5!!r1N)r.rreturnNone)r7rrr np.ndarray)r<strrrrt)rrrs)g?)rfrurgintrhrdrr list[str])rfrurrrx) __name__ __module__ __qualname____doc__r0r8r=r,rbrqr]r1r/rrs)8#:PZ*@X'>"r1rc,eZdZdZdddZddZd d dZy) SearchAppaA Flask-based web interface for semantic image search with natural language queries. This class provides a clean, responsive frontend that enables users to input natural language queries and instantly view the most relevant images retrieved from the indexed database. Attributes: render_template: Flask template rendering function. request: Flask request object. searcher (VisualAISearch): Instance of the VisualAISearch class. app (Flask): Flask application instance. Methods: index: Process user queries and display search results. run: Start the Flask web application. Examples: Start a search application >>> app = SearchApp(data="path/to/images", device="cuda") >>> app.run(debug=True) Nctdddlm}m}m}||_||_t |||_|tdt|jd|_ |jjd|jd d g y ) zInitialize the SearchApp with VisualAISearch backend. Args: data (str, optional): Path to directory containing images to index and search. device (str, optional): Device to run inference on (e.g. 'cpu', 'cuda'). z flask>=3.0.1r)Flaskrender_templaterequest)rr templatesz/images)template_folder static_folderstatic_url_path/GETPOST) view_funcmethodsN) r flaskrrrrsearcherryrresolveapp add_url_ruler*)r-rrrrrs r/r0zSearchApp.__init__sw >*99. &D@  't*,,.%   cTZZ%Qr1cg}|jjdk(rE|jjjddj }|j |}|j d|S)zCProcess user query and display search results in the web interface.rrfzsimilarity-search.html)rm)rmethodformr#striprr)r-rmrfs r/r*zSearchApp.indexsc <<  & (LL%%))'26<<>EmmE*G##$rsK# .37B%+ !"O"O"d5"5"r1