Fi;BddlmZddlZddlZddlZddlZddlmZddlm Z ddl Z ddl Z ddl mZmZmZddlmZmZmZmZmZmZddlmZmZddlmZdd lmZdd lm Z gd d d dhdgddddhdge jBdhdgddddhddgddddhdgddd d!hdge jBd"hdgd#d$d%d&hddd'Z"Gd(d)e jFjHZ%d0d*Z&d1d+Z'd2d,Z(Gd-d.e jFjHZ) d3 d4d/Z*y)5) annotationsN)Path)which)DetectPoseSegment)IS_DEBIAN_BOOKWORMIS_DEBIAN_TRIXIEIS_RASPBERRYPI IS_UBUNTULOGGERWINDOWS)check_apt_requirementscheck_requirements)onnx_export_patch) make_anchors) copy_attr)submul_2add_14cat_19g~8CA) layer_namesweights_memoryn_layers)rrrcat_21cat_22mul_4add_15g\EBAiip)rrrrgffffBAi i )detectposeclassifysegment)rmuladd_6cat_15gffffuCA)add_7rcat_17r&rr'cat_18gBAI)rr&r'r,g.CA)YOLO11YOLOv8c*eZdZdZdfd ZdZxZS)FXModelaA custom model class for torch.fx compatibility. This class extends `torch.nn.Module` and is designed to ensure compatibility with torch.fx for tracing and graph manipulation. It copies attributes from an existing model and explicitly sets the model attribute to ensure proper copying. Attributes: model (nn.Module): The original model's layers. imgsz (tuple[int, int]): The input image size (height, width). cjt|t|||j|_||_y)zInitialize the FXModel. Args: model (nn.Module): The original model to wrap for torch.fx compatibility. imgsz (tuple[int, int]): The input image size (height, width). Default is (640, 640). N)super__init__rmodelimgsz)selfr:r; __class__s ^/var/www/pod-logistic/pod-ai/venv/lib/python3.12/site-packages/ultralytics/utils/export/imx.pyr9zFXModel.__init__Ks, $[[  c 4g}|jD]{}|jdk7rMt|jtr||jn#|jDcgc]}|dk(r|n||c}}t|trt j t||_dttj|jDcgc] }||jjdz "c}d|jdD\|_|_t!|t"urt j t$||_t!|t(urt j t*||_||}|j-|~|Scc}wcc}w)aXForward pass through the model. This method performs the forward pass through the model, handling the dependencies between layers and saving intermediate outputs. Args: x (torch.Tensor): The input tensor to the model. Returns: (torch.Tensor): The output tensor from the model. c3@K|]}|jddyw)rN) transpose).0xs r> z"FXModel.forward..ks#(KK1%(srC)dimg?)r:f isinstanceintrtypes MethodType _inferencertorchcatr;stride unsqueezeanchorsstridestyper pose_forwardforwardrsegment_forwardappend)r<rFymjss r>rWzFXModel.forwardXsF  Assby(c2AaccFYZY\Y\8]TUa2g1Q49O8]!V$$// A> () tzz"R!1qxx'9'9"'=#="RXYZ\]\d\dfi($ 19 Aw$!,,\1= Aw'!!,,_a@ !A HHQK# $9^ #Ss F%F))r^__name__ __module__ __qualname____doc__r9rW __classcell__r=s@r>r6r6?s  r?r6c|j|j|d|jjd|jz}|j dd|dj jdddfS)z5Decode boxes and cls scores for imx object detection.boxesrrCscores) decode_bboxesdflrSrRrTrDsigmoidpermute)r<rFdboxs r>rNrNzsn   dhhqz2DLL4J4J14M NQUQ]Q] ]D >>!Q 8!4!4!6!>!>q!Q!G GGr?c |djd}t|d|j}tjt |j Dcgc]+}|j|||j||d-c}d}t|dr|j|jk7rn|jd}|j||jd|jddz|}|ddddddddf}|j||j|}tj||}|j|}g||jdddScc}w)zBForward pass for imx pose estimation, including keypoint decoding.r nk_outputrArCrhN)shapegetattrnkrOrPrangenlcv4viewhasattrrp kpt_shaperrW kpts_decoderm)r<rFbsnk_outikptspatialpred_kpts r>rVrVs/ 1AB T; 0F ))U477^T[TXXa[1&++B;TVX YCt[!dnn&?))B-hhr4>>!,dnnQ.?!.CWM!QQ,hhr477G,tQA$H (A (x1a( ((Us0Ec |j|d}|jd}tjt |j Dcgc]5}|j |||j||jd7c}d}tj||}g||jdd|Scc}w)z"Forward pass for imx segmentation.rrArhrC) protorrrOrPrurvrwrxnmrrWrD)r<rFpr|r~mcs r>rXrXs 1Q4A B U477^TKDHHQK!%**2tww;TVW XBtQA $A $r||Aq! $1 $$Us :B;cFeZdZdZ d dfd ZdZxZS) NMSWrapperz?Wrap PyTorch Module with multiclass_nms layer from edge-mdt-cl.cht|||_||_||_||_||_y)aInitialize NMSWrapper with PyTorch Module and NMS parameters. Args: model (torch.nn.Module): Model instance. score_threshold (float): Score threshold for non-maximum suppression. iou_threshold (float): Intersection over union threshold for non-maximum suppression. max_detections (int): The number of detections to return. task (str): Task type, one of 'detect', 'pose', or 'segment'. N)r8r9r:score_threshold iou_thresholdmax_detectionstask)r<r:rrrrr=s r>r9zNMSWrapper.__init__s6"  .*, r?c "ddlm}|j|}|d|d}}||||j|j|j }|j dk(ry|d}tj|d|jjdjdd|jd}|j|j|j|fS|j dk(r|d|d } } tj| d|jjdjdd| jd} |j|j|j| | fS|j|j|j|j fS) z:Forward pass with model inference and NMS post-processing.r)multiclass_nms_with_indicesrC)rgrirrrr#rhrAr%)'edgemdt_cl.pytorch.nms.nms_with_indicesrr:rrrrrOgatherindicesrRexpandsizergrilabelsn_valid) r<imagesroutputsrgri nms_outputskptsout_kptsrrout_mcs r>rWzNMSWrapper.forwardssW**V$ GAJv1 00,,..   99 1:D||D![-@-@-J-J2-N-U-UVXZ\^b^g^ghj^k-lmH$$k&8&8+:L:LhV V 99 ! GAJB\\"a)<)<)F)Fr)J)Q)QRTVXZ\ZaZabdZe)fgF$$k&8&8+:L:LfV[[ [  +"4"4k6H6H+J]J]]]r?)MbP?gffffff?i,r") r:torch.nn.ModulerfloatrrrrKrstrr_res@r>rrsOI "'"!      0^r?rc  tjddgddjj} t j d| } | rt | jdnd} | dk\sJd  t%dt%dt'|r|n|}ddl} ddl} ddlm}tjd |d| j0d|fd}|dd}| j2j5}t6d|j9vrdnd|j:}t=t?|jA|dvr tCd|dD]B}|jE| j2jFjHjK|gdD| j2jM| j2jOd !| j2jQd"|#}| j2jS|d$%}|r@| jTjW|||| jTjYd&d'd'(||)dn"| jZj]|||||*d}|j:d+k7rt_||xsd,|||j:-}ta|}|jcdd.|d/z }te5| jfji|||0ddd| jk|}|xsijmD]7\}}|jnjq}|ts|c|_:|_;9| jy||tatzj|j~}|trd1nd2z }|js td2}|s td3tjts|d4ts|d5ts|d6d7gd8t|d9z d:d;<5}|j|jjmD cgc] \}} | d  c} }dddts|S#ttjtf$rgtstr&tjd |d td gn1t st"r%tjd |d tdgYnwxYw#1swYxYwcc} }w#1swYts|SxYw)=aExport YOLO model to IMX format for deployment on Sony IMX500 devices. This function quantizes a YOLO model using Model Compression Toolkit (MCT) and exports it to IMX format compatible with Sony IMX500 edge devices. It supports both YOLOv8n and YOLO11n models for detection, segmentation, pose estimation, and classification tasks. Args: model (torch.nn.Module): The YOLO model to export. Must be YOLOv8n or YOLO11n. output_dir (Path | str): Directory to save the exported IMX model. conf (float): Confidence threshold for NMS post-processing. iou (float): IoU threshold for NMS post-processing. max_det (int): Maximum number of detections to return. metadata (dict | None, optional): Metadata to embed in the ONNX model. Defaults to None. gptq (bool, optional): Whether to use Gradient-Based Post Training Quantization. If False, uses standard Post Training Quantization. Defaults to False. dataset (optional): Representative dataset for quantization calibration. Defaults to None. prefix (str, optional): Logging prefix string. Defaults to "". Returns: (str): Path to the exported IMX model directory. Raises: ValueError: If the model is not a supported YOLOv8n or YOLO11n variant. Examples: >>> from ultralytics import YOLO >>> model = YOLO("yolo11n.pt") >>> path = torch2imx(model, "output_dir/", conf=0.25, iou=0.7, max_det=300) Notes: - Auto-installs Java>=17, model-compression-toolkit, imx500-converter, and related packages if not present - Only supports YOLOv8n and YOLO11n models (detection, segmentation, pose, and classification tasks) - Output includes quantized ONNX model, IMX binary, and labels.txt file javaz --versionT)checkcapture_outputz(?:openjdk|java) (\d+)rCrzJava version too old z! installing Java 21 for Ubuntu...zopenjdk-21-jrez2 installing Java 17 for Raspberry Pi or Debian ...zopenjdk-17-jre)z model-compression-toolkit>=2.4.1zedge-mdt-cl<1.1.0zedge-mdt-tpc>=1.2.0zpydantic<=2.11.7zimx500-converter[pt]>=3.17.3N) get_target_platform_capabilitiesz0 starting export with model_compression_toolkit z...c38K|D]}|d}|dz }|gyw)Nimggo@) dataloaderbatchrs r>representative_dataset_genz-torch2imx..representative_dataset_gens. E,C+C%K sz4.0imx500) tpc_version device_typeC2PSAr3r4rz9IMX export only supported for YOLOv8n and YOLO11n models.r ) num_of_images)concat_threshold_update)mixed_precision_configquantization_configbit_width_configr)riF)n_epochsuse_hessian_based_weightsuse_hessian_sample_attention)r:representative_data_gentarget_resource_utilization gptq_config core_configtarget_platform_capabilities) in_modulerrrrr$r)r:rrrr)parentsexist_okzmodel_imx.onnx)r:save_model_path repr_datasetzimxconv-pt.exez imxconv-ptzDimxconv-pt not found. Install with: pip install imx500-converter[pt]z-iz-oz--no-input-persistencyz--overwrite-output)rz labels.txtwzutf-8)encoding)F subprocessrunstdoutdecoderesearchrKgroupFileNotFoundErrorCalledProcessErrorAssertionErrorr r r inforr r rcallablemodel_compression_toolkitonnx edgemdt_tpcr __version__coreBitWidthConfig MCT_CONFIG__str__rlenlistmodules ValueErrorset_manual_activation_bit_widthcommonnetwork_editorsNodeNameFilter CoreConfig MixedPrecisionQuantizationConfigQuantizationConfigResourceUtilizationgptq+pytorch_gradient_post_training_quantizationget_pytorch_gptq_configptq"pytorch_post_training_quantizationrrmkdirrexporterpytorch_export_modelloaditemsmetadata_propsaddrkeyvaluesavesys executableparentrexistsropen writelinesnames)!r: output_dirconfioumax_detmetadatardatasetprefix java_output version_match java_versionmctrrrtpcbit_cfg mct_config layer_nameconfigresource_utilization quant_model onnx_model model_onnxkvmetabin_dirimxconv labels_file_names! r> torch2imxrs\ 7 nnfk%:$W[\ccjjl  ";[I 6Cs=..q12 r!9#99! 56#G,gi'G+< KK"VHLS__L]]`ab.5 +u( SChh%%'G5==?(BHQRWR\R\]J 4  !J)??TUU /r //1P1P1_1_`j1k0lnpqrXX "xxHHWYHZHH77PT7U !F 8877zRbGc7d  <<$>(<88]b9), =   WW 7 7$>(<), 8  * zzZ  ME"  j!JTD1..J    ))zHb *  :&JR&&()1((,,. #a&$*) IIj*%3>>"))GW),GG >>  %  fggNN WtS_dC OE]_st j<'w ?Q;5;;;L;L;NO44& OPQ z?g z<r.s#   88nnO7.3 @*c  ](c  %'"&&seT?'c  (='c  Y(c  %'"&&rdS<'c  '% P8ehhoo8vH )"%1^1^t! f ff f  f  f  f f f fr?