Fi~ ddZddlmZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl ZddlZddlmZmZddlmZmZddlmZdd lmZdd lmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&dd l'm(Z(m)Z)m*Z*m+Z+m,Z,dd l-m.Z.dd l/m0Z0ddl1m2Z2m3Z3e%dz ddddddddf dZ4GddZ5GddZ6y)az Benchmark YOLO model formats for speed and accuracy. Usage: from ultralytics.utils.benchmarks import ProfileModels, benchmark ProfileModels(['yolo26n.yaml', 'yolov8s.yaml']).run() benchmark(model='yolo26n.pt', imgsz=160) Format | `format=argument` | Model --- | --- | --- PyTorch | - | yolo26n.pt TorchScript | `torchscript` | yolo26n.torchscript ONNX | `onnx` | yolo26n.onnx OpenVINO | `openvino` | yolo26n_openvino_model/ TensorRT | `engine` | yolo26n.engine CoreML | `coreml` | yolo26n.mlpackage TensorFlow SavedModel | `saved_model` | yolo26n_saved_model/ TensorFlow GraphDef | `pb` | yolo26n.pb TensorFlow Lite | `tflite` | yolo26n.tflite TensorFlow Edge TPU | `edgetpu` | yolo26n_edgetpu.tflite TensorFlow.js | `tfjs` | yolo26n_web_model/ PaddlePaddle | `paddle` | yolo26n_paddle_model/ MNN | `mnn` | yolo26n.mnn NCNN | `ncnn` | yolo26n_ncnn_model/ IMX | `imx` | yolo26n_imx_model/ RKNN | `rknn` | yolo26n_rknn_model/ ExecuTorch | `executorch` | yolo26n_executorch_model/ Axelera AI | `axelera` | yolo26n_axelera_model/ ) annotationsN)deepcopy)Path)YOLO YOLOWorld) TASK2DATA TASK2METRIC)export_formats) Segment26) ARM64ASSETS ASSETS_URL IS_DOCKER IS_JETSONLINUXLOGGERMACOSTQDM WEIGHTS_DIRYAML)IS_PYTHON_3_13 check_imgszcheck_requirements check_yolo is_rockchip) safe_download) file_size) get_cpu_info select_devicez yolo26n.ptFcpuMbP?c $t|}t|tr|d|dk(sJd ddl} | jj d| jj d| jjd| jjd| jjd| jjdt|d }t|ttfr t|}|xst|j }t"|j } g} t%j$} |j'}|r(t)t+d }||vsJd |d |d t-t+j/D]\}}}}}}d\}} |r||k7r|dk(r|j dk7sUJd|dk(rt0rt2r=Jd|dk(rt0r+t2r%Jd|dk(rt4st0rt2rJd|dk(r t6rJd|dvrt|t8rJd|dk(rFt|t8rJd|j dk7sJdt0rt:r t4sJd|dk(rt|t8rJd |d!k(rt|t8rJd"|d#k(rEt|t8rJd$|j d%vsJd&d'|j=vsJd(|d)k(r5t|t8rJd*t0sJd+t?rJd,|d-k(rt|t8rJd.|d/k(rpt|t8rJd0t0r t2r t@rJd1|j d2k(r1tCd3|jDjGDrJd4d5|jHvr |sJd6d7|jHvr |sJd8|d9k(r4|jJxs|jLxs |jN}tQ|}nNtQ|jRdi||||||dd:| }t||j ;}|t|vsJd<d=}|j d>k7s |dk7sJd?|d@vsJdA|dk7stUjVdBk(sJdC|d/k7sJdD|jYtZdEz |||dF|j]|d|d|||ddGH }|j^| |j`dI}}tcdJ||zz dK}| je|dLtctg|dtc|dMtc|dK|gtq|P| js| dQdRdS| dTdUgdVW}|judXdY}|jw| jyj{| j|jd9}|jN}t%j$| z }dZ} d[|d\|d]|d^|d_d`| da|da }!tmj|!tdbdcdddef5}"|"j|!ddd|rCt|tr3|| j}#|$ty$fdg|#Ds Jdh$|S#th$rp}|rtI|tjus JdN|dO|tmjndN|dO|| je||tctg|ddddgYd}~d}~wwxYw#1swYxYw)jaBenchmark a YOLO model across different formats for speed and accuracy. Args: model (str | Path): Path to the model file or directory. data (str | None): Dataset to evaluate on, inherited from TASK2DATA if not passed. imgsz (int): Image size for the benchmark. half (bool): Use half-precision for the model if True. int8 (bool): Use int8-precision for the model if True. device (str): Device to run the benchmark on, either 'cpu' or 'cuda'. verbose (bool | float): If True or a float, assert benchmarks pass with given metric. eps (float): Epsilon value for divide by zero prevention. format (str): Export format for benchmarking. If not supplied all formats are benchmarked. **kwargs (Any): Additional keyword arguments for exporter. Returns: (polars.DataFrame): A Polars DataFrame with benchmark results for each format, including file size, metric, and inference time. Examples: Benchmark a YOLO model with default settings: >>> from ultralytics.utils.benchmarks import benchmark >>> benchmark(model="yolo26n.pt", imgsz=640) rTz'benchmark() only supports square imgsz.NASCII_BORDERS_ONLY_CONDENSEDF)verboseArgumentzExpected format to be one of z , but got 'z'.)❌Npbobbz.TensorFlow GraphDef not supported for OBB taskedgetpuz3Edge TPU export only supported on non-aarch64 Linuxtfjsz)TF.js export not supported on ARM64 Linuxcoremlz;CoreML export only supported on macOS and non-aarch64 Linuxz#CoreML not supported on Python 3.13>r+r.tfliter- saved_modelz;YOLOWorldv2 TensorFlow exports not supported by onnx2tf yetpaddlez,YOLOWorldv2 Paddle exports not supported yetzBPaddle OBB bug https://github.com/PaddlePaddle/Paddle/issues/72024z3Windows and Jetson Paddle exports not supported yetmnnz)YOLOWorldv2 MNN exports not supported yetncnnz*YOLOWorldv2 NCNN exports not supported yetimxz%YOLOWorldv2 IMX exports not supported>posedetectsegmentclassifyzbIMX export is only supported for detection, classification, pose estimation and segmentation tasksC2fz*IMX only supported for YOLOv8n and YOLO11nrknnz*YOLOWorldv2 RKNN exports not supported yetzRKNN only supported on Linuxz1RKNN Inference only supported on Rockchip devices executorchz0YOLOWorldv2 ExecuTorch exports not supported yetaxeleraz)YOLOWorldv2 Axelera exports not supportedzGexport is only supported on Linux and is not supported on ARM64 Docker.r8c3<K|]}t|tywN) isinstancer ).0ms ^/var/www/pod-logistic/pod-ai/venv/lib/python3.12/site-packages/ultralytics/utils/benchmarks.py zbenchmark..s;tYZJq)r.r-zinference not supportedDarwinz(inference only supported on macOS>=10.13z,inference only supported on Axelera hardwarezbus.jpg)rGrLrIr(r") rKbatchrGplotsrLrIrJr(conf inferenceu✅zBenchmark failure for : )rLFormatu Status❔z Size (MB)zInference time (ms/im)FPSrow)schemaorient )offsetudBenchmarks legend: - ✅ Success - ❎ Export passed but validation failed - ❌️ Export failedz Benchmarks complete for z on z at imgsz=z (z.2fzs)  zbenchmarks.logaignoreutf-8)errorsencodingc3TK|]}tj|r|kD!ywr?)npisnan)rAxfloors rCrDzbenchmark..sARXXa[1u9As( (z%Benchmark failure: metric(s) < floor )Err@listpolarsConfig set_tbl_cols set_tbl_rowsset_tbl_width_charsset_tbl_hide_column_data_typesset_tbl_hide_dataframe_shapeset_tbl_formattingrstrrrrrMr timelower frozensetr zipvaluesrr rrrr__str__rranymodelmodulestypept_path ckpt_path model_namerexportplatformsystempredictr val results_dictspeedroundappendr ExceptionAssertionErrorrerrorr DataFramewith_row_index with_columnsallcastString fill_nullinfoopenwritefloatto_numpy)%r{rKrGrIrJrLr(epsrHkwargsplkeyyt0 format_argformatsnamesuffixr!gpu_emojifilenameexported_modelresultsmetricrfpsedf df_displaydtlegendsfmetricsrhs% @rC benchmarkrEsF  E#-eT#:58uQx oFoo DII2II2II!!"%II,,T2II**40II  !?@ 65 1F%#t%U   (9UZZ(D ejj !C A BJN,Z89 d$A'+V`Uaac"dd -0.2B2I2I2K-L[U)ffc3%xY UjF2~zzU*\,\\*9$Ua,aa*6!!eY.YY,8#ut7tt5!)P+PP)KK%eY7v9vv7!%eY7g9gg7zzU*p,pp*iEp;pp9%eY7d9dd7%eY7e9ee7%eY7`9``7zz%NNxN /]1]]/%eY7e9ee7<< e;e e9!44 O6O O4X%):h)F rHr rFY& V(V V&  " "6I#5U6X\fk " l%(() G$005w}}[7QEF ,a0C HHdE58) HHdE58)>> benchmark = RF100Benchmark() >>> benchmark.set_key("your_roboflow_api_key") roboflowr)Roboflow)api_keyN)rrrr)rrrs rCset_keyzRF100Benchmark.set_key s :&%7+rctjjdr*tjdtj dfntj dtj dtj dttdt|d5}|D]} tjd|j\}}}}}|jj||d|} t| jsI|j j#|j%|j'|j)dnt+j,d |j.jtj0| z d z  d d d |j|j.fS#t2$rY)wxYw#1swY1xYw) aParse dataset links and download datasets. Args: ds_link_txt (str): Path to the file containing dataset links. Returns: (tuple[list[str], list[Path]]): List of dataset names and list of paths to dataset configuration files. Examples: >>> benchmark = RF100Benchmark() >>> benchmark.set_key("api_key") >>> benchmark.parse_dataset("datasets_links.txt") zrf-100zultralytics-benchmarksz/datasets_links.txtrarcz/+rFyolov8zDataset already downloaded.z data.yamlN)ospathexistsshutilrmtreemkdirchdirrrrresplitstriprrrr workspaceprojectversiondownloadrrrcwdr) r ds_link_txtfileliner_urlrrr proj_versions rC parse_datasetzRF100Benchmark.parse_datasets:<9Qx "((8"45WYW_W_`hWi  )* $789 + 0 D  ;=88D$**,;W8AtYMM((1&-Yay#9L -446)))4<>> benchmark = RF100Benchmark() >>> benchmark.evaluate("path/to/data.yaml", "path/to/val_log.txt", "path/to/eval_log.txt", 0) )u🚀u⚠️u💡r*namesrarc3&K|]}|v ywr?ri)rAsymbolrs rCrDz*RF100Benchmark.evaluate.._sA&v~Asr\c |dk7S)Nr#ri)rs rCz)RF100Benchmark.evaluate..bs #)rr^c 3~K|]4}|vs |dk(r)dvr%dvr!ddddddd d 6y w) rz(AP)z(AR)rr%rTrUrNri)rAr class_namesentriess rCrDz*RF100Benchmark.evaluate..dsl "K'AJ6;PU[cjUj")")!*#*1:%,QZ")!*!(!( "s:=Nr%zMultiple dicts foundrrrzSingle dict foundc3&K|] }|d yw)rNri)rAress rCrDz*RF100Benchmark.evaluate..ys>C3w<>sr_rV)rrr readlinesrzrrjfilterrextendlenrrnextrrr)r yaml_path val_log_file eval_log_filelist_ind skip_symbolsrlines eval_linesrmap_vallstrrrs @@@rCevaluatezRF100Benchmark.evaluateHs"9 ii *73 , 1 QKKMEJ ALAA**S/v&;WEF29:Q1774=:!! "% "    , z?Q  KK. /! +w<5(!'lG + KK+ ,>:>>G -w 7 ?1 GGt}}X./r'"= > ?W~7;  @ ?W~s*AE3E. E36%E?.E33E<?FN)rrs)zdatasets_links.txt)rrs)rr)rrsrrsrrsrint) __name__ __module__ __qualname____doc__rrr staticmethodrrrirrCrrs1"c, !/F##6rrceZdZdZ d ddZdZdZeddZedddZ dddZ ed Z ddd Z dd Z e dd Zedd Zy) ProfileModelsa*ProfileModels class for profiling different models on ONNX and TensorRT. This class profiles the performance of different models, returning results such as model speed and FLOPs. Attributes: paths (list[str]): Paths of the models to profile. num_timed_runs (int): Number of timed runs for the profiling. num_warmup_runs (int): Number of warmup runs before profiling. min_time (float): Minimum number of seconds to profile for. imgsz (int): Image size used in the models. half (bool): Flag to indicate whether to use FP16 half-precision for TensorRT profiling. trt (bool): Flag to indicate whether to profile using TensorRT. device (torch.device): Device used for profiling. Methods: run: Profile YOLO models for speed and accuracy across various formats. get_files: Get all relevant model files. get_onnx_model_info: Extract metadata from an ONNX model. iterative_sigma_clipping: Apply sigma clipping to remove outliers. profile_tensorrt_model: Profile a TensorRT model. profile_onnx_model: Profile an ONNX model. generate_table_row: Generate a table row with model metrics. generate_results_dict: Generate a dictionary of profiling results. print_table: Print a formatted table of results. Examples: Profile models and print results >>> from ultralytics.utils.benchmarks import ProfileModels >>> profiler = ProfileModels(["yolo26n.yaml", "yolov8s.yaml"], imgsz=640) >>> profiler.run() Nc ||_||_||_||_||_||_||_t|tjr||_ yt||_ y)a[Initialize the ProfileModels class for profiling models. Args: paths (list[str]): List of paths of the models to be profiled. num_timed_runs (int): Number of timed runs for the profiling. num_warmup_runs (int): Number of warmup runs before the actual profiling starts. min_time (float): Minimum time in seconds for profiling a model. imgsz (int): Size of the image used during profiling. half (bool): Flag to indicate whether to use FP16 half-precision for TensorRT profiling. trt (bool): Flag to indicate whether to profile using TensorRT. device (torch.device | str | None): Device used for profiling. If None, it is determined automatically. Notes: FP16 'half' argument option removed for ONNX as slower on CPU than FP32. N) pathsnum_timed_runsnum_warmup_runsmin_timerGrItrtr@torchrLr) rrrrrrGrIrrLs rCrzProfileModels.__init__sW4 ,.     *65<< @f mTZF[ rc |j}|stjdgSg}g}|D]}|jd}|jdvrt t |}|j|j|j}|jr]|jjdk7rD|js4|jd|j|j|jd}|jd |j|jd }n%|jd k(r|j!|}|}n|j#t |} |j%t |} |j'|j)|j*| | ||j'|j-|j*| | ||j/||S) aProfile YOLO models for speed and accuracy across various formats including ONNX and TensorRT. Returns: (list[dict]): List of dictionaries containing profiling results for each model. Examples: Profile models and print results >>> from ultralytics.utils.benchmarks import ProfileModels >>> profiler = ProfileModels(["yolo26n.yaml", "yolo11s.yaml"]) >>> results = profiler.run() z'No matching *.pt or *.onnx files found.z.engine.pt.yml.yamlrGr!engineF)rHrIrGrLr(onnx)rHrGrLr(z.onnx) get_filesrwarning with_suffixrrrsfuserrGrrLr}is_filerrIget_onnx_model_infoprofile_tensorrt_modelprofile_onnx_modelrgenerate_table_rowstemgenerate_results_dict print_table) rfiles table_rowsoutputr engine_filer{ model_info onnx_filet_enginet_onnxs rCrunzProfileModels.runs  NND EI  _D**95K{{66SY "ZZdjjZ9 88 0 0E 9+BUBUBW"',,'!YY"jj#{{ % #/#K"LL!**;;! ) '!55d;  223{3CDH,,S^ $ rc >g}|jD]}t|}|jrLgd}|j|Dcgc]*}t jt ||z D]}|,c}}j|j dvr|jt ||jt jt |tjdt|t|Dcgc] }t|c}Scc}}wcc}w)zReturn a list of paths for all relevant model files given by the user. Returns: (list[Path]): List of Path objects for the model files. )z*.ptz*.onnxz*.yamlrz Profiling: ) rris_dirrglobrsrrrrsorted)rr*r extensionsextrs rCrzProfileModels.get_filess JJ 3D:D{{}9  j`sTYYsSWZ]S]E_`Td`d`a 88 SY' TYYs4y12 3  k&-12'-e}5tT 55a6s /D ?Dcy)z\Extract metadata from an ONNX model file including layers, parameters, gradients, and FLOPs.)rrrrri)r/s rCr#z!ProfileModels.get_onnx_model_infos"rctj|}t|D]b}tj|tj|}}|||||zz kD||||zzkz}t |t |k(r|S|}d|S)a{Apply iterative sigma clipping to data to remove outliers. Args: data (np.ndarray): Input data array. sigma (float): Number of standard deviations to use for clipping. max_iters (int): Maximum number of iterations for the clipping process. Returns: (np.ndarray): Clipped data array with outliers removed. )rearrayrangemeanstdr)rKsigma max_itersrr=r> clipped_datas rCiterative_sigma_clippingz&ProfileModels.iterative_sigma_clippingsxx~y! A rvvd|#Dus{(:!:tdUUX[FX?X YZL< CI-  D   rcz|jrt|jsyt|}t j |j |j dftj}d}tdD]\}tj}t|jD]}|||j dtj|z }^tt|j||zz |jz|jdz}g} tt||D]8}|||j d} | j!| d j"d :|j%t j&| d d } t j(| t j*| fS) awProfile YOLO model performance with TensorRT, measuring average run time and standard deviation. Args: engine_file (str): Path to the TensorRT engine file. eps (float): Small epsilon value to prevent division by zero. Returns: (tuple[float, float]): Mean and standard deviation of inference time in milliseconds. )rrr)dtyperF)rGr(2descrrRrTr?r@)rrr"rrezerosrGuint8r<rtrmaxrrrrrrrBr;r=r>) rr-rr{ input_dataelapsedr start_timenum_runs run_timesrs rCr$z$ProfileModels.profile_tensorrt_model)svxxtK088:[!XXtzz4::q9J q /AJ4//0 Cj EB CiikJ.G  /uT]]gm.Qs#QSz#s+8q8Qs!#)r) tensor_shapes rC check_dynamiczProfileModels.check_dynamicNsQLQQQQrctdgddl}|j}|jj|_d|_|j||dg}i}|jD]}|j}|j|jrt|jdk7rC|j|jddr%td |jd |jt|jdk(rdd |j|jfndg|jdd} n |j} d |vrt j"} nbd |vrt j$} nMd|vrt j&} n8d|vrt j(} n#d|vrt j*} ntd|t!j,j.| j1| } |j} | || <|j3dj} d}t5d D]Z}t7j6}t5|j8D]}|j;| g|t7j6|z }\t=t?|j@||zz |j8z|jB}g}tEt5||D]R}t7j6}|j;| g||jGt7j6|z dzT|jIt!jJ|dd}t!jL|t!jN|fS)atProfile an ONNX model, measuring average inference time and standard deviation across multiple runs. Args: onnx_file (str): Path to the ONNX model file. eps (float): Small epsilon value to prevent division by zero. Returns: (tuple[float, float]): Mean and standard deviation of inference time in milliseconds. ) onnxruntimezonnxruntime-gpurNCPUExecutionProvider) providersrUr%zUnsupported dynamic shape z of rfloat16rdoubleint64int32zUnsupported ONNX datatype rrFrSrTrrH)(rrWSessionOptionsGraphOptimizationLevelORT_ENABLE_ALLgraph_optimization_levelintra_op_num_threadsInferenceSession get_inputsr}rUshaper ValueErrorrrGrer[float32float64r]r^randomrandastype get_outputsr<rtrr2rKrrrrrrBr;r=r>)rr/rort sess_optionssessinput_data_dict input_tensor input_type input_shape input_dtyperL input_name output_namerMrrNrOrPs rCr%z ProfileModels.profile_onnx_modelSs$ >?@!))+ 030J0J0Y0Y -,- )##I|H^G_#` OO- 5L%**J!!,"4"45|))*a/D4F4F|GYGYZ[Z\G]4^$'A,BTBTAUUYZfZkZkYl%mnn69,:L:L6MQR6RQ4::tzz2YZXt]i]o]opqpr]sXt+00 J& jj J& jj Z' jj J& hh J& hh  #=j\!JKK5<<[IJ%**J*4OJ '7 5:&&(+00 q /AJ4//0 9+8 9iikJ.G  /uT]]gm ? @ 11"((92EQZ[1\ wwy!266)#444rc|\}}}}d|dd|jd|ddd|ddd |ddd|ddd |d z dd|dd S) aGenerate a table row string with model performance metrics. Args: model_name (str): Name of the model. t_onnx (tuple): ONNX model inference time statistics (mean, std). t_engine (tuple): TensorRT engine inference time statistics (mean, std). model_info (tuple): Model information (layers, params, gradients, flops). Returns: (str): Formatted table row string with model metrics. z| 18sz | z | - | rz.1f±r%z ms | g.Az |r) rrr1r0r._layersparams _gradientsflopss rCr&z ProfileModels.generate_table_rows$.8*UC DJJ(pixels)zmAPval
50-95zSpeed
CPU (z) ONNX
(ms)zSpeed
z TensorRT
(ms)zparams
(M)zFLOPs
(B)|c3(K|] }d|d yw)r\NrirAhs rCrDz,ProfileModels.print_table..s:Q!A3a:sc3>K|]}dt|dzzyw)rFrTN)rrs rCrDz,ProfileModels.print_table..s"G!3#a&1*#5"Gsz N)rrE is_availableget_device_namerjoinrr)r+rheadersheader separatorrYs rCr)zProfileModels.print_tables05zz/F/F/Hejj((+e  # "!,.!1 @SE!2 3  sxx:':::S@#(("Gw"GGG#M  d6(O$ I C KK  r)d <iTTN)r list[str]rrrrrrrGrrIboolrrrLztorch.device | str | None)r/rs)rTr)rKz np.ndarrayr?rr@r)r")r-rsrr)r/rsrr)rrsr1tuple[float, float]r0rr.z!tuple[float, float, float, float])r+r)rr r r rr2rr r#rBr$rUr%r&r(r)rirrCrrsIF"!,0!\!\!\ !\  !\  !\!\!\*!\F4l6(""(#5JRRF5P  $ &  6  0  # & 6   4rr)7r  __future__rr5rrrrrtcopyrpathlibrnumpyre torch.cudar ultralyticsrrultralytics.cfgrr ultralytics.engine.exporterr ultralytics.nn.modulesr ultralytics.utilsr r rrrrrrrrrultralytics.utils.checksrrrrrultralytics.utils.downloadsrultralytics.utils.filesrultralytics.utils.torch_utilsrrrrrrirrCrs<#  '26,    nm5-E  $     l^JJZddr