Fi1ddlmZddlZddlZddlmZddlZddlmZm Z m Z m Z m Z m Z ddlmZmZddlmZmZd d dZe d d dZ d dd Zy)) annotationsN)Path) IS_JETSONLOGGER TORCH_VERSIONThreadingLockedis_dgx is_jetson)check_tensorrt check_version) TORCH_2_4 TORCH_2_9c~trKtjjjj dz }t r t|d}|rO|dz}nIdjtjddd}ddddd d d d d d ddddd d j|d}t||jjS) z@Return max ONNX opset for this torch version with ONNX fallback..N )z1.8z1.9z1.10z1.11z1.12z1.13z2.0z2.1z2.2z2.3z2.4z2.5z2.6z2.7z2.8)r torchonnxutils _constantsONNX_MAX_OPSETrminjoinrsplitgetdefsonnx_opset_version)rcudaopsetversions a/var/www/pod-logistic/pod-ai/venv/lib/python3.12/site-packages/ultralytics/utils/export/engine.pybest_onnx_opsetr)s   ++::Q> rNE  QJE((=..s3BQ78 #gr ! " udii224 55c |dg}|dg}trddini}tjj|||fd|d|||d|t |S)aExport a PyTorch model to ONNX format. Args: model (torch.nn.Module): The PyTorch model to export. im (torch.Tensor | tuple[torch.Tensor, ...]): Example input tensor(s) for tracing. output_file (Path | str): Path to save the exported ONNX file. opset (int): ONNX opset version to use for export. input_names (list[str] | None): List of input tensor names. Defaults to ``["images"]``. output_names (list[str] | None): List of output tensor names. Defaults to ``["output0"]``. dynamic (dict | None): Dictionary specifying dynamic axes for inputs and outputs. Returns: (str): Path to the exported ONNX file. Notes: Setting `do_constant_folding=True` may cause issues with DNN inference for torch>=1.12. imagesoutput0dynamoFT)verbose opset_versiondo_constant_folding input_names output_names dynamic_axes)r rrexportstr)modelim output_filer&r2r3dynamickwargss r( torch2onnxr<.sw6j !{ "+h F JJ    !    { r*c L  tds tr td ddl t j ddt j d d tjd | d j d|xst|jd} j jj} | r% jjj| _ j!| } | j#}t%xsddz}t% j j'ddddk\}|r,|dkDr'|j) j*j,|n |dkDr||_dt% j0j2z}| j5|}| j6xr|}| j8xr|}t:s t=dtj| dd|s |s t=d j>j@|_!t%|_"|jG jHjJ jM|| }|jO|stQd|tS|jTDcgc]}|jW|}}tS|jXDcgc]}|j[|}}|D]@}tj| d|j\d|j^d|j`B|D]@}tj| d|j\d|j^d|j`B|r{| jc}d|dddf}g|ddfd|ddD}|D]!}|je|j\|||#|jg||r|s|ji|tj| d |rd!nd"|rd#nd$zd%||r|jG jHjj jljn|_8G fd&d' jr}||tut|jd()|_;n'|r%|jG jHjx|r| j{||}| tQd*t}|d+5}| `tj| }|jt|jd,d-d.|j|j|j|dddtu|S| j||5}t}|d+5}| `tj| }|jt|jd,d-d.|j|j|j|jddddddtu|S#t$rtddl Y6wxYwcc}wcc}w#1swYtu|SxYw#1swYZxYw#1swYtu|SxYw)/amExport a YOLO model to TensorRT engine format. Args: onnx_file (str): Path to the ONNX file to be converted. output_file (Path | str | None): Path to save the generated TensorRT engine file. workspace (int | None): Workspace size in GB for TensorRT. half (bool, optional): Enable FP16 precision. int8 (bool, optional): Enable INT8 precision. dynamic (bool, optional): Enable dynamic input shapes. shape (tuple[int, int, int, int], optional): Input shape (batch, channels, height, width). dla (int | None): DLA core to use (Jetson devices only). dataset (ultralytics.data.build.InfiniteDataLoader, optional): Dataset for INT8 calibration. metadata (dict | None): Metadata to include in the engine file. verbose (bool, optional): Enable verbose logging. prefix (str, optional): Prefix for log messages. Returns: (str): Path to the exported engine file. Raises: ValueError: If DLA is enabled on non-Jetson devices or required precision is not set. RuntimeError: If the ONNX file cannot be parsed. Notes: TensorRT version compatibility is handled for workspace size and engine building. INT8 calibration requires a dataset and generates a calibration cache. Metadata is serialized and written to the engine file if provided. )jetpackz10.15rNz>=7.0.0T)hardz!=10.1.0z5https://github.com/ultralytics/ultralytics/pull/14239)msg z starting export with TensorRT z...z.enginei@rr z.DLA is only available on NVIDIA Jetson devicesz enabling DLA on core ztDLA requires either 'half=True' (FP16) or 'int8=True' (INT8) to be enabled. Please enable one of them and try again.zfailed to load ONNX file: z input "z " with shape z output " rc3VK|] }ttdxsd|z"yw)rN)intmax).0d workspaces r( zonnx2engine..s&"Vq3s1in1'='A#B"Vs&))roptrHz building INT8FP1632z engine as cReZdZdZ d d fd Zd dZd dZd dZddZddZ y)%onnx2engine..EngineCalibratoraCustom INT8 calibrator for TensorRT engine optimization. This calibrator provides the necessary interface for TensorRT to perform INT8 quantization calibration using a dataset. It handles batch generation, caching, and calibration algorithm selection. Attributes: dataset: Dataset for calibration. data_iter: Iterator over the calibration dataset. algo (trt.CalibrationAlgoType): Calibration algorithm type. batch (int): Batch size for calibration. cache (Path): Path to save the calibration cache. Methods: get_algorithm: Get the calibration algorithm to use. get_batch_size: Get the batch size to use for calibration. get_batch: Get the next batch to use for calibration. read_calibration_cache: Use existing cache instead of calibrating again. write_calibration_cache: Write calibration cache to disk. cjj|||_t||_j j nj j|_|j|_ t||_ y)z;Initialize the INT8 calibrator with dataset and cache path.N) IInt8Calibrator__init__datasetiter data_iterCalibrationAlgoTypeENTROPY_CALIBRATION_2MINMAX_CALIBRATIONalgo batch_sizebatchrcache)selfrWr`dlatrts r(rVz.onnx2engine..EngineCalibrator.__init__ss ##,,T2& !%g++AA00CC %// !%[ r*c|jS)z%Get the calibration algorithm to use.)r]ras r( get_algorithmz3onnx2engine..EngineCalibrator.get_algorithms yy r*c"|jxsdS)z*Get the batch size to use for calibration.r)r_res r(get_batch_sizez4onnx2engine..EngineCalibrator.get_batch_sizeszzQ&r*c t|jddz }|jjdk(r|j dn|}t |j gS#t$rYywxYw)zOGet the next batch to use for calibration, as a list of device memory pointers.imggo@cpur%N)nextrYdevicetypetorGdata_ptr StopIteration)ranamesim0ss r( get_batchz/onnx2engine..EngineCalibrator.get_batchse /6>D.2kk.>.>%.G4776?TD 011$  sA A## A/.A/c|jjr4|jjdk(r|jjSyy)zSUse existing cache instead of calibrating again, otherwise, implicitly return None..cacheN)r`existssuffix read_bytesres r(read_calibration_cachez.EngineCalibrator.read_calibration_cache s?::$$&4::+<+<+H::0022,I&r*c:|jj|}y)z Write calibration cache to disk.N)r` write_bytes)rar`_s r(write_calibration_cachez=onnx2engine..EngineCalibrator.write_calibration_cachesJJ**51r*N))r`r6returnNone)rztrt.CalibrationAlgoType)rrG)rzlist[int] | None)rz bytes | None)r`bytesrr) __name__ __module__ __qualname____doc__rVrfrhrtrzr~)rbrcsr(EngineCalibratorrSs? .  ) )  )" ! '  3  2r*rrv)rWr`z3TensorRT engine build failed, check logs for errorswblittle) byteordersigned)Gr r r tensorrt ImportErrorr __version__rinfor with_suffixLoggerINFOSeverityVERBOSE min_severityBuildercreate_builder_configrGr!set_memory_pool_limitMemoryPoolType WORKSPACEmax_workspace_sizeNetworkDefinitionCreationFlagEXPLICIT_BATCHcreate_networkplatform_has_fast_fp16platform_has_fast_int8r ValueError DeviceTypeDLAdefault_device_typeDLA_coreset_flag BuilderFlag GPU_FALLBACK OnnxParserparse_from_file RuntimeErrorrange num_inputs get_input num_outputs get_outputnameshapedtypecreate_optimization_profile set_shapeadd_optimization_profileset_calibration_profilerNProfilingVerbosityDETAILEDprofiling_verbosityrUr6int8_calibratorFP16build_serialized_networkopenjsondumpswritelento_bytesencode build_engine serialize)! onnx_filer9rKhalfint8r:rrbrWmetadatar/prefixloggerbuilderconfigworkspace_bytesis_trt10flagnetworkparseriinputsoutputsinpoutprofile min_shape max_shaperenginetmetarcs! ` ` @r( onnx2enginer]sXvxw#//948#//:3jk KK"VH;COO;LCPQGi!g67O3??((a034:HOa'$$S%7%7%A%A?S 1 $3! C55DDE ED$$T*G  ) ) 2dD  ) ) 2dD MN N vh4SE=>DG &)^^%7%7"c(445^^GV ,F  ! !) ,7 {CDD-2'2D2D,E Fqg" FF F.3G4G4G.HIw!!!$IGIV vhhsxxj SYYKq TUVW vhizcii[#))UVW557a"b) WeBQiW"VERSRTI"VW  QC   chhI5i  P Q''0   * *7 3 KK6(*tVSW9X$YYdepdqrs ,,-%(%;%;%D%D"? 2s22? 2D"2d9o11(;<"  ,,-11'6B >TU U +t $ #zz(+D **1*NO & GGFO   {   ! !'6 2 (fd;PT>U (YZ#zz(+D **1*NO & GGF$$& '  ( ( { C \GIJ  {   ( ( ( { sOY*Y$Y)A4Y. Z)BZ+ZY! Y!.ZZ ZZ#)F)rztypes.ModuleTyper%boolrrG)rNNN)r7ztorch.nn.Moduler8z'torch.Tensor | tuple[torch.Tensor, ...]r9z Path | strr&rGr2list[str] | Noner3rr: dict | Nonerr6) NNFFF)rrNNNFr)rr6r9zPath | str | NonerK int | Nonerrrrr:rrztuple[int, int, int, int]rbrrrr/rrr6rr6) __future__rrtypespathlibrrultralytics.utilsrrrrr r ultralytics.utils.checksr r ultralytics.utils.torch_utilsr rr)r<rr*r(rsB#  bbB>6< $(%)+ +/++  + " + # ++ ++`&* '7  RR"RR  R  R  R %R RRR R Rr*