FiICtddlmZddlZddlZddlZddlmZddlmZddl m Z ddl m Z ddl ZddlZddlmZddlmZddlmZmZmZdd lmZdd lmZmZmZdd lmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&dd l'm(Z(m)Z)dd l*m+Z+m,Z,ddl-m.Z.ddl/m0Z0GddejbZ2GddZ3GddejhjjjlZ7ddZ8 d ddZ9 d d dZ: d! d"dZ; d#dZ< d$ d%dZ=y)&) annotationsN)Iterator)Path)Any)urlsplit)Image)Dataset dataloader distributed)IterableSimpleNamespace)GroundingDataset YOLODatasetYOLOMultiModalDataset)LOADERSLoadImagesAndVideosLoadPilAndNumpyLoadScreenshots LoadStreams LoadTensor SourceTypes autocast_list) IMG_FORMATS VID_FORMATS)RANKcolorstr) check_file) TORCH_2_0c@eZdZdZdfd ZddZd dZdZdZxZ S) InfiniteDataLoaderaDataLoader that reuses workers for infinite iteration. This dataloader extends the PyTorch DataLoader to provide infinite recycling of workers, which improves efficiency for training loops that need to iterate through the dataset multiple times without recreating workers. Attributes: batch_sampler (_RepeatSampler): A sampler that repeats indefinitely. iterator (Iterator): The iterator from the parent DataLoader. Methods: __len__: Return the length of the batch sampler's sampler. __iter__: Yield batches from the underlying iterator. __del__: Ensure workers are properly terminated. reset: Reset the iterator, useful when modifying dataset settings during training. Examples: Create an infinite DataLoader for training >>> dataset = YOLODataset(...) >>> dataloader = InfiniteDataLoader(dataset, batch_size=16, shuffle=True) >>> for batch in dataloader: # Infinite iteration >>> train_step(batch) cts|jddt| |i|tj |dt |jt|!|_ y)zHInitialize the InfiniteDataLoader with the same arguments as DataLoader.prefetch_factorN batch_sampler) rpopsuper__init__object __setattr___RepeatSamplerr"__iter__iterator)selfargskwargs __class__s X/var/www/pod-logistic/pod-ai/venv/lib/python3.12/site-packages/ultralytics/data/build.pyr%zInfiniteDataLoader.__init__=sS JJ($ / $)&)4.ASAS2TU(* c@t|jjS)z1Return the length of the batch sampler's sampler.)lenr"samplerr+s r/__len__zInfiniteDataLoader.__len__Es4%%--..r0c#lKtt|D]}t|jyw)zICreate an iterator that yields indefinitely from the underlying iterator.N)ranger2nextr*)r+_s r/r)zInfiniteDataLoader.__iter__Is-s4y! &At}}% % &s24c t|jdsy|jjD]#}|js|j %|jj y#t $rYywxYw)zKEnsure that workers are properly terminated when the DataLoader is deleted._workersN)hasattrr*r;is_alive terminate_shutdown_workers Exception)r+ws r/__del__zInfiniteDataLoader.__del__Nsg 4==*5]]++ "::<KKM " MM + + -   sA0)A0,A00 A<;A<c.|j|_y)zIReset the iterator to allow modifications to the dataset during training.N) _get_iteratorr*r4s r/resetzInfiniteDataLoader.resetZs**, r0)r,rr-rreturnintrGr) __name__ __module__ __qualname____doc__r%r5r)rBrE __classcell__)r.s@r/rr%s!.+/& -r0rc eZdZdZddZddZy)r(a.Sampler that repeats forever for infinite iteration. This sampler wraps another sampler and yields its contents indefinitely, allowing for infinite iteration over a dataset without recreating the sampler. Attributes: sampler (torch.utils.data.Sampler): The sampler to repeat. c||_y)zDInitialize the _RepeatSampler with a sampler to repeat indefinitely.N)r3)r+r3s r/r%z_RepeatSampler.__init__is  r0c#LK t|jEd{7w)z=Iterate over the sampler indefinitely, yielding its contents.N)iterr3r4s r/r)z_RepeatSampler.__iter__ms#DLL) ) ) )s $"$N)r3rrI)rJrKrLrMr%r)r0r/r(r(_s*r0r(cXeZdZdZ d d dZd dZd dZd dZd dZy)ContiguousDistributedSamplera\Distributed sampler that assigns contiguous batch-aligned chunks of the dataset to each GPU. Unlike PyTorch's DistributedSampler which distributes samples in a round-robin fashion (GPU 0 gets indices [0,2,4,...], GPU 1 gets [1,3,5,...]), this sampler gives each GPU contiguous batches of the dataset (GPU 0 gets batches [0,1,2,...], GPU 1 gets batches [k,k+1,...], etc.). This preserves any ordering or grouping in the original dataset, which is critical when samples are organized by similarity (e.g., images sorted by size to enable efficient batching without padding when using rect=True). The sampler handles uneven batch counts by distributing remainder batches to the first few ranks, ensuring all samples are covered exactly once across all GPUs. Args: dataset (Dataset): Dataset to sample from. Must implement __len__. num_replicas (int, optional): Number of distributed processes. Defaults to world size. batch_size (int, optional): Batch size used by dataloader. Defaults to dataset.batch_size or 1. rank (int, optional): Rank of current process. Defaults to current rank. shuffle (bool, optional): Whether to shuffle indices within each rank's chunk. Defaults to False. When True, shuffling is deterministic and controlled by set_epoch() for reproducibility. Examples: >>> # For validation with size-grouped images >>> sampler = ContiguousDistributedSampler(val_dataset, batch_size=32, shuffle=False) >>> loader = DataLoader(val_dataset, batch_size=32, sampler=sampler) >>> # For training with shuffling >>> sampler = ContiguousDistributedSampler(train_dataset, batch_size=32, shuffle=True) >>> for epoch in range(num_epochs): ... sampler.set_epoch(epoch) ... for batch in loader: ... ... Nc|*tjrtjnd}|*tjrtjnd}| t |dd}||_||_d|_||_t||_ ||jk\rdn||_ tj|j|jz |_y)zHInitialize the sampler with dataset and distributed training parameters.Nr batch_size)distis_initializedget_world_sizeget_rankgetattr num_replicasrankepochshuffler2 total_sizerXmathceil num_batches)r+datasetr^rXr_ras r/r%z%ContiguousDistributedSampler.__init__s  484G4G4I4..0qL <&*&9&9&;4==?D   ,:J(   g,)T__SYZ [ii"77#dii:SS "77  $//1 i$//14??C'!!r0cd|j\}}tt||}|jrmt j }|j |jt jt||jDcgc]}|| }}t|Scc}w)zAGenerate indices for this rank's contiguous chunk of the dataset.) generator) rplistr7ratorch Generator manual_seedr`randpermr2tolistrR)r+rnroindicesgis r/r)z%ContiguousDistributedSampler.__iter__s!335 7uY01 <<!A MM$** %+0>>#g,RS+T+[+[+]^awqz^G^G}_s B-c2|j\}}||z S)z2Return the number of samples in this rank's chunk.)rp)r+rnros r/r5z$ContiguousDistributedSampler.__len__s !335 7""r0c||_y)zSet the epoch for this sampler to ensure different shuffling patterns across epochs. Args: epoch (int): Epoch number to use as the random seed for shuffling. N)r`)r+r`s r/ set_epochz&ContiguousDistributedSampler.set_epochs  r0)NNNF) rfr r^ int | NonerXrr_rraboolrGNone)rGztuple[int, int]rIrF)r`rHrGr) rJrKrLrMr%rpr)r5r~rSr0r/rUrUsspD$(!% HH!H H  H  H H2"& # r0rUctjdz}tjj |tj|y)zGSet dataloader worker seed for reproducibility across worker processes.lN)rt initial_seednprandomseed) worker_id worker_seeds r/ seed_workerrs1$$&.KIINN; KK r0c4|rtnt}|||j||dk(||jxs||jxsd|j xsd||dk(rdndt |d|j|j||dk(r|jSdS) zBBuild and return a YOLO dataset based on configuration parameters.trainNF?: ?)img_pathimgszrXaugmenthyprectcache single_clsstridepadprefixtaskclassesdatafraction) rrrrrrrrrr) cfgrbatchrmoderr multi_modalrfs r/build_yolo_datasetrs(3# G ii  XX ii4>>*U7?C4& $ XX !% 7: r0c Vtdid|d|d|d|jd|d|dk(d|d |jxs|d |jxsd d |jxsd d|d|dk(rdnddt |dd|j d|jd|dk(r|jSdS)zFBuild and return a GroundingDataset based on configuration parameters.r json_file max_samplesrrXrrrrrNrFrrrrrrrrrrrS) r rrrrrrrr)rrrrrrrrs r/build_groundingrs      ii          XX  ii4 >>*U  7?C 4& $ XX    "&!  7:! r0ct|t|}tjj }tt j t|dz|}|dk(rdn#|rtj||n t|} tj} | jdtzt|||xr| du|| |dkDrdnd|dkDxr|t|ddt | |xrt||zdk7 S) a_Create and return an InfiniteDataLoader for training or validation. Args: dataset (Dataset): Dataset to load data from. batch (int): Batch size for the dataloader. workers (int): Number of worker processes for data loading. shuffle (bool, optional): Whether to shuffle the dataset. rank (int, optional): Process rank in distributed training. -1 for single-GPU training. drop_last (bool, optional): Whether to drop the last incomplete batch. pin_memory (bool, optional): Whether to use pinned memory for dataloader. Returns: (InfiniteDataLoader): A dataloader that can be used for training or validation. Examples: Create a dataloader for training >>> dataset = YOLODataset(...) >>> dataloader = build_dataloader(dataset, batch=16, workers=4, shuffle=True) rWN)ralUU*UU*r collate_fn) rfrXra num_workersr3r! pin_memoryrworker_init_fnrr drop_last)rhr2rtcuda device_countos cpu_countmaxr DistributedSamplerrUrurvrrr]r) rfrworkersrar_rrndnwr3rrs r/build_dataloaderrs8 s7| $E  "B R\\^s2qz )7 3B 2:   + +GW E )' 2  !I -45 +GtO!V6(j7L$7"9G u 4 9  r0cd\}}}}}t|tttfrt|}|j }|j d}|rt |jn|jddttzv}|jxs|jdxs|xr| }|dk(}|r|rt|}nt|trd}nyt|tt frt#|}d}nUt|t$j$t&j(frd}n(t|t*j,rd}n t/d||||||fS) aCheck the type of input source and return corresponding flag values. Args: source (str | int | Path | list | tuple | np.ndarray | PIL.Image | torch.Tensor): The input source to check. Returns: source (str | int | Path | list | tuple | np.ndarray | PIL.Image | torch.Tensor): The processed source. webcam (bool): Whether the source is a webcam. screenshot (bool): Whether the source is a screenshot. from_img (bool): Whether the source is an image or list of images. in_memory (bool): Whether the source is an in-memory object. tensor (bool): Whether the source is a torch.Tensor. Examples: Check a file path source >>> source, webcam, screenshot, from_img, in_memory, tensor = check_source("image.jpg") Check a webcam source >>> source, webcam, screenshot, from_img, in_memory, tensor = check_source(0) )FFFFF)zhttps://zhttp://zrtsp://zrtmp://ztcp://.rz.streamsscreenTzZUnsupported image type. For supported types see https://docs.ultralytics.com/modes/predict) isinstancestrrHrlower startswithrpath rpartitionrr isnumericendswithrrrstuplerrrndarrayrtTensor TypeError) sourcewebcam screenshotfrom_img in_memorytensor source_loweris_urlis_files r/ check_sourcerUs=.7X3FJ)V&3T*+V||~ (()`a288L)..lVVWZ[\^_ + % !!#^vz'B^vG]V]R]!X- g'F FG $ FT5M *v& FU[["**5 6 FELL )tuu 6:xF BBr0c$t|\}}}}}} |r |jn t|||| } | r t|} nF|r|} nA|rt ||||} n/|rt ||} n|rt ||} nt||||} t| d| | S)aqLoad an inference source for object detection and apply necessary transformations. Args: source (str | int | Path | list | tuple | np.ndarray | PIL.Image | torch.Tensor): The input source for inference. batch (int, optional): Batch size for dataloaders. vid_stride (int, optional): The frame interval for video sources. buffer (bool, optional): Whether stream frames will be buffered. channels (int, optional): The number of input channels for the model. Returns: (Dataset): A dataset object for the specified input source with attached source_type attribute. Examples: Load an image source for inference >>> dataset = load_inference_source("image.jpg", batch=1) Load a video stream source >>> dataset = load_inference_source("rtsp://example.com/stream", vid_stride=2) ) vid_stridebufferchannels)r)rrr source_type) rrrrrrrrsetattr) rrrrrstreamrrrrrrfs r/load_inference_sourcers6?K6>R;FFJ)V(1&$${6:W_ag7hKV$  fFU]^ !&8< !&8<%fEj[cd G]K0 Nr0)rrHrGr)rF F)rr rrrrHrzdict[str, Any]rrrrrrHrrrGr )rFrP)rr rrrrrrHrrrrrrHrrHrGr )TrFT)rrHrrHrarr_rHrrrrrGr)rIstr | int | Path | list | tuple | np.ndarray | Image.Image | torch.TensorrGz(tuple[Any, bool, bool, bool, bool, bool])rWrWF) rrrrHrrHrrrrH)> __future__rrcrrcollections.abcrpathlibrtypingr urllib.parsernumpyrrttorch.distributedr rYPILrtorch.utils.datar r ultralytics.cfgr ultralytics.data.datasetr rrultralytics.data.loadersrrrrrrrrultralytics.data.utilsrrultralytics.utilsrrultralytics.utils.checksrultralytics.utils.torch_utilsr DataLoaderrr(utilsrSamplerrUrrrrrrrSr0r/rs# $! ==3YY   <,/37-..7-t**(c5;;#3#3#;#;cL          H         F4 44 4  4  444n/C U/C-/Ch / U/ //  /  /r0