GiddlZddlZddlZddlmZddlmZddlmZddl Z ddl m Z m Z ddlmZddlmZdd lmZdd lmZdd lmZmZmZmZdd lmZmZdd lmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)ddl*m+Z+ddl,m-Z-ddl.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7e%rddl8Z8e'r ddl9m:cm;cmZ>e(j~e@ZAdZBe#deBe-dGddeZCe$eCjeC_DeCjj8eCjjjdddeCj_Eyy)N)Callable)partial)Any) create_repois_offline_mode)validate_typed_dict)custom_object_save)TorchvisionBackend) BatchFeature)ChannelDimensionSizeDictis_vision_availablevalidate_kwargs)Unpack VideosKwargs) IMAGE_PROCESSOR_NAMEPROCESSOR_NAMEVIDEO_PROCESSOR_NAME TensorTypeadd_start_docstrings copy_funcis_torch_availableis_torchcodec_availableis_torchvision_v2_availableloggingsafe_load_json_file) cached_file)requires) VideoInput VideoMetadatagroup_videos_by_shapeinfer_channel_dimension_formatis_valid_video load_videomake_batched_metadatamake_batched_videosreorder_videos)PILImageResamplinga Args: do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the video's (height, width) dimensions to the specified `size`. Can be overridden by the `do_resize` parameter in the `preprocess` method. size (`dict`, *optional*, defaults to `self.size`): Size of the output video after resizing. Can be overridden by the `size` parameter in the `preprocess` method. size_divisor (`int`, *optional*, defaults to `self.size_divisor`): The size by which to make sure both the height and width can be divided. default_to_square (`bool`, *optional*, defaults to `self.default_to_square`): Whether to default to a square video when resizing, if size is an int. resample (`PILImageResampling`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the video. Only has an effect if `do_resize` is set to `True`. Can be overridden by the `resample` parameter in the `preprocess` method. do_center_crop (`bool`, *optional*, defaults to `self.do_center_crop`): Whether to center crop the video to the specified `crop_size`. Can be overridden by `do_center_crop` in the `preprocess` method. crop_size (`dict[str, int]` *optional*, defaults to `self.crop_size`): Size of the output video after applying `center_crop`. Can be overridden by `crop_size` in the `preprocess` method. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the video by the specified scale `rescale_factor`. Can be overridden by the `do_rescale` parameter in the `preprocess` method. rescale_factor (`int` or `float`, *optional*, defaults to `self.rescale_factor`): Scale factor to use if rescaling the video. Only has an effect if `do_rescale` is set to `True`. Can be overridden by the `rescale_factor` parameter in the `preprocess` method. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the video. Can be overridden by the `do_normalize` parameter in the `preprocess` method. Can be overridden by the `do_normalize` parameter in the `preprocess` method. image_mean (`float` or `list[float]`, *optional*, defaults to `self.image_mean`): Mean to use if normalizing the video. This is a float or list of floats the length of the number of channels in the video. Can be overridden by the `image_mean` parameter in the `preprocess` method. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`float` or `list[float]`, *optional*, defaults to `self.image_std`): Standard deviation to use if normalizing the video. This is a float or list of floats the length of the number of channels in the video. Can be overridden by the `image_std` parameter in the `preprocess` method. Can be overridden by the `image_std` parameter in the `preprocess` method. do_convert_rgb (`bool`, *optional*, defaults to `self.image_std`): Whether to convert the video to RGB. video_metadata (`VideoMetadata`, *optional*): Metadata of the video containing information about total duration, fps and total number of frames. do_sample_frames (`int`, *optional*, defaults to `self.do_sample_frames`): Whether to sample frames from the video before processing or to process the whole video. num_frames (`int`, *optional*, defaults to `self.num_frames`): Maximum number of frames to sample when `do_sample_frames=True`. fps (`int` or `float`, *optional*, defaults to `self.fps`): Target frames to sample per second when `do_sample_frames=True`. return_tensors (`str` or `TensorType`, *optional*): Returns stacked tensors if set to `pt, otherwise returns a list of tensors. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output video. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: video in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: video in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input video. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input video. If unset, the channel dimension format is inferred from the input video. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: video in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: video in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: video in (height, width) format. device (`torch.device`, *optional*): The device to process the videos on. If unset, the device is inferred from the input videos. return_metadata (`bool`, *optional*): Whether to return video metadata or not. z!Constructs a base VideoProcessor.)vision torchvision)backendsceZdZdZdZdZdZdZdZdZ dZ dZ dZ dZ dZdZdZdZdZdZdZdZeZdgZdeeddffd Zdefd Zd d defd Z d?d ede dzde e!zdzfdZ" d?dedee#zde$dzde%dzde&d f dZ' d?dede(e)zdzde(dzde&d fdZ*e+e,dedeedefdZ- d@de&d de$de$de.ddde$d e.d!e$d"e!d#e$d$e!e&e!zdzd%e!e&e!zdzd&e(e/zdzdefd'Z0e1 dAd(e(e2jfzd)e(e2jfzdzd*e$d+e$d,e(e$zdzd-e(f d.Z4dBd/e(e2jfzd0e$fd1Z5e1d(e(e2jfzde6e#e(e7fe#e(e7fffd2Z8e1d3e#e(e7ffd4Z9de#e(e7fffd5 Z:de(fd6Z;d7e(e2jfzfd8Ze1dCd<Z?d@d=e(e&e(ze&e&e(zfd>Z@xZAS)DBaseVideoProcessorNTgp?Fpixel_values_videoskwargsreturnc $t|di|y)N)super__init__)selfr0 __class__s e/var/www/pod-logistic/pod-ai/venv/lib/python3.12/site-packages/transformers/video_processing_utils.pyr5zBaseVideoProcessor.__init__s "6"c (|j|fi|SN) preprocess)r6videosr0s r8__call__zBaseVideoProcessor.__call__stv000r9videoz torch.Tensorctj|}|jddk(s|ddddddfdkjs|S|ddddddfdz }d|ddddddfz dz|ddddddf|dddddddfzz}|S)z Converts a video to RGB format. Args: video (`"torch.Tensor"`): The video to convert. Returns: `torch.Tensor`: The converted video. .Ngo@r )tvFgrayscale_to_rgbshapeany)r6r?alphas r8convert_to_rgbz!BaseVideoProcessor.convert_to_rgbs$$U+ ;;r?a c1al(;c(A'F'F'HLc1al#e+U3a?++s2U3a?5KeTWY[Z[Y[]^`aTaNb5bb r9metadata num_framesfpsc | | td||n |j}||n |j}|j}|6|4| |j tdt ||jz |z}||kDrtd|d|d|*t j d|||z j }|St j d|j }|S)a% Default sampling function which uniformly samples the desired number of frames between 0 and total number of frames. If `fps` is passed along with metadata, `fps` frames per second are sampled uniformty. Arguments `num_frames` and `fps` are mutually exclusive. Args: metadata (`VideoMetadata`): Metadata of the video containing information about total duration, fps and total number of frames. num_frames (`int`, *optional*): Maximum number of frames to sample. Defaults to `self.num_frames`. fps (`int` or `float`, *optional*): Target frames to sample per second. Defaults to `self.fps`. Returns: np.ndarray: Indices to sample video frames. zc`num_frames`, `fps`, and `sample_indices_fn` are mutually exclusive arguments, please use only one!zAsked to sample `fps` frames per second but no video metadata was provided which is required when sampling with `fps`. Please pass in `VideoMetadata` object or use a fixed `num_frames` per input videoz(Video can't be sampled. The `num_frames=z` exceeds `total_num_frames=z`. r) ValueErrorrKrLtotal_num_framesinttorcharange)r6rJrKrLr0rOindicess r8 sample_framesz BaseVideoProcessor.sample_framess0 ?z5u $.#9Zt _c$((#44  #/8<<#7 h-  2x+X>*1'%%eGn5 ''1  2 $F-N~%% q *&)T*#'"3"3F";KK Ou!3!3E!: OUVW$$o ~%%*.):):6Uf):)g&~%%!Ps3D! D# D!D!input_data_formatdevicec^g}|D]}t|tjr#tj|j }| t |}|tjk(r"|jddddj }||j|}|j||S)z: Prepare the input videos for processing. rrBr ) r^npndarrayrQ from_numpy contiguousr#r LASTpermutetor])r6r=rirjprocessed_videosr?s r8_prepare_input_videosz(BaseVideoProcessor._prepare_input_videos*s +E%,((/::<!($B5$I! $4$9$99 aAq1<<>!(  # #E *! +" r9c t|jt|jjjdgzt |j||jjD]}|j |t||d!|jd}|jd}|jd}|jd}|rt|jfi|nd}|j||||\}}|j|||}|jd i|}|jd i||jd |jd } |jd d |i|} | r|| d<| S) Nreturn_tensors)captured_kwargsvalid_processor_keysrirVrjrU)rUrVrW)r=rirj data_formatreturn_metadatar=r3)rkeysr_ valid_kwargs__annotations__r setdefaultgetattrpoprrTrhru_standardize_kwargs_validate_preprocess_kwargs _preprocess) r6r=r0 kwarg_namerirVrjrUrWr{preprocessed_videoss r8r<zBaseVideoProcessor.preprocessGs "KKM!%d&7&7&G&G&L&L&N!OScRd!d D--v6++;; KJ   j'$ D*I J K#JJ':;!::&89H%$45EUGD$6$6A&A[_!%!?!? )-/ "@"  ++6M^gm+n)))3F3(((262  =! **%67.d..GfGG 4B  0 1""r9do_convert_rgb do_resizesizeresamplez7PILImageResampling | tvF.InterpolationMode | int | Nonedo_center_crop crop_size do_rescalerescale_factor do_normalize image_mean image_stdrwc t|\}}i}|jD]3\}}|r|j|}|r|j|||}|||<5t ||}t|\}}i}|jD]4\}}|r|j ||}|j ||| | | | }|||<6t ||}td|i| S)N)rrr/)data tensor_type)r"itemsrIresizer( center_croprescale_and_normalizer )r6r=rrrrrrrrrrrrwr0grouped_videosgrouped_videos_indexresized_videos_groupedrFstacked_videosresized_videosprocessed_videos_groupedrts r8rzBaseVideoProcessor._preprocessvs$0EV/L,,!#%3%9%9%; ; !E>!%!4!4^!D!%^$QY!Z,: "5 )  ; ((>@TU0E^/T,,#% %3%9%9%; = !E>!%!1!1.)!L!77 NL*V_N/= $U + =**BDXY"79I!JXfggr9pretrained_model_name_or_path cache_dirforce_downloadlocal_files_onlytokenrevisionc ||d<||d<||d<||d<|||d<|j|fi|\}}|j|fi|S)aK Instantiate a type of [`~video_processing_utils.VideoProcessorBase`] from an video processor. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained video hosted inside a model repo on huggingface.co. - a path to a *directory* containing a video processor file saved using the [`~video_processing_utils.VideoProcessorBase.save_pretrained`] method, e.g., `./my_model_directory/`. - a path to a saved video processor JSON *file*, e.g., `./my_model_directory/video_preprocessor_config.json`. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model video processor should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the video processor files and override the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. To test a pull request you made on the Hub, you can pass `revision="refs/pr/"`. return_unused_kwargs (`bool`, *optional*, defaults to `False`): If `False`, then this function returns just the final video processor object. If `True`, then this functions returns a `Tuple(video_processor, unused_kwargs)` where *unused_kwargs* is a dictionary consisting of the key/value pairs whose keys are not video processor attributes: i.e., the part of `kwargs` which has not been used to update `video_processor` and is otherwise ignored. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. kwargs (`dict[str, Any]`, *optional*): The values in kwargs of any keys which are video processor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are *not* video processor attributes is controlled by the `return_unused_kwargs` keyword parameter. Returns: A video processor of type [`~video_processing_utils.ImagVideoProcessorBase`]. Examples: ```python # We can't instantiate directly the base class *VideoProcessorBase* so let's show the examples on a # derived class: *LlavaOnevisionVideoProcessor* video_processor = LlavaOnevisionVideoProcessor.from_pretrained( "llava-hf/llava-onevision-qwen2-0.5b-ov-hf" ) # Download video_processing_config from huggingface.co and cache. video_processor = LlavaOnevisionVideoProcessor.from_pretrained( "./test/saved_model/" ) # E.g. video processor (or model) was saved using *save_pretrained('./test/saved_model/')* video_processor = LlavaOnevisionVideoProcessor.from_pretrained("./test/saved_model/video_preprocessor_config.json") video_processor = LlavaOnevisionVideoProcessor.from_pretrained( "llava-hf/llava-onevision-qwen2-0.5b-ov-hf", do_normalize=False, foo=False ) assert video_processor.do_normalize is False video_processor, unused_kwargs = LlavaOnevisionVideoProcessor.from_pretrained( "llava-hf/llava-onevision-qwen2-0.5b-ov-hf", do_normalize=False, foo=False, return_unused_kwargs=True ) assert video_processor.do_normalize is False assert unused_kwargs == {"foo": False} ```rrrrr)get_video_processor_dict from_dict) clsrrrrrrr0video_processor_dicts r8from_pretrainedz"BaseVideoProcessor.from_pretrainedstn({#1 %5!"%z  #F7O'Cs'C'CDa'lek'l$fs}}1>. ) ?>2BBe!fg g NT2 #ZZ(8$?NjjN,@,@,Mb,QRG!'CDCFCKKG#99.I     ' t^D A')ggll>CW&X# 56 /0K/LMN   ' ' -jj) ( ,,,r9c J|jdd}|jdd}|jdd}|jdd}|jdd}|jdd}|jd d } |jd d} |jd d} d | d} | | | d<tr|stjdd}t |}t j j|} t j j|r|}d}d} nXt} t|t|||||| || d }|tfDcgc]}t|||||||| || d x} |}}|r|dnd}d}|t|}d|vr|d}| | t|}|td|d|dd| rtjd|||fStjdd|||fScc}w#t$rt$rtd|d|d|dwxYw)a From a `pretrained_model_name_or_path`, resolve to a dictionary of parameters, to be used for instantiating a video processor of type [`~video_processing_utils.VideoProcessorBase`] using `from_dict`. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`): The identifier of the pre-trained checkpoint from which we want the dictionary of parameters. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. Returns: `tuple[Dict, Dict]`: The dictionary(ies) that will be used to instantiate the video processor object. rNrFproxiesrrr subfolder_from_pipeline _from_autovideo processor) file_typefrom_auto_classusing_pipelinez+Offline mode: forcing local_files_only=TrueT) filenamerrrrr user_agentrr%_raise_exceptions_for_missing_entriesrz Can't load video processor for 'z'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'z2' is the correct path to a directory containing a z filevideo_processorzloading configuration file z from cache at )rrrrstrrrisdirrrrrrOSError Exceptionr)rrr0rrrrrrr from_pipelinerris_localresolved_video_processor_fileresolved_processor_filevideo_processor_filer resolved_fileresolved_video_processor_filesrprocessor_dicts r8rz+BaseVideoProcessor.get_video_processor_dict6s$JJ{D1 $4e<**Y- 7D)!::&8%@::j$/JJ{B/  #3T:  **\59#4Y  $+8J' (  %5 KKE F# (+,I(J%77==!>? 77>>7 8,I )&* #H#7 2 +61+'#1#%5)%':? +'&:;O$P2 )49%-&/+9$+-="''1%-&/BG *    "2.2*:X215]a.& $ " .01HIN N2'56G'H$ ( 49M9U#67T#U  '23P2QR55R4ST++?*@G   KK56S5TU V $V++ KK-.B-C?SpRqr $V++2.  67T6UV99V8WX//C.DEK s)$G: "G5/ G:5G::(H"rc |j}|jdd}|j|jDcic]!\}}||jj vs||#c}}|di|}g}t t|jD]V}t||s||jj vs)t|||j|d|j|X|r&tjd|jd|dtjd||r||fS|Scc}}w) a Instantiates a type of [`~video_processing_utils.VideoProcessorBase`] from a Python dictionary of parameters. Args: video_processor_dict (`dict[str, Any]`): Dictionary that will be used to instantiate the video processor object. Such a dictionary can be retrieved from a pretrained checkpoint by leveraging the [`~video_processing_utils.VideoProcessorBase.to_dict`] method. kwargs (`dict[str, Any]`): Additional parameters from which to initialize the video processor object. Returns: [`~video_processing_utils.VideoProcessorBase`]: The video processor object instantiated from those parameters. return_unused_kwargsFNzImage processor z : kwargs z were applied for backward compatibility. To avoid this warning, add them to valid_kwargs: create a custom TypedDict extending ImagesKwargs with these keys and set it as the `valid_kwargs` class attribute.zVideo processor r3)copyrupdaterr}r~reversedr_r|hasattrsetattrr]r warning_once__name__r) rrr0rkvr extra_keyskeys r8rzBaseVideoProcessor.from_dicts7" 488:%zz*@%H##flln$ndaSM]M]MmMmHmQT$no5 45 D/0 'C,C This API is experimental and may have some slight breaking changes in the next releases. Args: auto_class (`str` or `type`, *optional*, defaults to `"AutoVideoProcessor "`): The auto class to register this new video processor with. rNz is not a valid auto class.) r^rrtransformers.models.automodelsautorrNr)r auto_class auto_modules r8register_for_auto_classz*BaseVideoProcessor.register_for_auto_classsC *c*#,,J66{J/ |+FGH H$r9video_url_or_urlsc d}tstjdd}t|tr0t t |Dcgc]}|j ||c}St|||Scc}w)z Convert a single or a list of urls into the corresponding `np.array` objects. If a single url is passed, the return value will be a single object. If a list is passed a list of objects is returned. torchcodecz`torchcodec` is not installed and cannot be used to decode the video by default. Falling back to `torchvision`. Note that `torchvision` decoding is deprecated and will be removed in future versions. r+rZ)backendrW)rwarningswarnr^r_r[rcr%)r6rrWrxs r8rczBaseVideoProcessor.fetch_videos4sx&( MMI $G ' .ars\]d//EV/Wstu u/Tef ftsA2 )NNr;)NFFNmain)F)AutoVideoProcessor)Br __module__ __qualname__rrrrr size_divisordefault_to_squarerrrrrrrrVrLrKrUr{rr}model_input_namesrr5r r>r rIr!rPfloatrTdictboolrr_rhrr rurBASE_VIDEO_PROCESSOR_DOCSTRINGr<rrr classmethodrPathLikerrtuplerrrrrrrrrrc __classcell__)r7s@r8r.r.s9 KHJI DLIINJNLN CJNOL./# !5#$#1L1 8"&"& 33$J35[4  3r)--1 &&&&&,&&+ && $d? && n  &&V<@!   !11D8 d  n   :&*#*#&*#  *#*#t37+h^$+h+h +h  +h L +h+h+h+h+h+hDK'$.+h4;&-+hj(4/+h !+hZ/3$!&#'`='*R[['8`=$t+`= `=  `= TzD `=`=`=D.-cBKK.?.-d.-`{,,/"++,={, tCH~tCH~- .{,{,z&#T#s(^&#&#P c3h  G G 03+< 0D+sR[['8++$%%2gcDIoT#Y.Ogr9r.rrzvideo processor file)object object_class object_files)Grrrcollections.abcr functoolsrtypingrnumpyrmhuggingface_hubrrhuggingface_hub.dataclassesrdynamic_module_utilsr image_processing_backendsr image_processing_utilsr image_utilsr rrrprocessing_utilsrrutilsrrrrrrrrrrr utils.hubrutils.import_utilsr video_utilsr r!r"r#r$r%r&r'r(rQ$torchvision.transforms.v2.functional transformsv2 functionalrDr) get_loggerrrr'r.r__doc__formatr3r9r8rEsD $8;490 3    #(    66/   H %A" H'" ,-p g+p g.  p gf"++=+I+I!J!!))5-?-K-K-S-S-Z-Z /CRh.[.""*6r9