RU2839709C1 - Method for spatially parameterized estimation of motion vectors - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to image processing, and more specifically to estimation of motion vectors in a sequence of photographic or video images. Method for estimating motion on an image field comprises steps of: obtaining at least two images from an input bit stream comprising a plurality of images, performing preliminary processing of images, estimating a map of M parameters of a motion estimation algorithm (ME) for said pair of images and performing motion estimation for said pair of images; wherein said preprocessing comprises, at least, brightness adjustment. applying preconfigured prediction model of spatial parameters of motion estimation algorithm to said pair of images, storing said map of parameters of the motion estimation algorithm in the form of an array having dimensions corresponding to the desired field of motion vectors (MVF); motion estimation is performed depending on the above motion estimation algorithm parameters map and the motion vector field is output in the form of two-dimensional (2D) array of displacements from the position of each pixel on the first image to at least one position of the corresponding pixel on the second image in the said pair of images. Also disclosed is a system which realizes this method.

EFFECT: high accuracy of estimating motion vectors over an image field.

17 cl, 8 dwg

Description

Field of technology to which the invention relates

The proposed invention relates to the field of image processing, and more specifically to the evaluation of motion vectors in a sequence of photo or video images. The invention can be used in various technologies where the evaluation of a dense field of motion vectors (MVF) is involved. In particular, the invention can be used in the field of computer vision, the formation of photo and video images, for example in the formation of images with an extended dynamic range (HDR) or in multi-frame shooting.

State of the art

Currently, when shooting high dynamic range (HDR) photos and videos, one of the possible technologies used is multi-frame shooting. However, since moving objects may be present in the scene when shooting several frames sequentially, the motion of these moving objects must be compensated for in order to form a high-quality final HDR image, especially when shooting at night at relatively long shutter speeds. Modern image processing algorithms use an estimate of the dense optical flow (motion vector field, MVF), which consists of a two-dimensional vector in each pixel of the image showing the movement of this pixel, in particular, using the so-called block matching.

Currently known image processing algorithms suffer, in particular, from the problem caused by the assumption of constant illumination when shooting a sequence of images.

Well-known motion estimation (ME) algorithms are based on the assumption of constant illumination parameters. However, the following characteristics of image shooting conditions in reality do not correspond to this assumption:

a) different exposure and image sensor gain settings on different frames within the HDR bracketing frame sequence;

b) flickering of electric lighting;

c) changes in natural lighting from frame to frame due to movement;

d) blurring due to movement of objects and/or the camera, as well as liquids flowing in the frame or flying steam;

d) automatic adjustment of white balance in the camera and/or shifting of color correction from frame to frame.

Another problem inherent in existing approaches to motion estimation is the generation of noise on the image sensor. Known block-matching-based motion estimation algorithms can estimate the motion vector field (MVF) with high accuracy, but they can be highly sensitive to the quality of the image content and the noise levels in it. In video images captured in low-light conditions, the applied recursive temporal noise reduction (TNR) process generates moving artifacts (the so-called “boiling noise”), which significantly reduce the perceived image quality. In night shooting of HDR images, frames captured with different exposure parameters may have different noise levels after brightness adjustment due to different combinations of shutter speed and image sensor sensitivity. The presence of noise and differences in noise levels across frames complicates image block matching, which is the core of block-matching-based ME algorithms.

When shooting under artificial lighting conditions, for example, with electric light sources, the scene being imaged may contain flickering, which is erroneously perceived by the motion vector estimation algorithm as motion in the frame. In the process of temporal noise reduction, the said noise generated on the image sensor may also lead to the formation of false motion vectors, to the accumulation of errors from frame to frame, and as a result to the violation of the contours of objects present in the image being formed, due to the said incorrect estimation of motion vectors.

The ME algorithm, based on matching blocks by the exhaustive search method, is widely known in technology. For each block in the first frame, the corresponding block in the second frame is found based on the minimum value of the error functional. SAD is often used as the error functional: the sum of the absolute values of the difference in the intensity of each pixel in the block. The disadvantages of the exhaustive search method are high computational costs and low accuracy of motion estimation, due to the fact that when matching blocks by exhaustive search, the physical nature of motion in the frame is not taken into account. To improve the accuracy of ME, error functional regularization methods are used, for example, a limitation is introduced on the total variation of the motion vector field.

The source US 10341574 B2 (Apple Inc., published 02.07.2019) discloses a method for capturing an image of a scene with an extended dynamic range (HDR), in which a plurality of images with a first image exposure level are captured and stored. Next, a first image with a second image exposure level is captured, and a second image is selected from said plurality of captured images. Based on the first and second images, a composite image of a scene with an extended dynamic range (HDR) is formed. For this purpose, image alignment is used based on the binarization of the brightness channel using a plurality of threshold values, wherein the image alignment is performed at full resolution and at least at one reduced image scale. Block matching is performed with respect to the above-mentioned pairs of images and essentially consists of comparing significant features in the images. The disadvantages of this method include the lack of regularization, which can cause the general problems described above associated with noise, false motion vectors, etc. The result of implementing this known method can be closer in characteristics to ME algorithms with a complete search.

US 20210306528 A1 (Oppo Mobile, published 09/30/2021) discloses a motion estimation method in which a plurality of interblock error functional values is obtained for a block to be compared and each of the reference blocks. At least one candidate block is determined among the reference blocks based on said plurality of errors. A range of random numbers is determined according to the offset between the block to be compared and each of the plurality of reference blocks. A plurality of random motion vectors are obtained using the motion vector between the block to be compared and each of the at least one candidate block as a base motion vector, as well as random numbers within said range of random numbers as increments. A target motion vector corresponding to the block to be compared is determined according to said random motion vectors. This block matching algorithm uses a matching loss function that is a combination of SAD, block edge gradient, Euclidean distance, and penalty term. Candidate blocks can be sorted based on different components of the mentioned loss function. The disadvantages of this known solution include the generation of too many random candidate motion vectors based on matching with the mentioned candidate blocks, which limits the performance of the system that uses this algorithm.

The source US 9153027 B2 (Nvidia Corp., published 06.10.2015) discloses a method for performing fast non-rigid alignment of at least two images in a stack of images to form an image with an extended dynamic range. The method includes the steps of forming a warped image based on a set of corresponding pixels, analyzing the warped image to detect unreliable pixels in the warped image, and forming corrected pixel values for each unreliable pixel in the warped image. The set of corresponding pixels includes a plurality of pixels of the original image, wherein each of said plurality of pixels is associated with a potential feature in the original image and is compared with a corresponding pixel in the reference image, which essentially corresponds to a pixel in the original image. The disadvantages of this known solution include the fact that the above-described real-time processing requires the use of a graphics processing unit (GPU) having characteristics corresponding to the level of desktop computers. In addition, this known solution directly performs motion compensation without constructing optical flow. This solution is not suitable for frame rate upconversion (FRUC) and other similar applications that require true motion estimation.

The source CN 115619816 A (Xiamen University, published 01/17/2023) discloses a method for dynamically measuring offset to eliminate interference from illumination, based on dynamic correction of a map of variable spatial illumination. The method comprises extracting a low-frequency illumination component from an image using an illumination map, constructing a two-dimensional gamma correction function and combining it with said illumination map, and tracking target objects in the corrected image based on template matching, wherein the method is characterized by accurately establishing the coordinates of pixels on the template. The intensity of dark zones in the space of illumination values is adjusted, due to which it is possible to display details of the input image under conditions of low uneven illumination. The disadvantages of this known solution include the limiting assumption that the true image is a low-frequency component, and light is a high-frequency modifier, which does not always correctly reflect the conditions of the real world when performing photo and video shooting.

The source CN 114862902 A (Liaoning University of Science and Technology, published 08/05/2022) discloses a method for extracting and matching features from an object-relational database (ORB) with self-adaptation to the illumination parameters of the imaged scene, wherein the method comprises obtaining parameters of the camera movement state, constructing an image pyramid (i.e. obtaining reduced images at scales of ½, ¼, etc. from the original image) in accordance with the camera movement state, extracting keypoints, adding or discarding them based on an improved quadratic tree algorithm, obtaining pairs of matched feature points, discarding erroneous pairs of feature points using a random sampling model parameter estimation algorithm (RANSAC). The disadvantages of this known solution include the fact that the method results in the formation of a sparse optical flow (MVF) (the full field of motion vectors is not provided), and the use of the RANSAC algorithm does not allow the use of this method in real time in the case of shooting images with 4K+ quality.

GB 2599217 B (Intel Corp., published 03/29/2023) discloses a method for estimating optical flow based on deep learning and merging matching portions of the optical flow. The method processes pairs of images, wherein the first and second images of each pair are divided into first and second sets of fragments, respectively. The process of generating optical flow based on the deep learning model is applied either to the third and fourth fragments of the first and second sets of fragments, respectively, or to the downsampled images corresponding to the first and second images, respectively, to generate second optical flow generation results, after which the first and second optical flow generation results are combined to form an optical flow map for the pair of images. The disadvantages of this known solution include the fact that deep learning-based motion estimation methods are memory and computing power intensive, which limits their use in mobile devices. In addition, the implementation of this method requires a special means for combining optical flows in matching fragments.

US 20210099727 A1 (Intel Corp., published 01.04.2021) discloses a method for estimating motion in a structure of adjacent blocks for encoding video data. It uses image blocks of variable sizes, as well as vertically oriented, horizontally oriented, and square shapes. Specified sets of candidates for different block sizes are used. The number of candidates is constant for each block size. However, this method does not provide the ability to adapt to the content of the image.

WO 2021025375 A1 (Samsung Electronics Co., Ltd., published 11.02.2021) discloses a motion estimation method based on a "pyramidal" exhaustive search method, after which the motion vector field is refined by removing outlier motion vectors using quadratic constraints for a vertex grid based on the image structure. The method receives a reference image and a non-reference image, divides the reference image into a plurality of blocks, determines a motion vector map using an algorithm for first coarsely estimating and then more accurately estimating the motion vectors, and generates an output frame using the motion vector map, the reference image, and the non-reference image.

The disadvantages of this method include the fact that it is based on a complete enumeration (a complete search over the entire image field) to find the original motion vectors, although it does use physically reliable regularization to correct the found motion vectors.

The source US 11343526 B2 (Realtek Semiconductor Corp., published 05/24/2022) discloses a method for processing video images, including dividing a frame into a plurality of blocks, forming a motion vector for each block from the plurality of blocks based on the current block of this frame and the corresponding block of the previous frame. Next, a global motion vector is formed according to the obtained set of motion vectors, the sum of the absolute pixel differences is calculated for each block of the current frame based on said global motion vector. The distribution of the sum of the absolute pixel differences in the area containing the set of blocks of the current frame is compared with a set of models to determine the most suitable model, and based on the selected most suitable model, each block in said area of the current frame is marked as either a foreground block or a background block. In this known method, motion segmentation is based on the SAD distribution, as well as on refining the field of motion vectors in the boundary area of the moving object. The disadvantages of this method include the need to first perform a global motion estimate, as well as an SAD estimate for each block relative to the global motion, and a search for a model that best fits the distribution mentioned.

The solution according to source US 20210306528, described above, can be considered as the closest analogue of the proposed invention.

Given the shortcomings of the state of the art, a solution is needed to improve the accuracy of optical flow (motion vector field, MVF) estimation under changing illumination conditions and over a wide range of image noise levels.

Disclosure of invention

This section, which discloses various aspects of the claimed invention, is intended to provide a brief overview of the claimed objects of the invention and their embodiments. Below are detailed characteristics of the technical means and methods by which the combinations of features of the present invention are realized. Neither this disclosure of the invention, nor the detailed description given below together with the accompanying drawings, should be considered as determining the scope of legal protection of the claimed invention. The scope of legal protection of the invention is determined only by the following claims.

The technical problem solved by the present invention consists in reducing the influence of complex shooting conditions (different illumination, different exposure and different noise levels between frames, flickering electric lighting) on the assessment of optical flow when shooting a photographic image with an extended dynamic range (HDR) or a video image.

The technical result achieved by using the claimed invention consists in increasing the accuracy of motion vectors estimation across the image field. In addition, high system performance is ensured when estimating motion vectors across the image field.

The problem solved by the invention consists in creating a method and system for spatially parameterized evaluation of the field of motion vectors (formation of optical flow), in which the shortcomings of the above-described known solutions from the state of the art would be at least partially or completely eliminated.

In a first aspect of the present invention, said problem is solved by a method for estimating motion in a sequence of images, comprising the steps of: obtaining at least two images from an input bitstream containing a plurality of images, performing pre-processing of the images comprising at least a brightness adjustment; estimating a map M of parameters of a motion estimation algorithm (ME) for said at least two images using a pre-configured model for predicting spatial parameters of a motion estimation algorithm for said at least two images, wherein the dimensionality of the map M of parameters of the motion estimation algorithm depends on the method of dividing the image into blocks; storing the map M of parameters of the motion estimation algorithm; performing motion estimation by applying the motion estimation algorithm and the parameters from the map M of parameters of the motion estimation algorithm to each block of said at least two images, wherein the map M of parameters of the motion estimation algorithm determines the parameters of the motion estimation algorithm separately for each block; and outputting a motion vector field (MVF) in the form of a two-dimensional (2D) array of offsets from the position of each pixel in the first image to at least one position of a corresponding pixel in the second image of said at least two images.

Preliminary processing of said at least two images may comprise a step of image scaling with formation of an image pyramid with at least two reduced scales. Image scaling may comprise a step of performing spatial filtering of pixels of the raw image. Obtaining said at least two images from a bitstream may further comprise a step of obtaining at least one of a preliminary MVF estimate, coded metadata related to image shooting parameters, sensor data not related to image formation, or a step of obtaining at least one of brightness channel data, three-channel color images in RGB or YUV color spaces.

In one or more embodiments, the pre-configured model for predicting spatial parameters of the motion estimation algorithm has the architecture of a convolutional neural network or a discriminative or regressive machine learning model. The motion estimation algorithm can be selected from at least one of an algorithm based on matching blocks by an exhaustive search method, using the sum of the absolute values of the difference in the intensity value of each pixel in the block (SAD) as the minimum value of the error functional, a 3D recursive search (3DRS) algorithm, a block erosion algorithm with sub-pixel refinement of motion vectors (MV), a sub-pixel discretization algorithm of candidate blocks.

In a second aspect, said problem is solved by a system for estimating motion in a sequence of images, comprising: an image pre-processing unit configured to pre-process at least two images obtained from an input bitstream containing a plurality of images; a unit for generating a map of M parameters of a motion estimation algorithm (ME) for said at least two images, configured to generate a map of M parameters (ME) using a pre-configured model for predicting spatial parameters of the motion estimation algorithm and storing the map of M parameters, wherein the dimensionality of the map of M parameters of the motion estimation algorithm depends on the method for dividing the image into blocks; a motion estimation unit configured to estimate motion on said at least two images by applying a motion estimation algorithm and parameters from the map of M parameters of the motion estimation algorithm to each block of said at least two images, wherein the map of M parameters of the motion estimation algorithm determines the parameters of the motion estimation algorithm separately for each block; and outputting a motion vector field (MVF) in the form of a two-dimensional (2D) array of offsets from the position of each pixel in the first image to at least one position of a corresponding pixel in the second image of said at least two images.

The system may additionally comprise a unit for training a model for predicting spatial parameters of the motion estimation algorithm, configured to train said model for predicting spatial parameters of the motion estimation algorithm by iteratively executing the parameter estimation algorithm with N parameter variants and predicting the parameter value for each of the image blocks that yields the best result. The parameter value for each of the image blocks that yields the best result may be determined by calculating the loss function between the vectors of the motion vector field and the reference motion vectors.

The system can be further configured to obtain at least one of the MVF preliminary estimate, coded metadata related to the image shooting parameters, sensor data not related to image formation. The system can be configured to obtain a pair of images in the form of at least one of the brightness channel data, three-channel color images in the RGB or YUV color spaces.

The pre-configured model for predicting spatial parameters of the motion estimation algorithm may have the architecture of a convolutional neural network or a discriminative or regressive machine learning model. In one or more embodiments of the invention, the motion estimation algorithm may be selected from at least one of an algorithm based on matching blocks by an exhaustive search method, using the sum of the absolute values of the difference in the intensity value of each pixel in the block (SAD) as the minimum value of the error functional, a 3D recursive search algorithm (3DRS), a block erosion algorithm with sub-pixel refinement of motion vectors (MV), an algorithm for sub-pixel discretization of candidate blocks.

In a third aspect, the invention relates to a method for training a model for predicting spatial parameters of a motion estimation algorithm, comprising the steps of: selecting an index k of a pair of frames; calculating an estimated map of parameters y: y=Y(SRC_k, REF_k), where SRC_k- source frame, REF_k- reference frame; form the motion vector field MVF_k(y) from MVF_{i, k}, where i=1..N, relative to the estimated parameter map y by calculating the weighted sum MVF_1..N, to evaluate the output of the spatially adaptive ME algorithm; the loss function L between the vectors of the motion vector field MVF is calculated_{y, k}and reference motion vectors GT_k; updating the prediction model of spatial parameters of the motion estimation algorithm by backpropagating the gradient of the loss function L; determining whether the required convergence in the prediction model of spatial parameters of the motion estimation algorithm has been achieved; and storing the prediction model of spatial parameters of the motion estimation algorithm in memory.

At the stage of forming the field of motion vectors, weighting coefficients obtained by applying the Softmax function can be used to calculate the weighted sum.

In a fourth aspect, the invention relates to a system for training a model for predicting spatial parameters of a motion estimation algorithm, comprising: a rendering unit configured to render frame sequences with a reference field of motion vectors (GT); a motion estimation unit (ME) configured to estimate motion for said frame sequences with each set of parameters from a map M of motion estimation algorithm parameters; a unit for training a model for predicting spatial parameters of a motion estimation algorithm, configured to train a model for predicting spatial parameters of a motion estimation algorithm by calculating a loss function L between vectors of the field of motion vectors generated by the motion estimation unit and the reference motion vectors, updating the model for predicting spatial parameters of a motion estimation algorithm by backpropagating the gradient of the loss function L, determining whether the required convergence in the model for predicting spatial parameters of a motion estimation algorithm has been achieved; and a unit for storing weight coefficients of the trained model for predicting spatial parameters of a motion estimation algorithm.

It is obvious to those skilled in the art that the inventive concept is not limited to the aspects set forth above, and the invention may take the form of other inventive objects, such as at least one device, a computer program or a computer program product comprising a computer-readable medium on which the computer program is recorded. Additional features that may characterize specific embodiments of the present invention will be obvious to those skilled in the art from the detailed description of the embodiments given below.

Brief description of the drawings

The drawings are provided herein to facilitate an understanding of the present invention. The drawings are schematic and not drawn to scale. They are for illustrative purposes only and are not intended to define the scope of the present invention.

Fig. 1 is a block diagram of a method for estimating movement across an image field according to one or more embodiments of the invention;

Fig. 2 is a block diagram of the process of configuring a model for predicting spatial parameters of a motion estimation algorithm in accordance with one or more embodiments of the invention;

Fig. 3 is a block diagram of the process of training a model for predicting spatial parameters of a motion estimation algorithm in accordance with one or more embodiments of the invention;

Fig. 4 is a schematic representation of possible variants of the sizes and orientations of the motion block windows in the adaptive motion estimation algorithm according to various embodiments of the invention;

Fig. 5 is a schematic diagram of a system for estimating motion in a sequence of images according to one or more embodiments of the invention;

Fig. 6 is a schematic diagram of a system for training a model Y for predicting spatial parameters of a motion estimation algorithm according to one or more embodiments of the present invention;

Fig. 7 is a schematic representation of the structure of a model for forming a map of spatial parameters of a motion estimation algorithm according to at least one embodiment of the invention;

Fig. 8 is a schematic representation of the structure of a model for forming a map of spatial parameters of a motion estimation algorithm according to at least one other embodiment of the invention.

Implementation of the invention

Illustrative embodiments of the present invention are described in detail below and are illustrated in the accompanying drawings, in which the same or similar reference numerals designate the same or similar elements or elements that have the same or similar functions. Illustrative embodiments described with reference to the accompanying drawings are illustrative only and are used only to explain the present invention, and should not be considered in terms of any limitations on the scope of the invention.

According to the invention, an algorithm for estimating a dense optical flow or motion vector field (MVF) is proposed, which uses block matching. Such an algorithm may be in demand in many areas of application, such as computer vision, as well as for reducing noise in images, compressing video data, reconstructing 3D scenes, SLAM, recognizing/detecting/segmenting objects in a video image, in the field of augmented or virtual reality (VR/AR), controlling the movement of autonomous vehicles, etc. The algorithm according to the invention may be implemented as at least one software module, at least one hardware block, or any combination of software modules and hardware blocks that may be obvious to a person skilled in the art. It should be understood that the invention is not limited to a specific combination of hardware and software blocks used for its implementation in each possible embodiment of the invention.

According to the invention, the input data for estimating the motion vector field (MVF) are at least 2 frames of a photo or video image. The output data are a motion vector field (optical flow), which is a two-dimensional (2D) array of displacements from the position of each pixel in the first frame to the position of the corresponding pixel in the second frame.

In this case, as will be shown below, the accuracy of the MVF estimation in accordance with the invention is ensured by using adaptive motion estimation (ME), which allows, in particular, to adapt the algorithm used to:

- different noise levels in different frames due to different ISO/gain/shutter times when shooting a series of frames in HDR bracketing;

- different exposure parameters in a series of frames to obtain an HDR image;

- changes in natural lighting between frames of a series caused by movement;

- motion blur in one or more frames;

- automatic white balance adjustment and/or in-camera color correction shifts between frames; and

- the magnitude (intensity) of movement.

The proposed invention uses an improved ME algorithm based on block matching. For each block in the first frame, the corresponding block in the second frame is found based on the minimum value of the error functional. SAD is often used as the error functional: the sum of the absolute values of the difference in the intensity of each pixel in the block.

In addition, the proposed algorithm is characterized by high performance due to the elimination of the need for additional ME iterations and is based on the use of a lightweight model with direct connections for predicting a spatial map of the parameters of the motion estimation algorithm, on the basis of which the motion is adaptively estimated in each block using the selected parameter of the algorithm.

Method for Estimating Motion in Image Sequences

In a first aspect, the present invention relates to a method for estimating motion over an image field, receiving at least a pair of images from an input bitstream containing a plurality of images at the input, and outputting at the output a field of motion vectors in the form of a two-dimensional (2D) array of offsets from the position of each pixel in the first image to at least one position of the corresponding pixel in the second image in said pair of images. It should be noted that in specific embodiments of the invention, as an alternative or in addition to said at least two images, the method may comprise obtaining at least one image frame. In one or more embodiments of the invention, obtaining a pair of images may further comprise a step of obtaining at least one of a preliminary MVF estimate, encoded metadata related to image shooting parameters, sensor data not related to image formation. In one or more embodiments, obtaining a pair of images may comprise obtaining at least one of brightness channel data, three-channel color images in RGB or YUV color spaces.

In general, in one or more embodiments of the invention, the method according to the first aspect of the present invention comprises steps S1-S12, which will be discussed in detail below with reference to Fig. 1.

At step S1, a pair of images is obtained from an input bitstream containing a plurality of images. Those skilled in the art should understand that in specific non-limiting embodiments of the invention, the input bitstream may contain both a video image and a sequence of one or more still (photo) images. As described above, the pair of images from the input bitstream may be represented by frames (images) in an unprocessed and uncompressed format (as a non-limiting example, the Color Filter Array - CFA format), or in one of the other known formats, wherein the images may represent, as a non-limiting example, three-channel color images in the RGB color space (red, green and blue color channels) or YUV (containing a luminance component Y and two color difference components U and V, wherein the YUV components can be determined based on the RGB components as YCbCr).

In step S2, pre-processing of the pair of images obtained in step S1 is performed. As a non-limiting example, the pre-processing may comprise at least one of brightness adjustment and at least one scaling step.

Thus, in one or more non-limiting embodiments, the brightness adjustment of a pair of images containing a reference (REF) image and a source image (SRC) may be performed taking into account the following metadata:

- exposure index EV _ref , EV _src

- values of exposure time T _ref , T _src

- values of the total gain coefficient G _ref , G _src

- bit resolution per pixel BPP

- black level BL

Brightness adjustment is necessary for approximate alignment of pixel intensities with different exposure levels to enable block comparison, for example, using the SAD function. For example, if the first frame is taken with exposure 1, and the second frame with exposure 16 times less, then the pixel intensities of the second frame must be multiplied by 16.

Brightness adjustment can be done, for example, as follows:

Zooming out is essentially the process of forming a "pyramid" of images with different levels of zoom out:

- having a reference image in the CFA Î _ref format (Bayer mosaic), at least a first frame with a reduced scale and a second frame with a reduced scale in the YUV format are formed (in the preferred embodiment, ½ and ¼ of the scale of the original image, respectively): Î _ref ¹ , Î _ref ²

- having an original image in the CFA format Î _src (Bayer mosaic), at least a first frame with a reduced scale and a second frame with a reduced scale in the YUV format are formed (in the preferred embodiment, ½ and ¼ of the scale of the original image, respectively): Î _src ¹ , Î _src ² .

At step S3, a pre-configured model Y of the prediction of spatial parameters of the motion estimation algorithm is received at the input. The process of training the model Y of the prediction of spatial parameters of the motion estimation algorithm according to one or more specific embodiments of the invention is described below. It should be noted that in one or more embodiments of the invention, the model Y of the prediction of spatial parameters of the motion estimation algorithm can be stored in the same system that implements the method according to the invention, or can be received from external sources, in particular from one or more external servers, computing devices, etc. via one or more network connections.

At step S4, a prediction of a map M of parameters of the motion estimation algorithm (ME) is performed for said pair of images. For these purposes, a pre-configured model for predicting spatial parameters of the motion estimation algorithm is applied to said pair of images.

The dimension of the map M of the parameters of the motion estimation algorithm depends on the method of dividing the image into blocks. It should be noted here that the scope of legal protection of the invention is not limited to specific methods of dividing images into blocks, which are themselves well known in this field of technology. The sizes and shapes of the blocks can also be different, as a non-limiting example - the blocks can be square and/or rectangular, can have the same shape and size throughout the image or different shapes and sizes depending on a specific section of the image, etc. In this case, the map M of the parameters of the motion estimation algorithm provides a prediction of the parameter for each block in the image regardless of the shape, size of a specific block, its location in the image, etc.

Various models can be used as a model for predicting the spatial parameters of the motion estimation algorithm, including, but not limited to, models with a convolutional neural network architecture, but essentially any suitable classifier can be used to predict the spatial parameters of the motion estimation algorithm, such as a machine learning model, including, but not limited to, a logistic regression model, a decision tree, etc.

In addition, in one or more non-limiting embodiments of the invention, an algorithm from the family of 3D recursive search (3DRS) algorithms may be used for motion estimation, comprising forming a "pyramid" by using an image with the original resolution, scaling the image two- or four-fold with a decrease in resolution (which is also called downscaling in the context of the present invention) and alternately estimating the motion in the obtained frames; wherein at a certain level of the said "pyramid" a different structure of motion vector candidates and a number of iterations are used.

In addition, a block erosion algorithm with subpixel motion vector (MV) refinement or a subpixel candidate block sampling algorithm can be used.

In at least one non-limiting embodiment, for estimating the map M, a pair of input frames (reference and original) are obtained at a first spatial resolution: Î _ref 1 , Î _src 1. From these frames, the brightness channel data are extracted: Y _ref , Y _src . A pre-configured model, such as, for example, a convolutional neural network, can be applied to this data. The input data of the model are a 2-channel tensor (2xHxW) formed from Y _ref , Y _src (where H, W are the height and width of the processed image fragment). At the output, the model produces a tensor N x H/8 x W/8 (where N is the power of the set of admissible values of the set of parameters ME). The map M of parameters is obtained by executing the argmax function over N channels, and thus M is a matrix H/8 x W/8. Each element of the matrix M can take one of N values.

The resulting parameter map is saved as an array M of data, for example, of integer type.

At stage S5, the iteration counter of the motion estimation (ME) algorithm is initialized - S=0. At stage S6, the indices of the blocks of the processed images are initialized - x, y=0, 0.

In step S7, motion estimation (ME) is performed for each block (x, y) using the parameter M[x, y] from the map M of motion estimation algorithm parameters estimated in step S4.

For this purpose, in one or more embodiments of the invention, a 3D recursive search (3DRS) algorithm is used. The algorithm is initialized, and a first spatial resolution (as a non-limiting example, an image scale of ¼) is selected for the frames of the image "pyramid" formed in the image preprocessing step S2.

At the scale of ¼, the motion from the first frame to the second frame in the corresponding frame pair is estimated using the parameter downsampled for the corresponding block with index x, y from the parameter map M. At the scale of ¼, the motion from the second frame to the first frame in said frame pair is estimated, again using the parameter downsampled for the corresponding block with index x, y from the parameter map M. Then, a second spatial resolution is selected for the frames of the image "pyramid" formed in the step S2 of image preprocessing (as a non-limiting example, the image scale of ½). The motion from the second frame to the first frame in said frame pair is estimated at the scale of ½ using the parameter from the parameter map M in accordance with the index of this block in the 3DRS algorithm. The main parameter of the 3DRS motion estimation algorithm is the absolute value of the per-pixel sum of absolute differences of blocks (SAD), to which a certain threshold clipping value (SadClippingValue) is applied, which in one or more embodiments of the invention may be in a certain range (as a non-limiting example - 0..19 (N=20)).

At this stage, the motion vector for the current block (x, y) is selected from the candidate vectors.

As alternative parameters of the adaptive ME algorithm in one or more embodiments, in addition to the above-mentioned SAD cutoff value, various other parameters can be used, such as, as a non-limiting example, the size or shape of the SAD window, the structure (pattern) or orientation of the windows or image blocks (see Fig. 4). In particular, the location of the motion vectors (MV) candidates. The sizes and shapes (aspect ratio of the rectangle) of the windows of the motion estimation blocks in accordance with various embodiments of the invention can be selected from: 8×8, 16×16, 16×24, 24×16, 24×24 or other values of the number of pixels on two sides of the rectangle (N=5). The value N denotes the power of the set of admissible values of the adaptive ME parameter.

In one or more non-limiting embodiments of the invention, the ME algorithm parameter may be a structure of MV candidates, including the number and arrangement of MV candidates. The number may be, for example, equal to 5 or 10 MV candidates per block. In addition, various types of MV candidates may be used - spatial, temporal, randomly updated (RandomUpdate), hierarchical, etc. The ME algorithm can also take as a parameter the range of the random number generator (random search range) of MV candidates, as well as the priority of the candidates (e.g., spatial MV candidates bordering neighboring blocks, or MV candidates from neighboring frames (i.e., temporal MV candidates, or candidates from another image scaling level (e.g., from the level of the "pyramid" of lower-resolution images)). The above-mentioned different types of MV candidates can be ranked by priority and considered in different orders.

The type of loss function for matching blocks may be different in different parts of the image, and in one or more embodiments, different functions may be used adaptively. According to the invention, the model is configured to determine which function should be used in a given block based on a statistical criterion obtained during training of the model. For example, the sum of absolute differences (SAD) function may be used:

In one or more other embodiments of the invention, a Euclidean difference sum function may be used:

Also, for example, the Census loss function (see, for example, https://courses.cs.duke.edu/spring06/cps296.1/handouts/Zabih%20Woodfill%201994.pdf) can be used, which generalizes the local structure of the image, for example:

or, as a special case of it, the Triple Census loss function:

At step S8, the indices of the blocks x, y are updated in order to further estimate the motion vectors for the following blocks.

At step S9, it is checked whether all blocks have been processed by the algorithm. If not, the process returns to step S7 and is repeated for blocks with the next values of the x, y indices. If yes, the process goes to step S10, where the motion estimation (ME) algorithm iteration counter is updated (1 is added to the value of S), initialized at step S5, and the process goes to step S11, where it is checked whether all iterations of the motion estimation algorithm cycle have been completed. If yes, the final motion vector field (MVF) is output at step S12. If not, the process goes to step S6 of initializing the indices of the blocks of the x, y images being processed.

It should be noted that the method according to the invention is not limited to any specific parameters of the ME algorithm given above or their combinations. In addition, other parameters of the ME algorithm and their combinations may be obvious to specialists in this field of technology, which can also be used in the process of estimating the motion vectors according to the invention, without going beyond the scope of legal protection of the invention.

Configuring the spatial parameter prediction model of the motion estimation algorithm

Fig. 2 shows a flow chart of the process of preparing a training data set for configuring a model for predicting spatial parameters of a motion estimation algorithm in accordance with one or more embodiments of the invention.

At step S11, a synthetic training set of original data is received as input data, comprising K training examples, each of said examples comprising at least one reference frame, one original frame and a true motion vector field (GT). In addition, each training example may comprise one or more additional reference frames, metadata described above and data from sensors not associated with image capture. At step S12, the counter of elements of the data set k=0 is initialized. At step S13, a pair of synthetic frames REF _k (reference frame) and SRC _k (original frame) is obtained from the synthetic data set. At step S14, the set of parameters ME P={p1, p2, ..pN}, i=1 is initialized.

In step S15, motion estimation (ME) is performed using the parameter p. In step S16, the result of motion estimation is stored in the motion vector field MVF _{i, k} . In step S17, the value of i is updated by incrementing by one. Comparison of the value of i with N (where N is the cardinality of the set of admissible parameters of ME) yields the result either i < N, in which case the process returns to step S15 and ME is performed again using the parameter p with the incremented i, or i=N, in which case the process proceeds to step S18, where the value of k is updated by incrementing by one.

Comparison of the current value of k with the value of K for the synthetic training set of motion data used may yield the result k < K, in which case the process returns to step S13, where a pair of synthetic frames REF _k and SRC _k are obtained from the synthetic data set and repeated again, or k=K, in which case the process proceeds to step S20 of transmitting the prepared training set of data to the model training system for further training.

Training a model for predicting spatial parameters of a motion estimation algorithm

Fig. 3 shows a block diagram of the process of training a model for predicting spatial parameters of a motion estimation algorithm in accordance with one or more embodiments of the invention. As a result of training, the model is capable of constructing a field of motion vectors from motion vectors obtained at different parameter values, resulting in a control signal for deep training of the model. In one or more non-limiting embodiments, the model for predicting spatial parameters of a motion estimation algorithm has a convolutional neural network architecture. In addition, as indicated above, another discriminatory or regression machine learning model may be used in the method according to the invention.

At step S21, a model Y for predicting spatial parameters of the motion estimation algorithm is initialized. In one or more specific non-limiting embodiments, the model may be a convolutional neural network. The input data at the model initialization step is a 2-channel tensor ([C x F] x 128×128) (where the value of F corresponds to the number of input images, for example F=2 for the case of one reference image plus one original image, and the value of C corresponds to the number of color channels in the input images, for example C=1 for the case of using one brightness channel of the input images). The output data is a tensor [2 x N] x 32×32 (where the value "2" corresponds to the dimension of the 2-dimensional motion vector, and N is the power of the set of admissible ME parameters).

At step S22, the index k of a pair of frames is selected.

At step S23, the estimated parameter map is calculated:

y=Y(SRC _k , REF _k )

where SRC _k is the source frame, REF _k is the reference frame.

To do this, a random element is selected from the training data set. A 128×128 pixel image area is cropped, and a corresponding motion vector field (MVF) of dimension 32×32 is formed from the reference motion vector field (GT) and MVF _1…N .

At step S24, the motion vector field MVF _k (y) is constructed from MVF _{i, k} , where i=1..N, relative to the estimated parameter map y calculated at step S23.

This stage contains the calculations of the so-called forward pass through the model. The output data is passed through the Softmax function over the entire channel dimension. The output data of the layer implementing the Softmax function are used to calculate the weighted sum of the MVF_1..N, to evaluate the output of the spatially adaptive ME algorithm.

Next, in step S25, the loss function (error) L is calculated between the vectors of the motion vector field MVF _{y, k} and the motion vectors for the reference image GT _k . The above-mentioned weighted sum MVF _1..N obtained in the previous step is compared with the motion vector field for the reference image using one of the loss functions known in the art (as a non-limiting example, the SmoothL1Loss function).

In step S26, the model Y of the prediction of the spatial parameters of the motion estimation algorithm is updated by backpropagating the gradient of the error L. In this case, the weight coefficients of the model are updated, as is known in the art. In this case, in one or more embodiments of the invention, in particular, the Adam optimization algorithm can be used.

In one or more specific non-limiting embodiments of the invention, the block matching algorithm may use an ME algorithm with an exhaustive search, or an algorithm with an exhaustive search over the entire “pyramid” of images, or other suitable algorithms known in the art, such as the PatchMatch algorithm for finding correspondences between regions.

In step S27, it is determined whether the required convergence in the model Y of the spatial parameter prediction of the motion estimation algorithm has been achieved. If convergence has been achieved, the process proceeds to step S28; if not, the process returns to step S22 of selecting the index k of the pair of frames and is repeated again.

At step S28, the obtained model Y of predicting the spatial parameters of the motion estimation algorithm is stored in memory. The obtained model is configured to construct a field of motion vectors from the motion vectors obtained for different values of the said parameters, thus providing a control signal for deep learning of the model.

As a non-limiting example, a model Y for predicting spatial parameters of a motion estimation algorithm may be configured as follows.

First, pairs of frames are formed in which movement is present, using any suitable means of generating 3D graphics (as a non-limiting example, the Blender program).

The following requirements are imposed on the personnel:

- Images in frames must contain a hemispherical background with extended dynamic range (HDR) and high resolution, as well as moving 3D objects;

- The camera movement must be configured in an appropriate program (e.g. Blender software) in such a way as to simulate the shaking of the user's hands holding the camera;

- The speed of objects must also be configured in the corresponding software in such a way that it corresponds to the range of characteristic speeds of objects in the real world;

- Frames must be rendered at a resolution appropriate to the intended use, such as, but not limited to, 3000x4000 x 16 bits per pixel;

- The optical field of the reference image at the output must be configured in the appropriate software (i.e., for example, Blender );

- Rendered frames must be reverse processed to simulate a CFA image from the camera sensor with appropriate noise and exposure parameters.

The set of ME parameters in this context should be understood as a set of categories (e.g. for the SAD cutoff - P={0, 1, 2, …, 19}, where the numbers indicate the class index of the parameter value). Motion estimation is performed for each generated frame pair, and with each parameter specified in the set P. The resulting MVF is stored separately for each combination of frame pair and parameter. The final training dataset contains the following elements:

- reference frame;

- original frame;

- MVF of the reference image;

- MVF for parameter 1;

- MVF for parameter 2;

...

- MVF for parameter N.

In one or more embodiments of the invention, a set of parameters P={p1, p2, ..pN} is initialized. In this case, the parameters p belong to the categorical class. A loop is initialized that goes through the entire set of parameters P (index i). Motion estimation is performed for frames REF _k , SRC _k using the global parameter p for the ME algorithm. The obtained field of motion vectors is stored for the tensor MVF _{i, k} . The loops end upon reaching the values i and k. In this way, at least a training data set is formed from MVF _{i, k} data tensors, where i=1..N; k=1..K.

The motion vector field MVF _k(y) is constructed from MVF _{i, k} , i=1..N with respect to the map parameter y. To ensure the possibility of backpropagation of the error gradient during training, MVF _k(y) is constructed as a weighted sum of MVF _{i, k} , where the corresponding value of y is a weighting coefficient. During testing (application) of the model, MVF _k(y) is constructed by applying the agrmax operation in the section i=1..N.

Fig. 7 shows a schematic representation of the structure of the model for forming a map of spatial parameters of the motion estimation algorithm according to at least one embodiment of the invention, where two values of the brightness channel (Y) are entered at the input of the model (for the reference and original frames, respectively), and a map of parameters M of the motion estimation algorithm is output at the output. Fig. 8 shows a schematic representation of the structure of the model for predicting spatial parameters of the motion estimation algorithm according to another non-limiting embodiment of the invention, which differs from the previous embodiment in that a pre-estimated field of motion vectors MVF is entered into the model, which is subjected to processing by a series of convolutional filters up to a filter block size of 128x32x32 and is concatenated with the values of the filter block of brightness channels of the same dimension.

Motion assessment system in a sequence of images

In a second aspect, the present invention relates to a system for estimating motion in a sequence of images, which is designed to implement the method according to the first aspect of the invention described above. A basic diagram of the system for estimating motion in a sequence of images is shown in Fig. 5.

Referring to Fig. 5, the system for estimating motion in a sequence of images comprises at least the following elements. Reference 101 denotes a lens of a camera capturing an image for which it is necessary to perform motion estimation (original image). Reference 102 denotes an image sensor on which an unprocessed image of a three-dimensional scene of the real world is formed on the basis of optical radiation captured by the camera through the lens 101, preferably in the CFA format corresponding to the image from the image sensor without processing and compression. Reference 103 denotes a CFA buffer in which the images in CFA format captured by the camera are temporarily stored. Reference 104 denotes an image pre-processing unit that receives images from the CFA buffer 103 for performing image pre-processing, in particular consisting of adjusting the brightness and/or other image parameters, as well as converting the images into other formats suitable for further processing, as a non-limiting example - into the YUV format.

The YUV buffer is designated by position 105, in which the images that have undergone transformation in the preliminary image processing block 104 are temporarily stored. The block for forming a spatial map of the parameters of the motion estimation algorithm is designated by position 106, and the block for storing the spatial map of the parameters of the motion estimation algorithm, formed by the block 106 for forming a spatial map of the parameters of the motion estimation algorithm, is designated by position 107.

The position 108 denotes a motion estimation unit (ME), configured with the possibility of forming and outputting a field of motion vectors in the form of a two-dimensional (2D) array of offsets from the position of each pixel in the first image to at least one position of the corresponding pixel in the second image in the mentioned pair of images. The position 109 denotes a buffer of motion vector fields (MVF), configured with the possibility of temporarily storing MVFs formed by the ME unit 108.

The symbols 1...8 in Fig. 5 indicate a sequence of data transmission between the blocks of the motion estimation system in a sequence of images in accordance with one or more embodiments of the invention. The symbol RAM indicates that the blocks 103, 105, 107 and 109 described above can be implemented by means of one or more random access memories (RAM). The part indicated in Fig. 5 by the symbol CPU indicates that the corresponding blocks 104, 108 can be implemented by means of at least one processor, such as a general-purpose central processor. The part indicated in Fig. 5 by the symbol GPU/NPU/DSP indicates that the corresponding block 106 can be implemented by means of at least one specialized processor, such as a graphics processor (GPU), a neural processor (NPU) or a digital signal processor (DSP).

It should be understood by those skilled in the art that the scope of the invention is not limited to the above-listed material and technical means implementing the components of the system according to the invention, and other means can be used to implement the above-listed blocks, such as one or more integrated circuits (ASIC), field programmable gate arrays (FPGA), one or more microprocessors, etc. In addition, other components can be provided in the system in addition to those listed above, for example, at least one read-only memory (ROM), which can be implemented by one or more types of memory known to those skilled in the art. It should also be understood that the above-listed blocks can be implemented in practice in the form of various combinations of hardware and software, as applicable in the context of the present invention.

Model Y learning system for predicting spatial parameters of motion estimation algorithm

In a further aspect, the present invention may relate to a system for training a model Y for predicting spatial parameters of a motion estimation algorithm, which is configured to be used in the method according to the first aspect and/or in the system according to the second aspect of the present invention for training the model Y for predicting spatial parameters of a motion estimation algorithm to form a motion vector field (MVF) in the respective embodiments of the present invention. A schematic diagram of the system for training the model Y for predicting spatial parameters of a motion estimation algorithm according to one or more embodiments of the present invention is shown in Fig. 6.

Referring to Fig. 6, the system for training the model Y for predicting spatial parameters of the motion estimation algorithm comprises at least the following elements. The reference numeral 201 in Fig. 6 denotes a rendering workstation. The reference numeral 202 denotes a server processor node (CPU), the reference 203 denotes a server processor node with a graphics accelerator (GPU), the reference 204 denotes a long-term memory. The symbols 1...8, 7' in Fig. 6 indicate a sequence of data transmission between the units of the system for training the model Y for predicting spatial parameters of the motion estimation algorithm in accordance with one or more embodiments of the invention.

Position 210 denotes a rendering unit configured to render frame sequences with a reference field of motion vectors (GT). Position 211 denotes a unit for storing linear TIFF files obtained as a result of the rendering program; unit 212 is a unit for storing binary MVF files corresponding to the reference fields of motion vectors (GT). Position 221 denotes a unit for inverse processing for modeling an image in the CFA format. Position 222 denotes a unit for motion estimation (ME) with each set of parameters. Position 223 denotes a unit for storing CFA files (with their inherent noise, exposure parameters), position 224 denotes a unit for storing motion vector fields MVF _{i, k} .

The model training unit is designated by reference numeral 231 in Fig. 6. The data set generation unit is designated by reference numeral 232. The trained model weight coefficient storage unit is designated by reference numeral 233. As in the system according to the second aspect of the present invention described above, those skilled in the art should understand that the scope of the invention is not limited to the above-listed material and technical means implementing the components of the system according to the invention, and other means can be used to implement the above-listed units, such as one or more integrated circuits (ASIC), field programmable gate arrays (FPGA), one or more microprocessors, etc. In addition, other components can be provided in the system in addition to those listed above, for example, at least one random access memory (RAM), which can be implemented by one or more types of memory known to those skilled in the art. It should also be understood that the above-listed units can be implemented in practice in the form of various combinations of hardware and software, as far as applicable in the context of the present invention.

The above-described system for training the Y model for predicting spatial parameters of the motion estimation algorithm is designed with the ability to obtain training examples of the ME algorithm execution taking into account various sets of parameters. In this case, the proposed method, in the implementation of which the system for training the Y model for predicting spatial parameters of the motion estimation algorithm is used, can be used in a wide range of different block matching algorithms. In this case, the method according to the invention and the corresponding systems implementing the method do not depend on the nature of specific parameters. A wide selection of parameters and their combinations can be used in a network with unified training, non-limiting examples of which are given above.

In specific applications, the invention can provide the following useful effects. For example, when reducing noise in a video image under very low lighting, the invention, with a slight increase in computational complexity compared to traditional block matching algorithms for motion estimation, allows processing video images with a frame repetition rate (FPS) of 30 frames per second, with a resolution of 4K in real time. At the same time, the GPU uses approximately ~250 thousand weighting coefficients and a typical convolutional model with one forward pass.

Night-time HDR imaging provides a small increase in computational complexity compared to traditional block-matching ME algorithms while still allowing HDR photo processing to be completed in a time frame that is reasonable for a commercial product.

It will be clear to those skilled in the art that the invention may be implemented by various combinations of hardware and software, and none of these specific combinations limit the scope of protection of the present invention. The technology according to the invention may be implemented by one or more computers, processors (CPU), such as general-purpose processors or specialized processors, such as digital signal processors (DSP), or one or more ASICs, FPGAs, logic elements, etc. Alternatively, one or more of its elements or method steps may be implemented as software, such as, for example, a program or programs, computer program elements or modules that control one or more computers, CPUs, etc. These software tools may be implemented on one or more machine-readable media that are well known to those skilled in the art, may be stored in one or more memory units such as ROM, RAM, flash memory, EEPROM, etc., or, if necessary, transmitted, for example, from remote servers over one or more wired and/or wireless network connections, the Internet, an Ethernet connection, local area networks (LANs), or other local or global computer networks.

Those skilled in the art should understand that only some of the possible examples of technologies and material and technical means by which embodiments of the present invention can be implemented are described and shown in the drawings above. The detailed description of embodiments of the invention given above is not intended to limit or determine the scope of legal protection of the present invention.

Industrial applicability

In specific embodiments of practical implementation, the invention can be used in the field of computer vision, in image processing to improve image quality, remove noise from images, compress video images, restore three-dimensional (3D) scenes, simultaneously determine the location and map (SLAM), recognize/detect/segment objects and actions in a video image. In addition, the invention can be used in virtual reality (VR) or augmented reality (AR) systems, in autonomous vehicle control systems, etc.

In particular, in video frame rate upconversion (FRUC) applications, it is very important to have information about the actual motion vectors for interpolation of the intermediate frame. In video data compression, the better the motion vector is estimated, the higher the compression ratio.

However, video images that contain a significant amount of noise typically require a higher data rate than normal images, so increasing the compression ratio through more accurate motion estimation results in greater bandwidth savings when transmitting a video stream.

In VR/AR applications, asynchronous reprojection technology is used to improve the headset's response to user motion. This uses one or more previously rendered frames and newer motion information from the headset's sensors to extrapolate the previous frame into a prediction of what a normally rendered frame should look like. In robotics, autonomous vehicles, and medical devices, the invention can enable simultaneous location and display.

Those skilled in the art should understand that only some of the possible examples of technologies and material and technical means by which embodiments of the present invention can be implemented are described and shown in the drawings above. The detailed description of embodiments of the invention given above is not intended to limit or determine the scope of legal protection of the present invention.

Other embodiments within the scope of the present invention may be envisaged by those skilled in the art upon careful reading of the above description with reference to the accompanying drawings, and all such obvious changes, modifications and/or equivalent substitutions are considered to be included within the scope of the present invention. All prior art references cited and discussed herein are hereby incorporated by reference into this description to the extent applicable.

While the present invention has been described and illustrated with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes may be made in form and specific details thereof without departing from the scope of the present invention, which is defined only by the following claims and their equivalents.

Claims (41)

1. A method for estimating motion in a sequence of images, comprising the steps of: obtaining at least two images from an input bitstream containing a plurality of images, performing pre-processing of images, comprising at least brightness adjustment; estimating a map M of parameters of a motion estimation algorithm (ME) for said at least two images using a pre-trained model for predicting spatial parameters of a motion estimation algorithm to said at least two images, wherein the dimension of the map M of parameters of the motion estimation algorithm depends on the method of dividing the image into blocks; save the map M of parameters of the motion estimation algorithm; performing motion estimation by applying a motion estimation algorithm and parameters from a motion estimation algorithm parameter map M to each block of said at least two images, wherein the motion estimation algorithm parameter map M determines the motion estimation algorithm parameters separately for each block; and outputting a motion vector field (MVF) in the form of a two-dimensional (2D) array of offsets from the position of each pixel in the first image to at least one position of the corresponding pixel in the second image of the at least two images. 2. The method according to claim 1, wherein the preliminary processing of said at least two images comprises a step of reducing the scale of the images with the formation of a pyramid of images with at least two reduced scales. 3. The method according to claim 2, wherein the image scaling comprises a step in which spatial filtering of the pixels of the raw image is performed. 4. The method according to claim 1, wherein obtaining said at least two images from the bitstream further comprises a step of obtaining at least one of a preliminary MVF estimate, coded metadata related to image capture parameters, and sensor data not related to image formation. 5. The method according to claim 1, wherein obtaining said at least two images from the bitstream comprises a step in which at least one of the brightness channel data, three-channel color images in the RGB or YUV color spaces are obtained. 6. The method according to claim 1, wherein the pre-configured model for predicting spatial parameters of the motion estimation algorithm has the architecture of a convolutional neural network or a discriminative or regression machine learning model. 7. The method according to claim 1, wherein the motion estimation algorithm is selected from at least one of an algorithm based on comparing blocks by the exhaustive search method, using as the minimum value of the error functional the sum of the absolute values of the difference in the intensity value of each pixel in the block (SAD), a 3D recursive search algorithm (3DRS), a block erosion algorithm with sub-pixel refinement of motion vectors (MV), an algorithm for sub-pixel discretization of candidate blocks. 8. The method according to claim 1, further comprising a step in which a model for predicting spatial parameters of the motion estimation algorithm is trained, wherein training the model for predicting spatial parameters of the algorithm comprises steps in which: receiving as input a synthetic training set of initial data, including K training examples, wherein each of the training examples includes at least one reference frame, one original frame and a true motion vector (GT) field; a pair of synthetic frames is obtained from said synthetic data set, containing a reference frame REF _k and a source frame SRC _k ; select index k of a pair of frames; calculate the estimated map of parameters y: y=Y(SRC _k , REF _k ); form a motion vector field MVF _k (y) from MVF _{i, k} , where i = 1…N, relative to the estimated parameter map y by calculating the weighted sum MVF _1…N , in order to estimate the output of the spatially adaptive ME algorithm; calculate the loss function L between the vectors of the motion vector field MVF _{y, k} and the reference motion vectors GT _k ; update the prediction model of spatial parameters of the motion estimation algorithm by backpropagating the gradient of the loss function L; determine whether the required convergence has been achieved in the spatial parameter prediction model of the motion estimation algorithm; and store the weight coefficients of the trained model for predicting spatial parameters of the motion estimation algorithm in memory. 9. The method according to item 8, in which, at the stage of forming the field of motion vectors, weighting coefficients obtained by applying the Softmax function are used to calculate the weighted sum. 10. A system for estimating motion in a sequence of images, comprising: an image pre-processing unit configured to perform pre-processing of at least two images obtained from an input bit stream containing a plurality of images, wherein the image pre-processing unit is configured to at least adjust brightness; a block for generating a map of M parameters of a motion estimation algorithm (ME) for the at least two mentioned images, configured to generate a map of M parameters (ME) using a pre-trained model for predicting spatial parameters of the motion estimation algorithm and storing the map of M parameters, wherein the dimension of the map of M parameters of the motion estimation algorithm depends on the method of dividing the image into blocks; a motion estimation unit configured to estimate motion on said at least two images by applying a motion estimation algorithm and parameters from a motion estimation algorithm parameter map M to each block of said at least two images, wherein the motion estimation algorithm parameter map M determines the motion estimation algorithm parameters separately for each block; and outputting a motion vector field (MVF) in the form of a two-dimensional (2D) array of offsets from the position of each pixel in the first image to at least one position of a corresponding pixel in the second image of said at least two images. 11. The system according to item 10, further comprising a unit for training a model for predicting spatial parameters of a motion estimation algorithm, configured to train said model for predicting spatial parameters of a motion estimation algorithm by iteratively executing the parameter estimation algorithm with N parameter variants and predicting the parameter value for each of the image blocks that gives the best result. 12. The system according to claim 11, wherein the value of the parameters for each of the image blocks that gives the best result is determined by calculating the loss function between the vectors of the motion vector field and the reference motion vectors. 13. The system of claim 10, further configured to receive at least one of a preliminary MVF estimate, coded metadata related to image capture parameters, and sensor data not related to image formation. 14. The system according to claim 10, configured to obtain a pair of images in the form of at least one of the brightness channel data, three-channel color images in the RGB or YUV color spaces. 15. The system of claim 10, wherein the pre-configured model for predicting spatial parameters of the motion estimation algorithm has the architecture of a convolutional neural network or a discriminative or regression machine learning model. 16. The system of claim 10, wherein the motion estimation algorithm is selected from at least one of an algorithm based on block matching by the exhaustive search method, using as the minimum value of the error functional the sum of the absolute values of the difference in the intensity value of each pixel in the block (SAD), a 3D recursive search algorithm (3DRS), a block erosion algorithm with sub-pixel refinement of motion vectors (MV), an algorithm for sub-pixel discretization of candidate blocks. 17. The system according to claim 11, wherein the training unit for the model for predicting spatial parameters of the motion estimation algorithm additionally comprises: a rendering unit configured to render sequences of frames with a reference motion vector (GT) field; a motion estimation unit (ME) configured to estimate motion for said frame sequences with each set of parameters from the map M of parameters of the motion estimation algorithm; a motion estimation algorithm spatial parameter prediction model training unit configured to train the motion estimation algorithm spatial parameter prediction model by calculating the loss function L between the vectors of the motion vector field generated by the motion estimation unit and the reference motion vectors, updating the motion estimation algorithm spatial parameter prediction model by backpropagating the gradient of the loss function L, determining whether the required convergence has been achieved in the motion estimation algorithm spatial parameter prediction model; and block for storing weight coefficients of the trained model for predicting spatial parameters of the motion estimation algorithm.