FOLLOW
1 Reviews   |   2177 Views   |   2 Followers

How to Pixel a Video

IP

US8958478

Owner

TU Berlin

Filing Date

December 03, 2010

Priority Date

June 07, 2010

Portfolio

Video - TU Berlin

Intro

Video encoding methods save motion vector fields locally in a buffer to predict the following frames, thus reducing the size of the error pattern that is needed to decode the frames. Besides, this so-called motion-compensated prediction further reduces noise by reducing the signal-to-noise-ratio (SNR), regardless of the bit rate. Prior art is a spatial approach that reduces edge artifacts from blocking. This invention combines the spatial approach with a temporal trajectory filter to reduce noise as well as the data rate. Noise is filtered along temporal trajectories for every pixel which has two-dimensional coordinates describing the motion of pixels from frame to frame. The algorithm used here is less efficient for sources with a high quantizer, therefore an automatic shutoff is implemented for quantizers above 45. With the help of this new in-loop filtering algorithm, the picture quality could be improved considerably. This was proofed by the inventors using the H.264/AVC codec. In this way it was possible to reach a bit rate reduction up to 12 % (the average being 4 %).

Claims

1. A method for processing the pixel amplitude of at least one image pixel contained in a current frame of a video sequence, said method comprising th...
  1. A method for processing the pixel amplitude of at least one image pixel contained in a current frame of a video sequence, said method comprising the steps of constructing an individual motion trajectory comprising motion-shifted versions of the at least one image pixel over a plurality of neighboring frames, and filtering the pixel amplitude with an individual filter length based on the individual motion trajectory that results from a pixel-wise decision concerning the filter length, wherein said step of constructing said individual motion trajectory comprises the steps of: choosing the at least one image pixel of the current frame as a start pixel of the individual motion trajectory, and adding motion-shifted versions of the at least one image pixel of preceding and/or subsequent frames to the individual motion trajectory; wherein for each motion-shifted version to be added, it is determined whether a difference value, which indicates the change of the pixel amplitude of the respective motion-shifted version between consecutive frames, is smaller than an amplitude threshold value, which is individually assigned to the current frame or a frame portion of the current frame comprising said at least one image pixel, and wherein said step of adding further motion-shifted versions is stopped if said difference value exceeds said amplitude threshold value, and wherein said amplitude threshold value is allocated to the current frame by carrying out the following steps: a) defining an amplitude threshold value range comprising a plurality of amplitude threshold values; (b) processing the pixel amplitudes of a plurality of pixels contained in the current frame based on a given amplitude threshold value of said amplitude threshold value range, and generating a processed frame; (c) comparing the processed frame with a reference frame and determining an image difference value; d) repeating steps (b) and (c) for each amplitude threshold value of said amplitude threshold value range and determining the respective image difference value; (e) determining the amplitude threshold value providing the smallest image difference value; and (f) allocating the determined amplitude threshold value to the current frame.
  2. The method of claim 1 wherein the step of adding a motion-shifted version to the individual motion trajectory includes: determining the pixel amplitude of said motion-shifted version; determining the pixel amplitude of the motion-shifted version in the next preceding and/or subsequent frame; determining a difference value, which indicates the difference between the pixel amplitude of said motion-shifted version and the pixel amplitude of said motion-shifted version in the next preceding and/or subsequent frame; comparing said difference value with said amplitude threshold value, if the difference value is smaller than said amplitude threshold value, elongating the individual motion trajectory by adding the motion-shifted version to the individual motion trajectory, and if the difference value exceeds said amplitude threshold value, completing the construction of the individual motion trajectory.
  3. The method of claim 1 wherein the at least one image pixel and the motion-shifted versions thereof belong to a pixel block, said pixel block comprising a plurality of pixels belonging to the same frame section.
  4. The method of claim 1, wherein said amplitude threshold value range ranges to at most 5% of the maximum intensity value that can be assigned to pixel amplitudes.
  5. The method of claim 1, wherein a bit stream is generated, the bits of said stream describing the frames of the video sequence and the amplitude threshold value for each of said frames.
  6. The method of claim 1 wherein the frames of the video sequence comprise pixel blocks, each pixel block comprising a plurality of pixels belonging to the same frame section; wherein the video sequence comprises a flag for each block, said flag indicating whether or not the amplitude threshold value of the frame is allocated to the pixels of the respective pixel block; and wherein for pixels of blocks, to which no amplitude threshold value is allocated, the construction of trajectories is skipped.
  7. The method of claim 1 comprising the steps of: (a) defining an amplitude threshold value range comprising a plurality of amplitude threshold values, and a block threshold value range comprising a plurality of block threshold values; (b) processing the pixel amplitudes of a plurality of pixels contained in the current frame based on a given amplitude threshold value of said amplitude threshold value range and a given block threshold value of said block threshold value range, and generating a processed frame; (c) comparing the processed frame with a reference frame and determining an image difference value; (d) repeating steps (b) and (c) for each amplitude threshold value of said threshold value range and each block threshold value of said block threshold value range, and determining the respective image difference value; (e) determining the combination of amplitude threshold value and block threshold value providing the smallest image difference value; and (f) allocating the determined amplitude threshold value and the determined block threshold value to the current frame.
  8. The method of claim 1 wherein the frames of the video sequence comprise pixel blocks, each pixel block comprising a plurality of pixels belonging to the same frame section; wherein said video sequence comprises a flag for each block, said flag indicating whether or not the amplitude threshold value of the frame and the block threshold value of the frame are allocated to the pixels of the respective pixel block; and wherein for pixels of blocks, to which no amplitude threshold value and no block threshold value is allocated, the construction of trajectories is skipped.
  9. The method of claim 1 wherein said step of processing the pixel amplitude based on the individual motion trajectory comprises the step of combining the pixel amplitudes of the motion-shifted versions of said image pixel along the individual motion trajectory using a weighting function, to form a processed pixel amplitude of said image pixel.
  10. The method according to claim 1 wherein frames of said video sequence are filtered and/or predicted by incorporating said processed pixel amplitude of said image pixel into the video sequence.
  11. The method according to claim 1 wherein further video sequences having spatially adjacent frames that comprise image pixels corresponding to those of said video sequences, are taken into account to form said processed pixel amplitude of said image pixel.
  12. Device capable of carrying out the method according to claim 1.
  13. The method of claim 3 comprising the steps of: for each motion-shifted version to be added, determining the number of pixel blocks, which are adjacent to said pixel block and have similar or identical motion vectors, comparing said number of pixel blocks with a block threshold value; and stopping said step of adding further motion-shifted versions to the individual motion trajectory if said number of pixel blocks is smaller than said block threshold value.
  14. The method of claim 7 wherein said amplitude threshold value range ranges to at most 5% of the maximum intensity value that can be assigned to pixel amplitudes; and wherein said block threshold value range ranges to at most 50% of the maximal number of blocks adjacent to a given block.
  15. The method of claim 7 wherein a bit stream is generated, the bits of said stream describing the frames of the video sequence, the amplitude threshold value for each of said frames, and the block threshold value for each of said frames.
  16. Device of claim 12, wherein the device is an in-loop filter.
  17. Device of claim 12, wherein the device is a decoder.
  18. Device of claim 12, wherein the device is an encoder.  
read more

Abstract

Method and device for processing pixels contained in a video sequence including processing the pixel amplitude of at least one Image pixel contained i...
Method and device for processing pixels contained in a video sequence including processing the pixel amplitude of at least one Image pixel contained in a current frame of a video sequence. The method includes the steps of constructing an individual motion trajectory including motion-shifted versions of the image pixel over a plurality of neighboring frames, and processing the pixel amplitude based on the individual motion trajectory. The step of the constructing the individual motion trajectory includes the steps of choosing at least one image pixel of the current frame as a start pixel of the individual motion trajectory, and adding motion-shifted versions of the image pixel of preceding find/or subsequent frames to the individual motion trajectory.
read more

Get more Information

Here are the reviews from the crowd!

Rating categories
WOW!
GOTCHA!
DEAL!
Where should the donation go to?
Conroy
October 18, 2018

Video image processing is a highly competitive, and with the rise ML/AI based on the date of submission there is a strong possibility this technique could have superseded newer more novel solutions. In the highly lucrative defense and government surveillance industry it may be to prove if this solution is being used.
Applications: This solution likely valuable in within an existing or even new wholly/mostly software (not ML/AI) based solution.
Companies: Wholly software based video companies and Cloud based video service providers

LIKE       COMMENT


Bastian October 18, 2018

Thank you for sharing your insights!!!

Rate the patent & write your own review!

Rating categories
WOW!
GOTCHA!
DEAL!
Where should our donation go to?
Your review
Write about potential applications
Write about potentially interested companies
Choose a name

Contact us!

Your email
Your message