See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/284325035

p2693-wu

Data · November 2015

CITATIONS READS
0 112

4 authors:

Yuxing Wu Xinghao Jiang

Shanghai Jiao Tong University Shanghai Jiao Tong University
2 PUBLICATIONS 44 CITATIONS 107 PUBLICATIONS 732 CITATIONS

SEE PROFILE SEE PROFILE

Tanfeng Sun Wan Wang

Shanghai Jiao Tong University Shanghai Jiao Tong University
107 PUBLICATIONS 646 CITATIONS 7 PUBLICATIONS 210 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

NSF of China, "Video Content Management Contral" View project

Detection Scheme Based on AI Forgery View project

All content following this page was uploaded by Tanfeng Sun on 22 November 2015.

The user has requested enhancement of the downloaded file.

 2014 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP)

EXPOSING VIDEO INTER-FRAME FORGERY BASED

ON VELOCITY FIELD CONSISTENCY

Yuxing Wu, Xinghao Jiang*, Tanfeng Sun and Wan Wang

School of Electronic Information and Electrical Engineering,

Shanghai Jiao Tong University, Shanghai, China
{wuyuxing, xhjiang, tfsun, wangwan}@sjtu.edu.cn

ABSTRACT optical flow. They found that inter-frame forgery operations

would cause discontinuity in optical flow sequence.
In recent years, video forensics has become an important In this paper, we propose a new approach to detect sur-
issue. Video inter-frame forgery detection is a significant veillance video inter-frame forgery based on the consistency
branch of forensics. In this paper, a new algorithm based on of velocity field. This method is able to distinguish the tam-
the consistency of velocity field is proposed to detect v ideo pered video, identify the forgery types (i.e., consecutive
inter-frame forgery (i.e., consecutive frame deletion and con- frame deletion, consecutive frame duplication) and locate the
secutive frame duplication). The generalized extreme manipulated positions in forged videos as well. Our algo-
studentized deviate (ESD) test is applied to identify the for- rithm follows three steps. First, obtain velocity field
gery types and locate the manipulated positions in forged sequence by applying block-based cross correlation. Then,
videos. Experiments show the effectiveness of our algorithm. calculate the corresponding relative factor sequence from
velocity field sequence. Finally, determine the authenticity,
Index Terms— Video forensics, Inter-frame forgery de- the forgery type and manipulated locations with generalized
tection, Velocity field, Generalized extreme s tudentized extreme studentized deviate (ESD) algorithm.
deviate
2. VELOCITY FIELD IN VIDEO FORGERY DETECTION
1. INTRODUCTION
Velocity field is a term induced from Particle Image Veloci-
Nowadays, surveillance camera systems have been widely metry (PIV) technique [11]. The key point of PIV is to
deployed in many circumstances to monitor illegal activities. compare adjacent video frames and estimate their displace-
Surveillance videos have already been regarded as the judi- ments caused by time separation. It is considered that any
cial proofs in the court. However, with the development of inter-frame operations, like frame deletion and duplication
advanced video editors, their integrity cannot be guaranteed will enlarge the displacements. In this section, we will show
anymore. Therefore, how to authenticate the surveillance how to form the velocity field sequence and illustrate traces
videos has become a significant issue. left in it after different forgery operations.
So far, many video forensics techniques have been
studied [1]. [2]-[4] proposed to detect double compression, 2.1. Velocity field sequence estimation
[5]-[7] detected video forgery with sensor noise patterns,
and [8]-[10] exposed forgery based on the videos’ content. The velocity field computation is done by PIVlab [12]. Its PIV
In the aspect of inter-frame forgery detection, Wang and algorithm is set to FFT window deformation with one-pass
Farid [2] first exposed the frame deletion or insertion by pre- 16 16 pixel interrogation window and 75% overlap factor. (1)
diction error. They discovered that frames moving from one and (2) are the mathematical descriptions of the computation
group of picture (GOP) to another will have larger motion process.
estimation errors. However, their method would fail if a com-
plete GOP is deleted. Mondaini et al. [5] proposed to detect

RC  u,v   F 1  F  I  i, j,t   F  I  i, j,t  1 
*
 (1)

frame insertion/duplication by the photoresponse non- arg max ReRC  u,v  (2)
u,v
uniformity noise (PRNU) fingerprinting technique. Chao et al.
[10] proposed to detect frame deletion and insertion through where I  i, j, t  and I  i, j, t  1 are the interrogation windows
at  i, j  location in t and  t  1 frame respectively. F , F 1
are 2-D Fourier transform operator and inverse Fourier trans-

* Corresponding Author is Xinghao Jiang form operator respectively, * is the complex conjugate

978-1-4799-2893-4/14/$31.00 ©2014 IEEE 2693

function, Re  obtains the real part of its parameter. Ac-
cording to these formulas,  u,v  is regarded as the
displacement (also called velocity vector) between the two
interrogation windows. To express accurately, we denote
 u,v  as u  i, j, t  , v i, j , t  , which indicates the velocity vector
at  i, j  location of t frame. Therefore, we can define the
velocity field intensity (VFI) as follows: (a)

VFI  t h   u  i, j , t  ;VFI t v   v i, j , t  (3)

i j i j

where VFI  t h , VFI  t v indicate the horizontal and vertical

velocity field intensity respectively. We then denote
VFI t h t  1, L  1 and VFI t v t  1, L  1 as the horizon-
(b)
tal and vertical VFI sequence, where L is the number of
frames.
The max-sample technique is used to exclude those
frames with extremely low VFI. The low VFI is probably
caused by the similarity of two neighbor frames in data,
which was introduced by camera coding error. Every three
frames are sampled into one frame with the maximum VFI.
(c)
And the sample process starts at the position where the
Fig. 1. The horizontal (left) and vertical (right) relative factor se-
number of the remaining frames can be divided by 3. Then
quences. (a) original video; (b) frame deletion video; (c) frame
we have the new VFI sequences as follows:
duplication video.
SVFI t  h
t [1,T],SVFI  t v t [1,T]
where SVFI  t  denotes VFI sequence after max-sampling,
  represents round down function and T   L  1 / 3 .
The consistency of the VFI sequences in both direc-
tions will be destroyed if the video is manipulated by some
(a) (b) (c) (d)
inter-frame forgery operations. Therefore, the relative factors
Fig. 2. Four representative frames of a video. Over 600 consecutive
RFh and RFv are defined to reveal these changes.
frames have been deleted between (a) and (d) to cover a suspicious
SVFI  t  1h  SVFI  t  1h
RFh  t    SVFI  t h (4) man walking out of the elevator. There will be no visual differences
SVFI  t  1h  SVFI  t  1h before and after forgery process.
SVFI  t  1v  SVFI  t  1v
RFv  t    SVFI  t v (5)
SVFI  t  1v  SVFI  t  1v 2.2.2. Consecutive frame deletion video
In the relative factor sequences RF  t h t  2,T 1 and These videos are tampered by deleting consecutive frames.
After the forgery process, two originally unrelated frames
RF t  v  , the discontinuity peaks
t   2,T 1 introduced by the have become neighbors, generating a salient increase in the
forgery operations will be obviously highlighted. VFI sequence. Therefore, one discontinuous peak would be
observed in the relative factor sequence.
2.2. Traces in relative factor sequence
2.2.3. Consecutive frame duplication video
In this paper, two types of forgeries, consecutive frame dele- These videos are modified by duplicating consecutive
tion and consecutive frame duplication are considered. frames from one time point to another. Hence, two discontin-
Different forgery operations will introduce different numbers uous peaks would be observed.
of discontinuous peaks in the relative factor sequence. Fig. 1 Again, note that we only consider the videos recorded
shows the corresponding relative factor sequences of a giv- by static surveillance cameras. That is to say, only the inter-
en video before and after manipulation. frame forgery operations will introduce the obvious disco n-
tinuity in the relative factor sequence. In addition, we focus
2.2.1. Original video on detecting videos with meaningful forgeries, which mean
These videos are directly from surveillance cameras without no visual differences will be perceived before and after for-
any modifications. And there is no discontinuous peak in the gery process. Fig. 2 demonstrates an example of meaningful
relative factor sequence in this type of video. consecutive frames deletion forgery.

2694
 3. VIDEO FORGERY IDENTIFICATION to improve the identification accuracy. Let N h and N v de-
note the detected number of the discontinuity peaks in
The discontinuous peaks in the relative factor sequence are horizontal and vertical sequence, respectively. The flowchart
regarded as the evidence of video forgery. The generalized of the identification algorithm is given in Fig. 3.
ESD test has been applied to extract the peaks and identify
the forgery types. The detail of the identification algorithm is Y
Video clip Nh  Nv  1 Original
described in this section.
N
3.1. Generalized ESD test Y
Generate Nh  Nv  2 Deletion
We find that the probability distribution of the relative factor relative factor sequence
sequence follows an approximate normal distribution. Hence, N

Generalized ESD test [13] is able to be employed in our iden- Y

N h  1; N v  1
tification algorithm.
There are two important parameters in the test, the upper Generalized ESD test N
bound number of outliers r and significance level  . First
compute R1 from Duplication

Ri  max i xi  x / s (6)
Fig. 3 Flowchart of the identification algorithm
where x and s denote the mean and standard deviation of
the n samples respectively. Remove the observation that Finally, the tampered location range is determined based
maximizes xi  x / s and then re-compute the above statistic on the relative factor sequence generation process in section
with n  1 observations. Repeat this process until 2. The upper bound of the range is Ru   P  1  3  mod  LVFI ,3 ,
R1,R2 ,...,Ri have all been computed. Finally pick the corre- where P is the location of the detected peaks in relative fac-
sponding r critical values i at the chosen confidence level tor sequence, “mod” is modulo operation, and LVFI is the
 . The number of outliers is determined by finding the larg-
est i such that Ri  i . length of the corresponding VFI sequence. Hence, the tam-
In order to determine the exact number of peaks in the pered range is  Ru  2,Ru  .
relative factor sequence, we have fine-tuned the critical val-
ues i by multiplying a coefficient  , and the new definition
is as follows:
t p , n  i 1   n  i 
i '   (7)
n  i 1  t     n  i  1
2
(a) (b) (c) (d)
p , n  i 1 Fig. 4. Four source videos with different scenes. (a) scene 1; (b)
 scene 2; (c) scene 3; (d) scene 4.
p 1 (8)
2   n  i  1
4. EXPERIMENTS
where t p , n i 1 is the pth percentile of a t distribution with
 n  i  1 degrees of freedom.
4.1. Video database
Some fake peaks might be found in the relative factor
sequence. Comparing with real forgery peaks, these peaks
To the best of our knowledge, there is no open database for
are with relatively low intens ities, which were probably in-
detecting video inter-frame forgery. Therefore, we have in-
troduced by camera noise or video encoding. The fake peaks
vited some volunteers to build one. Our four different scenes
would be determined as outliers with the original i , while
source videos (see in Fig. 4) are downloaded from TRECVID
the fine tuning, which slightly raises the critical values is
surveillance event detection evaluation [14]. Each source
helpful to refuse these fake peaks and pick out the forgery
video split out 10 video clips. Each clip contains about 3000
peaks accurately as well.
frames with 720  576 resolution. Then the 40 video clips
were delicately tampered to generate 40 frame deletion vide-
3.2. Identification algorithm
os and 40 frame duplication videos (defined in section 2.2).
Hence, there are totally 120 video clips in our final inter-frame
According to the description in section 2.2, there are at most
forgery detection database. Note that all the tampered video
two discontinuity peaks in the relative factor sequence,
clips were MPEG-2 re-coded with the same coding standard
hence we set the upper bound number of outliers r  2 .
and parameters as the source videos.
Moreover, the generalized ESD test is carried out on both
horizontal and vertical relative factor sequences , which helps

2695
4.2. Results and analysis compression), 2, 3 are shown in Table IV. When re-
compressing with Qscale=2, the bit rate has averagely de-
The configurations of identification algorithm are as follows. creased by 3%, so the accuracy is the same with the result of
The upper bound number of outliers is r  2 , the signifi- Qscale=2. While when re-compressing with Qscale=3, the bit
cance level is   0.05 and the coefficient of the critical rate dropped a lot (with 30%), the duplication identification
values is    . accuracy have slightly decreased. The reason is that the
intensity gap between the two detected peaks enlarged after
4.2.1. Detection accuracy under random deletion re-compression, which makes it easy to be identified as frame
This experiment is to test the sensitivity of our algorithm by deletion forgery. Anyway, the accuracies report the robust-
computing the detection accuracies when frames were ran- ness of our algorithm to some degree of compression.
domly deleted. Table I shows the detection accuracies for
randomly deleting 1 frame, 3 consecutive frames and 5 con- 5. CONCLUSION
secutive frames from original videos. The result illustrates
that our algorithm could have good accuracy when detecting We have proposed a new algorithm to detect video inter-
frame deletion forgery with a few frames removed. frame forgery. This method is based on the consistency of
velocity field. With consecutive frame deletion and frame
4.2.2. Identification accuracy under meaningful forgery duplication forgery operations, some discontinuity peaks
The confusion matrices for the four scenes of video clips can be observed in VFI sequence. And the generalized ESD
and the overall accuracy are given in Table II. The relatively test is applied to extract the peaks and identify the forgery
low accuracies for frame duplication identification are due to type. Experiments show the effectiveness of our algorithm.
the large intensity gaps between their two detected peaks, all
the incorrect identification videos are identified as frame de- ACKNOWLEDGMENT
letion forgery. However, the result demonstrates the
effectiveness of our algorithm with overall 90.0%, 85.0% and The work of this paper is sponsored by the National Natural
80.0% accuracies for identifying original video, frame dele- Science Foundation of China (No.61272439, 61272249), the
tion video and frame duplication video. If we only consider Specialized Research Fund for the Doctoral Program of High-
whether a video is tampered or not, the overall identification er Education (No.20120073110053), and the Fund of State
accuracy for the tampered videos is 96.3%, with 10% false Key Laboratory of Software Engineering, Wuhan University
positives. We did not do comparison experiments because (No.SKLSE2012-09-12). It was also under the Project of In-
no papers were found on identifying consecutive frame dele- ternational Cooperation and Exchanges supported by
tion and duplication forgeries. Shanghai Committee of Science and Technology (No.
12510708500).
4.2.3. Location accuracy under meaningful forgery
Table I Detection accuracies for random deletion.
The location is considered to be incorrectly identified if one
of the detected peaks in both horizontal and vertical VFI s e- Deleted frame number 1 3 5
quences is not in the expected range described in section 3.2. Accuracy 40% 65% 80%
The location accuracies for correctly identified forged videos Table III Location accuracies under meaningful forgery.
are given in Table III. All the locations of detected peaks in Forgery type deletion duplication
forged videos are correctly identified due to the s tatistics-
Accuracy 100%(34/34)a 100%(32/32)
based generalized ESD algorithm. a
(n/m) indicates n of m locations are correctly identified.

4.3. Robustness against compression Table IV Detection accuracies under different Qscales (%).
Qscale 1 2 3
The robustness against lossy compression is tested in this original 90.0 90.0 90.0
experiment. Each video clips was re-compressed by ffmpeg deletion 85.0 85.0 85.0
software with different Qscales (a parameter to control video duplication 80.0 80.0 62.5
quality). The identification results with Qscale=1 (lossless
Table II Confusion matrix for each scene and their overall accuracy (%). – denotes value 0.
Video Scene 1 Scene 2 Scene 3 Scene 4 Overall
Forgery type oria delb dup c ori del dup ori del dup ori del dup ori del dup
ori 80.0 10.0 10.0 90.0 10.0 - 100 - - 90.0 10.0 - 90.0 7.5 2.5
del 20.0 70.0 10.0 - 90.0 10.0 10.0 90.0 - - 90.0 10.0 7.5 85.0 7.5
dup - 30.0 70.0 - 20.0 80.0 - 30.0 70.0 - - 100 - 20.0 80.0
a
Original video type. b Frames deletion video type. c Frames duplication video type.

2696
 REFERENCES

[1] M ilani S., Fontani M ., Bestagini P. et al., “An overview on

video forensics,” APSIPA Transactions on Signal and Information
Processing, 2012.

[2] Wang W. H. and Farid H., “Exposing digital forgeries in video

by detecting double M PEG compression,” In proceedings of the 8th
ACM workshop on M ultimedia and Security, pp.37-47, 2006.

[3] Chen W. and Shi Y. Q., “Detection of double M EPG video

compression using first digits statistics,” Digital Watermarking,
Springer Berlin Heidelberg, pp.16-30, 2009.

[4] Wang W. H. and Farid H., “Exposing digital forgeries in video

by detecting double quantization,” In proceedings of the 11th ACM
Workshop on M ultimedia and Security, pp.39-47, 2009.

[5] M ondaini N., Caldelli R., Piva A. et al., “Detection of malevo-

lent changes in digital video for forensic applications,” In
proceedings of the society of photo-optical instrumentation engi-
neers, vol.6505, pp.T5050-T5050, 2007.

[6] Hsu C.C., Hung T.Y., Lin C.W. et al., “Video forgery detection
using correlation of noise residue,” IEEE 10th Workshop on M ulti-
media Signal Processing, pp.170-174, 2008.

[7] Kobayashi M ., Okabe T., Sato Y., “Detecting Forgery From

Static-Scene Video Based on Inconsistency in Noise Level Func-
tions, ” IEEE Transactions on Information Forensics and Security,
vol.5, pp.883-892, 2010.

[8] Wang W. H. and Farid H., “Exposing Digital Forgeries in Video

by Detecting Duplication,” In proceedings of the 9th ACM Work-
shop on M ultimedia and Security, pp.35-42, 2007.

[9] Conotter V., O'Brien J.F., Farid H., “Exposing Digital Forgeries
in Ballistic M otion,” IEEE Transactions on Information and Securi-
ty, vol.7, pp.283-296, 2012.

[10] Chao J., Jiang X. H. and Sun T. F., “A novel video inter-frame
forgery model detection scheme based on optical flow consisten-
cy,” International Workshop on Digital Forensics and Watermaking,
pp.267-281, 2012.

[11] Grant I., “Particle image velocimetry: A review,” In proceed-

ings of the Institution of M echanical Engineers, vol. 211, pp.55-76,
1997.

[12] Thielicke W., Stamhuis E. J., “PIVlab- Time-Resolved Digital

Particle Image Velocimetry Tool for M ATLAB version 1.32,” 2010.

[13] Iglewicz B., Hoaglin D.C., “How to Detect and Handle Outli-
ers,” M ilwaukee (Wisconsin): ASQC Quality Press, vol. 16, 1993.

[14] TREC Video Retrieval Evaluation, http://trecvid.nist.gov/

View publication stats

2697