Privacy Protected Video Surveillance Sen-ching Samson Cheung Center for Visualization

1
Privacy Protected Video SurveillanceSen-ching
Samson Cheung Center for Visualization
Virtual EnvironmentsDepartment of Electrical
Computer Engineering University of
Kentuckyhttp//www.vis.uky.edu/mialab
2
Acknowledgements

• Graduate Students
• Vijay Venkatesh Mahalingam
• Jian Zhao
• Jithendra K. Paruchuri
• Research Support
• Department of Homeland Security
• Oak Rridge Associated Universities

3
What is privacy?
the right most valued by all civilized men the
right to be let alone. - U.S. Supreme Court
Justice Louis Brandeis, 1890

• To develop human excellence without interference
  Aristotles Politics 350 B.C.
• Control over information about oneself Warren
  and Brandeis 1890

4
Todays privacy concerns

• Electronic Voting
• E-commerce
• Medical Records
• Financial Records
• Cyber Activities

5
Tomorrows privacy concerns

• Smart video surveillance
• Biometric theft
• Multimedia processing
• RFID tracking

6
Privacy Protection Technology

• Technologies that aim at protecting personal
  privacy without compromising the legitimate use
  of information.
• Main PPT include the followings
• Encryption Tools
• Platform for Privacy Preferences (P3P)
• Automated Privacy Audit
• Anonymizer
• Privacy Protected Data mining

Is privacy protection of multimedia any different?
7
Challenges from Multimedia

• What to protect?
• Identify semantic objects for protection
• How to protect it?
• Reliable protection without losing perceptual
  utility
• How to control it?
• Flexible control and secure authentication of
  privacy data

8
Talk Overview

• Video Surveillance
• Subject Identification
• Optimal Camera Network
• Video Obfuscation
• Privacy Data Management
• Portable AV devices
• Evaluation of audio privacy protections
• Secure Distributed Processing

9
Overview
Subject Identification Module
Obfuscation
Object Identification and Tacking
Secure Camera System
Secure Data Hiding
Surveillance Video Database
Privacy Data Management System
10
Talk Overview

• Video Surveillance
• Subject Identification
• Optimal Camera Network
• Video Obfuscation
• Privacy Data Management

11
Video Obfuscation
Black Out
Original
Pixelation/ Blurring
12
Challenges of Video Inpainting
13
Dynamic Object In-painting

• Basic idea Using object template extracted form
  other time instant to complete a conceptually
  consistent sequence.
• Steps
• 1. Similarity based on optimal alignment
• 2. Motion continuity
• 3. Positioning of templates

?
...
...
...
14
Motion Continuity
?
?
15
Object-based Video In-painting

• Better motion in-painting by better registration
  and task separation
• Capable to in-paint partially and completely
  occluded objects
• Improved computational performance (Matlab)

Number of frames with complete occlusion
Number of frames with partial occlusion
16
Public-domain Sequences
17
Complex Sequences
18
Talk Overview

• Video Surveillance
• Subject Identification
• Optimal Camera Network
• Video Obfuscation
• Privacy Data Management

19
Privacy Data Management
Subject A
Producer
Subject B
Subject C
Key question How does a client know which
subject to ask?
20
Three-agent architecture
Mediator Agent PKM,SKM
AES(Vu K)
Subject Agent PKU,SKU
Client Agent PKC,SKC
AES(Vu K)
21
Keeping sensitive information
Medium Method Pro Pro Con
Separate File Encryption Cryptographic signature Standard Technology Storage efficiency Pervious to attacker Difficult to distribute with the modified video Separate authentication for modified video Pervious to attacker Difficult to distribute with the modified video Separate authentication for modified video

Meta-data Encryption Cryptographic signature Standard Technology Storage efficiency Less pervious to attacker Depend on format
Data hiding Encrypted watermark Impervious to attacker Inseparable from data Joint authentication May need more storage May affect visual quality
22
Data Hiding for Privacy Data Preservation
23
Data Hiding

• Data hiding/Stenography/Watermarking
• Active research in the past fifteen years
• Typical applications include authentication, copy
  detection, monitoring
• Challenges in our application
• Picture-in-picture large embedding capacity
• Compatibility with existing compression scheme
• Minimal visual distortion

24
Optimal Data Hiding
Block-based Rate-Distortion Calculation
Psycho-visual Modeling
Discrete Optimization
Solve constrained optimization
25
Proposed Data Hiding
DCT(i,j) watermark_bit 2round(DCT(i,j)/2)
Privacy protected video
DCT
Entropy Coding
Motion Compensation
H.263
H.263
J. Paruchuri S.-C. Cheung Rate-Distortion
Optimized Data Hiding for Privacy Protection
submitted to ISCAS 2008
26
R-D framework

• Target cost function
• Ri Increase in Bandwidth of Block i
• Di Perceptual Distortion in Block i
• d Relative Weight
• Greedy embedding of P data bits in Block i
• Lagrangian optimization determine the optimal Pi
  and ? to embed the target number of data bits

27
Examples 1/2
Distortion 637 kbps 81 kbps data
119kbps No data
Rate Distortion 562 kbps 81 kbps data
Rate only 370 kbps 81 kbps data
28
Examples 2/2
Distortion 743 kbps 81 kbps data
406.3kbps No data
Rate Distortion 678 kbps 81 kbps data
Rate only 610 kbps 81 kbps data
29
Conclusions

• Privacy Protecting Video Surveillance
• Visual Tagging for subject identification
• Optimized camera network for visual tagging
• In-painting for video obfuscation
• Privacy Data Management
• R-D optimized watermarking
• Current Research
• Video In-painting in crowded environment
• Performance Evaluation for PPT
• Secure Reversible Modification
• Audio Privacy Protection
• Signal processing in encrypted domain