Академический Документы
Профессиональный Документы
Культура Документы
Introduction
See beginning of Lecture 5
Overview of LECTURE 6
PART 1: Need to be able to formally describe video content in
terms of objects and events in order to make a query to a video database, e.g. specify who is doing what. Subrahmanians Video SQL
Segment Retrieval: find all video segments where an exchange of a briefcase took place at Johns house Object Retrieval: find all the people in the video sequence (v,s,e) Activity Retrieval: what was happening in the video sequence (v,s,e) Property-based Retrieval: find all segments where somebody is wearing a blue shirt
Recall that SQL is a database query language for relational databases; queries expressed in terms of:
SELECT (which attributes) FROM (which table) WHERE (these conditions hold)
FindVideoWithObjectandProp(o,p,z)
FindObjectsAndPropsInVideo(v,s,e)
EXAMPLE 1
Find all video sequences from the library CrimeVidLib1 that contain Denis Dopeman SELECT vid : [s,e] FROM video : CrimeVidLib1 WHERE (vid,s,e) IN FindVideoWithObjects(Denis Dopeman)
EXAMPLE 2
Find all video sequences from the library CrimeVidLib1 that show Jane Shady giving Denis Dopeman a suitcase
EXAMPLE 2
SELECT vid : [s,e] FROM video : CrimeVidLib1 WHERE (vid,s,e) IN FindVideoWithObjects(Denis Dopeman) AND
EXAMPLE 3
Which people have been seen with Denis Dopeman in CrimeVidLib1
EXAMPLE 3
SELECT vid : [s,e], Object FROM video : CrimeVidLib1 WHERE (vid,s,e) IN FindVideoWithObject(Denis Dopeman) AND Object IN FindObjectsInVideo(vid,s,e) AND Object = Denis Dopeman AND type of (Object, Person)
Exercise 6-1
Given a video database of old sports broadcasts, called SportsVidLib, express the following users information needs using the extended SQL as best as possible. You should comment on how well the extended SQL is able to capture each users information need and discuss alternative ways of expressing the information need more fully.
Bob wants to see all the video sequences with Michael Owen kicking a ball Tom wants to see all the video sequences in which Vinnie Jones is tackling Paul Gascoigne Mary wants to see all the video sequences in which Roy Keane is arguing with the referee, because Jose Reyes punched Gary Neville, while Thierry Henry scores a goal, and then Roy Keane is sent off.
Bob wants to see all the video sequences with Michael Owen kicking a ball
Tom wants to see all the video sequences in which Vinnie Jones is tackling Paul Gascoigne
Mary wants to see all the video sequences in which Roy Keane is arguing with the referee, because Jose Reyes punched Gary Neville, while Thierry Henry scores a goal, and then Roy Keane is sent off.
Think about
What metadata would be required in order to execute these kinds of video query? How could this be stored and searched most efficiently?
Need to be able to describe temporal relationships between intervals formally and make inferences about temporal sequences
X equal Y =
Another Example
If A contains B, and B < C then what relationships can hold between A and C? BBBBB ?CC? ?CCCC? ?CCCCC? AAAAAAAAAAAAA?CCCCC? ?CCCCC? Possibilities: A < C ; A overlaps C; A meets C; A contains C; A is finished by C
EXERCISE 6-2
For an video database application domain of your choosing write five video queries that use some of Allens 13 temporal relationships If event A is before (<) event B, and event B is during event C, then what relationships could hold between A and C? How do you think such reasoning about temporal could be used in a video database?
OPTIONAL READING
Dunckley (2003), pages 38-39; 393-395. For details of the extended video SQL, see: Subrahmanian (1998). Principles of Multimedia Databases - pages 191-195. IN LIBRARY ARTICLE COLLECTION For temporal relationships:
Allen (1983). J. F. Allen, Maintaining Knowledge About Temporal Intervals. Communications of the ACM 26 (11), pp. 832-843. Especially Figure 2 for the 13 relationships and Figure 4 for the full transitivity table. [In Library on shelf] For causal relationships: Roth (1999). Volker Roth, Content-based retrieval from digital video. Image and Vision Computing 17, pp. 531-540. [Available online through library eJournals]