-
Notifications
You must be signed in to change notification settings - Fork 1
/
preliminaries.tex
194 lines (148 loc) · 10.6 KB
/
preliminaries.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
% \thesistitle{
% Pruning Large Search Spaces using Context Networks
% }
\thesistitle {
Discovering Real World Context to Tag Personal Photos
}
\degreename{Doctor of Philosophy}
% Use the wording given in the official list of degrees awarded by UCI:
% http://www.rgs.uci.edu/grad/academic/degrees_offered.htm
\degreefield{Computer Science}
% Your name as it appears on official UCI records.
\authorname{Arjun Satish}
% Use the full name of each committee member.
\committeechair{Professor Ramesh Jain}
\othercommitteemembers
{
Dr. Amarnath Gupta\\
Professor Nalini Venkatasubramanian\\
Professor Deva Ramanan\\
Professor Bill Tomlinson
}
\degreeyear{2013}
\copyrightdeclaration
{
{\copyright} {\Degreeyear} \Authorname
}
% If you have previously published parts of your manuscript, you must list the
% copyright holders; see Section 3.2 of the UCI Thesis and Dissertation Manual.
% Otherwise, this section may be omitted.
% \prepublishedcopyrightdeclaration
% {
% Chapter 4 {\copyright} 2003 Springer-Verlag \\
% Portion of Chapter 5 {\copyright} 1999 John Wiley \& Sons, Inc. \\
% All other materials {\copyright} {\Degreeyear} \Authorname
% }
% The dedication page is optional.
\dedications
{
\textit{And it's whispered that soon, if we all call the tune,} \\
\textit{Then the piper will lead us to reason} \\
\textit{And a new day will dawn for those who stand long} \\
\textit{And the forest will echo with laughter} \\
\textit{} \\
\setlength{\parindent}{6.3cm} \texttt{Jimmy Page \& Robert Plant, } \\
\setlength{\parindent}{7cm} \texttt{from Stairway to Heaven, 1970} \\
}
\acknowledgments
{
I would like to thank my advisors, Ramesh Jain and Amarnath Gupta for teaching me the values and fundamental principles of the scientific method. Without their continuing support, a project like CueNet would have barely made this far. The many technical and personal discussions I had with them during the last few years are going to be greatly missed. I owe a lot to Ramesh for accepting me as a graduate student, and introducing me to his beloved projects and ideas and letting me play with them as I seemed fit.
Special thanks to my committee members, Deva Ramanan, Nalini Venakatasubramian and Bill Tomlinson to take the time out to provide feedback on my work and suggesting very interesting experiments which helped me gain some major insights into it. My fellow graduate students at ISG were always a source of inspiration and encouragement to work on challenging cross-domain problems.
My friends at the Experiential Systems Laboratory: Setareh, Hamed, Laleh and Siripen deserve a huge reward for putting up with me these past few years. They have always been extremely supportive and took the time to provide enormous amounts of feedback and ideas to improve all the projects I have worked on. Many thanks to them for helping shape the ideas presented in the following pages.
A special thanks to Alex Thornton, Chen Li and Mike Carey to help me understand the teacher's side of the table at universities. Working with them has helped me understand how I can change the world in small steps by guiding students the right way. Big thanks to the students whom I worked with during the last five years here.
I owe my best friends Shekhar and Karthik for patiently bearing my tantrums for the last decade. Without Shekhar’s persistent pushes, I would have never applied to graduate school. Big thanks to them for introducing me to the world of music, which colors many of the gray days.
Hearty thanks to Amrish, Ronen and Liat to introducing me to various events during my first days at the USA. They helped me explore various aspects of life in UCI, which I would have never been exposed to otherwise.
A big thanks to my dear friends Alex, Nick, Stephanie, Vinnie, Joel, Ashley, Sameer, Alegria and Reuben for all the barbecues, wine nights, dessert parties, concerts and all the other kinds of activities which made the last few years so memorable. A small apology to Alex and Nick for stopping them halfway each time we decided to run a few miles together.
Sincerest thanks to my family, and friends, Shweta, Sidharth and Natasha for helping me through the ups and downs of life. This dissertation would have been impossible without your help.
}
% Some custom commands for your list of publications and software.
\newcommand{\mypubentry}[3]{
\begin{tabular*}{1\textwidth}{@{\extracolsep{\fill}}p{4.5in}r}
\textbf{#1} & \textbf{#2} \\
\multicolumn{2}{@{\extracolsep{\fill}}p{.95\textwidth}}{#3}\vspace{6pt} \\
\end{tabular*}
}
\newcommand{\mysoftentry}[3]{
\begin{tabular*}{1\textwidth}{@{\extracolsep{\fill}}lr}
\textbf{#1} & \url{#2} \\
\multicolumn{2}{@{\extracolsep{\fill}}p{.95\textwidth}}
{\emph{#3}}\vspace{-6pt} \\
\end{tabular*}
}
% Include, at minimum, a listing of your degrees and educational
% achievements with dates and the school where the degrees were
% earned. This should include the degree currently being
% attained. Other than that it's mostly up to you what to include here
% and how to format it, below is just an example.
\curriculumvitae
{
\textbf{EDUCATION}
\begin{tabular*}{1\textwidth}{@{\extracolsep{\fill}}lr}
\textbf{Doctor of Philosophy in Computer Science} & \textbf{2013} \\
\vspace{6pt}
University of California, Irvine & \emph{Irvine, CA} \\
\textbf{MS in Computer Science} & \textbf{2011} \\
\vspace{6pt}
University of California, Irvine & \emph{Irvine, CA} \\
\textbf{BE in Electronics and Communications} & \textbf{2002-2006} \\
\vspace{6pt}
Sir MV Institute of Technology & \emph{Bangalore, Karnataka, India} \\
\end{tabular*}
\vspace{12pt}
\textbf{RESEARCH EXPERIENCE}
\begin{tabular*}{1\textwidth}{@{\extracolsep{\fill}}lr}
\textbf{Graduate Research Assistant} & \textbf{2007--2013} \\
\vspace{6pt}
University of California, Irvine & \emph{Irvine, California} \\
\textbf{Research Intern} & \textbf{2010} \\
\vspace{6pt}
Microsoft Research & \emph{Mountain View, California} \\
\end{tabular*}
\vspace{12pt}
\textbf{TEACHING EXPERIENCE}
\begin{tabular*}{1\textwidth}{@{\extracolsep{\fill}}lr}
\textbf{Teaching Assistant} & \textbf{2007--2013} \\
\vspace{6pt}
University of California, Irvine & \emph{Irvine, California} \\
\end{tabular*}
\vspace{12pt}
\textbf{SELECTED HONORS AND AWARDS}
\begin{tabular*}{1\textwidth}{@{\extracolsep{\fill}}lr}
\textbf{Google, Ph.D. Fellowship} & \textbf{2012} \\
\vspace{6pt}
Google, Inc.\\
\textbf{Hitec Octane Entrepreneurship Competition} & \textbf{2008} \\
\vspace{6pt}
Second Prize & \emph{Irvine, California} \\
\end{tabular*}
\pagebreak
\textbf{REFEREED CONFERENCE PUBLICATIONS}
\mypubentry{CueNet: a Context Discovery Framework to Tag Personal Photos}{Mar 2013}{ICMR 2013}
\mypubentry{Visualizing Progressive Discovery}{Mar 2013}{ICMR 2013}
\mypubentry{Tolkien: An Event Based Storytelling System}{Aug 2009}{VLDB 2009}
\mypubentry{A New Approach for Adding Browser Functionality}{Jun 2008}{Hypertext 2008}
\vspace{12pt}
\textbf{TECHNICAL REPORTS}
\mypubentry{Context Networks for Annotating Personal Media}{May 2013}{ESL.UCI.EDU-TR 2013-May/01}
\mypubentry{Tagging Personal Photos Using Contextual Information}{Aug 2012}{Whitepaper}
\mypubentry{Lives: A System for Creating Families of Multimedia Stories}{Aug 2010}{MSR-TR-2011-65}
\mypubentry{Tolkien: Weaving Stories from Personal Media}{Jun 2010}{ESL.UCI.EDU-TR 2010/04/11}
\vspace{12pt}
\textbf{SOFTWARE}
\mysoftentry{CueNet}{https://github.com/wicknicks/cuenet/}
{Polyglot implementation of the CueNet ecosystem to tag faces in photos.}
}
% The abstract should not be over 350 words, although that's
% somewhat of a soft constraint.
\thesisabstract
{
Automatic annotation algorithms assign one or more labels from a candidate search space to a given input object. The collection of these labels is either constructed manually, or derived from a set of sources. It is known that the accuracy of annotation algorithms is typically a decreasing function of the number of labels in the search space. In annotating real-world multimedia objects, this search space quickly explodes if not carefully curated. In this work, we present a technique to prune search spaces yet retain a very large number of correct tags for a multimedia object. Specifically, we shall address pruning of search spaces for the problem of tagging faces in the photo media.
Photos capture the momentary state of real-world entities. Any knowledge of the state of entities in the real world at the time of photos capture provides useful context to eliminate candidates in the search space. For example, people in Japan cannot appear in photos taken at Paris, and photos taken at an academic conference will largely contain people who work in the same field.
In the real world, relations between entities change rapidly. It is not possible to use a static model of entity relationships to reason over faces in photo instances taken across a wide range of time. Thus, it is imperative that any system first \textit{discover} the set of relationships which are true at the time of photo capture before pruning the search space.
In this dissertation, we adopt a \textbf{relation-centric view} of real-world contextual information to design and implement a framework, \textbf{CueNet} to discover real-world context. The primary contribution of this dissertation is a \textbf{progressive discovery algorithm} to identify relationships between real-world events and objects. CueNet uses these relationships to reason which objects could appear in a given photos, and uses image features to confirm its hypotheses. People often co-occur across photos. We present a method to \textbf{propagate} objects from neighboring photos to a new untagged photo. We shall extend the concepts behind the PageRank algorithm, used to rank web pages, to order real-world objects, and design score propagation techniques based on event semantics. We present \textbf{experiments} on both real-world photos and simulated events. We implement a system using CueNet to tag thousands of personal photos captured by multiple users, to study the efficacy of using context discovery in pruning search spaces. We also present simulation experiments to measure the time and space complexity of large scale discovery operations. We present the convergence characteristics and results on our real-world datasets to show the utility of the propagation algorithm.
Finally, we conclude the dissertation by presenting challenging problems for future research and ideas for new applications which can be designed using the foundations of context discovery techniques.
}
%%% Local Variables: ***
%%% mode: latex ***
%%% TeX-master: "thesis.tex" ***
%%% End: ***