As the title implies, on 8-13 of January 2017 I attended “The 1st Winter School in Computer Science and Engineering on Computer Vision” At the Israel Institute for Advanced Studies in Givat Ram Campus in Jerusalem, Israel.
This conference was very interesting, inspiring and motivating.They had a dream team of speakers from the vision community. I wrote a lot of notes but here is a brief summary.
The goal of this post is to summarize the contents and provide a source for all of the useful links from the winter school.
Video lectures are available on youtube in this Link.
Day 1
Prof. Jitendra Malik from UC Berkley gave a talk via skype called “Deep Visual Understanding from Deep Learning” surveying some of his most recent work on
- Feedback Architecture
- 3D understanding
- Predicting and self-supervision
Selected quote: “Object detection is nearly solved’
Next Prof. Rob Fergus from NYU and Facebook gave a very good “Introduction to Convolutional networks” starting with LeCun’s work and working his way through non-linearity, pooling, and choosing learning rates. He stated that the main challenge in this field is the question of “how to design the network?” (How many layers, filter size, number of elements in each layer etc.) The main conclusion is that deeper networks are better.
Selected quote: ” Its not about the parameters, Its about the depth”
After lunch, there was a great panel which presented classic papers in computer vision. The panel included the legendary Takeo Kanade from Carnegie Mellon (which later came to talk to me about my poster and gave me some great ideas), Shree K. Nayar from Columbia and Michal Irani from The Weizmann Institute of Science.
On this day I presented my poster, including a short spotlight presentation ( 5 minutes).
Best poster of the day (in my opinion) was Shiry Ginosar “Century of Portraits: A Visual Historical Record of American High School Yearbooks“. Here is a small teaser: she made an Afro haircut classifier (those who knew me during 2001-2002 know why this is very sentimental 🙂 ).
Day 2
Prof. Yair Weiss from The Hebrew University of Jerusalem opened the day with a very interesting talk titled ” Neural Networks, Graphical Models, and Image Restoration”. He presented his work on denoising and the BM3D. He emphasized that graphical models have the advantage of being interpretable and modular. Looking into the future he said that intelligent system should perform unsupervised learning and incorporate between neural networks and graphical models because we now expect papers that do more than just improve the performance of a single task.
Selected quote: “In 1998, the keywords that were most likely to predict rejection from NIPS were: Neural Networks”
Prof. Michal Irani gave a talk about ” “Blind” visual inference”. She presented several works on the following topics
- Blind Optics
- Blind Deblurring
- Blind Dehazing
- Detecting similar objects and actions
- Analyzing and segmenting video without constraint
All of these papers were significantly different from each other yet seem to share the principle of searching for patch \ regions reoccurrence and similarity.
Selected quote: “while this has nothing to do with deep neural networks, If we want to scale up we need to learn how to incorporate these principals into neural networks”.
After lunch Prof. Takeo Kanade gave an amazingly inspiring talk titled “Fun Research in Computer Vision”. He talked about his moments of fame with his Eyevision system for the 35th super Bowl and his guest appearance in the Bruce Willis movie “surrogates”. Next, he addressed fundamental questions such as ” what is your wish as a researcher\engineer?” he presented the problematic “expert thinking” and said
selected quote: “Think like an amateur, do as an expert“
He then continued to blow our minds with grandiose projects (from the late 70’s – present) of his autonomous car, 3D Dome, 4D digitalization, car lights that avoid rain or snow reflections and much more.
Another selected quote: “Newness is not virtue, usefulness is”
He finished with the story of how he had a student that did some very simple work, so simple he thought it wasn’t worth publishing but because the student insisted he approved its publication in an obscure conference. It turned out this was the famous Lucas-Kanade optical flow algorithm that has over 14000 citations.
Finally, he signed off with Allen Newell quote: “Problems are waiting for you to solve”.
INSPIRING! ! !
(A photo of Takeo, Yair, Michal and me – Only on the last day I decided I can’t go home without one, now I have a picture with celebrities ! ! ! )
After this amazing talk, there were some more student spotlights and poster sessions followed by a visit to Mobileye, where they presented their cutting edge technology followed by Orcam.
Day 3
The day started with a great talk by Ira Kemelmacher from the university of Washington titled “People modeling from Historical Footage”. Her goal was to create 3D models that move and act like real people. Her 3D Obama and holocaust survivors was amazing. I also loved the HoloChess virtual and augmented reality capstone project. I can’t wait to start playing around with the Face Movie feature in picasa (which I did not even know exists).
selected quoute : “Rembrandt was the creator of the selfie”
Next, Cordelia Schmidt from INRIA Grenoble gave a talk about “Action Recognition and automatic video understanding”. she explained that the main difficulties are in large variations in appearance, manual collection of training data and that action vocabulary is not well defined. She then presented work on the subject.
After lunch, Cordelia Schmidt, Rob Fergus and Alexei (Alyosha) Efros held a panel about “Deep Learning in Computer Vision” in which Alyosha emphasized the importance of data and Rob presented a predictive model of a simple world “Physnet” to predict falling cubes. During the Q&A portion of the panel, Yair Weiss said (when asked about the future of deep learning) ” Neural nets would be like matrix multiplication – a tool like many tools”
Selected quotes (by Alyosha): “The Revolution will not be Supervised”, “ConvNets is a particular useful black box”.
Best poster of the day (in my opinion) was by Tal Remez titled “Deep Class Aware Denoising” where he used convolutional networks to denoise images according to their class – very simple with some interpretation of what the layers are doing. A close second best was Roy Or-El with a poster titled “Learning Detailed Face Reconstruction from a Single Image”.
Day 4
The day started with a MIND BLOWING (!!!) talk by Prof. Shree K. Nayar titled “Computational imaging”. He said that there is a paradigm shift from traditional imaging to computational imaging which uses optics that encode additional useful information with new functionality and reduced complexity. During the talk, Shree answered the following questions (the answers to all of the questions is yes and a link to each his paper\ product in brackets)
- Can a camera capture everything around it? (Catadioptrics)
- Can a camera capture bright skies and dark shadows? (Dynamic Range: Assorted Pixels)
- Can a camera take a photo with a billion pixels? (Gigapan demo, Gigapixel)
- Can a camera see what you see? (The world in an eye)
- Can we refocus a photograph?
- Can a photo be focused everywhere? (Focal Sweep)
- Can a camera separate direct and indirect light? (Direct and Global)
- Can a camera power itself? (Eternal video)
- Can a camera be flexible? (Elastic Optics)
- Can a camera Educate? (BigShot ,Cambits– I can’t wait for my son to be old enough)
He finished the lecture by declaring that ” The future of Imaging is bright”.
Selected quote(s):
“We talked about neural networks … one of the enabling factors is the sensors“
“We are in the middle of an imaging revolution”
“SIGGRAPH is a lifestyle, it is not a conference” (joking)
Next Prof. William (Bill) T.Freeman from MIT and Google gave a talk about ” Accidental Imaging and motion magnification”. He talked about the well known PhotoScan App, Motion magnification (which has a TEDx talk) but also about
- Pinhole and Pinspeck cameras
- Edge cameras
- Surface cameras
He showed some results for “Seeing around the corner”.
What I found fascinating most about this talk is the way he looked at problems differently (Generating images through shadows rather than light)
Selected quote: “I want to take a picture of the earth by looking at the moon” (using a satellite is cheating!)
After lunch, we went on a tour of the old city of Jerusalem and took a great photo near the Kotel \ Western wall\ Wailing wall (There were a lot of other great photos but i think this one was the best one)
After the tour, we went to visit Intel in the Perceptual computing group. They presented fun demos of applications that use their RealSense RGBD camera. The demos included Asus zenbo, Project alloy (which I didn’t get to try because the line was sooooo long, I can only guess how awesome it was) and a fun balloon popping game using oculus Rift. Here is a picture of me trying to pop some balloons:
Finally, we all went out to Machne Yehuda and drank some beer as it was the last night together for most of us.
Day 5
The day started with a talk by Richard (Rick) Szeliski (which is literally the guy who wrote the book on computer vision) titled “3D Reconstruction for Image Based Rendering Computational photography. He talked about
- Image-based rendering (Lumigraph)
- Virtual viewpoint video
- 360 video
- Immersive video stabilizer
- Large scale environmental capture (Photo Tourism, PhotoSynth)
- Reflection and transparency
He also mentioned the state-of-the-art in 3D reconstruction:
SLAM (Simultaneous Localization and Mapping)
- Orb-SLAM
- PSO
Multi-View Stereo
Next Alexei (Alyosha) Efros gave a talk titled “Self-supervised Deep Learning with application to image Synthesis. He started by saying “Life is good in Deepland” and formalized the framework for deep learning: Get data → Define Cost function → Train your network → Train your network → Sell your startup and become a millionaire
Alyosha focused on the first two steps and said ” King Midas did not think enough about his cost function”. He emphasized the importance of finding a way to perform self supervised learning and demonstrated a way to get context and semantics from a non-semantic task. Next, he demonstrated their ColorizeBot. He also recommended avoiding a common phenomenon caused in many DNN related tasks – The “Graduate Student Decent”. he then continued with their work on GANS.
The final panel of the conference was titled “Academia and Industry” and hosted Rick Szeliski, Ira Kemelmacher and Bill Freeman . It was not recorded (in order to allow the speakers to speak freely, or as Ira said it “This session never happened”). Each of them presented their unique story of how they juggle academic appointments and industry. Bill referred us to two sources of information on “How to do research” and “Getting an Academic Job“. I will not be very specific (just to make sure i don’t write something they are not comfortable with) I will just say that the thing I liked most about this panel is that it talked about issues I was (and I’m sure many other students in the audience) struggling with such as balancing family and work, failures and dead ends (knowing that the best people in the world had some is comforting).
selected quote (Bill): “Go and be the star”
After these great talks and panel the final poster session was held. The best poster of the day (in my opinion) was “Single Image 3D Interpreter Network” presented by Jiajun Wu.
Finally, Takeo summarized the winter school and said
“I hope you learned the passion of the speakers”
and
“You are our dream team of students”
On a personal note
This winter school was a one once in a lifetime experience. I met some amazing people which I’m sure I will see and hear from again. I learned from the best minds. During the school I felt like a little kid, meeting celebrities and getting to talk to them over lunch.
Most importantly I was inspired and motivated.
I wish to thank the organizers: Michael Rabin( General Director), Takeo Kanade(Director), Alexei (Aloysha) Efros (Codirector), Yair Weiss (Codirector). Thank you for this amazing experience.
And to all the readers who got so far – I hope you find this useful (All the links here are pure gold – hopefully they won’t break too fast). If you feel that I have missed something important, should add some more photos, have an additional link that I should add – please contact me.
Update 30.1.17: Recently I found out that one of the students that I met during the winter school also started a blog. So give Oran Shayer’s blog a visit.