Sramana Mitra: You did a PhD as well at MIT?
John Underkoffler: I did. I stuck it through.
Sramana Mitra: What was your PhD in?
John Underkoffler: My PhD was in a set of systems called the IO bulb and luminous room. The intent was to show, through prototypes, what would happen if we abandon the old keyboard and mouse paradigm for UI. By that time, the GUI and the keyboard and mouse that supported it were 15 years old. It seemed to be that we can, usefully and valuably, move past that. Inventing and using a radically more capable UI will let people do new kinds of things entirely.
Sramana Mitra: What did you do after you finished your PhD?
John Underkoffler: Just as I was finishing, the lab was visited by a delegation from the film Minority Report, which Spielberg was directing. The film’s design team led by Production Designer Alex McDowell was romping around the country looking for just emerging technologies that they could import into the film. The UI that I and others were working on at the MIT Media Lab solved for Alex the biggest problem that the film had. What were computers going to look like and how would people use them 50 years forward? That led to my becoming the Technology Advisor for Minority Report.
Sramana Mitra: It must have been great fun.
John Underkoffler: It was.
Sramana Mitra: What year was this?
John Underkoffler: This was 2000. I moved to Los Angeles on Thanksgiving and worked for the next year on the film designing and knitting together all of the future tech that appears in the film. The bulk of what I worked on was the gesture-driven massive scale computer interfaces that the characters use to detect future crimes.
Sramana Mitra: What happens after the project was over?
John Underkoffler: That film came out. As is always the case when you work on a film, it can seem fantastic day to dau, but you never really know until you’re sitting in the audience watching it whether it will be a success or not. Audiences really reacted very strongly to those scenes of future computation. That was the light bulb moment for a bunch of us that there was a real commercial opportunity here. These next generation UI systems that had started in academia and had had a public unveiling also needed to come into the world for real as a commercial offering. So we founded Oblong Industries.
Sramana Mitra: What was the genesis of Oblong?
John Underkoffler: After three of four calls from Fortune 50 companies asking if the technology that they’ve seen was real, I did realize that we should build them and make them commercially available.
Sramana Mitra: What kinds of companies were calling you?
John Underkoffler: There were a variety. Our earliest customers were Boeing, General Electric, and Saudi Aramco.
Sramana Mitra: What were the use cases for which they were looking for this technology?
John Underkoffler: They were radically different and that was part of what was exciting and would later on become a clue for us about what the product was. The stakeholders that were approaching us realized that they had problem domains where the solution was not just faster CPUs or bigger databases, but actually better human-in-the-loop UIs.
With Boeing, we built a large-scale, gesture-driven control system for a global scale simulation. For Saudi Aramco, we built a front-end for the world’s largest oil and gas reservoir simulator. For GE, we built supervisory systems involving lots and lots of pixels and new kinds of inputs around smart grid energy management.
Sramana Mitra: Interesting. These three use cases were quite diverse? What would be a reasonable product to build based on these customer inputs?
John Underkoffler: As we built each of these systems and watched how the customer used them, we did begin to see that. The thread that was common to all of them was that these systems, uniquely, allowed people to work together. For Saudi Aramco, it meant that, for the first time, you could get together the Drilling Engineer, Site Manager, and Chemical Engineer in a room all looking at the same stuff taking in a vast amount of visual information, and all of them able to manipulate it by reaching in with a gesture.
For Boeing, some of the stakeholders would stay in the room every once in a while and do their monthly budgeting plans. It was the only room in the entire facility where a bunch of people could throw up their laptop screens side-by-side and see an adequate amount of information at the same time. The idea of collaboration emerged as the genesis of the new product called Mezzanine.