Stanley Osher: Mathematician with an Edge

Stanley Osher is an extraordinary mathematician who has made fundamental contributions to applied mathematics, computational science and scientific computing and who has cofounded three companies based, in part, on his research. He has applied his pioneering work on level set methods and other numerical methods for partial differential equations to the field of image processing and, in particular, to video image enhancing and movie animation. He has been featured prominently in the scientific and international media such as Science News, Die Zeit and Los Angeles Times. He is perhaps the most highly cited researcher in the field of scientific computing.

He received the NASA Public Service Group Achievement Award, Japan Society of Mechanical Engineers Computational Mechanics Award and the SIAM Pioneer Prize. He was an invited speaker at the International Congress of Mathematicians.

He is currently Director of Special Projects at the Institute for Pure and Applied Mathematics (IPAM) at the University of California at Los Angeles and Director of Applied Mathematics.

The Editor of Imprints interviewed Stanley Osher on 17 December 2003 when he was an invited guest at the Institute’s program on image processing and gave a public lecture. The following is based on an edited transcript of an interview in which he talked about the fascination of applied mathematics and his total dedication to research and applications.

I: Thank you very much, Professor Osher, for agreeing to be interviewed. In which area did you do your Ph D?

O: I did my Ph D in an esoteric area in functional analysis, which is in pure mathematics. I left it immediately and switched to numerical analysis after my thesis. I was lucky enough to talk to some people, including Peter Lax, who suggested the numerical stuff.

I: Did you find your real inclinations in applied mathematics?

O: Yeah, but it did not happen until after I got my degree. I liked everything and I specialized more after my Ph D.

I: Did you use functional analysis later on in your work?

O: Yes. The first thing I did in numerical analysis was an application of Toeplitz Matrices. It used functional analysis and was short and elegant.

I: You switched to applied mathematics and then eventually to something very practical like applying to movie animation.

O: This just happened – that’s the way research leads you. You cannot predict these things. Together with colleagues and students, I was using the level method, which is a way of determining how surfaces such as bubbles move in three dimensions, how they merge and so on. You can simulate the flow of bubbles, planes and things like that. It happened that people in the movie industry got interested in this stuff.

I: Are these are pure mathematics problems?

O: It’s a way of representing surfaces and has connections with differential geometry. People prove theorems about these things. They have applications in many areas including fluid dynamics and quantum mechanics. They arise in the movie industry

because you want to see how things merge and split like in explosions or rising bubbles. You know, our Governor, Schwarzenegger, used in his latest movie, a lot of these methods done by my former student Ron Fedkiw.

I: Did you have to use the mathematics of many different areas to solve those problems?

O: Sure. The key thing I would say is partial differential equations, elementary differential geometry, numerical analysis. The language of applied mathematics is differential equations.

I: How did you get into the movie animation business?

O: We had a week of movie industry people coming into UCLA (University of California at Los Angeles) giving lectures about what they did and, in fact, imaging science was highlighted by the American Mathematical Society one year. There was a week in that stuff. We invited people from the local movie industry and they were interested in what we were doing. They wound up arguing with me. The water in the “Titanic”, which won many Academy awards, was very bad. It was old-fashioned stuff. Their people came to talk about that and so we decided we could do better than that. In recent movies, the water is much more realistic. The first movie that actually used sophisticated water was “Ants”. Now level sets are used by movies like “Shrek”, “Terminator” and many blockbusters. My former student Ron Fedkiw is doing this movie animation stuff very well. The stuff that my colleagues and I do has applications in many other places besides the movie industry.

I: I believe that another of you dramatic achievement is the use of mathematics in catching criminals. How did this come about?

O: Well, I was living in Los Angeles when the city went up in smoke. There was a big riot in Los Angeles after this guy (Rodney King) was beaten up by the police.

The riot resulted in people being arrested for looting and beating up passers-by. There was a video recording of the bad guys beating up truck driver Denny and it showed a speck on the arm of a man throwing a brick at Denny.

It turned out that I had a friend who knew the District Attorney or somebody, and I

was then doing video image enhancement with my colleague L. Rudin, We were able to resolve the speck into a rose tattoo and it was a great application of what we were doing. After the Denny case trial (the tattoo led to the conviction of the suspect) we had a lot of media publicity and our company specialized in the area of image enhancement. Eventually I sold my share of the company to Rudin. He has a package on video image enhancement which is used by the police around the world, and he’s quite successful.

I have left this business. It was quite fun and related to mathematics. Image processing is the real world and the graphics that is manufactured is the fake world. You want to find out what the image really is.

///

I: Do you actually go and seek out those problems or do people come looking for you?

O: It’s hard to say. Sometimes it’s serendipity. Things just happen. I was lucky. In all my years of doing science, I managed to work with the right people who knew what the problems were. For example, I knew nothing about image processing at all after my PhD. Then this guy Rudin came over to me and asked me about some work I had done in fluid dynamics on supersonic flow and shock waves. I asked him what he wanted to know and I got fired up. He was a computer scientist and he realized that shock waves had something to do with imaging. It was a fantastic observation and our collaboration worked out well.

I: The scope of movie animation has just opened up, hasn’t it?

O: Yeah. But I’m not sure this is the best field of application for somebody to work in because the market is small except for video games which is a big business. But video games require real time imaging. What we do is not real time; it’s too slow. That might change.

I: It could be just a matter of computing power.

O: Yeah, yeah, and also hardwiring in level set methods, which may come in time.

I: Pattern recognition used to be sort of very big.

O: I’m not sure of the definition of pattern recognition, but image processing is related to it. The key idea in everything we do is about “edges”, which characterize images. Edges, and now, textures. If you look at a table, for example, the flat part is not very interesting but the boundary of it is. It’s the discontinuity. If I look at you, I can see you because of the outline. The outline is very important and the mathematics that was used in other areas of science like fluid dynamics specialise in things like edges and boundaries. And now images.

I: Does it mean that if you have a vague or blurred object, you can always refine it?

O: Yeah, but it’s usually difficult to do so when you have edges because near the edges you will get spurious oscillations. The techniques we have developed, which came originally from fluid dynamics, are now useful for removing those unreal artifacts.

I: How do you know that what you get is the actual object?

O: That’s a good question. But you can get enough science behind it and you can prove theorems, algorithms converge and stuff like that.

I: You look at the picture and it’s so blurred. And then you do this and you get that. There’s a lot of faith in that.

O: I won’t disagree with that completely. But when I drop something, I expect it to fall and not go up. The probability is not zero that it might go up. But with a high degree of certainty, you can say that this is a realistic picture.

I: Does that mean there is a probabilistic element in it.

O: Oh, yes. That’s always true with nature.

I: For example, in the police case, couldn’t the guy dispute your picture?

O: Yes, but the jury believed us. Fingerprints are also subjective. Fingerprints are very good examples of image processing of a certain sort.

I: What about problems in voice recognition?

O: It’s a different kind of mathematics. I’m not an expert in it. The techniques we use in image processing involving differential equations have never been used for sound. But I understand there’s some very new stuff along these lines. It’s only beginning and just developing. The Institute for Pure and Applied Mathematics (IPAM) at UCLA is running a program on sound and how the ear works and the related mathematics.

I: What about artificial intelligence?

O: It’s highly dubious. Among the things I do is designing computer chips and you use mathematics which is not intuitive. Engineers cannot intuit what the right shape should be. One of my engineering friends said this is artificial intelligence. The computer would do something the engineer cannot imagine. That’s not the usual definition of artificial intelligence. If you have solid mathematics, you can do things that are intuitively not obvious. But the experienced engineer makes a leap of understanding. If you call that artificial intelligence, so be it. But it’s not the kind that people use to talk about.

I: The imaging business is also very important in astronomy, isn’t it? They take pictures which are so faint.

O: Very much so. Astronomers have done very good work in this area. They had to over the years. The early work in this stuff in astronomy was very fascinating. They did things which are precursors of what is going on now.

I: You mean the astronomers actually did something mathematical?

O: Yes. That’s very often the case. The good thing about being an applied mathematician is that when you work in different areas you find very brilliant people in other areas of science develop mathematical algorithms without realising what they are doing and which can be generalized to other areas. So it’s a question of language.

I: Are more astronomers talking to mathematicians now?

O: Actually we’ll be running a program on computational astronomy in IPAM a year from now. Astronomy is just one example. Throughout science mathematics is playing more and more of the key role. In the Institute in which I am involved, its mission is to do interdisciplinary work. People from different areas of science have problems which they think are mathematical. And our goal is to make mathematics out of it. Now they believe that we can do something, mainly because of the computer.

I: That’s interesting. So now mathematics is also contributing towards understanding the origin of the universe. This is something not many people are aware of.

O: Well, I’m not an expert in the field, but yes, absolutely. I think that the world is governed by differential equations.

I: Your work involves a lot of algorithms. Do you invent the algorithms?

O: That’s what we do. That’s the fun part of it. I was interviewed by the LA Times (so was Tony Chan for the same article) and I said that I wrote the algorithms that make the computer sing. I am the Barry Manilow of mathematics.

I: Talking about algorithms, some people consider algorithms to be inventions.

O: Yeah, they are. You can actually patent them now. It used to be that a patent has to contain a device with wires and everything. But I understand the US Patent Office is now more liberal. I’m no lawyer, but I know that as the years go by, the Patent Office seems more and more interested in giving legal protection to algorithms.

I: You could be rich. Hollywood would be paying you millions.

O: People work for salaries. There is money, ego and fun. It’s a very nonlinear function. I don’t know which is most important. Fun is very important. It’s a very good life. I would recommend people going into this stuff now. If you have the talent for it, it’s the best life.

I: It’s something very different, something, how do you say, non-academic?

O: In some sense, yes. You learn things, you read stuff and you learn new ideas, and you are fired up. Sometimes you deliver something different from what you have found. You have a vague idea that something interesting is going to come up. You wander around and something happens. Then you get very excited. It’s like opening a door and you don’t know what good things are behind it. You’re not sure where it’s going to end and what the level of success it’s going to be. It’s very exciting. Everyday I can’t wait to go to work. People often asked me, “What kind of life is this that work is so important?” People go on vacation. My work is vacation.

I: Do you have many Ph D students?

O: Many, disproportionately Chinese. We have many good Chinese students at UCLA.

I: Have you used your methods for something more serious like weather forecasting and earthquake prediction?

O: Yeah. The differential equations and some stuff I used to do in fluid dynamics and done by many other very good people are absolutely useful in weather prediction. I still do work on explosives and multi-phase flows and ray tracing. Physical phenomena apart from imaging is very much a part of my research with my colleagues and students.

I: What about the theory of turbulence?

O: Ah, turbulence is too dangerous. If you touch turbulence, you get burnt. One of my mentors once said he had great respect for people in turbulence, which is far more than they have for each other. Turbulence is too controversial. Turbulence has a probabilistic aspect to it, it’s statistical and that’s not my thing.

I: It’s not algorithmic?

O: It’s a different kind of algorithm, a different kind of intuition. I do understand it, but I’ve never been a big user of that stuff. It’s very dangerous even to discuss it, it’s like politics. Everybody condemns the other guy’s theory. It’s somewhat like religion.

I: That’s surprising. The methods are mathematical, the equations are mathematical.

O: Yeah. The problems are very hard and complicated. You get some very good mathematics coming out of it but I’m not sure the physics is understood in analysing what really happens.

I: I’d like to ask you a philosophical question. How does your work affect your view of life?

O: In terms of how research affects my philosophy? The basic idea is to try to make order out of this life that we live. Everyday you encounter things and it’s a messy world. The goal is to take this mess that we see and somehow “mathematize” it and make a prediction. In that sense, research has certainly affected my philosophy. I try to figure out what is going on. The most complicated thing is how our human nature operates. It will be fun to understand that. Many people I know at UCLA and elsewhere are using medical imaging to understand how the topology and shape of the brain affects its function. They use the mathematics that I am involved in. The greatest mystery of all is human behaviour and maybe it can be explained by level sets.

I: Would you say that your present interest and activity is, in some way, directed by your own personal philosophy towards finding order?

O: Yeah, absolutely. I came from being a poor boy in Brooklyn. I wanted some order in my life, to become middle-class and to have a life that I enjoy. Then I stumbled onto this thing, and wow, that’s very good for creating order. I entered graduate school in New York University in 1962 when it was a fantastic place for applied mathematics, maybe one of the best ever in the world,, All the top people from Goettingen wound up in New York and it was so exciting.

I: Was that the Courant Institute?

O: Yes, the Courant Institute. In 1962, when I entered, it was incredible. The people who were there and the atmosphere. You felt that you were doing something important. It had a very great influence on me and many other people.

I: Was Courant still there?

O: He was still around, old but still functioning. He added many people all of whom I thought of being old, something like 20 years younger than I am now. They were great people and it was a fantastic time. There were many people like me who were ethnic New Yorkers and who view becoming a mathematician as a way of becoming middle-class American citizens.

I: What attracted you to UCLA?

O: In truth, there was a guy who I was working with: Andy Majda. He is an excellent applied mathematician. He was there and they were building an applied mathematics group. Also I liked California. When I was in New York, I was always dreaming about the Beach Boys. Sunshine and California together with math was great – everything that I wanted. Over the years we had very nice people. The atmosphere is extremely good in UCLA. People who visited us commented on how well we got along each other, which is unusual in academia, It has worked out quite well.

I: You have given us a very interesting and illuminating view about mathematics. Thank you very much.

26 February 2004