when you know why and shifting the paradigm

A friend of mine visited me in Madison last spring. She is a community college biology professor near Detroit. The week after she visited, she was teaching her students about levers for anatomy class. So while I was working at school, she spent some time trying to understand these simple machines.

She and I talked about how we both have a hard time using information we have learned until we feel there are no gaps in our understanding. For me, if I wasn't careful at understanding why something is the way it is, I have to take giant leaps of faith in actually applying it. And being a skeptic, I'm not particularly good at leaps of faith! You can ask my sewing teacher-- I'm always questioning the instructions she gives me. But sometimes, all the inside-out seams and pattern matching suddenly make sense, and I realize she was right all along!

When some fundamental thing just clicks, it's not always obvious why things suddenly make sense. We all have a paradigm which we accept, and once something new fits into that paradigm, we feel it "click." But this is also a hindrance to shifting the paradigm itself.

I have been reading a very good book about incorporating Structured Controversy into the classroom, by Johnson Johnson & Smith 1996. Intellectual conflict is believed to be a central part of learning to have an open mind and to be a critical thinker. Piaget in 1950 proposed that cognitive dissonance-- or inconsistency among a student's understandings of the world and what they see and hear from others-- is what motivates transitions from one level of reasoning to higher level which is more broad and flexible, and which causes "a shift from egocentrism to accommodation of other people's
perspectives."

It should be the lifetime goal of the researcher to achieve a balance between on one hand having a paradigm that allows you to incorporate and use new knowledge efficiently and on the other hand being open to shifts in your paradigm in order to follow the really breakthrough ideas.

when a proof becomes a proof

Just this week, a researcher at HP labs named Vinay Deolalikar has claimed to solve the P ≠ NP problem. It has been conjectured long ago that P ≠ NP, and probably every researcher in algorithms, signal processing, and computer science has some work that relies on the belief that P ≠ NP, so if he has proved it there will be world wide celebrating. It is also one of the Clay Math prize problems, so (again, if the proof is accepted) Deolalikar will win the million dollar prize.

What I find so exciting is now the process of accepting or rejecting the proof as a community of researchers. This blog post discusses possible issues in the proof that have been raised by various computer scientists and mathematicians. Hopefully this process will help the public understand something about the way that math and research really works-- as that blog quotes from mathematician Yuri Manin, "A proof only becomes a proof after the social act of accepting it as a proof." Engineering, science and even math are social endeavors in many ways, but the most important is that ideas only begin to impact the world when the whole community begins to understand and accept them. The point in time when the community embraces the ideas, and begins to voice the implications enthusiastically, is the point at which the idea really guarantees its spot in the library of scientific knowledge.

compressed sensing

I created this audio illustration of compressed sensing as part of a demo for an NPR bit about compressed sensing, which Jordan Ellenberg was asked to participate in after the NPR folks saw his Wired article. Well yesterday the piece was on All Things Considered! You can listen here.

Authenticity

I watched this talk by Jesse Schell, a professor at CMU. In it he talks to game developers about how games are moving forward into the future. He has some very interesting ideas about bringing gaming into our every day lives-- even to help us become better people. But what I found most interesting was his reference to a book called "Authenticity." Here is a transcript of this part.


"We talk about realism, and we know yeah realism can make your game better sometimes if it's the right kind of realism. But normally we're thinking about visual realism. There's a lot of other kinds of realism that can come into a game. ... Guitar hero you play on a real guitar, and webkins with a real stuffed animal, and even the X-Box achievements is outside the reality of the game-- there's a higher level scoring system.

"But it's not just us that were snuck up on by this reality thing, and it's not just happening to us. Go look at TV-- the people on TV, their heads are spinning--everything has turned into reality TV. Go to the grocery store-- it's not just groceries anymore, it's organic groceries-- more genuine, more real groceries. Go to McDonald's, well you could get a Big Mac, OR you could get a real burger, the Angus burger, made with real this and that and whatever. Everything is suddenly about reality.

"Now, what's going on? Is this just how it's always been? Well I found this really interesting book, called Authenticity, by the guys who wrote the Experience Economy. Gilmore and Pine put forth this interesting concept-- that the most valuable thing in products today is "Are they real, are they authentic"-- which is a bold hypothesis. Then they go further and they say, well, why is it? Why now? It didn't always used to be this way-- certainly that's not what sold stuff in the 80s. It wasn't reality and authenticity that sold stuff then.

"What is it now that people are demanding reality and demanding authenticity? They're arguing that all this virtual stuff that's been creeping up on us over the last 20 years has really cut us off from nature. We're cut off from nature, we're cut off from self-sufficiency. We couldn't be self-sufficient if we wanted to. We don't know how to do it. We live in a bubble of fake bullshit, and we have this hunger to get to anything that's real-- Even if the best we can do is a Starbucks mocha with real Swiss chocolate, we'll take it, Oh it's real! Look how real that seems to me relative to what I'm used to.

"So there's this idea that maybe there's this hunger for reality. You might believe it or not, but once I read this authenticity book, I started seeing it everywhere, everywhere I looked--every ad was about their product being the real one. And I go to see the movie Avatar, and you might say Oh Avatar, that was cool special effects and that was really a lot of fluff, but it's the movie that's made the most money of all time and it's got a good shot at best picture-- What's this movie about? This movie is about the question of: We know technology cuts us off from the real world, and the movie addresses the question of can we then use that technology in order to penetrate back into reality and back into something genuine. So it may be this movie is not just fluff, it may be this movie is resonating with something important for people."

Women in Physical Education: Title IX

In 1974, Kathleen Saunders-Nordeen was appointed as the first Director of Women's Intercollegiate Athletics. In 1975 the Oral History Archive did an interview. Here are some excerpts.

Interviewer: "I do know that women have had a hard time at the University from way back-- the whole coeducational idea was vigorously fought at the turn of the century by the administration. Chadburn was for really considerable segregation of the sexes in terms of practically all activities."

KSN: "I think to a great extent, women perpetuated that."

Interviewer: "How do you mean?"

KSN: "Back in the late 20s or early 30s, we had some of the strongest leaders in the country here on our campus as far as Physical Education is concerned. And women's athletics have grown out of Physical Education, because that's where the leadership was, but they've also been slowed down a whole lot by Physical Education leaders and the feeling that it was wrong to put all your efforts and all your facilities and all your finances into making a small number of people really highly skilled and really concentrating all your efforts on a small number of people.

"The philosophy was that it was far better to have a thousand women who were really mediocre athletes, given the opportunity, and it was as if you couldn't possibly do both."

Interviewer: "Why not?"

KSN: "That's the point. Now maybe at that point resources were so limited that you couldn't, or maybe it was just real narrow kind of thinking. But this thinking, and I think Blanche Trilling who was here on our campus and does deserve a lot of credit in Physical Education, has been one of the leaders in the country who kept women's athletics from progressing.

"From then on, for 40 or 50 years, nothing happened in women's athletics. Before that, we had very strong women's teams in a lot of sports-- there were 250 women out for rowing in 1920. Of course they were largely intramural teams, but the interest was there, and the skill was there.

"But things just kind of died for about 40 or 50 years, and during that time the men's programs were still growing. And now when women are trying to get back into facilities and back into budgets, it's very difficult. We're trying to crack back into something that's been allowed to grow and we've been just dragging our feet."

==========

KSN: "The women's crew has done really very well, they've gotten themselves recognized nationally, which is very good in 3 or 4 years."

Interviewer: "That recently? Even though they've been rowing since the 20s?"

KSN: "Well, they rowed in the 20s, and then there was no rowing again until about three years ago when several women students from one of the lakeshore dorms decided it might be kind of fun to get out there in those shells. They came to me at Lathrop and we sent them to the men's crew coach who said, OK, We'll give it a try, see how many people you have interested. I don't know that he thought they'd make it through the first winter, because he really zapped it to them with the same kind of training that the guys have-- which meant running several miles in below zero weather, working in the tank rowing, and working on the odometers and lifting weights and doing all these things. Instead of that having that make them lose interest, it grew, because women really did want to push themselves and see how good they could become."

==========

KSN: "I'm sure the women's movement has helped a lot, because it's okay for women to be good at almost anything they want now. Physical activity just happens to be one of those things. It's kind of a new freedom or something."

==========

KSN: "Right now, it looks very much like the interpretation of Title IX is going to be that if there are scholarships given for men in any institution, then you've got to do it for women also. Right now the income producing sports are not exempted from the guidelines of Title IX; It looks to me that they probably will be exempted, if the guidelines come out at all.

"Now my understanding is that they're getting kicked around so much because of pressure groups like NCAA who don't want athletics included in it at all, that we may never get the guidelines out, which would be really a shame.

It was pointed out to me a couple weeks ago that even if the guidelines never come out, never get signed by the president, the impact that they've had and the threat that they've had to institutions has done an awful lot to programs for women getting started, just the idea that they were there. I think also some institutions, we have to give some credit, it's not just the threat behind Title IX but the concept that was pointed out so vividly by Title IX that women's athletic programs really have been short-changed and it really is discrimination."

Shattering Coefficient and VC Dimension

Here is an excerpt from a book called "Reliable Reasoning, Induction and Statistical Learning Theory." I read this book about a year ago and really liked its description of VC dimension and shattering coefficient in statistical learning. These aren't easy concepts, so I wanted to put the description here for reference.

The setup is that you have a set of hypotheses or rules, C, that are used in order to classify data. For example, if we are trying to decide whether or not a patient is healthy, we may look at his resting pulse and systolic blood pressure. Then I could make a rule that says if his pulse is above 100 and his systolic blood pressure above 140, then he is not healthy. A rule like this helps us make a decision whether or not the patient is healthy. Now assume you do not have expert information (that high blood pressure is bad) but you have data points-- measurements of pulse and blood pressure-- and labels for each data point which say "healthy" or "not healthy". From here we would like to learn a rule that can classify other patients as healthy or not healthy, using the same measurements. From p23:

"How can we use data to choose a good rule from C? One obvious idea is to select a rule from C with the least error on the data. Then we use that rule in order to classify new data. This is basically the method of enumerative induction. Our question then, is: How good is this version of enumerative induction for choosing a rule from C?

"Clearly, it depends on what rules are in the set C from which the rule is to be chosen. If all possible rules are in that set, then there will be many rules with least error on the data, and these will give completely different advice about new cases. ...

"Of course, restricting the rules in C runs the risk of not including the best of all possible rules, the rule with the least expected error on new cases. ...

"There are other possible inductive methods for choosing rules-- methods that do not just choose the rule with the least error on the data. One such method balances data-coverage against something else, such as the simplicity of a given rule. ... but now let us concentrate on what is needed for the sort of enumerative induction that simply chooses the rule in C with the least error on the data.

"So now consider the question of how the rules in C might be restricted if enumerative induction in this sense is to be guaranteed to work, given enough evidence, no matter what the background statistical probability distribution.

"The answer to this question is one of the great discoveries of statistical learning theory-- the discovery of the importance of the Vapnik-Chervonenkis dimension, or VC dimension, of a set of rules. The VC dimension is a measure of the "richness" of the set of rules and it is inversely related to the degree of falsifiability of the set.

... from p45: "What has to be true of the set C in order to guarantee that, with probability approaching 1, given more and more data, the expected error for the rules that enumerative induction endorses at each stage will approach the minimum value of expected error for rules in C?

"You might wonder whether this sort of convergence isn't guaranteed by the statistical law of large numbers. That principle implies that with probability approaching 1, the empirical error of any particular rule will approach the expected error of that rule, given more and more data. But this is not the same as what is wanted. The trouble is that, given infinitely many rules, as more and more data are taken into account, the rules endorsed by enumerative induction can change infinitely often. Even if the empirical error for each rule approaches a limit, that does not imply anything about the limit of the empirical error of the varying rule endorsed by enumerative induction at each stage. ...

"What is needed then is not just that the empirical error of each rule should converge to its expected error but also that the empirical error of the varying rules endorsed by enumerative induction should approach the value of the expected error of that rule in the limit. If c_n is a rule endorsed by enumerative induction after n data points, then what is needed is that the empirical error of the rule c_n after n data points should approach the expected error of c_n in the limit. ...

"This will happen if (with probability approaching 1) the empirical error of the rules in C converge uniformly to their expected error. ...

"What has to be true of the set of rules C for such uniform convergence? Vapnik and Chervonenkis (1968) show (in effect) that this condition is met for classification rules if and only if the set of classification rules C is not too rich, where the richness of the set is measured by what has come to be called its "VC dimension." ...

"Suppose that some set of N points in the feature space is shattered by rules in C in the sense that, for any possible labeling of those points, some rule in C perfectly fits the points so labeled. Then the VC dimension of the set of rules C is at least N. More specifically, the VC dimension of a set of rules C is the largest number N such that some set of N points in feature space is shattered by rules in C. ...

"Notice that the definition of VC dimension refers to some set of N points being shattered, not to all sets of N points being shattered. Consider the set of all linear classifications of points in the plane where YESes and NOs are separated by a straight line. The VC dimension of this set of classification rules is 3, because some sets of three points in the plane can be shattered by this class of rules and no set of four points can be shattered. Three collinear points (i.e., three points on the same straight line) cannot be shattered by this class of rules, because there is no such rule that can classify the middle point as a YES and the outer points as NOs. But three points that are not collinear can be shattered... And no four points can be shattered by this class of rules, so the VC dimension of these linear rules is exactly 3. ...

"Some other examples: The VC dimension of the set of all linear separations in D-dimensional space is D+1. The VC dimension of the set of all inner rectangles in the plane is 4. The VC dimension of the set of all unions of rectangles in the plane is infinite.

"So, that is what VC dimension comes to. Vapnik and Chervonenkis (1968) show roughly that enumerative induction is guaranteed to work no matter what the background probability distribution if and only if the classification rules in C have a finite VC dimension."

orthogonal

My friend Todd posted this on facebook:


' I get the same reaction when I use the word "orthogonal". '