If you need a laugh

I ran across this hilarious story from the onion 21 years ago: Scientists Abandon AI Project after seeing The Matrix. I can't believe that was 2004 and the first Matrix came out in 1999. What!! By the way I realized recently that funny links are basically impossible to find now because of SEO. Every time I search something that's not extremely specific, the first 100 links have mostly ad-ridden "magazine" pages. If you have suggestions on great places to find people being clever and insightful and funny without one million ads, please let me know.

Sarah Goddard Power award

Today I was honored to receive the Sarah Goddard Power award from the University of Michigan. Here are the remarks I gave at the ceremony, with some additional shout-outs [in brackets].

It is truly an honor to receive the Sarah Goddard Power award from the Academic Women's Conference and the Center for the Education of Women+. I follow many amazing women engineers who are an inspiration to me – including past winners Jenna Wiens, Valeria Bertacco, Rada Milhalcea, Dawn Tilbury, and Martha Pollack. I can only strive to make an impact on as many lives as these women have, inspiring women to bring their skills and voices into engineering. Right now this work is critically important in my field, as technology in machine learning, artificial intelligence, and computing changes our world on a daily basis. Technology is often thought of as an objective pursuit, where the goals are clear and well-defined, and only those who are “math geniuses” can make a contribution. This couldn’t be further from the truth – we are constantly defining the goals and values of our technology, and diverse voices are key to creating technology that lifts us up as a whole society.

I have been studying or practicing engineering and computer science since I became an electrical engineering major as a freshman in college. I was very lucky early in this career to have some of the most compassionate and supportive male engineers as my educators and bosses. [Thank you especially: Don Johnson, Rich Baraniuk, Rob Nowak, Ed Knightly, John Treichler, and Mani Srivastava! What an impressive list of strong mentors I had before I even started my PhD.] They saw an ability in me that went beyond book learning, and they encouraged me to cultivate it. Without them I would definitely not be here today. At the same time, I have to point out that I had not a single female professor in any of my technical classes until my second semester of PhD. For years I had been telling people that I had supportive male professors who inspired me to be a professor. But it was Gloria Mari-Beffa, my math professor at the University of Wisconsin, who showed me that this dream was a reality. It's really hard to understand the power of role models who "look like you" until you experience it yourself. [I absolutely loved that she wore barettes in her hair. It made me feel like I could wear barettes and be a professor, too! And the way she supported me, as an engineer in her math class. Ask me for another story about her if you're curious.] Having Gloria as a professor created a desire in me to use my career to be that role model -- to find the young women who have the ability, and give them the confidence to keep going.

I want to tell you a little about two important women in my life -- my grandmothers. Neither of my grandmothers had schooling past high school, but both of them took work very seriously. My mom's mom was the only person in her family who had a job during the great depression, working at a doctor's office. She was married at age 25 and had three children under three years old when my grandfather was shipped to Europe in WW2. While she would have liked to work when they were older, my grandfather expected her to do all the work at home on top of any other work, and so she never tried. But she pushed her own three daughters in their schooling, and made sure they all went to college.

My dad's mom was very good at math. She tutored other kids in her school. She wanted to go to college, but her father didn't think that was a good idea, so she got a job in retail. She was married at age 26 and had two sons. When my dad was 12, she took a class in the comptometer, a specialized computing device that was used in many financial fields. Then without telling my grandfather, she went to apply for jobs at the banks in downtown Cleveland. She told me this story many times. She went to each one with her credentials, and one by one they turned her down. One of them told her straight away that she was too old (at 42 years old). As she approached the last bank, Key Bank, she thought to herself she might as well not even try. She crossed the street to head back home, changed her mind and crossed back again, crossing the street three times before she finally walked in. They hired her and she worked there for 20 years. Her boss always bragged about how lucky he was that he found her.

Both my grandmothers encouraged me to study hard and go to college. My mother saw that I was good in math and found ways to encourage me every step of the way. My mother is the most important woman in my life, and she certainly has her own story, but for me her biggest gift was this encouragement. In 8th grade I took an advanced math class at the high school, and there were many 8th grade girls in that class. But the teacher would try to trip us up, and he would high-five with boys when girls got answers wrong. It really ticked me off, but I always got the answers right [lucky breaks] so I didn’t have the wherewithal to see it for the discrimination it was. When I was deciding what high school to attend, my mom met all the math teachers at that public school. They were all age 60+ white men. So she gently steered me toward a private school, which happened to have one of the state's most decorated female math teachers. I loved her class and I flourished. [Thank you Ms. Hund, now Mrs. Radiel!]

When I was thinking about what I wanted to do in college, my mom said I could do anything but nursing and teaching. I could be a doctor or a professor or anything else, but since she was only allowed to do nursing and teaching, she wanted me to consider a broader range of options.

Today, less than 10% of electrical engineering professionals are women. Our undergraduate program has under 25% women, and our faculty has 15% women. I wish I could tell you these numbers have changed in the 25 years since I started college, but they have not. But there are pockets of hope -- I see many of my college female friends excelling in their careers, starting companies, and encouraging more women to join them. In today's climate, this work is so critical. There are many girls and young women who are capable of becoming the country's best engineers if simply given opportunities and encouragement.

I would like to thank my colleagues here at Michigan who have been so supportive of me, especially Jeff Fessler, Herb Winful, and Fred Terry who graciously nominated me for the award, as well as the staff Shelly Feldkamp and Kathy Austin [and Beth Lawson, who provided strong guidance for me at the very start of my faculty career], who do such a great job supporting the faculty in our area. Thank you to my cohort of amazing female colleagues who provided a community for me in Electrical Engineering and Computer Science: Johanna Mathieu, Emily Mower-Provost, Necmiye Ozay, and Jenna Wiens. And finally, I want to thank my husband and daughter who are here today. My husband supports me in every way possible, and I know his pride in my work shines through to our two beautiful daughters. And I am of course so proud of my daughters, who give me hope for the future. My daughter made me this bracelet, which I wear to remind myself of my motivation to leave a better world for all our children. Thank you again for this honor.

Black Lives Matter

Professor Rob Sellers, the Vice Provost for Equity and Inclusion and Chief Diversity Officer at University of Michigan, wrote a candid and unguarded reflection on the struggles of African-Americans today and throughout history. He wrote it 4 days after the death of George Floyd, as protests were spreading across the nation and the globe. Here is an excerpt:

"Some people argue that this country, while being built substantially by us, was never meant for us. (They are not wrong.) As such, some of these same people believe that other-worldly optimism is a sign of weakness and is ultimately what has sealed our fate as a people. They question the wisdom in holding out such faith and hope for change in a system (in a society) that has time and time again demonstrated that Black dignity, Black bodies, and Black lives matter a little less. (It is hard to argue with the logic of the question.)

"These times really do raise for me the question of how long must we wait, plan, work, march, agitate, forgive, and vote before we have a society in which all lives matter equally, regardless of race or color? In my bone-weary tired state this morning, before I even got out of bed, I asked myself why should I continue to fight to try to change a system that has proven time and time again that it simply does not regard me and people who look like me as fully human."

Professor Sellers' essay is not like the statements denouncing racism that I received from the university president, dean of my college, and chair of my department. It was honest, heartfelt, and raw. It touched me deeply and brought me sorrow.

It also made me proud to work alongside people like Professor Sellers in support of turning this country into a better place for everyone, especially African-Americans who have struggled for so long. Here are some existing projects at the University of Michigan with which you could potentially get involved -- or start your own.

Wolverine Pathways
M-STEM
Michigan Engineering Zone
AI4All

I believe engineering can provide many opportunities for underserved people, especially Black people, to build better lives and communities. I want to do that work alongside them, and I hope you join me.

US Women's Soccer -- World Cup Champions

I enjoyed reading several articles over the last 24 hours about the US Women's team's victory in the World Cup. This was one of my favorite.

From the article, a quote from Megan Rapinoe: “Getting to play at the highest level at a World Cup with a team like we have is just ridiculous,” Rapinoe said, “but to be able to couple that with everything off the field, to back up all of those words with performances, and to back up all of those performances with words, it’s just incredible.”

When I was a little girl playing soccer with a magazine spread of Mia Hamm's bicycle kick on my wall, I knew that women's soccer was something really special in the United States. But I wouldn't have guessed that it would someday become part of a bigger moment for equal rights in America (and the world), and that is now my hope.

I believe that we will win!

Blank page

“I'm writing a first draft and reminding myself that I'm simply shoveling sand into a box so that later I can build castles.”

― Shannon Hale

Writing can be very hard when you start with a blank page. I love this quote to help me get started.

Faster multiplies with (what else) the FFT

A friend sent me a great writeup of the new fastest known algorithm for multiplying two large numbers. Just like past fastest algorithms, it uses the Fast Fourier Transform, a clever algorithm for computing the Fourier Transform efficiently that was developed in 1965 (and a re-discovery of something Gauss originally developed in 1805). The Fourier Transform is simply a way of changing coordinate system of a signal to the Fourier domain, a beautiful and elegant coordinate system for representing the frequencies of a signal.

As the article points out, this new algorithm may only be used in a limited setting, as hardware implementation constraints dictate. But in cryptography or scientific computing, one may need to multiply extremely large numbers, and the hardware overhead of transferring to an on-chip FFT implementation may become worthwhile. The article itself points out that even the relative computational cost of multiplication versus addition has changed at the hardware level.

That said, the new algorithm achieves what people believe is the lower bound for multiplicative computations. Now on to proving that it is, indeed, the lower bound.

From the article:

On March 18, two researchers described the fastest method ever discovered for multiplying two very large numbers. The paper marks the culmination of a long-running search to find the most efficient procedure for performing one of the most basic operations in math.

...

In 1971 Arnold Schönhage and Volker Strassen published a method capable of multiplying large numbers in n × log n × log(log n) multiplicative steps, where log n is the logarithm of n. For two 1-billion-digit numbers, Karatsuba’s method would require about 165 trillion additional steps.

Schönhage and Strassen’s method, which is how computers multiply huge numbers, had two other important long-term consequences. First, it introduced the use of a technique from the field of signal processing called a fast Fourier transform. The technique has been the basis for every fast multiplication algorithm since.

Second, in that same paper Schönhage and Strassen conjectured that there should be an even faster algorithm than the one they found — a method that needs only n × log n single-digit operations — and that such an algorithm would be the fastest possible. Their conjecture was based on a hunch that an operation as fundamental as multiplication must have a limit more elegant than n × log n × log(log n).

“It was kind of a general consensus that multiplication is such an important basic operation that, just from an aesthetic point of view, such an important operation requires a nice complexity bound,” Fürer said. “From general experience the mathematics of basic things at the end always turns out to be elegant.”

Schönhage and Strassen’s ungainly n × log n × log(log n) method held on for 36 years. In 2007 Fürer beat it and the floodgates opened. Over the past decade, mathematicians have found successively faster multiplication algorithms, each of which has inched closer to n × log n, without quite reaching it. Then last month, Harvey and van der Hoeven got there.

Their method is a refinement of the major work that came before them. It splits up digits, uses an improved version of the fast Fourier transform, and takes advantage of other advances made over the past forty years. “We use [the fast Fourier transform] in a much more violent way, use it several times instead of a single time, and replace even more multiplications with additions and subtractions,” van der Hoeven said.

March is upon us

With March Madness right around the corner, I found this cool visualization of where top high school basketball players end up in the NBA: https://pudding.cool/2019/03/hype/.

I love the visualization. This is taking a single feature for each player -- at what level they are playing -- and visualizing its change over time with bouncing balls. It's not clear to me if the balls spend time at each relevant level, or if they just drop to the "current" or "final" level that the player achieved. The level is discretized, so this also allows for clear visualization. You can see that high school rank is correlated with final level -- but it's not perfect, and indeed the visualizations with lower-ranked players are also enlightening.

I'm looking forward to seeing where our Michigan players end up in the next few years and beyond!

An article in the Wall Street Journal last Saturday discussed Google's AI system called ARDA (Automated Retinal Disease Assessment) for diagnosing diabetic retinothapy. This sounds like an awesome tool -- and what I love to see -- showing that machine learning will make people's lives better in so many cases. Though, as many machine learning algorithms can be, it's sensitive to data quality. From the article:

"While ARDA is effective working with sample data, according to three studies including one published in the Journal of the American Medical Association, a recent visit to a hospital in India where it is being tested showed it can struggle with images taken in field clinics. Often they are of such poor quality that the Google tool stops short of producing a diagnosis—an obstacle that ARDA researchers are trying to overcome.

"The stakes are high. If diabetic retinopathy is caught early it can be kept at bay through monitoring and management of the diabetes, said R. Kim, an Indian ophthalmologist who runs the Aravind Eye Hospital in Madurai, Tamil Nadu, where Google is testing ARDA. More advanced stages need laser surgery that can stop progression. If it isn’t treated, the condition can cause blindness."

and later in the article:

"If Google allowed the algorithm to make a diagnosis from blurred images, it could miss small lesions that appear in the early stages of the condition, she said. Google must decide how bad an image can be before ARDA refuses to grade it. “It’s a trade-off. We want them to be able to use cameras that are a little harder to use but at some point it should move into something where it is ungradable,” Dr. Peng said."

This seems like an ideal setting for active learning, where the algorithm could request input from doctors when analyzing certain images. The algorithm should also train on blurrier or lower-quality but doctor-labeled images, so that it can learn some of the higher-level features that are indicative of retinopathy.

Still, props to all the companies working hard to improve the health of people around the world. It's a long road and I'm excited people are starting down it.

Dogs can't operate MRI scanners...

But catscan!! I need to start posting more -- what better way than to start with a bunch of puns! (That post is dead, let's try this one and see how long it lives)

like a girl

It's been a long time since I posted! Guess what? I got married! and had a baby! These last few months with my baby girl I have been reminded about the "like a girl" ad from a few years ago. “Yes I kick like a girl, and I swim like a girl, and I walk like a girl, and I wake up in the morning like a girl... because I am a girl.” I can only hope my baby learns how to derive manifold optimization algorithms like a girl.

for today

“You will never reach your destination if you stop and throw stones at every dog that barks.” — Winston Churchill

Google's AlphaGo beats a Go champion

For awhile now artificial intelligence has been really good at generating winning game play based on the rules of games. Tic tac toe, checkers, sudoku, chess; in just a few short weeks, one of my students in DSP made an AI that was very good at connect-4. (It didn't beat me-- I was only willing to play it once, though!) However, checking the outcome of every possible move will never be possible for a game like Go unless a completely new paradigm of computing comes about. Still, Google has designed a player that can win. Instead of checking the outcome of moves, AlphaGo learns how best to move based on historical data of real games and its own simulations of games. It keeps trying out new games and it sees what happens. Then it keeps a memory of what to play based on the board configuration using deep learning networks, a sort of dimensionality reduction technique that encodes this experience in several layers of equations that can take any input and give the output of what to play next.

This discussion of what AlphaGo means to the future of AI takes several perspectives on what the implications of a Go-winning AI really are. I agree most with Professor Brunskill. She says, "Go is a fixed game: The rules, possible moves and observable information about the game are all prespecified. AlphaGo is not allowed to invent a new move, nor gain new insight by quizzing its opponent. Fortunately the real world is not like this. From the Hubble telescope to vaccinations, people constantly invent new ideas that allow us to transform how we monitor and shape the universe and achieve previously unimaginable outcomes."

I would add that furthermore, not only because humankind can innovate and create new realities do the rules of life change out from under us. New challenges face us every day, like the disappearance of the Malaysian Airlines flight MH370 two years ago this month, or the Zika virus, or locked cell phones of terrorists. Humans have evolved over centuries to face this adversity head-on and adapt for survival; this is exactly why we do innovate, create new measurement technologies, new drugs, and new security protections. Hopefully computers will be able to help us with this process in the not-too-distant future. But before that, machine learning research needs to face the hurdle of learning in a dynamic world.

Chess game with the Devil

I had been meaning to read this article about Terry Tao for the last several months and I finally got around to it on a cozy Friday night at home. I really like it for the way it describes him as such a genial and friendly guy, and the story it tells about even Terrence Tao being intimidated when he arrived at Princeton. My favorite quote though is this one about what it's like to be a mathematician:

"The true work of the mathematician is not experienced until the later parts of graduate school, when the student is challenged to create knowledge in the form of a novel proof. It is common to fill page after page with an attempt, the seasons turning, only to arrive precisely where you began, empty-handed — or to realize that a subtle flaw of logic doomed the whole enterprise from its outset. The steady state of mathematical research is to be completely stuck. It is a process that Charles Fefferman of Princeton, himself a onetime math prodigy turned Fields medalist, likens to ‘playing chess with the devil.’ The rules of the devil’s game are special, though: The devil is vastly superior at chess, but, Fefferman explained, you may take back as many moves as you like, and the devil may not. You play a first game, and, of course, ‘he crushes you.’ So you take back moves and try something different, and he crushes you again, ‘in much the same way.’ If you are sufficiently wily, you will eventually discover a move that forces the devil to shift strategy; you still lose, but — aha! — you have your first clue."

Einstein memorial

Last weekend I was in DC and I visited the Einstein memorial along with all the other national memorials on or near the national mall. There are so many great quotes memorialized on these walls, but this is one of my favorites:

"The right to search for truth implies also a duty: one must not conceal any part of what one has recognized to be true.”

Probability in your profession

What does probabilistic terminology really mean when people use it in your field?

Here is my favorite (image from the linked site above, used without permission, but for educational purposes obviously):

In mine we say "with probability 1-delta" and that's exactly the probability we mean. Except don't calculate delta.

Gömböc

I recently learned that when you win the Smale Prize you get a gömböc!

My favorite part of the gömböc is its relationship to a turtles' righting response: "The balancing properties of the gömböc are associated with the 'righting response', their ability to turn back when placed upside down, of shelled animals such as tortoises and beetles."

the entry in which I admit I am reading the phantom tollbooth

"That's absurd," objected Milo, whose head was spinning from all the numbers and questions.

"That may be true," [the Dodecahedron] acknowledged, "but it's completely accurate, and as long as the answer is right, who cares if the question is wrong?"

everyone gather round for a physics joke

Thanks to my friend Jim Hall and this reddit.

Heisenberg, Schrödinger, and Ohm are together in a car, driving down the road.

They get pulled over. Heisenberg is driving and the cop asks him "Do you know how fast you were going?" "No, but I know exactly where I am" Heisenberg replies. The cop says "You were doing 55 in a 35." Heisenberg throws up his hands and shouts "Great! Now I'm lost!"

The cop thinks this is suspicious and orders him to pop open the trunk. He checks it out and says "Do you know you have a dead cat back here?" "We do now, asshole!" shouts Schrödinger.

The cop moves to arrest them. Ohm resists.

big data is very often bad data

In my research I study issues with big data that make them messy. Missing data, corrupted data, uncalibrated sensors, biased human participants, inaccurate information on where or when the measurement was taken, measuring X when you really wanted to know Y... etc. A lot of big data proponents act like it's easy peasy to milk all the information possible out of these datasets. It's not!

What a perfect time to learn this lesson but during March Madness, via a winning prediction algorithm that does so well because it figured out what were the best data to use. This is called "variable selection" in statistics, and we try to do it in an automated way, but often (as was the case here) it's really domain expertise that allows one to figuring out which data are the best for inference.

One of my favorite lists of problems with big data can be found here-- including the fact that "although big data is very good at detecting correlations, ... it never tells us which correlations are meaningful." Here is another nice article from last summer on the limitations of big data -- through ok cupid and facebook user experiments. And of course my earlier blog post that gives props to IEEE for discussing the same.

Every statistical inference procedure -- from simply calculating p-values, to predicting class labels with SVM, to estimating system dynamics with filters -- has assumptions that may or may not hold in practice. Understanding the implications of that is crucial to figuring out how to use big data.

parents staying at home

This is an interesting article by a young man named Ryan Park who chose to take a year off to stay at home with his daughter. Many times statistics are quoted about women taking a future pay cut by having children, but men getting a bonus for the same. This article points to research that indicates a different cause:

"The fatherhood bonus also dissipates when men become more involved at home. ... Men who decrease their work hours for family reasons suffer a 15.5 percent decline [in future salary], while women’s salaries decline by just 9.8 percent. In other words, having a family helps men in the workplace only if they submit to their traditional gender role."

Park was a clerk for Justice Ruth Bader Ginsburg. I really enjoyed the video interview with Justice Ginsburg. Here is an interesting excerpt:

Park: "You mentioned your work at the ACLU women's right's project. ... Many commenters, the first thing they talk about was the tactical brilliance of bringing these cases where men were complaining about gender distinctions in the law that they believed harmed them. Can you talk a little bit about that strategic choice? Was it a strategic choice? Do you think the course of jurisprudence would have been different had you only brought cases with a female plaintiff?"

Ginsburg: "I have been complemented for mapping out a strategy, in truth it's the cases that came trooping into the ACLU.

"The turning point case, Reed v Reed, was a woman plaintiff. A law that discriminated blatantly against a woman, said 'As between persons equally entitled to administer a decedent's estate, males must be preferred to females.' Great first case.

"But then, let's talk about Stephen Wiesenfeld case, because that's a perfect example of what's wrong with drawing rigid lines based on gender. So Stephen was married to a teacher who had a very healthy pregnancy. She was in the classroom until the ninth month. She went to the hospital for the birth of the child. The doctor came out and told Stephen you have a healthy baby boy, but your wife died of an embolism. So he at that moment decided that he would not work full time until the baby Jason was going to school full time. And he had heard that the social security system has benefits for a sole surviving parent with a child under the age of 12 in his care. He applied for that benefit and was told, well we're sorry Mr. Wiesenfeld, these are mothers' benefits. Why mothers? Because mothers take care of children.

"Where did this discrimination begin? It began with the woman as wage earner. She paid the same social security tax as a man paid, but her family did not get the same protection that a man's family would. That was the majority view of the court, that it was really discrimination against the woman. And then one, who later became my chief, he was then Justice Renquist, said, this is utterly irrational from the point of the baby. Why should the baby have the opportunity for the care of a sole surviving parent if that parent is female, but not if the parent is male? So that was my example of how these rigid gender lines in the law hurt everybody."