Q&A with Eugene Fama on the Bell Curve in finance, posted without commentary

It would be very enlightening if you would comment on the Nassim Nicholas Taleb (“The Black Swan”) attack on the use of Gaussian (normal bell curve) mathematics as the foundation of finance. As you may know, Taleb is a fan of Mandelbrot, whose mathematics account for fat tails. He argues that the bell curve doesn’t reflect reality. He is also quite critical of academics who teach modern portfolio theory because it is based on the assumption that returns are normally distributed. Doesn’t all this imply that academics should start doing reality-based research?

EFF: Half of my 1964 Ph.D. thesis is tests of market efficiency, and the other half is a detailed examination of the distribution of stock returns. Mandelbrot is right. The distribution is fat-tailed relative to the normal distribution. In other words, extreme returns occur much more often than would be expected if returns were normal.

There was lots of interest in this issue for about ten years. Then academics lost interest. The reason is that most of what we do in terms of portfolio theory and models of risk and expected return works for Mandelbrot’s stable distribution class, as well as for the normal distribution (which is in fact a member of the stable class). For passive investors, none of this matters, beyond being aware that outlier returns are more common than would be expected if return distributions were normal.

For other applications, however, the difference can be critical. Risk management by financial institutions is a good example. For example, portfolio insurance, which was the rage in the early 1980s, bombed in the crash of October 1987, because this was an event that was inconceivable in their normality based return model. The normality assumption is also likely to be a serious problem in various kinds of derivatives, where lots of the price is due to the probability of extreme events. For example, news story accounts suggest that AIG blew up because its risk model for credit default swaps did not properly account for outlier events.

KRF: I agree with Gene, but want to make another point that he is appropriately reluctant to make. Taleb is generally correct about the importance of outliers, but he gets carried away in his criticism of academic research. There are lots of academics who are well aware of this issue and consider it seriously when doing empirical research. Those of us who used Gene’s textbook in our first finance course have been concerned with this fat-tail problem our whole careers. Most of the empirical studies in finance use simple and robust techniques that do not make precise distributional assumptions, and Gene can take much of the credit for this as well, whether through his feedback in seminars, suggestions on written work, comments in referee reports, or the advice he has given his many Ph.D. students over the years.

The possibility of extreme outcomes is certainly important for things like risk management, option pricing, and many complicated “arbitrage” strategies. Investors should also recognize the potential effect of outliers when assessing the distribution of future returns on their portfolios. None of this implies, however, that the existence of outliers undermines modern portfolio theory or asset pricing theory. And the central implications of modern portfolio theory and asset pricing—the benefits of diversification and the trade-off between risk and return—remain valid under any reasonable distribution of returns.

Source

Did Common Core work?

Chalkbeat’s Matt Barnum does such a wonderful job reporting on edu research. Here he is, taking on a fascinating study that attempted to understand what effect the US’s Common Core standards had. Surprising results!:

How do you a study a policy as far-reaching as the Common Core, particularly one that was introduced alongside a host of other school reforms?

It’s not easy, but Song and her colleagues reasoned that some states were more affected by the switch to “college and career ready standards,” which meant Common Core in almost all cases. So they categorized states by the “rigor” of their previous standards and how similar those standards were to the Common Core.

They divided states into those more affected by the switch (because their prior standards were deemed less rigorous or less similar to the Common Core) and those less affected. Then they compared how each group’s scores changed on fourth and eighth grade NAEP tests between 2010 and 2017.

Common Core didn’t seem to help students’ scores, and over time the standards may have had an increasingly negative effect, according to the study, which has not been formally peer-reviewed.

Other researchers consulted by Chalkbeat, including Laura Hamilton of the RAND Corporation, say the study’s approach is a credible one.

“I’m not ready to conclude that the adoption of rigorous content standards is bad for student learning,” said Hamilton. “But I don’t look at this and think this looks totally wrong. It definitely looks plausible.”

Still, the approach has limitations. Most important is that the study is comparing two groups of states that adopted the standards — so if Common Core universally helped or hurt the states that adopted it, this study would miss that effect.

More, on an unpublished report:

Joshua Bleiberg, a doctoral student at Vanderbilt University, is also studying the impact of Common Core on NAEP scores. His study starts examining student scores when new standards hit classrooms, not when states decided to formally adopt the Common Core. He also excluded states that ultimately dropped the standards.

That all seems to lead to different results. In preliminary findings shared with Chalkbeat, Bleiberg finds that the Common Core had small positive effects on NAEP scores through 2013. His study has not been released publicly, so it can’t be fully examined.

“This is not going to be the type of thing that is going to turn around the whole ship really quickly,” Bleiberg said of the standards. “I would think about [the effects] as quite small.”

I don’t know what to make of all of this. I’m very worried that my internal compass has recently been getting out of whack, that I’m getting too pessimistic about any possible major change. At the same time I’m increasingly happy to talk about fundamental issues with the way things are done. (For example, that math is frequently not serving kids particularly well in high school.) But — back to the pessimism — there isn’t much to do about.

I guess I try to keep both of those ideas in mind at once, and to work in the tension between them.

Addendum: Though! Check out another piece of Matt Barnum reporting, about how reducing pollution increased test scores.

The situation seems to be that ed reform is less effective than you think it’ll be, but that improvements to quality of life tend to have more educational implications than you’d think. (The main driver of US educational improvements? GDP.)

My ed reform platform: capital improvements to schools, air conditioning in every school, better food for kids, cut pollution.

No, I don’t think the job market should decide whether or not we teach math.

I understand where the confusion is coming from, but I don’t think school should just reflect job market needs.

I also don’t think that science is a more meaningful context for math.

I think pure math can be meaningful, and it’s easier to apply the math to new situations if it’s mastered in a more abstract, contextless form. (Of course, that abstraction needs to be meaningful to students.)

What I do believe is that high school math goes beyond the mathematics that is meaningful and broadly useful for everybody to learn. With the exception of something like exponential functions (which in the US appears in Algebra 1), I think we’re requiring too much. And there are two — well, three — reasons that you’d require more math than everybody needs.

First, because the content you’re requiring is awesome and deep and wonderful.

Second, because it keeps open career pathways for more students who would not elect to take the courses if they were optional.

While math is wonderful, I don’t think high school students are having awesome, deep, and wonderful experiences in their coursework. I don’t actually think this is intrinsic to the content of algebra — I’m a high school math teacher, after all! I think it’s more that we’re requiring too much learning to happen and all previous learning is prerequisite; if you’re behind by 9th Grade, it usually gets worse.

So let’s cut the math! Let’s change the curriculum! Yes, but then will students still have every opportunity open at the end of high school?

Really, the idea that you can keep every option open to every student the way we currently school our children is a myth. But it’s one that is widely held and makes it hard to make any change in the curriculum — learning about rearranging rational expressions is a universal right! You hear these things.

That was when I said to myself, OK, let’s keep those options open for kids. Instead of requiring math, let’s require the subjects that we are supposedly preparing them for careers in. Let them teach the math that you need for those subjects as units in those courses, or even as the first half of the year if that’s what’s best. True, science teachers these days don’t teach math very well…but they also aren’t charged with it. I’m sure they could figure it out as well as high school math teachers can.

Why would this be a better experience for students? Well, maybe we could pare down the curriculum while simultaneously keeping those pathways open. And while math is beautiful and wonderful, so is science. And while students certainly deserve the opportunity to experience the kind of thinking that makes mathematics unique…come on, they’ve already got 8 or 9 years of school to study pure math. At some point you have to stop requiring it!

All these speculative ideas are tough to judge because they’re all fantasy. Which fantasy is more realistic? I don’t know. Maybe it’s better to fantasize that we’ll replace Algebra 2 with Graph Theory, or that we’ll drastically cut the algebra requirements while beefing up support systems so that every student can success in the algebra sequence. Maybe we should require coding or statistics courses.

(Honestly, requiring statistics or coding as math classes might be a fantastic compromise in our current system, a little bit less fantastical. For me the key question is how dependent success in a course is on success in previous coursework. Students, especially in high school, would benefit from more courses that represent something like a fresh start. Otherwise the failures just compound.)

But, probably, none of this will happen, and the reason is because the real role that math is playing in high school isn’t about the importance of the subject as a humanity or its value on the job market — it’s a third reason: sorting students and signalling their academic potential to universities. And that should be a major concern for mathematicians and math educators. While I do love math and would love to share more of what I love with students, if cutting math meant I could lower the stakes for math students in high school in a significant way, I would do it with no hesitation.

Not that the plan I articulating in a few vague sentences is anything like a solution. Other people have other ideas. But I hope we can get on the same page about what the problems are.

The Mathematical Modelers’ Hippocratic Oath

The Financial Modelers’ Manifesto was a proposal for more responsibility in risk management and quantitative finance written by financial engineers Emanuel Derman and Paul Wilmott. The manifesto includes a Modelers’ Hippocratic Oath.

The Modelers’ Hippocratic Oath

I will remember that I didn’t make the world, and it doesn’t satisfy my equations.

Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.

I will never sacrifice reality for elegance without explaining why I have done so.

Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.

I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension

Benoit Mandelbrot, Beauty and Finance

Is there anything that shouts “mathematical beauty!” quite like fractals? Fractals are a core pillar of how mathematicians and educators sell math to the public. That makes sense — they really are beautiful in a way that is both visual and intellectually pleasing.

In fact, just a few hours ago I showed this image to my 4th Graders, and I got exactly the “ooohs” and “ahhs” I was hoping for.

sierp-det.GIF

The name associated most strongly with fractals is Benoit Mandelbrot’s. Mathematician, visionary, Jewish WW2 refugee, early adopter of computational visualizations, really a very cool person, and discoverer of the Mandelbrot Set. He seems likely to join that canon of Great Mathematicians. (There is such a canon, right?)

Plus, there is an awesome song about him:

And it was only since I’ve started reading more about finance that I realized there was a part of the story that wasn’t being told. Because while there is no denying the beauty of fractals, Mandelbrot’s study of fractals is caught up in the “wild randomness” that he sees in financial markets.

Just to be clear, I’m not saying that this is some sort of big secret. Mandelbrot wrote and spoke frequently about this, and I came across this nearly immediately while studying finance. Because within finance, Mandelbrot emerges as a critic of the sort of quantitative financial engineering that I’m trying my best to understand.

(Shoutout, Nassim Nicholas Taleb.)

The sort of financial models I read about in textbooks all represent the movement of a stock as a kind of random walk, weighted by certain probabilities. So IBM may be more likely to go up or go down, but essentially its price at a given time has to do with how the stock travels through this field of uncertainty — and it is moving randomly. Maybe its price is more likely to rise than fall, but still it’s governed by the same laws as coin flips.

In other words, the value of IBM follows a normal distribution, i.e. the conventional bell curve.

That’s how most quants (apparently) think about finance, but Mandelbrot thought this was a big mistake. He thought the markets were governed by wild randomness, a propensity to run towards the extremes. Here is a piece where Mandelbrot (co-written with Taleb) describes the difference between his model and the conventional one:

These two models correspond to two mutually exclusive types of randomness: mild or Gaussian on the one hand, and wild, fractal or “scalable power laws” on the other. Measurements that exhibit mild randomness are suitable for treatment by the bell curve or Gaussian models, whereas those that are susceptible to wild randomness can only be expressed accurately using a fractal scale.

I’m not quite ready to try to describe what he means by “fractal scale.”

Does it mean anything that the financial side of Mandelbrot’s work is less often shared with students and the public? Maybe not. It certainly doesn’t exhibit the same gut-punch beauty as his images, and finance is probably not the quickest way to gain an appreciation for fractals.

And yet…so often, mathematicians and educators are eager to emphasize the beauty of mathematics. And I agree — math can be beautiful! But it doesn’t seem quite right to say that mathematicians simply chase beauty. Mandelbrot wouldn’t have studied fractals if he wasn’t trying to model financial data, and the story of how math at all levels is entangled with finance is deeply undertold.

How much real-world complexity can we tolerate in math class?

I saw this, and I have a reaction:

I think of this as a matter of the complexity that we are exposing students to. Part of what makes these math modeling experiences engaging is that they are relatively simple. Not to say of course that these problems are easy for students. They aren’t. But when they are engaging it’s because the context is new, but the structure is discernible to students. That’s what engagement is: that feeling of novelty, along with the feeling of I-can-do-this.

And the reason why students can do this is typically because we’ve chosen a dataset that matches either a proportional, linear, exponential or quadratic function, or a pair of these functions. (True: it’s hard to choose the appropriate variables, and if you choose the wrong variables you won’t have very much fun. For that reason teachers usually make the smart choice to help students choose useful variables before they have time to tackle some task.)

Anyway I don’t want the point of this to be that the modeling experiences we create in math class are easy or pointless. That’s not what I’m trying to say. What I am trying to say is that part of what makes them engaging is that they are relatively simple modeling problems. And what makes them relatively simple is what makes them mathematical modeling problems as opposed to scientific ones. What I mean is, students don’t know the context in depth, we are telling them that they know enough to make predictions — they are relatively protected from the complexity.

What makes scientific modeling a different kind of engaging is that it’s in a way more serious contextual work. You learn to see new variables and new factors, and then you’re trying to coordinate them into newer, more powerful models. But there is a sense in which this is more serious work than mathematical modeling, which (by dint of being part of math class) engages less with the context.

I’m not trying to hate on mathematical modeling here, but I am trying to articulate something that I’m moving towards. I really do think kids deserve a chance to connect with applied math problems, “real world” math. At the same time, I’ve been frustrated with what passes for “real world” in math because it doesn’t take the context seriously. I have a great respect for complexity and people who study it. I’d hate for students to get the message that math can just march in and math all over the place and solve everything. You have to have some humility and learn about the world before diving in with an equation!

One question I’m asking myself these days is whether there is a way to take the contexts more seriously in math class. Is there a way I could bring more applied math into my classes without getting lost in the complexity, or ignoring it entirely?

Trying to write arbitrage puzzles

I’ve been trying to write little arbitrage puzzles. I’m hoping to make them accessible to kids. Here’s one I’m working on now.

It might rain on April 15th. Then again, it might not.

Your first friend agrees to make a bet. If it rains, you get $10. If it doesn’t, you have to pay your friend $5.

You have a second friend who wants the opposite bet. If it rains, you pay your friend $5. If it doesn’t, they’ll pay you $10.

What do you do, and why?

My take on a solution in the comments.

Is this OK?

Is it OK to abstract away all the history (and suicide) for the sake of a good puzzle?

Forget for a second whether or not this particular video is ok. (I wouldn’t show it to my students; you never knows what someone is dealing with.) But doesn’t this say something about mathematical culture? What are we training mathematical people to ignore?

Reading Chaim Grade’s “The Yeshiva”

2277135.jpg

This is a book that I first read when I was 18 and studying in yeshiva in Israel. But I didn’t really read it very well back then. This is for two reasons. First, I wasn’t sleeping well at all, and I read Grade’s novel only between the hours of 11 and 1. Second, I was a ball of anxieties about secular/religious tensions, and ill-equipped to make sense of the world of ideas Grade was constructing.

In Yiddish the novel is called Tzemach Atlas, and we are quickly introduced to the man himself. Tzemach Atlas is an extreme adherent of the Navarodok branch of the musar movement. He believes that man’s desires need to be entirely uprooted through extreme self-reflection. He can find the flaws and hypocrisies in both common people as well of in great rabbis. He obsesses constantly over his own faults, and has gained a reputation as a brave teacher who is willing to “save” his students for the yeshiva. And yet, on the very first page, we are told that he is haunted by his doubts about the existence of God.

Then a lot of things happen — I won’t burden you with them all! — but we’re eventually introduced to Chaikl Vilner, i.e. Graim Chade, i.e. this is just a stand-in for the author Chaim Grade himself.

Tzemach Atlas eventually takes Chaikl into his upstart musar yeshiva. (Chaim Grade himself spent time studying in a Navarodok musar yeshiva.) Misery ensues for everyone involved, especially Tzemach Atlas and Chaikl.

And then, in the last fifth of the novel, the great gaon and Talmudic scholar Rabbi Avraham-Shaye shows up, vacationing in the small town where the yeshiva is. He takes a liking to Chaikl and invites him to be his personal study partner — this is shocking, as it is wildly unlike a great Torah scholar to take a young study partner.

Amazingly, though, all of this is more or less true. Rabbi Avraham-Shaye is a stand-in for Rabbi Avrohom Yeshaya Karelitz, who published under the pseudonym “Hazon Ish.” Chaim Grade was his study partner for many years.

I’m rereading the book now — I just finished Volume 1, and am ready to start Volume 2 — and the biggest surprise for me has been just how clearly Grade sides with Karelitz. Grade himself left the religious life and fell in with the secular Vilna Circle. Nevertheless, his portrait of Karelitz is almost hagiographical — it’s beaming and admiring.

Though it’s reductive to put it like this, it seems to me as if Grade wanted to write this book simply to take his teacher’s side in an argument about the value of musar.

The real Karelitz was a critic of musar and its ability to improve a person’s character. As Lawrence Kaplan writes in a piece about Karelitz’s views, the musarniks had the whole thing backwards:

The Hazon Ish was of the opinion that the fundamental Musar approach of working on oneself, of turning inward, in order to develop one’s spiritual personality and overcome the obstacles standing in the way of proper observance was fundamentally misguided.

In the novel, Tzemach Atlas, the great man of musarslowly comes apart at the seams. He is a failure in every single aspect of his life. He is unable to keep a “proper” engagement, and runs off with a wealthy secular woman. And though he shows moments of clear moral insight — and impresses his wife with his clarity and moral standards — he alienates himself from an entire town. He then runs off to start his own yeshiva, and finds that he is unable to influence his students in the least.

But from the perspective of the novel, this is wholly due to his own faulty ideology and personality. When Chaikl starts spending the night with a local girl (bad Chaikl!) Atlas reprimands him, but succeeds only in inciting Chaikl’s rage.

In contrast is the positive influence of Rabbi Avraham-Shaye. He is even able to influence the child Melechke, who goes about bragging about every little accomplishment for the first 330 pages of the novel. I don’t want to go too hard on him — he’s just 11! — but really he’s a greedy, sniveling braggart who annoys everyone.

But after he visits Avraham-Shaye? He’s a changed boy!:

As soon as the visitor and his escorts withdrew, the younger students surrounded Melechke. Although he had grown and matured somewhat and could already read a chapter of the Talmud, Melechke was still considered a spoiled only child and a show-off who trust himself into the midst of older people so they would test him in Talmud and pet him. His friends couldn’t understand why he hadn’t told them that he had visited Reb Avraham-Shaye.

“I went to see Reb Avraham-Shaye Kosover to discuss Torah with him,” Melechke replied, “and I probably asked him about a dozen questions about the first page of the Tractate Kiddushin. So Reb Avraham-Shaye told me, ‘Come into my house and we’ll open my copy of the Kiddushin and look at that matter together. Then you cn ask what you have to ask.’ So we studied the page together, and all my questions were answered of their own accord. When we left his room, Reb Avraham-Shaye led me to the low window which faced the forest and told me, ‘Climb out. Let’s see if you can climb out of a window as nimbly and quickly as you can ask questions.'” Melechke spread his hands like an old Torah scholar and concluded, “Since I didn’t hear any new interpretations from him, what was there to tel you? How I crawled out the window in the woods, and how he crawled out after me?”

Later, once Chaikl is invited to be the personal study partner of the great gaon, he finds himself unable to remain the night with his landlady’s daughter:

“How could he look Reb Avraham-Shaye straight in the eye and debate lofty matters with him when he did such improper things at night? His shame at deceiving a saintly gaon changed to fear: Perhaps Reb Avraham-Shaye knew? A man like that had divine intuition…”

Later, when the gaon debates Tzemach Atlas in the closing pages of the book, Avraham-Shaye makes the case that through studying the Talmud and its commentaries one essentially gains moral influencers along these lines: “A youngster pores over his Talmud in Vilna and muses that he’s in Babylonia, sitting in the great Talmudic academy of Nahardea, in the beth medresh of Rashi and his scholarly descendants.” So, through Torah study one obtains moral exemplars who effect a profound moral influence. (Or maybe it’s Torah itself that exerts the influence?) Greater than whatever results from the intense moral introspection and trials that the mussarniks recommend.

Two things seem remarkable about this to me. First, that this is essentially the actual view of Karelitz. Second, Chaim Grade seems to have written this novel so as to adopt this perspective wholeheartedly. In the final lines of this book, Tzemach Atlas walks away a broken man, having been absolutely seen-through by the visiting gaon.

There is one place, however, where Grade seems to go beyond his teacher. Late in the book, with nowhere else to go, Chaikl goes to the beautiful wooden shul to sit alone. (Sidenote: the wooden shuls of Poland really are a wonder.) He sits there contemplating — but he is not alone, as Tzemach Atlas has been performing moral introspection up above, in the women’s section. He descends to accost Chaikl.

“Are you studying Musar by yourself?” Tzemach asks.

Chaikl responds: “I came into the shul to look at the carvings. The man who carved the lions, eagles, deer and leopards is a great artist.”

I think the strong suggestion of the book is that Chaikl’s obsession with beauty (and metaphor, as he is the author and poet as a young man) is roughly equivalent to Karelitz’s obsession with Torah. Both are capable of providing a strong moral influence on the subject, something that Tzemach Atlas is unable to comprehend. The book tells us again and again that Tzemach not only doubts God; he also does not comprehend beauty, and (as a consequence) has no respect for humanity — in himself or in others. He is a moral failure because he is a religious and aesthetic failure.

All of these themes are made much more explicit in Grade’s remarkable short story, My War With Hersh Rasseyner. He writes there — and Hersh Rasseyner is an adherent of musar who sounds quite a bit like Tzemach Atlas:

Karelitz believes in simple faith and an obsession devotion to Torah study to guide one’s moral behavior. Grade in The Yeshiva basically agrees — but extends that basic moral perspective to devotion to the humanities, in general.

css.php