A Very Brief Primer on Chaos Theory
Perhaps you might have heard of the "butterfly effect" which, in the original formulation, is the idea that a butterfly can flap its wings in Brazil and cause a tornado in Texas. Personally I find this a more confusing than helpful representation of what we'll term "chaotic behavior." But it's what you may be familiar with so by the end of this section hopefully you'll understand what is actually meant by this.
The basic idea of chaos theory is that there are some systems where small differences in their initial conditions can eventually dominate the behavior of the system. Let's look at a very simple example, the Logistic Map. The logistic map is governed by a very simple function. Start with a value x0 between 0 and 1, and then choose another value r. Now compute the value r (x0)(1 - x0) and call this x1. Now repeat this process with x1 in place of x0 to get x2. And so on and so on. In this system it's convenient to consider what these terms could mean. A common interpretation is that this equation models the population of a species. Each time you run the equation, say to produce x1, you compute a new population, and the result is based on the population of the preceding generation, in this case x0 and the constant r.
What happens when you calculate the population after many iterations? Well it all depends on what you choose for r. If you choose r between 0 and 1, you originally will get to 0, regardless of what value of x0 you start with. If you choose r between 1 and 3, the population will eventually stabilize at some value. It doesn't matter what we start with we always end in the same value. When you start increasing r beyond 3 you start seeing oscillations. For example if you take r = 3.2 regardless of what x0 you start with, you will eventually start alternating between about 0.513 and 0.799. These oscillatory systems are still considered stable, but not in the same way as the simple system before. As you continue to increase r, you eventually start oscillating between 4 values, and then 8, and somewhere around r = 3.57, all hell breaks loose. Before we look at those crazy values, we should first look at what happens when r is greater than 4. Here, regardless of what value of x0 you start with, you will eventually shoot off into infinity.
So what happens between r = 3.57 and r = 4? After some large number of iterations you will wind up somewhere between 0 and 1, you will not shoot off into infinity. Exactly where though is impossible to predict. The reason it's hard to predict is that even small errors in your initial value can bring you into wildly different places. For example, let's look at r = 3.6 and two different values for , one of which is 0.3 and the other is 0.301, a difference of one part in one thousand.What I've plotted below is the difference between our two values as a function of iteration number. Things seem nice and stable at the very beginning, but around generation 20, you start seeing very large differences between the two values. There's another period between about 35 and 45 where they are correlated, but after about 45 they diverge again. As time increases more this becomes even more chaotic and there is no correlation between the two values for any later generations.
 | |
|
Now we get to what it actually means for the system to be "chaotic" as I described in the previous paragraph. For a system to be chaotic the error needs to be exponential. The typical mathematical quantity used here is the Lyapunov Exponent, and this governs how chaotic a system behaves. You can indeed calculate the Lyapunov Exponent for our system by fitting the curve before it starts exhibiting chaotic behavior. And you get a value of roughly 0.2.
It turns out many of the systems in the world exhibit chaotic behavior. One of the most familiar ones is the weather. Weather forecasting involves taking a lot of information from sensors around the globe and feeding them into complicated meteorological models in order to predict future weather. However each of these measurements has some error, and even if all our measurements were perfectly accurate, we don't sample every molecule forcing us to fill in the gaps. Ignoring errors in our model, these sampling errors would eventually grow and dominate any system. This is why forecasting the weather beyond a few days is usually useless. It also explains the butterfly example shown above. It's not that the butterfly literally causes a tornado, but rather a small error, such as that caused by not considering the motion of a butterfly's wings, can eventually cause the entire global weather system to deviate from prediction after enough time.
Some chaotic systems only display the behavior after very long times. It turns out that the planetary motions of Venus and Mercury are chaotic due to the gravitational forces exerted by all the other planets. We can predict the motion of Venus and Mercury only to about 4-5 million years. The planetary system is governed by extremely simple laws, and there are only a finite number of objects to consider. Yet, it turns out that all you need is 3 planetary bodies to create a chaotic system.
One question I have always wondered about is whether the human brain is chaotic. It's not something I've ever gotten a good answer too, but perhaps with some extra research, it can be a topic for the future.
Quantum Uncertainty
If you are ever trading incredibly nerdy jokes, here's one you can offer up.
Werner Heisenberg is driving in his car when a cop pulls him over. The cop asks, "Do you know how fast you were going?" To which Heisenberg replies, no clue, but I know exactly where I am.If this joke made sense to you, great. If it didn't, hopefully it will after you read the rest of this section. Heisenberg is notable for many physical concepts, but the one referenced here is commonly referred to the Heisenberg Uncertainty Principle. It basically states that there is a minimum error to which you can know something. The error in a particle's momentum multiplied by the error in the particle's position must be greater than some number. How big? About 5.27 * 10-35 m2*kg/s. Just in case you're not familiar with the notation, that's 0.0000000000000000000000000000000000527. Really small.
However it turns out that this fundamental error can actually matter in reality. Let's see how. I give you a single atom of radon-222. That's a nucleus with 86 protons and 136 neutrons. If you've ever had radon detection in your basement, it's looking for this guy. At some point in the near future, this radon nucleus will eject an "alpha particle" which is a small nucleus of two protons and two neutrons (a helium-4 nucleus) leaving a polonium-218 nucleus. When will it eject it? That I can't tell you. But what I can tell you is the probability that it will have ejected the alpha particle at some point in time from now. Specifically I can tell you that there's a 50% chance that in 3.82 days, it will have ejected the alpha particle.
Let's say one day has passed and the radon molecule has not decayed, how long will it be until it has a 50% chance of decaying? If you said 2.82 days, you are wrong. It's still 3.82 days. The past is irrelevant, all that matters is the current state.
So what's actually going on? Generally when people picture nuclei they thing of a solid ball of protons and neutrons glued together. This is how they're often displayed in science textbooks. But that's misleading. The protons and neutrons are not fixed in location, they're moving. They are held together by the strong nuclear force and pushed apart by electromagnetic repulsion (among other things). Peering into the nucleus itself is impossible, that's because the scales are small enough that Heisenberg's Principle comes into play. One way to picture the decay is to imagine that glob of 2 protons and 2 neutrons existing somewhere in the nucleus and bouncing around. Be aware, this is not exactly what's going on, just use it as a simplified model. Usually when our glob gets to the edge the strong nuclear force pulls it back in. At some point though, it gets just enough energy that it manages to escape out. Since we can't peer into the nucleus we have no idea when this might happen for any given nucleus beyond the probabilities that we learn from observing many different nuclei.
The alpha decay of a radioactive nucleus is an example of a fundamental uncertainty that we can't do anything about. We are limited by fundamental laws of physics and are forced to deal with our radon nucleus entirely probabilistically. In most real world applications you are not dealing with a single nucleus but rather a very large number of nuclei. In these cases statistics are your friend and you can very accurately predict that in 3.82 days half of your large number of radon molecules will have decayed. Yet, a single alpha particle is detectable, and it is possible to engineer situations where that decay of the single radon molecule matters.
When we combine quantum uncertainty with our knowledge of chaotic system we run into a situation where many systems cannot be predicted at all beyond a certain time. There is a limit to how accurate we can measure something, and even small inaccuracies can cause big differences given enough time. It is very important to keep these effects in mind when people lament about how eliminating active gods who are manipulating parameters behind the scenes creates deterministic worlds. It appears our universe has indeterminism built into the very fabric of it.
Going back to the joke we started with. If you don't get blank stares, you can continue with the following response to Heisenberg's wisecrack:
The cop, clearly having none of it, punches Heisenberg. "Ow," says Heisenberg "do you know how hard you hit me?" "No," says the cop, "but I know exactly when I did it."Why this is funny is left as an exercise for the reader.
Can Anything be Fully Understood?
In the last two sections we talked about uncertainties in physical measurements. The results came along with real world analogies and it was fairly obvious we are talking about how stuff happens in the universe we inhabit. It turns out, that even in artificially constructed systems, there are things that evade understanding. This is a somewhat profound thought. Even when you strip away all the messy issues that come about with real world data, stuff like measurement errors, chaotic systems and quantum uncertainty, and limit yourself to purely mathematical systems. There are still things that you just can't know.
This idea comes from perhaps one of the most misunderstood mathematical theorems, due to the fact that it's more often discussed by philosophers than mathematicians. I'll gladly add myself to the list of non-mathematicians who probably misunderstand Goedel's Incompleteness Theorem.
According to my understanding the theorem states the following. Let's you have a set of consistent axioms, like for example those involving real numbers, addition and multiplication. Then you can use this set of axioms to evaluate whether a statement is true or not. For example, you could verify that 2+3=5 using the definitions of integers and addition. Goedel's Incompleteness Theorem states that there are statements that are true in this system that cannot be proven from the axioms alone.
Like all modern mathematics, Goedel's theorem is precise in what it actually says. And I have been somewhat less precise, since precision often prevents understanding to non-mathematicians (including me!) But I think this gives a general gist of what's going on. The important point in the purest of mathematical systems, uncertainty abounds. It is something we can't get away from, so we best learn to live with it.
Summing up
Hopefully I've convinced you that uncertainty is something that we're stuck with. Running after 100% certainty is a fool's errand, there are lesser certainties that are good enough. How close is good enough depends greatly on the system. In a chaotic system it turns out that even 99.9999999% isn't good enough given long enough times. In a stable system, or over shorter time scales, you can get away with far less certainty.
I might also have convinced you that there are weird and amazing things to discover in the universe. I hope to revisit some of these in the future. I hope you'll join me for those excursions as well.
Finally, I predict (with 95% certainty) that in the not too distant future someone will comment that I've misstated Goedel's theorem.
Regarding the brain: as someone with a some background (and a PhD) in neuroscience, my understanding is that although some of internal *machinery* of the brain is chaotic, overall it is pretty stable. Small scale molecular effects (ion channels within a neuron, availability of certain elements such as calcium, sodium), do fluctuate, as does the behavior of individual, or small populations of neurons. However, much of the architecture and large-scale functioning of the brain is specifically engineered ("designed" ? :) ) to be robust to these fluctuations/chaotic activity. You can recognize people, objects, buildings, places, etc every time you see them, despite significant variations in appearance, lighting, occlusions, and the subsequent fluctuations in brain activity due to these variations. As an analogy, imagine a town-hall meeting where some issue is presented to a large number of people for a vote. Yes, there will be some fluctuations/randomness in each person's behavior (how they feel about the issue that day may depend on recent news coverage, recent discussions with friends, and what they had for breakfast that morning), but overall, these random effects will average out (either within a person, or amongst groups of people), and the overall voting trend will be robust to these random effects, especially on the large scale. Similarly, population dynamics and winner-take-all behavior in populations of neurons tends to stabilize and average out the inherently noisy brain activity. (This does not negate the fact that small changes in "initial conditions" (e.g. differences based on genetic factors, parenting) or "small perturbations" (childhood memories, impressionable moments) can have lasting effects on one's thoughts and behavior, as can sudden "bifurcations" (epiphanies, formative discussions with friends) all of which are to be expected in a complex system.)
ReplyDeleteThe argument that quantum mechanics must have some effect on the brain is one of my pet peeves: both because the magnitude of quantum effects is completely negligible compared to events happening in individual neurons, and because the brain appears specifically designed to be robust to any tiny amounts of noise that any quantum effects might give rise to. My suspicion is that these arguments are often religiously motivated, and may be a (somewhat veiled) attempt to argue that we must have souls, and that they somehow interact with our brain and the world using quantum effects. I guess the reasoning goes something like: if quantum mechanics plays no observable, measurable part in brain activity, that means our brain activity/behavior must be deterministic, which is untenable from a religious perspective (free will, etc.). Therefore, quantum mechanics must be involved somehow... quantum events are inherently random and unpredictable, but maybe this is just from physical perspective, but perhaps not when the effects of the spiritual realm (ie. souls) are taken into account (for now, let's leave aside a more detailed discussion of hidden variables, Bell's theorem, etc). A similar approach, I guess, may be taken to argue for a more modern version of hashgacha - Hashem is somehow behind the quantum effects in the rest of the world, "playing dice with the universe", as Einstein would put it, except that every wavefunction collapse/quantum event is part of a master plan that we don't know, and which therefore appears random to us ... a little absurd, maybe, but just as unfalsifiable. Anyway, the quantum effects usually average out on the classical scale, which leaves Hashem rather impotent if all you leave him to work with is quantum randomness. The same goes for the brain - any quantum effects will be averaged out when looked at on the classical scale of neurons and the brain.
Thanks for the information. I do admit that in the time period between coming to the conclusion that I didn't believe in Judaism but before I would identify as an atheist, I tried to shoehorn God into quantum uncertainties. Unfortunately, as I learned more about quantum mechanics, I gave up that approach as hopeless also.
ReplyDeleteOn to the brain. Some of the difficulties I've run into is that there are tons of systems that seem stable in the short term but display chaotic behavior over the long term. The motion of the planets is an example of it. It would also make sense that the brain should behave stably to most stimuli, otherwise most animals and humans would not be able to function. There is probably a very strong evolutionary selection for just that kind of stability.
I sometimes pose the question as whether it's possible to fully model a brain state, so that it's possible to predict someones actions for any scenario. It's an impossible experiment to test, and we don't have the technology to build a single brain of any type anyway. But it really gets at what I've wondered about. If you have technical references on this topic, feel free to pass them along.
To clarify: the 'pet peeve' was not for disillusioned religious people trying to carve out a place for Hashem in a modern, scientific world (I too, was once a member of that club), but rather at people who put forward these theories (e.g. Stuart Hameroff and his rather wacky microtubule theory).
ReplyDeleteFor your second point - yes, I agree that the brain is complex enough that, as a dynamical system, it could display chaotic behavior at some levels (it's possible some of this chaotic activity could emerge from the surface during dreams, or be related to the nature of creativity, or inspiration). But on the whole, when the brain is performing most of the cognitive tasks done every day (perception, decision making, motor control), the evolutionary pressure will by necessity make it robust to any noise in the system (and there is a lot of noise in brain activity, again, more than enough to swamp any quantum effects by many orders of magnitude.).
For a reference, here's one that comes to mind: https://doi.org/10.1017/S095252380000715X ("A relationship between behavioral choice and the visual responses of neurons in macaque MT") In this study, the researchers had macaque monkeys watch random dot motion stimuli (moving dots are placed on a screen, with some proportion moving coherently left or right, while the rest move in random directions) and decide whether the coherent motion was to the left or to the right (an easy task when most are moving coherently, but which gets harder as the proportion of dots moving coherently gets smaller. (google "random dot motion" for more details). It's a very simple setup, but it involves a potentially difficult perception task, requiring integration of multiple sources of evidence across the visual field, leading to a decision process and a motor response. Without too much effort, one can draw analogies with decisions we make based on accumulating perceptual evidence, current thoughts, and preconceived notions, etc. which, in the end, are all encoded as neural activity in some form and feed into a decision-making process somewhere in the brain. There are regions in the brain specifically devoted to decision making (such as prefrontal cortex), and it is no great surprise to see correlations between patterns of activity in these brain regions and decisions/behavior. What's neat about this experiment is that the stimuli are simple enough that the researchers were able to find direct correlations between the monkey's choice (left vs right) and individual neuron activity in area MT, a visual motion processing area that is very early in the visual pathway, and which would have taken place up to a few hundred milliseconds before the corresponding decision-related activity in the prefrontal cortex, which is when the monkey would have been "aware" of making the decision (After looking around a little more, I also found this one: https://www.nature.com/news/2008/080411/full/news.2008.751.html where a similar study was done in humans, and actually addresses your question more directly). So yes, with access to internal brain activity, we can predict the behavior very accurately (the accuracy gets higher as we get more detail and look further along the processing stream, closer to where the decision is made). More broadly, while we are very far from being able to fully understand and completely model the brain, we have been making steady progress for decades, understanding more and more about each brain area and how its activity relates to our experiences. At no point have we encountered any weird 'black box' of activity which completely mystifies us, and calls for a non-physical, mystical soul, or weirdness from quantum mechanics. The brain is marvelously complex and fascinating enough without it.
Thanks again for another great post.
ReplyDeleteWith regards to the brain / chaos question, I don’t understand how that brain can be said to *not* be chaotic. The problem I keep hitting when I think about it is that the brain is so fundamentally linked to other systems that are obviously chaotic (e.g. your example of the weather. The weather affects my mood and decisions in fundamental ways, and it is chaotic as you describe.) So presumably the question is something like “is the brain chaotic given a precise set of inputs over time?” But again surely it is obvious that it is. We are so subtly affected by each other’s moods etc that surely even a small degree of difference in inputs will magnify over time. I really don’t see how there is any doubt as to the chaotic nature of the brain over time.
Now it being chaotic does not preclude being able to make some general predictions even far into the future and / or statistical predictions on groups of brains. As an allegory, you agree that the weather is chaotic but we can still predict that the summer of 2098 will be on average several degrees warmer than the winter of 2098 etc. Similarly while we can not with any accuracy predict the position of a given partical within the Earth in a years time, we can predict with great accuracy the overal average position of the planet at that point in time.
So to say a brain is “not chaotic” you would need to be in effect saying that given a reasonably granular model of the brain today and a reasonably accurate set of inputs over time, you are able to model things like: what my precise interactions with a specific person will be in 3 years time. I just can’t understand how you can even propose that this is doable while simultaneously stating as fiat that weather is chaotic. If either is in doubt, surely both are.
So, we should make sure we're all on the same page about how we define a chaotic system. As @AKefirah pointed out, a system is said to be chaotic if small changes in compound over time, so that tiny differences in initial conditions lead to very different trajectories. Probably one of the simplest chaotic systems is the double pendulum (check youtube for cool demos). We would probably all agree, though, that a simple (single) pendulum is *not* a chaotic system. But if you took your single pendulum on a bumpy car ride, it's motion would probably look chaotic, due to the random/chaotic external forces acting on it from outside. But that doesn't mean that the single pendulum is inherently a chaotic system - it's just that it's currently being affected by external chaotic forces. Similarly for the brain - just because the weather can affect your mood, doesn't mean that your mood is chaotic - just that it can respond to external stimuli. Imagine a typical day, and then imagine common, random, pertubations to that day. How do those changes affect the overall course of the day? I would argue that in the vast majority of cases, you would course-correct and the overall day would remain more-or-less the same. Out of cereal? pick a different breakfast. Hit some extra traffic? maybe you drive a little faster later on to try make up lost time. Bump into someone by accident? apologize (ideally) and move on. Yes, I'm sure we could all conceive of situations where a tiny change could compound and lead to huge differences (like in the movie "sliding doors"), but these are the exceptions that prove the rule - the vast majority of the day-to-day variations do not exhibit this "butterfly effect".
ReplyDeleteDespite the enormous complexity of the brain (which I understand might lead people to assume that it must be chaotic), what differentiates it from the (far-simpler, yet definitely chaotic) double pendulum is cognitive control - our conscious minds are goal-directed, and we try to do what we can to achieve our goals, and try to correct things when things go off-course.
So, perhaps, in the absence of this top-down control (e.g., when we're sleeping), you might classify brain activity (including dreams) as chaotic. And in certain mental disorders (psychosis) where top-down control is severly impaired, people might act erratically ('chaotically'). But in day to day interactions, the behavior of typical people is pretty stable, and robust to minor variations. So, yes, with enough computational power and with a precise enough brain model, and access to the exact sensory inputs, I don't see why you wouldn't be able to predict someone's behavior to a large degree of accuracy over the course of a day, or a week. (I'm not sure about 3 years - that might require pretty precise modeling, perhaps down to the molecular level to account for small molecular effects... similar to how you can predict quite well how your car will behave over the course of driving for a day or week, but might need to take into account much more physical detail (e.g. when the oil/lubricant will breakdown) to predict after exactly how many miles it will break down, but I'm sure you would agree that a car is a not a chaotic system).
I think Avi answered what I would answer before I could, but I have a few things to add, on the technical side.
ReplyDeleteYou said that we might not be able to predict the weather explicitly in a year, but we can predict it will be warmer in the summer than the winter. This is true, and what it implies is that the temperature is bounded. Or even more explicitly, you can accurately describe the temperature at any point in time in the future with a probability distribution function. You can do the same with the logistic equation. The most interesting chaotic systems (to me) are the ones that are bounded. It is also true that bounded systems cannot display the chaotically divergent behavior forever. At some point they hit the bounds. That is why in order to calculate things like Lyapunov exponents you need to restrict yourself to regions far smaller than the bounds. A key point here is that as long as you are far from the bounds, a chaotic system will display divergent behavior at all scales.
Avi already talked about external forcing, and that's the other main point. Simple chaotic systems, such as the double pendulum, or the logistic equation, are chaotic without any external influences. Weather is impossible to isolate, since the weather is affected by solar behavior. Yet you can restrict yourself to periods of time in which the solar behavior is constant and if you were a mathematically minded meteorologist, you could still show that the weather is chaotic. Measuring this basically means looking for deviations over short time scales and seeing if they obey an exponential law.
One more point, that of the planets. The orbits of the inner planets are chaotic but it takes many years for these effects to come about. However, if you imagine that our solar system was constantly invaded by large comets that deflect the orbits slightly, then these comets would outweigh the very slowly manifesting chaotic behavior. However, we can still declare the system to be mathematically chaotic, even if in reality it's impossible to measure because of all the interference.
So to sum up, the hallmark of a chaotic system is that extremely minute deviations can manifest into larger ones giving enough time. Another key point is that you can see exponential divergence at all scales (much smaller than the bounds)
What Avi has informed me about the brain is that it does not behave chaotically at these small scales. Small deviations in the brain's electrical signals do not cause different neurons to fire. There is no exponential divergence.
However, it is almost certainly true that our brains live in a mathematically chaotic environment (if from the weather alone). But like Avi's pendulum in a car example, which is a really good one, this does not imply that the brain is chaotic.