So I was working out too hard, or something, maybe looking a bit anaemic, and this big, cut, burly dude wanders over all respectful-like, and says, “Hey bro’, if you work out like that every day you will end up walking fairies across the river Styx.”
Now why would a big bracing guy like him know about Greek mythology?
And I said to myself, “Dude! That was a pretty obvious thing you just said, but hell, you said in in such a luscious way, I could hug you for being so …,” I searched for the word in my mind, ” … so erudite.” I thought. Maybe “original” was what I meant? I’ve since lost the soul of that moment.
But all I said to him was, “Yeah.”
And then I’m, like, “Well I bet he thought I was profound.”
The cynic might bemoan, “when did people lose such erudition?” They might be fantasising about a past time when people spoke how Shakespeare wrote, and moreover understood what they were saying.
Wrong question! There was never such a time. Ever since the Stone Age grunts of our distant ancestors were first uttered in anger or sorrow or joy, humans, as a whole, have relentlessly and progressively been becoming more and more erudite.
This is not a myth of “inevitable progress”. It’s not some romantic ideal that humans have a purpose which we are all unknowingly working towards. But the fact is that human society has always been progressing.
It’s not a smooth process of course. There are plenty of historical pockets of attrition and retrogression. We can even see them in current affairs. The Taliban regime in Afghanistan doesn’t exactly shower human civilisation with praise. Probably lucky we are not under observation by sentient aliens. Regimes like these would be embarrassing:
“Oh yeah, the Taliban. They were adopted.”
(Thor-ish quips aside, “Adopted by who?” I’d like to ask. Sheeesh. )
But Hey! Don’t dwell on such human embarrassments. There is a lot more to rejoice in. Just open your eyes. Open your heart. Check out my previous post on The Better Angels of Our Nature for a start.
You know what? It’s even intellectually sexier if you pass the facts about human progress through the filters of mathematics. You get this sort of intoxicating honeyed-mead dripping through. (I guess. I don’t drink, so wouldn’t know. When people mention mead I imagine something that tastes sweet like comb-honey, with bite like whisky, and inducing delirium like LSD. Fantasy is better than reality?)
Do you know where I’m going to take you next? Go on. Guess. 🙂
STATISTICS are amazing.
Bet you wouldn’t have guess that.
By “amazing” I do not mean stuff like calculating averages, working out standard deviations, plotting histograms, and that sort of thing. That’s all applied statistics, and it is (I admit) fairly boring, since any dunce with a stats computer package or spreadsheet can get that sort of job done.
What I mean by “Statistics is amazing” is deeper — it is the stuff of reality, the essence of life in our universe, and the (possible) source of free will. (This is what they should teach in school not all the stiflingly boring spreadsheet puppetry.)
The deal is this: we live in a universe of quantum mechanics. This means everything that happens is unpredictable. But not random. What’s the difference?
Completely random things cannot be analysed statistically, they obey no laws, not even statistical laws. “But,” you might protest, “everything statistical has a random element to it right?” The answer is no, not really. Statistics deals with unknown quantities, but which are not random. They have an element of randomness because we are missing information which might provide a more certain and complete description.
Consider a roll of a dice. For all practical purposes, a casino manager would hope, this is a random process, with a statistical outcome from among six numbers (or faces of the die). But if you know the laws of physics, and you can rapidly measure the exact position and orientation and spin and velocity of the dice just as the casino croupier rolls it, then you can use classical mechanics to predict with almost arbitrary precision, which face will land upright. It’s not really all that hard, just enormously hard to do without a fast computer, if you wish to predict in real time before all betting is closed. (I don’t know, do casinos allow betting when the dice are already in the air … I suspect not, but that’s not my point.)
The point is that within the small universe of six possible futures (faces which could land up on the dice) there is a practical sense in which the toss is random, but only because we have no practical way of measuring the information we would need to make the prediction. That’s in a Newtonian clockwork universe.
The perturbing thought I want you to now ponder is that our actual real universe is more random than this. We live in a quantum physical universe. There is randomness built into the fabric of our universe that no amount of information can remove. Our universe is always unpredictable. What this means, in very simple and yet profoundly realistic terms, is that while all events that take place in our universe, from the collisions of air molecules to the decisions your brain makes and on to the evolution of stars and galaxies, all of it obeys certain laws of nature, we cannot ever know precisely the conditions necessary to predict anything. All we can manage is a statistical answer.
All questions of physics reduce ultimately to statistical answers. Yet this is not the same as rolling a dice. Quantum physics is very different. If our whole universe was a dice, then it would be like this: if as soon as we measure the state of the dice while it is in motion, it always seems (tho’ unseen) to do something internally random to re-mix itself up, so we can never be certain of how it will land. That’s how weird our universe is. It is not random seeming. It is random being. It injects randomness into everything. Yet it injects randomness into our lives in a very special way — in a consistent way, that is, consistent on average with Newtonian laws of physics (and as updated by Einstein — the universe described by Albert Einstein obeys the same exactness laws as Newton’s clockwork universe, “all” Einstein did was modify the fine details, pretty epic modification though it was!).
Got it yet? Our universe is irreducibly random, and yet on average it is consistent with classical deterministic clockwork Einstein-Newton physics.
So how come our world seems so predictable? The Sun always rises in the east — not randomly, not merely on average, but all the time — a day is always 24 hours long (always, not just on average), people are born the same way (in general), all people die, clouds condense overhead and then fade and then reform and then rain upon us, rivers flow downhill, trees breathe CO2, animals breath O2, and so many things are so predictable. Stock market crashes notwithstanding, haha! (They are fairly predictable actually, only greedy money managers seem to not foresee crashes.) Not many phenomena are perfectly predictable, and you might put that down to the dice-rolling type of statistics, where the small degree of unpredictability is due to insufficient information. But you’d be wrong. There is an essential type of randomness in all quantum mechanical phenomena, which no amount of measurable information can fix, and all phenomena are quantum mechanical in essence. So again — why are things so predictable?
Here is one reason: the laws of physics have structure. Lots of structure. Almost heavenly beautiful structure to those who know. The unpredictability in quantum mechanics comes from selection of possibilities within this structure. That’s why things are so predictable despite being irreducibly random.
So, for example, with the roll of a quantum dice at least you know one of the six faces will land upright. There will not be a seven dotted face, nor an eight dotted face, and so on, and there will only be dots in number from 1 to 6 turning face up after the roll. That’s the “structure”. The structure makes things emerge in long series averages in a predictable way, that is to say, statistically predictable, not perfectly clockwork predictable.
Absolute complete randomness (in terms of a dice roll) on the other hand, would be like having a dice with unknown and constantly varying numbers of dots on it’s faces. You would not even know if the dots were 1, 2, … through to 6 in number. They could have any number of dots from say 0 to infinity (which would be hard to read, but … whatever). The thing is you’d never know which six numbers were on the faces, nor which face had which number of dots. This is true Absolute Randomness. There’s something infinite and scary about Absolute Randomness. It’s like lucifer or something, maybe beelzebub? (Or my Greek-quoting gym guy might say it’s like Eris or a manifestation of Hades.)
And so it would be with a possible world of physics which was completely random. If it was absolutely random, then there would be no pattern, no stability, and really no life, or not life as we’d know it (hey Trekkies! was Scotty really the first character to say that, in fiction or not?)
The randomness in quantum mechanics is, by contrast, severely tamed. It’s rather wonderful actually, since it is tamed but has a residual wild character as well. Some people even say that the residual randomness in quantum physics is precisely what enables creatures like humans to exhibit free will — the capacity for autonomous conscious decision-making. I’m not validating such ideas, but I would admit there is something semi-plausible in them. The thing I wish to emphasise is that our physical reality is only random within a prescribed set of possibilities, and the modern job of physicists (and other scientists no doubt) is to discover the hidden constraints on the randomness in quantum physics.
Should I use the dice analogy again? May as well. So a quantum dice would be like one with all the normal six types of dot pattern. That structure would correspond to `laws of physics’. But when quantum dice are rolled there is absolutely no way in which anyone (not even God?) can tell in advance what face will land upright. No amount of measurement or computation would help. It is as if the dice, before it lands, has no knowledge of itself! Because, you see, if a thing cannot even know itself, nothing internal can dictate what it will do, and so nothing else can predict it’s behaviour either. Yes, that’s no exaggeration. This is the way it is with quantum mechanics. That’s our universe in a nutshell.
And yet whenever the dice stops rolling one of the six normal faces will be face-up. So it has a random character, but it is not the scary satanic Absolute Randomness. Thus, if you repeat the dice roll many, many times, then out of say sixty rolls you will get each different face popping up around ten times each, plus or minus a few due to fluctuating statistics. I stopped writing and did it myself just now, and I got 7 One’s, 6 Twos, 16 Thees, 8 Fours, 19 Fives, only 4 Sixes, and you’d get slight variations away from 10 if you did the experiment yourself. But you’d get something close to 10 outcomes for each face among the set of 60 rolls. I know 19 and 4 do not seem very close to 10, but they are reasonable fluctuations away from the expected. (It would be freaky if I had not rolled a single Six out of the sixty rolls, that would make me suspect the dice was loaded. But even with a fair dice, it would not be impossible to roll it sixty times and not get a particular face turn up. I think if you repeated the experiment a thousand times then in at least one lot of sixty rolls you’d probably be missing one face. I’m guessing, but could work out the probability for you if you like.)
In our real universe though, do the statistical outcomes appear (simply?) because of the “insufficient information” type of randomness?. No, the reality is that quantum mechanics has a deeper sort of uncertainty. The name of Heisenberg is often mentioned. And this is precisely what is associated with quantum uncertainty. Heisenberg’s uncertainty principle. It says that incompatible things in our world can never by 100% certain. And since the position and speed of flight of a rolled dice are two incompatible observables, Heisenberg’s Principle applies, which means we can never predict the outcome of the roll of a quantum dice, no matter how mush information we can collect beforehand.
Remember, in a classical Newtonian universe there really is no chance. You could collect enough information quite easily and with it predict exactly what face on the dice will land upright. So you’d be able to reduce statistics to certainties.
Not so in our universe ruled by quantum mechanics.
In our quantum universe statistics really does matter. (That’s a geek-joke btw — matter gets screwed by indeterminacy, get it?) We need statistics. We cannot live without statistics. All is uncertain, but all is structure. So all can be analysed statistically and we can make sense of the world. We can predict the sunrise and the rotation of the Earth, and our lives and deaths, because all the uncertainty which exists in nature is quantum uncertainty, not absolute uncertainty. And in physics, we do not predict absolutes either. We predict only within uncertain limits.
So the Sun really might not rise tomorrow, but according to standard quantum mechanics the probability the Sun will not rise on time and in the expected place in the east, is something minuscule in chance, something like,
and that’s a percentage probability! (haha! AIIMADD) (I just guessed this number, but from my background knowledge I can tell you it is honestly probably a lot smaller!) So yes, there’s a tiny realistic chance we will have no star which our planet orbits tomorrow. But no one is going to lose any sleep over this possibility.
If quantum mechanics was truly thoroughly random then the probability the Sun will rise tomorrow would be nearly zero, and the probability it will not rise would be near to 100%, this is because there are many more ways the universe could exist by pure chance without our Sun than with It.
Now here is the amazing thing about statistics. Even though you cannot predict what will happen when a female gamete (an embryonic egg cell) meets a sperm cell, and the one fertilizes the other, using statistics you can obtain almost 90% certainty of the form of life that will ensue. You can use statistics to find out with alarming precision how this life form will evolve. You can use statistics to place fairly tight limits on the baby’s lifespan, and probable occupation and interests and the language she or he will speak, and millions of other things.
You can use statistics to predict the crime rate increase in a city within any number of years into the future given only (a) it’s current crime rate, and (b) projections of how it’s population will increase (and yes, I say any number of years, do you see the power! It’s not as coarse as weather forecasting, it is prescribed and eternal in extent). You can tell me where it would be unwise to build a house near a major river, you can tell me which school would best suit my child, you can tell me how much money I will likely earn next year, you can tell me what the fate of our solar system will be in a billion years from now. You can even have a pretty good guess at what I am thinking about if you had access to an fMRI scanner with my head resting inside it’s field — using statistics. (It’s not mind-reading, but hey, it’s pretty darned similar in outcome, so what the heck, let’s call that statistical mind-reading. It can be done with today’s technology.)
With the power of statistics you have gained knowledge of emergence. This is the most powerful knowledge in science.
The best example of emergence is the way an entire ant colony seems to have a collective life of it’s own, as if all the individual ants were just cells in some great organism, the organism called The Colony. E.O. Wilson wrote a definitive introduction to this in his Pulitzer Prize winning book The Ants. Douglas Hofstadter wrote a cute parable about a conscious ant Colony in Gödel, Escher, Bach (one of the best non-fiction reads of the Twentieth century IMO).
But the most awesome example of an emergent phenomena has to be human intelligence. The way it emerges from the neuron firings in our brains. It’s by far the greatest mystery in modern science, and no one yet knows if it is “solvable”, in fact no one knows what it even means to say that the problem of human consciousness is “solvable”, because no one knows what consciousness is, so they do not know what the “problem” of it is, in essence, they only know it arises somehow from our brains. In fact they do not even know that! All we know is that without a brain it seems “difficult” to display conscious thought. And that is about the entire summary of the science of consciousness at the present time.
Emergence refers to a wide variety of phenomena which describe how putting together many, many, many small and simple interacting parts, and given the right kind of general kick, this simple composite system can take on a life of it’s own, and new laws of nature will be seen to spring into form and life. It is the philosophical Principle of Holism but rendered into scientific terms: “the whole is greater than the sum of it’s parts”.
There is no way to describe emergent complexity other than using statistics, because the exact details of an emergent phenomena are totally unpredictable. They yet have a characteristic form and shape and general abstract properties. All cats, for example, have kitten-like character, do they not? But what is the character of an abstract cat? It is nothing you can point to in the physical world. It is pure idea. Pure abstraction. Soul. Spirit. Essence. Call it what you will. Science has it encompassed under the umbrella concept of an emergent complex system.
Yet there is no exact science of ‘catness’. There is only ‘statistics for cats’. That’s the power of statistics. It gives us a handle on emergence, and hence a tether to the world of abstract (dare I say spiritual?) reality. Don’t teach your kids averages and histograms until you teach them these ideas about cats. OK?!!
Have you figured out the truth about the myth of human progress yet?
No? Well let me give you my two cents worth. Statistics tell us that human civilisation is advancing in a huge variety of ways, and there is evidence of what could be called “progress”. Progress towards what? The answer is manifold: progress towards a more peaceful and just society, progress in eco-efficiency, progress in technology, progress in economic stabilization, progress in gender and racial equality. Yes, I know, these all sound like things (apart from the technology and equality issues perhaps, which I think most people will accept are improving) that the media would report often as getting worse. But the statistical data deny the de facto “truth” taught by the media.
Really, we (as a species and as a global civilisation) are improving for the better in so many ways that it is overwhelming. I am not trying to be a prophet of the church of optimism here. It’s just the cold hard truth told by statistics. Read them for yourself, don’t, for heavens sake, just blindly trust me.
But first, tell your children about the awesome might of statistics, that at least will contribute to making the world a better place. And tell your children that progress is not a myth. In fact, it is a prophecy, and one of the good kinds, it is self-fulfilling. If you believe in the possibility of advancement of civilisation and the objective way that progress can be quantified and measured, then you will have ample ways and means in your everyday life to turn your beliefs into a slice of reality. If you really wish to help make the world a better place then statistics will not stand in your way.
And maybe, in a few decades from now, people will tell you little grains of wisdom in ways that sound pleasing to your ear, maybe using poetry that is better than the Bard’s.
Here’s how to roll a dice sixty times in a microsecond, using the computer software package R (free software developed here in New Zealand at Auckland University).
> x= sample(c(seq(1,6)),60,replace=TRUE)
 2 1 4 6 6 1 1 1 5 5 3 1 4 6 4 6 4 4 5 5 4 6 3 6 3 1 5 6 5 5 4 3 1 5 5 5 3 4
 4 6 1 5 3 6 1 4 2 3 4 1 1 4 3 2 2 5 1 3 5 5
Here’s how to easily count the number of Fives (for example),