you can’t buy philosophical acumen

Believe me man, I wish it were possible–I’d sell you mine for a cool $M. We’d both be better off.

So I’m reading Probability and the Doomsday Argument (Mind, 1993) by William Eckhardt. Now, this guy is not a philosopher, it seems. Rather, he’s a (somewhat famous) futures trader. In fact he seems to be the same William Eckhardt I once read about in a book called “Rise of the Market Wizards” or something like that.

[[Personal aside: a friend with money wanted me to trade for him so he gave me this book in about 1994 to entice me….I ended up trying Datek with my own cash for about a year right around the crash in ’00 or so but found that I was very bad at it; moreover the market did not seem to be behaving the way it had for the previous ten years (buy the dip doesn’t work too well in a free fall). I did make some money but it wasn’t for me…rather a heinous person living in those dark days with those dark goals. Now I view the phenomenon of highly talented people going into finance as the archetypal  social ill of our age. Hedge funds owned by billion-heirs employing dozens of minor geniuses tasked with making them yet richer still whilst producing nothing whatsoever…makes you proud to be a human, does it not? Yes, it does not. By all manner of means.]]

At any rate, Eckhardt wants to discredit the Doomsday argument. Last sentence of his brief paper: “There may exist a plethora of reasons for supposing the human race to be doomed, but our own birth rank in the total human population cannot reasonably be counted among them.”

I applaud this conclusion, but how did he get to it? Well, he describes the usual Doomsday argument, where you assume that you were sampled uniformly from the pool of all humans, past current and future, then look at your birth order n and see how that reflects on

Dm(d) = there will be exactly d humans, ever.

With such a sampling scheme, lower values of d are favored relative to higher values. Except of course for values below n, which are ruled out. Indeed, the lower d is the more likely it will be ruled out by the revelation that birth rank is n, which accounts for why lower values get a bigger boost when they aren’t ruled out.

Here’s a cryptic though apparently crucial passage in Eckhardt’s counter-argument :

“However, the sampling arrangement in this example cannot truly be analogous to that of the Doomsday argument. In sampling equiprobable from a pool, only part of which currently exists, it is essential that one not invariably succeed in obtaining a sample item. Equiprobability entails that in some instances the sample ought to be one of the nonexistent items, in which case the procedure ought to yield a null result. A procedure that invariably yields an existent item cannot be equiprobable sampling, since in that case nonexistent members of the pool could not be receiving appropriate weight. Yet the sampling procedure employed in the Doomsday argument invariably yields a result-a human rank current at the time of the argument’s discovery. Hence, this cannot be equiprobable sampling from an ensemble only part of which currently exists.”

I have no firm idea what the above might mean. If (uniformly at random) Kirk wakes the members of Khan’s 200 member crew (one per day for 200 days) will Joachim, at the time of his awakening, opine “these wakenings cannot possibly have been generated uniformly at random, for not one of us who has been selected for awakening is still asleep!”

Fortunately, Eckhardt wastes no time getting to his novelty–a model no one else could dream up: “…let Samp(r) represent a sampling procedure that is the restriction to {1,2,…,d}  of a distribution that is independent of d.” Okay, so if this is the way my birth rank is decided then there is an N such that I am 99% likely to have a birth rank somewhere in {1,2,…,N}. (What if there are 2N people? If we get them together and they are all 99% confident that their birth rank is in {1,2,…,N}, we have a problem.) High birth ranks are pretty rare by this model’s lights. But if I am born in isolation and learn that there are 10^12 people, total, past present and future, should my birth rank distribution not be uniform on {1,2,…,10^12}? What if everyone is born in isolation and has experiences just like mine?

Contra Eckhardt, almost everyone else has it that, conditional on d, your prior over birth ranks has to be uniform. The Doomsday argument hinges rather on what your credences over values of d should be. Objective chance, or objective chance times number of observers (renormalized)? Eckhardt’s idea that one should abandon uniformity conditional on d is a bit on the audacious side, given that, conditional on d, there are just as many people with high birth rank (i.e. > d / 2) rank as with low.

Worse, Eckhardt’s model is prone to gratuitous violations of reflection. Long after Doomsday, there’s a banquet in heaven. All humans (no present and future, just past now) are invited. Go ahead and mingle. Count everybody there. Now you know d. What are your priors concerning your birth rank now? These are your priors, mind you. Before evidence you gathered from the fossil record and availability of Nintendo while you were alive. Uniform, I’m guessing. Otherwise, you must just think there’s something special about you. Reflection, then, says that they should be uniform now.

So how do we solve the Doomsday paradox? Well, first you need to assume that the total number of humans d is a finite-expectation random variable. You have a pool of birth-able human templates of size N >> E(d). From this pool, you choose d human templates at random (I see no reason not to allow repeats, though they will be rare…allowing repeats avoids problems arising from the rare case that d > N). Okay…now suppose you are a template in the pool. The probability of being birthed at all is E(d)/N. Conditional on being birthed, your posterior distribution on d is given by

Q(d=k) = [ k P(d=k) ] / E(d)

and your birth rank distribution, conditional on d, is uniform on {1,2,…,d}. Now conditional on birth rank n, we have

Q(d=k|n) = [Q(d=k)Q(n|d=k)] / SUM_{j>=n} Q(d=j) Q(n|d=j) = … = P( d = k | d >= n),

which is just what you’d expect. (No paradox…all a birth rank of n tells you is that d is at least n.)

This isn’t the only place Eckhardt falls prey to a reflection violation…or a failure to recognize the role of finite expectation. Consider for example his treatment of the shooting room paradox (which he, financial guru that he is, reworked to his comfort zone) in his book Paradoxes in Probability Theory:

“The Betting Crowd game consists of one or more rounds. For each round a certain number of players enter a region and they each bet even money against double sixes on a single roll of the dice (so they all win or lose together). If the players win, they leave the region and a number of new players are brought in that equals ten times the number of players that have won so far. The dice are rolled again. The rounds continue until the house wins on double sixes at which point the game is over. This guarantees that 90 percent of all those who play lose. Two trains of thought collide: (1) since double sixes occur less than 3 percent of the time and a player stands to win about 97 percent of the time, the bet is highly favorable; (2) since 90 percent of all players are destined to lose, the bet is highly unfavorable.”

Here is Eckhardt’s “solution”:

“A player ought to reason thus: 90 percent of all players will lose, but I have less than a 3 percent chance of belonging to that losing majority. This is no paradox; each player is prospectively likely to be in the minority, since he or she is prospectively likely to win and winning itself causes there to be enough subsequent players to guarantee the winner is in the minority.”

Let’s call this argument “NO PARADOX!” (NOPE!). Actually let’s not; it’s not that funny. Essentially, though, the argument is that there is no paradox “just because”. But there is a paradox here. Let’s change things slightly. You don’t actually learn whether you’ve won at the time you play. (You see them roll the dice, but the outcome is obscured.) You can’t see how many are playing at the same time, either. And, again, let’s have a banquet at the end of the experiment, to which all who’ve played are invited. Feel free to mingle, feel free to count the guests. How does Eckhardt like his odds now? Still 35/36? If he says “yes” well, let’s just say I’d love to be making side bets in that room! Just me and 10^N Eckhardt clones, me gettin’ rich at their expense fast.

But oh no…that’ll never happen. Nobody’s savvier when the money’s on the line, and old Bill will be flopping at the banquet faster than you can say arbitrage schmarbitrage. One tenth all the way, baby, and viola…there goes reflection.

Now, maybe it’s just me, but in my book violations of reflection are damn paradoxical. Accordingly, if your analysis of a paradox is saddled with a violation of reflection then your analysis has failed to resolve the paradox, protestations of NOPE! to the contrary notwithstanding. (Okay, fine, I couldn’t resist.)

So how do we resolve the paradox? Well, you’re not gonna like this. The expected number of players in the game has to be (queue yawns, you’ve already guessed it if you’ve been reading this blog) bounded in expectation.

I know, I know…groans all around. I’ll spare you the proof. But hey…don’t shoot the messenger. Don’t bet against him either…in this or any other Crowd. This is that rare occasion in which yours isn’t the smart money.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: