“Next stop: Leiden University Faculty of Law!”

Returning to a place from one’s old life is always a complex experience. I’ve spent a year studying in this town, a mere decade ago: yet today, it feels like an eternity or a past life. So much has changed since then, and I barely recognise the man I was.

Back when I lived in Leiden, I was attached to the law faculty, housed in Kammerlingh Onnes’s old lab. If you had told me then that a decade and a bit later I would be back, but as a data scientist working with a client nearby, I would have laughed. Data science wasn’t even a thing back then, and while I was always into statistics and maths, I never saw myself doing it as a career until (relatively) quite recently. And so, to return to a town that holds all these memories from a past life is strange to say the least. Strange – but not necessarily unpleasant!

Posted in Uncategorized

It’s complete!

It's complete! My amazing, gorgeous wife has just received her Master's degree. And I couldn't be prouder. I'm not east to impress. But it was impossible not to be impressed by the hard work Katie has put into this degree, and all the skill, talent and passion she displayed. She's a true fighter, and to all of us who helped her (only some of whom could be present), it was a privilege getting to watch her in action.

It’s complete! My amazing, gorgeous wife has just received her Master’s degree. And I couldn’t be prouder. I’m not east to impress. But it was impossible not to be impressed by the hard work Katie has put into this degree, and all the skill, talent and passion she displayed. She’s a true fighter, and to all of us who helped her (only some of whom could be present), it was a privilege getting to watch her in action.

Posted in Uncategorized

Ooops.

Turns out that being severely anaemic and the reduced partial pressure of oxygen in high altitude trans-continental flights don't mix. With a Hgb level barely above 6.5, I was expecting some shortness of breath and suchlike, but when my muscles started to burn like I just ran a marathon and my vision became woozy, I knew that things weren't ok. The wonderful <a rel=@unitedcabin crew handled the whole issue with the utmost professionalism and without any panic or exaggeration - they simply gave me an oxygen tank and let me fix myself, just as I prefer.Β  A few hours later, and I'm much, much better (although until my counts recover, I think I'll have to organise oxygen for future trips). We're an hour and a half out of Newark and have about 6 hours to go to DC, when we finally get to meet more of our awesome family! Katie and I couldn't be more excited - after a few fantastic days in Paris with my lovely parents, we're looking forward to visit our DC and Indiana kin. My sats are slowly back to normal, Katie is enjoying Finding Dory, and life is awesome. And that's the point here - by not panicking, trying to take this as just another part of the adventure, we sailed through what could otherwise have been a frightening (but really not particularly serious) experience. Sometimes, the biggest gift St. Christopher can give you for the road is good people and a solid sense of humour." width="600" height="600" />

Turns out that being severely anaemic and the reduced partial pressure of oxygen in high altitude trans-continental flights don’t mix. With a Hgb level barely above 6.5, I was expecting some shortness of breath and suchlike, but when my muscles started to burn like I just ran a marathon and my vision became woozy, I knew that things weren’t ok. The wonderful United Airlines cabin crew handled the whole issue with the utmost professionalism and without any panic or exaggeration – they simply gave me an oxygen tank and let me fix myself, just as I prefer.Β  A few hours later, and I’m much, much better (although until my counts recover, I think I’ll have to organise oxygen for future trips). We’re an hour and a half out of Newark and have about 6 hours to go to DC, when we finally get to meet more of our awesome family! Katie and I couldn’t be more excited – after a few fantastic days in Paris with my lovely parents, we’re looking forward to visit our DC and Indiana kin. My sats are slowly back to normal, Katie is enjoying Finding Dory, and life is awesome. And that’s the point here – by not panicking, trying to take this as just another part of the adventure, we sailed through what could otherwise have been a frightening (but really not particularly serious) experience. Sometimes, the biggest gift St. Christopher can give you for the road is good people and a solid sense of humour.

Posted in Uncategorized

How I predicted Trump’s victory

Introit

“Can you, just once, explain it in intelligible words?”, my wife asked.

We’ve been talking for about an hour about American politics, and I made a valiant effort at trying to explain to her how my predictive model for the election worked, what it took into account and what it did… but twenty minutes in, I was torn between either using terms like stochastic gradient descent and confusing her, or having to start to build everything up from high school times tables onwards.

Now, my wife is no dunce. She is one of the most intelligent people I’ve ever had the honour to encounter, and I’ve spent years moving around academia and industry and science. She’s not only a wonderful artist and a passionate supporter of the arts, she’s also endowed with that clear, incisive intelligence that can whittle down the smooth, impure rock of a nascent theory into the Koh-I-Noor clarity of her theoretical work.

Yet, the fact is, we’ve become a very specialised industry. We, who are in the business of predicting the future, now do so with models that are barely intelligible to outsiders, and some even barely intelligible to those who do not share a subfield with you (I’m looking at you, my fellow topological analytics theorists!). Quite frankly, then: the world is run by algorithms that at best a fraction of us understand.

So when asked to write an account of how I predicted Trump’s victory, I’ve tried to write an account for a ‘popular audience’. 1There is an academic paper with a lot more details forthcoming on the matter – incidentally, because republication is generally not permitted, it will contain many visualisations I was not able or allowed to put into this blog post. So just for that, it may be worth reading once it’s out. I will post a link to it here. That means there’s more I want to get across than the way I built some model that for once turned out to be right. I also want to give you an insight into a world that’s generally pretty well hidden behind a wall made of obscure theory, social anxiety and plenty of confusing language. The latter, in and of itself, takes some time and patience to whittle down. People have asked me repeatedly what this support vector machine I was talking about all the time looked like, and were disappointed to hear it was not an actual machine with cranks and levers, just an algorithm. And the joke is not really on them, it’s largely on us. And so is the duty to make ourselves intelligible.

Prelude

I don’t think there’s been a Presidential election as controversial as Trump’s in recent history. Certainly I cannot remember any recent President having aroused the same sort of fervent reactions from supporters and opponents alike. As a quintessentially apolitical person, that struck me as the kind of odd that attracts data scientists like flies. And so, about a year ago, amidst moving stacks of boxes into my new office, I thought about modelling the outcome of the US elections.

It was a big gamble, and it was a game for a David with his sling. Here I was, with a limited (at best) understanding of the American political system, not much access to private polls the way major media and their court political scientists have, and generally having to rely on my own means to do it. I had no illusions about the chances.

After the first debate, I tweeted this:

Also, as so many asked: post debate indicators included, only 1 of over 200 ensemble models predict a HRC win. Most are strongly Trump win.

– Chris (@DoodlingData), September 28, 2016

To recall, this was a month and a half ago, and chances for Trump looked dim. He was assailed from a dozen sides. He was embroiled in what looked at the time as the largest mass accusation of sexual misconduct ever levelled against a candidate. He had, as many right and left were keen on pointing out, “no ground game”, polling unanimously went against him and I was fairly sure dinner on 10 November at our home will include crow.

But then, I had precious little to lose. I was never part of the political pundits’ cocoon, nor did I ever have a wish to be so. There’s only so much you can offer a man in consideration of a complete commonsensectomy. I do, however, enjoy playing with numbers – even if it’s a Hail Mary pass of predicting a turbulent, crazy election.

I’m not alone with that – these days, the average voter is assailed by a plethora of opinions, quantifications, pontifications and other -fications about the vote. It’s difficult to make sense of most of it. Some speak of their models downright with the same reverence one might once have invoked the name of the Pythiae of the Delphic Oracle. Others brashly assert that ‘math says’ one or other party has ‘already won’ the elections, a month ahead. And I would entirely forgive anyone who were to think that we are, all in all, a bunch of charlatans with slightly more high-tech dowsing rods and flashier crystal balls.

Like every data scientist, I’ve been asked a few times what I ‘really’ do. Do I wear a lab coat? I work in a ‘lab’, after all, so many deduced I would be some sort of experimental scientist. Or am I the Moneyball dude? Or Nate Silver?

Thankfully, neither of those is true. I hate working in the traditional experimental science lab setting (it’s too crowded and loud for my tastes), I don’t wear a lab coat (except as a joke at the expense of one of my long-term harassers), I don’t know anything about baseball statistics and, thanks be to God, I am not Nate Silver.

I am, however, in the business of predicting the future. Which sounds very much like theorising about spaceships and hoverboards, but is in fact quite a bit narrower. You see, I’m a data scientist specialising in several fields of working with data, one of which is ‘predictive analytics’ (PA). PA emerged from combinatorics (glorified dice throwing), statistics (lies, damned lies and ~) and some other parts of math (linear algebra, topology, etc.) and altogether aims to look at the past and find features that might help predicting the future. Over the last few years, this field has experienced an absolute explosion, thanks to a concept called machine learning (ML).

ML is another of those notions that evokes more passionate fear than understanding. In fact, when I explained to a kindly old lady with an abundance of curiosity that I worked in machine learning, she asked me what kind of machines I was teaching, and what I was teaching them – and whether I had taught children before. The reality is, we don’t sit around and read Moby Dick to our computers. Nor is ML some magic step towards artificial intelligence, like Cortana ingesting the entire Forerunner archives in Halo. No, machine learning is actually quite simple: it’s the art and science of creating applications that, at least when they work well, perform better each time than the time before.

It is high art and hard science. Most of modern ML is unintelligible without very solid mathematical foundations, and yet knowledge has never really been able to substitute for experience and a flair for constructing, applying and chaining mathematical methods to the point of accomplishing the best, most accurate result.

Wait, I haven’t talked about results yet! In machine learning, we have two kinds of ‘result’. We have processes we call ‘supervised learning’, where we give the computer a pattern and expect it to keep applying it. For instance, we give it a set (known in this context as the training set) of heart rhythm (ECG) tracings, and tell it which ones are fine and which ones are pathological. We then expect the computer to accurately label any heart rhythm we give to it.

There is also another realm of machine learning, called ‘unsupervised learning’. In unsupervised learning, we let the computer find the similarities and connections it wants to. One example would be giving the computer the same set of heart traces. It would then return what we call a ‘clustering’ – a group of heartbeats on one hand that are fine, and the pathological heartbeats on the other. We are somewhat less concerned with this type of machine learning. Electoral prediction is pretty much a straightforward supervised learning task, although there are interesting addenda that one can indeed do by leveraging certain unsupervised techniques. For instance, groups of people designated by certain characteristics might vote together, and a supervised model might be ‘told’ that a given number of people have to vote ‘as a block’.

These results are what we call ‘models’.

On models

Ever since Nate Silver allegedly predicted the Obama win, there has been a bit of a mystery-and-no-science-theatre around models, and how they work. Quite simply, a model is a function, like any other. You feed it source variables, it spits out a target variable. Like your washing machine:

f(C_d, W, E_{el}, P_w) = (C_c)

That is, put in dirty clothes (C_d ), water (W ), electricity (E_{el} ) and washing powder (P_w ), get clean clothes (C_c ) as a result. Simple, no?

The only reason why a model is a little different is that it is, or is supposed to be, based on the relationship between some real entities on each side of the equality, so that if we know what’s on the left side (generally easy-to-measure things), we can get what’s on the right side. And normally, models were developed in some way by reference to data where we do have both sides of the equation. An example for this is the tool known as Henssge’s nomogram, which is a tool called a nomogram, a visual representation of certain physical relationships. That particular model was developed from hundreds, if not thousands, of measurements of (get your retching bag ready), butthole temperature measurements of dead bodies where the time of death actually was known. As I’m certain you know, when you die, you slowly assume room temperature. There are a million factors that influence this, and to calculate the time since death could certainly break a supercomputer. And it would be accurate, but not much more accurate than Henssge’s method. Turns out, a gentleman called Claus Henssge discovered, that three and a half factors are pretty much enough to estimate the time since death with reasonable accuracy: the ambient temperature, the aforementioned butthole temperature, the decedent’s body weight, and a corrective factor to take account for the decedent’s state of nakedness. Those factors altogether give you 95% or so accuracy – which is pretty good.

The Henssge nomogram illustrates two features of every model:

  1. They’re all based on past or known data.
  2. They’re all, to an extent, simplifications.

Now, traditionally, a model used to be built by people who reasoned deductively, then did some inductive stuff such as testing to assuage the more scientifically obsessed. And so it was with the Henssge nomogram, where data was collected, but everyone had a pretty decent hunch that time of death will correlate best with body weight and the difference between ambient and core (= rectal) temperature. That’s because heat transfer from a body to its environment generally depends on the temperature differential and the area of the surface of exchange:

Q = hA(T_a - T_b)

where Q is heat transferred per unit time, h is the heat transfer coefficient, A is the area of the object and T_a - T_b is the temperature difference. So from that, it then follows that T_a and T_b can be measured, h is relatively constant for humans (most humans are composed of the same substance) and A can be relatively well extrapolated from body weight.2The reasoning here is roughly as follows. Assume the body is a sphere. All bodies are assumed of being made of the same material, which is also assumed to be homogenous. The volume of a sphere V = \frac{4}{3} \pi r^3 , and its weight is that multiplied by its density \rho . Thus the radius of a sphere of a matter of known density \rho can be calculated as r = \sqrt[3]{\frac{3}{4} \frac{M}{\pi \rho}} . From this, the surface area can be calculated (A = 4 \pi r^2 ). Thus, body weight is a decent stand-in for surface area.

The entire story of modelling can be understood to focus on one thing, and do it really well: based on a data set (the training set), it creates a model that seeks to describe the essence of the relationship between the variables involved in the training set. The simplest suich relationships are linear: for instance, if the training set consists of {number of hamburgers ordered; amount paid}, the model will be a straight line – for every increase on the hamburger axis, there will be the same increase on the amount paid axis. Some models are more complex – when they can no longer be described as a combination of straight lines, they’re called ‘nonlinear’. And eventually, they get way too complex to be adequately plotted. That is often the consequence of the training dataset consisting not merely of two fields (number of hamburgers and the target variable, i.e. price), but a whole list of other fields. These fields are called elements of the feature vector, and when there’s a lot of them, we speak of a high-dimensional dataset. The idea of a ‘higher dimension’ might sound mysterious, but true to fashion, mathematicians can make it sound boring. In data science, we regularly throw around data sets of several hundred or thousand dimensions or even more – so many, in fact, that there are whole techniques intended to reduce this number to something more manageable.

But just how do we get our models?

Building our model

In principle, you can sit down, think about a process and create a model based on some abstract simplifications and some other relationships you are aware of. That’s how the Henssge model was born – you need no experimental data to figure out that heat loss will depend on the radiating area, the temperature difference to ‘radiate away’ and the time the body has been left to assume room temperature: these things more or less follow from an understanding of how physics happens to work. You can then use data to verify or disprove your model, and if all goes well, you will get a result in the end.

There is another way of building models, however. You can feed a computer a lot of data, and have it come up with whatever representation gives the best result. This is known as machine learning, and is generally a bigger field than I could even cursorily survey here. It comes in two flavours – unsupervised ML, in which we let the computer loose on some data and hope it turns out ok, and supervised ML, in which we give the computer a very clear indication of what approrpiate outputs are for given input values. We’re going to be concerned with the latter. The general idea of supervised ML is as follows.

  1. Give the algorithm a lot of value pairs from both sides of the function – that is, show the algorithm what comes out given a particular input. The inputs, and sometimes even the outputs, may be high-dimensional – in fact, in the field I deal with normally, known as time series analytics, thousands of dimensions of data are pretty frequently encountered. This data set is known as the training set.
  2. Look at what the algorithm came up with. Start feeding it some more data to which you know the ‘correct’ output, so to speak, data which you haven’t used as part of the training set. Examine how well your model is doing predicting the test set.
  3. Tweak model parameters until you get closer to higher accuracy. Often, an algorithm called gradient descent is used, which is basically a fancy way of saying ‘look at whether changing a model parameter in a particular direction by \mu makes the model perform better, and if so, keep doing it until it doesn’t’. \mu is known as the ‘learning rate’, and determines on one hand how fast the model will get to a best possible approximation of the result (how fast the modell will converge), and on the other, how close it will be to the true best settings. Finding a good learning rate is more a dark art than science, but something people eventually get better at with practice.

In this case, I was using a modelling approach called a backpropagation neural network. An artificial neural network (ANN) is basically a bunch of nodes, known as neurons, connected to each other. Each node runs a function on the input value and spits it out to its output. An ANN has these neurons arranged in layers, and generally nodes feed in one direction (‘forward’), i.e. from one layer to the next, and never among nodes in the same layer.

Neurons are connected by ‘synapses’ that are basically weighted connections (weighting simply means multiplying each input to a neuron by a value that emphasises its significance, so that these values all add up to 1). The weights are the ‘secret sauce’ to this entire algorithm. For instance, you may have an ANN set to recognise handwritten digits. The layers would get increasingly complex. So one node may respond to whether the digit has a straight vertical line. The output node for the digit 1 would weight the output from this node quite strongly, while the output node for 8 would weight it very weakly. Now, it’s possible to pick the functions and determine the weights manually, but there’s something better – an algorithm called backpropagation that, basically, keeps adjusting weights using gradient descent (as described above) to reach an optimal weighting, i.e. one that’s most likely to return accurate values.

My main premise for creating the models was threefold.

  1. No polling. None at all. The explanation for that is twofold. First, I am not a political scientist. I don’t understand polls as well as I ought to, and I don’t trust things I don’t understand completely (and neither should you!). Most of all, though, I worry that polls are easy to influence. I witnessed the 1994 Hungarian elections, where the incumbent right-wing party won all polls and exit-poll surveys by a mile… right up until eventually the post-communists won the actual elections. How far that was a stolen election is a different question: what matters is that ever since, I have no faith at all in polling, and that hasn’t gotten better lately. Especially in the current elections, a stigma has developed around voting Trump – people have been beaten up, verbally assaulted and professionally ostracised for it. Clearly asking them politely will not give you the truth.
  2. No prejudice for or against particular indicators. The models were generated from a vast pool of indicators, and, to put it quite simply, a machine was created that looked for correlations between electoral results and various input indicators. I’m pretty sure many, even perhaps most, of those correlations were spurious. At the same time, spurious correlations don’t hurt a predictive model if you’re not intending to use the model for anything other than prediction.
  3. Assumed ergodicity. Ergodicity, quite simply, means that the average of an indicator over time is the same as the average of an indicator over space. To give you an example:3I am indebted to Nassim Nicholas Taleb for this example. assume you’re interested in the ‘average price’ of shoes. You may either spend a day visiting every shoe store and calculate the average of their prices (average over space), or you may swing past the window of the shoe store on your way to work and look at the prices every day for a year or so. If the price of shoes is ergodic, then the two averages will be the same. I thus made a pretty big and almost certainly false assumption, namely that the effect of certain indicators on individual Senate and House races is the same as on the Presidency. As said, while this is almost certainly false, it did make the model a little more accurate and it was the best model I could use for things for which I do not have a long history of measurements, such as Twitter prevalence.

One added twist was the use of cohort models. I did not want to pick one model to stake all on – I wanted to generate groups (cohorts) of 200 models each, wherein each would be somewhat differently tuned. Importantly, I did not want to create a ‘superteam’ of the best 200 models generated in different runs. Rather, I wanted to select the group of 200 models that is most likely to give a correct overall prediction, i.e. in which the actual outcome would most likely be the outcome predicted by the majority of the models. This allows for picking models where we know they will, ultimately, act together as an effective ensemble, and models will ‘balance out’ each other.

A supercohort of 1,000 cohorts of 200 models each was trained on electoral data since 1900. Because of the ergodicity assumption (as detailed above), the models included non-Presidential elections, but anything ‘learned’ from such elections was penalised. This is a decent compromise if we consider the need for ergodicity. For example, I have looked at the (normalised fraction4Divide the smaller by the larger value, normalise to 1. of the) two candidates’ media appearances and their volume of bought advertising, but mass media hasn’t always been around for the last 116 years in its current form. So I looked at the effect that this had on smaller elections. All variables weighted to ‘decay’ depending on their age.

Tuning of model hyperparameters and deep architecture was attempted in two ways. I initially began with a classical genetic algorithm for tuning hyperparameters and architecture, aware that this was less efficient than gradient descent based algorithms but more likely to give you a diversity of hyperparameters and far more suited to multi-objective systems. Compared with gradient descent algorithms, genetic algorithms took longer but performed better. This was an acceptable tradeoff to me, so I eventually adapted a multi-objective genetic algorithm implementation, drawing on the Python DEAP package and some (ok, a LOT of) custom code. Curiously (or maybe not – I recently learned this was a ‘well known’ finding –  apparently not as well known after all!), the best models came out of ‘split training’: genetically optimised convolutional layers, genetically optimised structure but non-convolutional layers are trained using backpropagation.

Another twist was the use of ‘time contingent parameters’. That’s a fancy word of saying data that’s not available ab initio. An example for that would be post-debate changes of web search volumes for certain keywords associated with each candidate. Trivially, that information is not in existence until a week or so post-debate. These models were trained to ‘variants’. So if a particular model had information missing, it defaulted to an equally weighted model without the nodes that would have required that information. Much as this was a hacky solution, it was acceptable to me as I knew that by late October, every model would have complete information.

I wrote a custom mdoel runner in Python with an easy-as-heck output interface – I was not concerned with creating pretty, I was concerned with creating good. The runner first pulled all data it required once again, diffed it against the previous version, reran feature extractors where there was a change, then ran the models over the feature vectors. Outputs went into CSV files and simple outputs that looked like this (welcome to 1983):

CVoncsefalvay @ orinoco ~/Developer/mfarm/election2016 $ mrun –all

< lots of miscellaneous debug outputs go here >

[13:01:06.465 02 Nov 2016 +0000] OK DONE.
[13:01:06.590 02 Nov 2016 +0000] R 167; D 32; DNC 1
[13:01:06.630 02 Nov 2016 +0000] Output written to outputs/021301NOV2016.mconfdef.csv

That’s basically saying that (after spending the best part of a day scoring through all the models) 167 models were predicting a Republican victory, 32 a Democratic victory and one model crashed, did not converge somewhere or otherwise broke. The CSV output file would then give further data about each submodel, such as predicted turnout, predictions of the electoral college and popular vote, etc. The model was run with a tolerance of 1%, i.e. up to two models can break and the model would still be acceptable. Any more than that, and a rerun would be initiated automatically. One cool thing: this was my first application using the Twilio API to send me messages keeping me up to date on the model. Yes, I know, the 1990s called, they want SMS messaging back.

By the end of the week, the first models have phoned back. I was surprised: was Trump really that far ahead? The polls have slammed him, he seemed hopeless, he’s not exactly anyone’s idea of the next George Washington and he ran against more money, more media and more political capital. I had to spend the best part of a weekend confirming the models, going over them line by line, doing tests and cross-validation, until I was willing to trust my models somewhat.

But part of our story in science is to believe evidence with the same fervour we disbelieve assertions without it. And so, after being unable to find the much expected error in my code and the models, I concluded they must be right.

Living with the models

The unique exhilaration, but also the most unnerving feature, of creating these models was how different they are from my day-to-day fare. When I write predictive models, the approach is, and remains, quintessentially iterative. We build models, we try them, and iteratively improve on them. It is dangerous to fall in love with one’s own models – today’s hero is in all likelihood destined for tomorrow’s dungheap, with another, better model taking its place – until that model, too, is discarded for a better approach, and so on. We do this because of the understanding that reality is a harsh taskmaster, and it always has some surprises in store for us. This is not to say that data scientists build and sell half-assed, flawed products – quite the opposite: we give you the best possible insight we can with the information we’ve got. But how reality pans out will give us more new information, and we can work with that to move another step closer to the elusive truth of predicting the future. And one day, maybe, we’ll get there. But every day, if we play the game well, we get closer.

Predicting a one-time event is different. You don’t get pointers as to whether you are on the right track or not. There are no subtle indications of whether the model is going to work or not. I have rarely had a problem sticking by a model I built that I knew was correct, because I knew every day that new information would either confirm or improve my model – and after all, turning out the best possible model is the important part, not getting it in one shot, right? It was unnerving to have a model built on fairly experimental techniques, with the world predicting a Clinton win with a shocking unanimity. There were extremely few who predicted a Trump win, and we all were at risk of being labelled either partisans for Trump (a rather hilarious accusation when levelled at me!) or just plain crackpots. So I pledged not to discuss the technical details of my models unless and until the elections confirmed they were right.

So it came to pass that it was me, the almost apolitical one, rather than my extremely clever and politically very passionate wife, who stayed up until the early hours of the morning, watching the results pour in. With CNN, Fox and Twitter over three screens, refreshing all the time, I watched as Trump surged ahead early and maintained a steady win.

My model was right.

Coda

It’s the 16th of November today. It’s been almost a week since the elections, and America is slowly coming to terms with the unexpected. It is a long process, it is a traumatic process, and the polling and ‘quantitative social science’ professions are, to an extent, responsible for this. There was all kinds of sloppiness, multiplication of received wisdom, ‘models’ that in fact were thin confirmations of the author’s prejudices in mathematical terms, and a great deal of stupidity. That does sound harsh, but there’s no better way really to describe articles that, weeks before the election, state without a shade of doubt that we needed to ‘move on’, for Clinton had already won. I wonder if Mr Frischling had a good family recipe for crow? And on the note of election night menu, he may exchange tips with Dr Sam Wang, whom Wired declared 2016’s election data hero in an incredibly complimentary puff piece, apparently quite more on the basis that the author, Jeff Nesbit, hoped Wang was right rather than any indications for analytical superiority.

The fact is, the polling profession failed America and has no real reason to continue to exist. The only thing it has done is make campaigns more expensive and add to the pay-to-play of American politics. I don’t really see myself crying salt tears at the polling profession’s funeral.

The jury is still out on the ‘quantitative social sciences’, but it’s not looking good. The ideological homogeneity in social science faculties worldwide, but especially in America, has contributed to the kind of disaster that happens when people live in a bubble. As scientists, we should never forget to sanity check our conclusions against our experiences, and intentionally cultivate the most diverse circle of friends we can to get as many little slivers of the human experience as we can. When one’s entire milieu consists of pro-Clinton academics, it’s hard to even entertain doubt about who is going to win – the availability heuristic is a strong and formidable adversary, and the only way to beat it is by recruiting a wide array of familiar people, faces, notions, ideas and experiences to rely on.

As I write this, I have an inch-thick pile of papers next to me: calculations, printouts, images, drafts of a longer academic paper that explains the technical side of all this in detail. Over the last few days, I’ve fielded my share of calls from the media – which was somewhat flattering, but this is not my field. I’m just an amateur who might have gotten very lucky – or maybe not.

Time will tell.

In a few months, I will once again be sharing a conference room with my academic brethren. We will discuss, theorize, ideate and exchange views; a long, vivid conversation written for a 500-voice chorus, with all the beauty and passion and dizzying heights and tumbling downs of Tallis’s Spem in Alium. The election has featured prominently in those conversations last time, and no doubt that will be the case again. Many are, at least from an academic perspective, energised by what happened. Science is the only game where you actually want to lose from time to time. You want to be proven wrong, you want to see you don’t know anything, you want to be miles off, because that means there is still something else to discover, still some secrets this Creation conceals from our sight with sleights of hand and blurry mirrors. And so, perhaps the real winners are not those few, those merry few, who got it right this time. The real winners are those who, led by their curiosity about their failure to predict this election, find new solutions, new answers and, often enough, new puzzles.

That’s not a consolation prize. That’s how science works.

And while it’s cool to have predicted the election results more or less correctly, the real adventure is not the destination. The real adventure is the journey, and I hope that I have been able to grant you a little insight into this adventure some of us are on every hour of every day.

References   [ + ]

1. There is an academic paper with a lot more details forthcoming on the matter – incidentally, because republication is generally not permitted, it will contain many visualisations I was not able or allowed to put into this blog post. So just for that, it may be worth reading once it’s out. I will post a link to it here.
2. The reasoning here is roughly as follows. Assume the body is a sphere. All bodies are assumed of being made of the same material, which is also assumed to be homogenous. The volume of a sphere V = \frac{4}{3} \pi r^3 , and its weight is that multiplied by its density \rho . Thus the radius of a sphere of a matter of known density \rho can be calculated as r = \sqrt[3]{\frac{3}{4} \frac{M}{\pi \rho}} . From this, the surface area can be calculated (A = 4 \pi r^2 ). Thus, body weight is a decent stand-in for surface area.
3. I am indebted to Nassim Nicholas Taleb for this example.
4. Divide the smaller by the larger value, normalise to 1.
Posted in Uncategorized

Destiny and desolation

This is the story of how I lost my destiny and found a world without destinies. It’s a story of two viewpoints – the human and the institutional. It’s a story of desire, longing, loss and of new beginnings. It’s a story that perhaps is a little uplifting, but also, on the whole, fairly depressing. And that shall suffice by way of preamble.1The title, as well as this sentence, is of course an act of unbridled linguistic thievery committed against one of my favourite papers in moral philosophy by Rae Langton, but maybe the reader will accept a more charitable construction of homage.

Destiny

There are, to the best of my understanding, two ways of dealing with present adversity. One is to fight it in the here and now, and rage against it with all one’s might, and to hell with the consequences. Or, one might instead opt for a slower course, and bide one’s time. The present may be dark, one might say, but there will be a future that will no doubt vindicate oneself. Sometimes, facing overwhelming odds, where there is no chance of success of a fight in the present, biding one’s time might indeed be the only thing one can do.

Inevitably, it becomes an existential necessity to set one’s sights on a target, as much in the stars as one’s adversity-laden present is a gutter. That, that would be the final justification, the thing to set all things right, the just reward for not giving up hope and the well-earned prize of a steadfast hand.

Some people would call that pursuit an ‘ambition’. That, dear reader, is pure steaming bullshit coming from people who do not understand the depth of the emotion herein described. It’s not an ambition. It’s love. It’s infatuation. It’s a passionate desire, an intoxicating mixture of equal part love and madness, that does not admit to alternatives or silver medals. You, you’ve just got to have it.

And to a select few, that thing becomes a part of their identity, dwarfing much that others care about. From time to time, society encounters individuals thus obsessed (or perhaps possessed would be a more apt term?), and sees them at best as eccentrics and at worst as single-minded obsessives devoid of fundamental human equanimity, who have abstracted their human worth onto the single pinpoint pinnacle of reaching their goal, pouring everything into that goal in ways that might, to the so-called ‘sane’, seem strange to the point of insanity (indeed, a good number of history’s asylums were, and remain, filled with people of this mindset).

I’m not here to convince you that people who go through this emotion are sane. That would be, of course, partly false witness. And partly, it would be witnessing in my own case.

Devotion

An ambition is something you follow (or not, depending on how you’re feeling at the time). A Destiny is more like a plant, something one nourishes, and when one is out of water, it is the way of things that one would willingly shed one’s own blood to sustain that frail little flower, and to hell with one’s own survival. And with every day, it grows into a sustaining force of its own, a symbiotic entity of sorts, in need to be nourished much as it also nourishes and protects.

And sometimes, its very existence can help one survive the unsurvivable.

I will dispense with sordid details of past atrocities. All that needs to be said is, there were plenty. All that needs to be understood is that for a long, long time, the sole thing that kept me alive, through the dark night of the decade from age 8 to 18, was my destiny. I know this because I know those who went through the same night, and few made it out sane, never mind at all. Much of the terror of that night was compounded by a bitter lack of companionship, understanding and respect. And so, my destiny would be the place where I would find all three.

I would be a Fellow of All Souls.

Now, in case you have no idea what I’m talking about, here’s the Cliffsnotes version. Every year, All Souls College, the most elite college of Oxford and Cambridge and admitting only graduate students, elects one, sometimes two Prize Fellows (now called Fellows by Election). You can only try in the first few years after your BA, and generally, you need a top 1st – a top of the tops degree in your subject – to try. You sit a written examination, which is as insane as it is fun: two general papers and two subject papers. Many sit. Few, at most five, are chosen to a viva, an oral examination of sorts where your soul – and mind – is weighed by all current Fellows, many distinguished in their field to the point of being household names. Two at most are chosen.

To be a Prize Fellow is the greatest acknowledgement a young academic can hope for. I could wax lyrical about it, but frankly, there’s no point. It doesn’t matter what it was. It matters what it was to me.

To me, it was acknowledgement that I was worthy – the only thing I craved all my life. A simple, plain recognition that I was worthy of respect, of attention, of fostering and perhaps even of love. And a damning verdict on a world that repaid these needs of mine with rejection and abuse.

And so, when I took my pen to paper on a Saturday morning, the air thick with the sausage and hash browns of a whole university town at brunch, I wrote with the force and fervour of every ounce and grain of pain amassed over fifteen years, every insult and atrocity taken with a straight face and saved for this moment, every single one of those blinding flashes of grief and humiliation that are one’s lot – fifteen years of pent-up rage and anger and hate and that ultimate of human fundamental forces: the desire to be understood, loved and respected. I’m surprised the paper did not catch fire.

And a few days later, I got an e-mail. I made it into the viva. I was within the Final Five. I could see it, just inches away. Here was my prize, and by All Souls’ Day, a few days after the viva, it would be mine.

I came in from the river early, and ran for my room at a breakneck pace on the day of election. It was, we of the final five knew, that afternoon that the results would be communicated to us by the Warden (the head of the College).

And just as sure, the phone rang a few minutes after I entered my room. I picked up the receiver. As soon as I heard the Warden’s tone, I knew what the message was going to be. I was passed over. For whatever reason, and reasons are not really given in this stupid game, I’ve been found wanting.

I don’t care much for seven years’ free food and board. But that day, I lost my destiny.

+0

Losing your destiny is like permanently missing a body part. It’s not so much painful as it is an acute awareness of the fact that there ought to be something there, and it isn’t. The edges of the wound, from which a part of one’s soul was torn with the violence of a stellar explosion, are sore. They heal slowly. Five years on, they still are incredibly sore.

I don’t know to this day how I survived that day, and the following weeks. I fielded calls of sympathy and e-mails telling me how incredibly proud the college was of having me in the viva… well meant, but I frankly couldn’t have cared less. After two and a half decades of working through pain and fatigue and a non-specific sickness of the previous few years that would soon make its grand entrance; after all-nighters heaped upon all-nighters, after exceeding every single expectation, after sacrificing more than many will ever know, – and perhaps I ought to be ashamed of writing this, but I am not – I hung my head and wept for an hour.

I would never be the same.

+1825

Destinies are not like houses. You cannot build yourself a new one if the old one crumbles. You cannot buy a new one. They are crafted in fiery furnaces, and it’s exceptional enough to have one in a lifetime. You certainly don’t get a second one.

What’s left is to pick up the pieces and carry on. That’s indeed what I did. I made a moderate success of my BCL, but it was clear after this rejection that there was no way I could with any self-respect get a doctoral place in Oxford. That’s the cost of shooting your arrow to the sky: if you’re chastened, there’s a good chance you’ll be chastened with that arrow through your knee.

It’s been 1,825 days since the worst injustice of my entire life – worse than any other – the thing that could’ve made everything good. Or so I thought, anyway. I never said there was a trace of sanity in this. If you think I’m entitled, you are probably right – but then, that word also means ‘deserving of receiving something’. And were you not chastising me for just that?

Or, if you think I was insane to put so much into an abstract, not even objectively measurable process: once again, I did not give warranties of sanity.

A few months later, I would stop being able to eat, violently throwing up every bite of food. I carried on doing 20-hour days in utter physical agony. I would eventually come down with a disease so rare, it was the fifth haematologist to pick it up. Things looked pretty darn grim. Very little is known about HLH, but it was pretty well known that it does one thing pretty well: it kills most patients. 78%, according to some statistics. That, by the way, is with treatment. I did not exactly care a lot, but probably decided to do chemotherapy because it was the right thing to do, plus, I am constitutionally intolerant of not doing something about a problem.

In what struck me as utterly bizarre, I lived.

A lot of pieces have fallen into place since then. Towards the end of my chemotherapy, I met the girl who became my loving, devoted wife. If I can write about this experience with pain only, but no anger and resentment, that’s all due to her. I did not really need someone to explain to me how silly the whole thing was, but that does not make it any less real. My heart will never be whole, and I have no more choice in that than I have in having greenish-blue eyes. But whether I would let this poison the good in it or not, that was my choice. I doubt I would have recognised it without her.

She is everything to me. I don’t know if she is my destiny, and I don’t care. Destiny can kiss my ass.

Since then, I have made my own destiny. I married the girl I loved, and now, I can hear her soft breathing as she sleeps next to me, occasionally exuding a giggle from what must be a particularly amusing dream. We have a kitten, and she’s a joy, even if she decides to poop where she’s not supposed to. I’ve found my place in a new career in a new industry that I love, and perhaps I’m better off now. On more cogent days, I even realise I came closer to my dream than virtually everyone save the 1-2 people a year, and that in and of itself is an honour. Some put it on their CV, proudly. I’m not there yet.

I’ve had plenty of recognition, too, since, and I’ve found many like-minded people. And perhaps part of growth is understanding that dreams are just that. We have them and they get us through the night. But we can’t spend our days in dreams. Not unless we aim to have some Thorazine with our dinner, too.

I’m probably never going to have a destiny again, and most of the time, that’s fine with me. “Fine”, like an old injury is fine – painless most days, but makes itself felt every once in a while when moving funny or when the weather turns damp.

References   [ + ]

1. The title, as well as this sentence, is of course an act of unbridled linguistic thievery committed against one of my favourite papers in moral philosophy by Rae Langton, but maybe the reader will accept a more charitable construction of homage.
Posted in Uncategorized

Happy #nationalwombatday everybody!

Happy #nationalwombatday everybody! My fantastic wife <a rel=@katvcsefalvaydrew me this. It's a big bucket list item for me to get to spend some time working with wombats (and I totally have the skills, what with being a data scientist and computational epidemiologist! lol, not.). So she drew me with a whole BUNCH of my absolute favourite critters ever! Awwww-esome! ;)" width="600" height="600" />

Happy #nationalwombatday everybody! My fantastic wife @katvcsefalvaydrew me this. It’s a big bucket list item for me to get to spend some time working with wombats (and I totally have the skills, what with being a data scientist and computational epidemiologist! lol, not.). So she drew me with a whole BUNCH of my absolute favourite critters ever! Awwww-esome! πŸ˜‰

You can see more of Katie’s art, and buy some priceless originals, at her Etsy store.

Posted in Uncategorized

Congratulations, Genevieve!

Our friend Genevieve is now officially a published author and all! We just received the short story anthology with hers in it, and we couldn't be happier for her. :)

Our friend Genevieve is now officially a published author and all! We just received the short story anthology with hers in it, and we couldn’t be happier for her. πŸ™‚

Since this photo was taken, Ginny has published Spin, her first full-length book, which immediately shot to the bestseller lists on Amazon! Keep watching this spot – she is definitely destined for literary greatness!

Posted in Uncategorized