Herd immunity: how it really works

There are few concepts as trivial yet as widely misunderstood as herd immunity. In a sense, it’s not all that surprising, because frankly, there’s something almost magical about it – herd immunity means that in a population, some people who are not or cannot be immunized continue to reap the benefit of immunization. On its own, this may even be counter-intuitive. And so, unsurprisingly, like many evidently true concepts, herd immunity has its malcontents – going so far as to condemn the very idea as a ‘CDC lie’ – never mind that the concept was first used in 1923, well before the CDC was established.1

Now, let’s ignore for a moment what Dr Humphries, a nephrologist-turned-homeopath with a penchant for being economical with the truth when not outright lying, has to say – not because she’s a quack but because she has not the most basic idea of epidemiology. Instead, let’s look at this alleged ‘myth’ to begin with.

Herd immunity: the cold, hard maths

Our current understanding of herd immunity is actually a result of increasing understanding of population dynamics in epidemiology, towards the second half of the 20th century. There are, on the whole, two ways to explain it. Both are actually the same thing, and one can be derived from the other.

The simple explanation: effective R_0 depletion

The simple explanation rests on a simplification that makes it possible to describe herd immunity in terms that are intelligible at the level of high school maths. In epidemiology, R_0 (pron. ‘arr-nought‘, like a pirate), describes the basic reproduction rate of an infectious disease.2 To put it at its most simplistic: R_0 is the number of cases produced by each case. The illustration on the side shows the index case (IDX) and the first two generations of an infection with R_0 = 3.

Now, R_0 is a theoretical variable. It is usually observationally estimated, and don’t take measures intended to reduce it into account. And that’s where it gets interesting.

Consider the following scenario, where a third of the population is vaccinated, denoted by dark black circles around the nodes representing them. One would expect that of the 13 persons, a third, i.e. about. 4 , would remain disease-free. But in fact, over half of the people will remain disease-free, including three who are not vaccinated. This is because the person in the previous generation did not pass on the pathogen to them. In other words, preventing spread, e.g. by vaccination or by quarantine, can affect and reduce R_0. Thus in this case, the effective R_0 was closer to 1.66 than 3 – almost halving the R_0 by vaccinating only a third of the population.

We also know that for infections where the patient either dies or recovers, the infection has a simple ecology: every case must be ‘replaced’. In other words, if the effective R_0 falls below 1, the infection will eventually peter out. This happens quite often when everyone in a population is dead or immune after an infection has burned through it (more about that later).

Thus, the infection will be sustainable if and only if

R_{0} \geq 1

Under the assumption of a 100% efficient vaccine, the threshold value \bar{p_V} after which the infection will no longer be able to sustain itself is calculated as

\bar{p_V} = 1 - \frac{1}{R_0}

Adjusting for vaccine efficacy, E_V, which is usually less than 100%, we get

\bar{p_V} = \frac{1-\frac{1}{R_0}}{E_V} = \frac{R_0 - 1}{R_0 E_V}

For a worked example, let’s consider measles. Measles has an R_0 around 15 (although a much higher value has been observed in the past, up to 30, in some communities), and the measles vaccine is about 96% effective. What percentage of the population needs to be vaccinated? Let’s consider \bar{p_V}, the minimum or threshold value above which herd immunity is effective:

\bar{p_V} = \frac{R_0 - 1}{R_0 E_V} = \frac{15-1}{15 \cdot 0.96} = \frac{14}{14.4} \approx 97.22\%

The more complex explanation: \mathcal{SIR} models

Note: this is somewhat complex maths and is generally not recommended unless you’re a masochist and/or comfortable with calculus and differential equations. It does give you a more nuanced picture of matters, but is not necessary to understand the whole of the argumentation. So feel free to skip it.

The slightly more complex explanation relies on a three-compartment model, in which the population is allotted into one of three compartments: \mathcal{S}usceptible, \mathcal{I}nfectious and \mathcal{R}ecovered. This model makes certain assumptions, such as that persons are infectious from the moment they’re exposed and that once they recover, they’re immune. There are various twists on the idea of a multicompartment model that takes into account the fact that this is not true for every disease, but the overall idea is the same.3 In general, multicompartment models begin with everybody susceptible, and a seed population of infectious subjects. Vaccination in such models is usually accounted for by treating them as ‘recovered’, and thus immune, from t = 0 onwards.

Given an invariant population (i.e. it is assumed that no births, deaths or migration occurs), the population can be described as consisting of the sum of the mutually exclusive compartments: P = \mathcal{S}(t) + \mathcal{I}(t) + \mathcal{R}(t). For the same reason, the total change is invariant over time, i.e.

\frac{d \mathcal{S}}{d t} + \frac{d \mathcal{I}}{d t} + \frac{d \mathcal{R}}{d t} = 0

Under this assumption of a closed system, we can relate the volumes of each of the compartment to the transition probabilities \beta (from \mathcal{S} to \mathcal{I}) and \gamma (from \mathcal{I} to \mathcal{R}), so that:

\frac{d \mathcal{S}}{d t} = - \frac{\beta \mathcal{I} \mathcal{S}}{P}

\frac{d \mathcal{I}}{d t} = \frac{\beta \mathcal{I} \mathcal{S}}{P} - \gamma \mathcal{I}

\frac{d \mathcal{R}}{d t} = \gamma \mathcal{I}

Incidentally, in case you were wondering how this connects to the previous explanation: R_0 = \frac{\beta}{\gamma}.

Now, let us consider the end of the infection. If \mathcal{S} is reduced sufficiently, the disease will cease to be viable. This does not need every individual to be recovered or immune, however, as is evident from dividing the first by the third differential equation and integrating and substituting R_0, which yields

\displaystyle \mathcal{S}(t) = \mathcal{S}(0) e^{\frac{-R_0 (\mathcal{R}(t)-\mathcal{R}(0))}{P}}

Substituting this in, the limit of \mathcal{R}, as t approaches infinity, is

\displaystyle \lim_{t\to\infty}\mathcal{R}(t) = P - \lim_{t\to\infty}\mathcal{S}(t) = P - \mathcal{S}(0) e^{\frac{-R_0 (\mathcal{R}(t)-\mathcal{R}(0))}{P}}

From the latter, it is evident that

\displaystyle \lim_{t\to\infty}\mathcal{S}(t) \neq 0 \mid \mathcal{S}(0) \neq 0

In other words, once the infection has burned out, there will still be some individuals who are not immune, not immunised and not vaccinated. These are the individuals protected by herd immunity. This is a pretty elegant explanation for why herd immunity happens and how it works. There are three points to take away from this.

First, herd immunity is not unique to vaccination. The above finding in relation to the nonzero limit of \lim_{t\to\infty}\mathcal{S}(t) holds as long as \mathcal{S}(0) \neq 0, but regardless of what \mathcal{R}(0) is. In other words, herd immunity is not something artificial.

Two, for any i \in \mathcal{S} (that is, any susceptible person) at time t, the probability of which compartment he will be in at t+1 depends on whom he encounters. That, statistically, depends on the relative sizes of the compartments. In this model, the assumption is that the sample i will encounter will reflect the relative proportions of the individual compartments’ sizes. Thus if i meets n people at time t, each compartment will be proportionally represented, i.e. for any compartment \mathcal{C}, the proportion will be \frac{\mathcal{C}(t)}{P-1} for all \mathcal{C} \neq \mathcal{S}, for which the proportion will be \frac{\mathcal{S}(t) - 1}{P - 1}, since one cannot meet oneself. Given that the transition probability \beta_{i}(t) is assumed to equal the probability of meeting at least one element of \mathcal{I}, the following can be said. i‘s risk of infection depends on the relationship of n and \mathcal{I}(t), so that i is likely to get infected if

\displaystyle n \frac{\mathcal{I}(t)}{P-1} \geq 1

This elucidates two risk factors clearly, and the way to reduce them: reduce interactions (quarantine/self-quarantine), thereby reducing n, and reduce the proportion of infectious cases (\frac{\mathcal{I}(t)}{P-1}). The latter is where herd immunity from immunisation comes in. Recall that for a constant n, i‘s risk of infection at t rises as \mathcal{I}(t) rises.4 Recall also that while susceptible cases can turn into infectious cases, recovered (or vaccinated) cases cannot. And so, as \mathcal{R}(0) converges to P-1,5 i‘s risk of infection at any time t, denoted by \beta_{i}(t), falls. In other words,

\displaystyle \lim_{\mathcal{R}(0) \to P-1} \beta_{i}(t) = 0

Or to put it simply: the more are vaccinated at the start, the lower the probability, all things being equal, to meet someone who can pass on the infection.6

A final point to note is that this is primarily a model of statistical dynamics, and deals with average probabilities. It does not – it cannot – take account of facts like that some some susceptible people are just darn unlucky, and bump into a flock of unvaccinated, shiny-eyed snowflakes. Equally, in some places, susceptible people and infected people congregate, creating a viral breeding ground, also known as a Waldorf school. There are agent based models, which are basically attempts at brute force hacking reality, that can take account of such disparities. The takeaway is that herd immunity does not mean no susceptible individual will get infected. What it does mean is that their probability of getting infected is going to be significantly lower, for two reasons. First given a constant number of encounters (n), the likelihood of one of them being with an infectious individual is going to be much lower. More importantly, however, because of herd immunity, the disease is going to be able to persist in the population for a far shorter time – eventually it will burn through the small number of ‘accessible’ susceptible persons. Since the cumulative risk \beta_{i}^T for i \in \mathcal{S} for an infection that dies out after time T is defined as

\beta_i^T = \displaystyle \int\limits_0^T \beta_{i}(t) \, \mathrm{d}t

– the sooner the infection dies out, the smaller the likelihood that i will be infected. With that mathematical basis, let’s tackle a few of the myths about herd immunity.

Myth #1: herd immunity only works with naturally acquired immunity

This argument goes roughly along the following lines: herd immunity does exist, but it only exists if and where the immunity is acquired the ‘natural’ way, i.e. by surviving the disease. Case in point:

The $64,000 question, of course, is what the difference is between the residual immunity from a vaccine and the residual immunity from having survived the illness. A vaccine effectively ‘simulates’ the illness without actually causing the pathological symptoms. From the perspective of the immune system, it is largely irrelevant whether it has been exposed to an actual virus that can damage the body, or merely a capsid protein that is entirely harmless but will nonetheless elicit the same immune reaction. That should suffice to bust this myth, but it’s worth considering immunity quantitatively for a moment. As we have seen above, the source of immunity doesn’t matter. In fact, it doesn’t even have to be immunity: culling every animal except one in a herd is an entirely good way to reduce disease transmission. So is sealing oneself away from the rest of society and spending the evenings telling sexually explicit stories, as the heroes and heroines of Boccaccio’s Decameron have done, since we know that

\displaystyle n \frac{\mathcal{I}(t)}{P-1} \geq 1

Boccaccio’s crowd of assorted perverts knew nothing of all this, of course, but they did know that if they reduced n, the number of contacts with possibly infected persons, their chances of surviving the plague would increase. As it indeed did. Score one for medieval perverts. The bottom line is that it is entirely immaterial how immunity was obtained.

Myth #2: Herd immunity is a concept deriving from animals. It doesn’t work on humans.

This is one of the more outlandish claims, but shockingly, it actually has a tiny kernel of truth.

Now, much of the above is a veritable storehouse of insanity, but the point it makes in the beginning has some truth to it. In human populations, herd immunity sometimes behaves anomalously, because humans are not homogenously distributed. This is true a fortiori for humans who decide not to vaccinate, who – for better or worse – tend to flock in small groups. The term of venery for a bunch of anti-vaxxers is, in case you were wondering, a ‘plague’.7

Herd immunity was, in fact, observed in a range of species. Humans are different as we can knowingly and consciously decide to create herd immunity in our population and protect our fellow men, women and children, the last of whom are particularly susceptible to infectious diseases, from some of the worst killers.

Myth #3: If herd immunity can be obtained through natural immunity, surely we don’t need vaccines.

This argument has recently been peddled by the illustrious Kelly Brogan MD, who bills herself as a ‘holistic psychiatrist’ who threw away her script pad, which means she tends exclusively to the worried well and those with mild mental health issues where medication does not play as decisive a role as it does in, say, schizophrenia, severe PTSD, crippling anxiety disorders or complex neuropsychiatric post-insult phenomena.8 Here’s her foray into epidemiology, something she vaguely remembers studying in her first year of med school.

In this, Dr Brogan has successfully found almost a century old evidence for what everybody knew, namely that herd immunity can be naturally obtained. To anyone who has read the maths part above, this should evoke a sensation of ‘DUH!’. The problem is twofold. One, the ‘actual virus’ has an unsavoury fatality rate of 0.1%, not including the horribly tragic, heartbreaking late consequence of measles known as SSPE.9 Two, and perhaps more important: you don’t get lifelong, natural immunity if you die. This may have somehow escaped Dr Brogan’s attention, but part of the point of herd immunity is to protect those who would not survive, or would suffer serious sequelae, if they contracted the infection. What we don’t know, of course, how many of that 68% suffered permanent injuries, and how many are not included because they died. What we do know is that all 68% probably had a miserable time. Anyone who thinks measles is so fantastic should start by contracting it themselves.

Myth #4: Herd immunity means 95% need to be vaccinated to prevent a disease.

This one comes courtesy of Sarah aka the Healthy Home Economist,10, who, to what I presume must be the chagrin of her alma mater, states she has a Master’s from UPenn. Suspiciously enough, she does not state what in. I am somehow pretty sure it’s not public health.

The tedious conspiracy theory aside, it is quite evident just how little she understands of herd immunity. No – herd immunity is not based upon11 the idea that 95% must be vaccinated, and it is most definitely not based on the idea that 100% must be vaccinated. Indeed, the whole bloody point of herd immunity is that you do not need to vaccinate 100% to protect 100%. In fact, given the R_0 and vaccine efficacy E_V, we can predict the threshold vaccination rate for herd immunity quite simply, as demonstrated earlier: the threshold value, \bar{p_V}, can be calculated as

\bar{p_V} = \frac{R_0 - 1}{R_0 E_V}

As an illustration, the herd immunity threshold \bar{p_V} for mumps, with an efficacy of 88%12 and an R_0 of around 5.5, is \approx 92.98\%, while for Ebola, which has a very low R_0 around 2.0, herd immunity sets in once about 50% are immune.13

And those ‘conventional health authorities’? That’s what we call health authorities whose ideas work.

Myth #5: If vaccines work, why do we need herd immunity?

This argument is delightfully idiotic, because it, too, ignores the fundamental underlying facts of herd immunity. Quite apart from the fact that some people cannot receive some or all vaccines and other people can receive vaccines but may not generate sufficient antibody titres to have effective immunity, sometimes vaccines simply fail. Few things are 100%, and while vaccines are designed to be resilient, they can degrade due to inappropriate storage or fail to elicit a sufficient response for some other reason. Unlike wearing deodorant (or ‘deoderant’, as spelling-challenged anti-vaxxers would say), infections can sometimes be imagined as a chain of transmission. This is a frequently used model to explain the consequences of not vaccinating on others.

In this illustration, an index patient (IDX) is infected and comes in contact with G1, who in turn comes into contact with G2 who in turn comes into contact with G3. In the first case, G1, G2 and G3 are all vaccinated. The vaccine may have a small failure rate – 2% in this case – but by the time we get to G3, his chances of contracting the infection are 1:125,000 or 0.0008%. In the second case, G2 is unvaccinated – if G1’s vaccine fails, G2 is almost guaranteed to also fall ill. By not vaccinating, his own risk has increased 50-fold, from 0.04% to 2%. But that’s not all – due to G2’s failure to vaccinate, G3 will also be affected – instead of the lottery odds of 1:125,000, his risk has also risen 50-fold, to 1:2,500. And this 50-fold increase of risk will carry down the chain of potential transmission due to G2’s failure to vaccinate. No matter how well vaccines work, there’s always a small residual risk of failure, just as there is a residual risk of failure with everything. But it takes not vaccinating to make that risk hike up 50-fold. Makes that deodorant (‘deoderant’?) analogy sound rather silly, right?

Conclusion

Admittedly, the mathematical basis of herd immunity is complex. And the idea itself is somewhat counterintuitive. None of these are fit excuses for spreading lies and misinformation about herd immunity.

I have not engaged with the blatantly insane arguments (NWO, Zionists, Masonic conspiracies, Georgia Guidestones), nor with the blatantly untrue ones (doctors and public health officers are evil and guided just by money as they cash in on the suffering of innocent children). I was too busy booking my next flight paid for by Big Pharma.14 Envy is a powerful force, and it’s a good way to motivate people to suspect and hate people who sacrificed their 20s and 30s to work healing others and are eventually finally getting paid in their 40s. But it’s the myths that sway the well-meaning and uncommitted, and I firmly believe it’s part of our job as public health experts to counter them with truth.15

In every social structure, part of co-existence is assuming responsibility not just for oneself but for those who are affected by our decisions. Herd immunity is one of those instances where it’s no longer about just ourselves. Many have taken the language of herd immunity to suggest that it is some sort of favour or sacrifice done for the communal good, when it is in in fact the very opposite – it is preventing (inadvertent but often unavoidable) harm to others from ourselves.

And when the stakes are this high, when it’s about life and death of millions who for whatever reason cannot be vaccinated or cannot form an immune response, getting the facts right is paramount. I hope this has helped you, and if you know someone who would benefit from it, please do pass it on to them.

References   [ + ]

1.Topley, W. W. C. and Wilson, G. S. (1923). The spread of bacterial infection; the problem of herd immunity. J Hyg 21:243-249. The CDC was founded 23 years later, in 1946.
2.Why R_0? Because it is unrelated to \mathcal{R}, the quantity denoting recovered cases in \mathcal{S(E)IR} models – which is entirely unrelated. To emphasize the distinction, I will use mathcal fonts for the compartments in compartment models.
3.I hope to write about SIS, SEIR and vital dynamic models in the near future, but for this argument, it really doesn’t matter.
4.Technically, as \frac{\mathcal{I}(t)}{P - 1} rises, but since the model presupposes that P is constant, it doesn’t matter.
5.Since otherwise \mathcal{R} = P and \mathcal{S} = 0, and the whole model is moot, as noted above.
6.Note that this does not, unlike the R_0 explanation, presuppose any degree of vaccine efficacy. An inefficiently vaccinated person is simply in \mathcal{S} rather than \mathcal{R}.
7.Initially, ‘a culture’ was proposed, but looking at the average opponent of vaccination, it was clear this could not possibly work.
8.In other words, if you have actual mental health issues, try an actual psychiatrist who follows evidence-based protocols.
9.Subacute sclerosing panencephalitis is a long-term delayed consequence of measles infection, manifesting as a disseminated encephalitis that is invariably fatal. There are no adjectives that do the horror that is SSPE justice, so here’s a good summary paper on it.
10.As a rule, I don’t link to blogs and websites disseminating harmful information that endangers public health.
11.Correct term: ‘on’
12.As per the CDC.
13.Efficacy E_V is presumed to be 100% where immunity is not acquired via vaccination but by survival.
14.Anyone in public health is happy to tell you those things don’t merely no longer exist, they never even existed in our field.
15.And, if need be, maths.

Installing RStudio Server on Debian 9

Oh, wouldn’t it be just wonderful if you could have your own RStudio installation on a server that you could then access from whatever device you currently have, including an iPad? It totally would. Except it’s some times far from straightforward. Here’s how to do it relatively painlessly.

Step 1: Get a server

Choose a suitable (and affordable) server, pick a location near you in the drop-down menu on the bottom, and press Add this Linode! to set up your first Linode.

I use Linode,1 and in general, their 4096 server is pretty good. Linodes can be very easily resized, so this should not be a worry.

Step 2: Set up the server

Once your Linode is up, it will turn up on your dashboard with a random name (linode1234567, typically). If you click on it, you will see your Linode is ‘Brand New’, which means you need to configure it. I usually keep them in groups depending on purpose: blog servers, various processing servers, hosts, research servers. Each of them then gets a name. Choose whatever nomenclature fits your needs best.

Give your Linode a name you can recognise it by, and assign it to a category.

Next, click on the Rebuild tab, and configure the root password, the operating system (we’ll be using Debian 9), the swap disk size (for R, it’s a good idea to set this as large as you can) and finally, set the deployment disk size. I usually set that for 66%.

Step 3: Install R

The Remote Access field shows the SSH access command, complete with the root user, as well as public IPs, default gateways and other networking stuff.

SSH into your Linode and log in as root. You will find your Linode’s IP address and other interesting factoids about it under the Remote access tab. I have obscured some information as I don’t want you scallywags messing around with my server, but the IP address is displayed both in the SSH link (the one that goes ssh [email protected] or something along these lines) and below under public IPs.

Using vim, open /etc/apt/sources.list and add the following source:

# CRAN server for Debian stretch (R and related stuff)
deb http://cran.rstudio.com/bin/linux/debian stretch-cran34/

Save the file, and next install dirmngr (sudo apt-get install dirmngr). Then, add the requisite GPG key for CRAN, update the repository and install r-base:

sudo apt-key adv --keyserver keys.gnupg.net --recv-key 'E19F5F87128899B192B1A2C2AD5F960A256A04AF'

sudo apt update
sudo apt install r-base

At this point, you can enter R to test if your R installation works, and try to install a package, like ggplot2. It should work, but may ask you to select an installation server. If all is well, this is what you should be getting:

[email protected]:~# R

R version 3.4.4 (2018-03-15) -- "Someone to Lean On"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

In the prompt, enter install.packages("ggplot2"). Select a server if requested. Otherwise, watch ggplot2 (and a bazillion other packages) install. Then, quit R by calling q().

Step 4: Downgrade libssl

For some inscrutable reason, RStudio is currently set up to work with libssl1.0.0, whereas Debian 9 comes with libssl1.1.0 out of the box. Clearly that’s not going to work, so we’ll have to roll back our libssl. We’ll do so by creating a file called /etc/apt/sources.list.d/jessie.list using vim. And we’ll fill it with the following:

deb http://httpredir.debian.org/debian jessie main contrib non-free
deb-src http://httpredir.debian.org/debian jessie main contrib non-free

deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free

In case you’re curious: this creates a sources list called jessie, and allows you to draw from Debian Jessie. So let’s have apt get with the program (sudo apt update), and install libssl1.0.0 (sudo apt install libssl1.0.0). Ift may be prudent to also install the openssl tool corresponding to the libssl version (sudo apt install openssl/jessie).

Step 5: Time to install RStudio Server!

First, install gDebi, a package installer, by typing sudo apt-get install gdebi. Next, we’ll be grabbing the latest RStudio version using wget, and installing it using gDebi. Make sure you either do this in your home directory or in /tmp/, ideally. Note that the versions of RStudio tend to change – 1.1.442 was the current version, released 12 March 2018, at the time of writing this post, but may by now have changed. You can check the current version number here.

wget https://download2.rstudio.org/rstudio-server-1.1.442-amd64.deb
sudo gdebi rstudio-server-1.1.442-amd64.deb

If all is well and you said yes to the dress question about whether you actually want to install RStudio Server (no, you’ve been going through this whole pain in the rear for excrement and jocularity, duh) , you should see something like this:

(Reading database ... 55651 files and directories currently installed.)
Preparing to unpack rstudio-server-1.1.442-amd64.deb ...
Unpacking rstudio-server (1.1.442) ...
Setting up rstudio-server (1.1.442) ...
groupadd: group 'rstudio-server' already exists
rsession: no process found
Created symlink /etc/systemd/system/multi-user.target.wants/rstudio-server.service → /etc/systemd/system/rstudio-server.service.
● rstudio-server.service - RStudio Server
   Loaded: loaded (/etc/systemd/system/rstudio-server.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2018-03-31 21:59:25 UTC; 1s ago
  Process: 16478 ExecStart=/usr/lib/rstudio-server/bin/rserver (code=exited, status=0/SUCCESS)
 Main PID: 16479 (rserver)
    Tasks: 3 (limit: 4915)
   CGroup: /system.slice/rstudio-server.service
           └─16479 /usr/lib/rstudio-server/bin/rserver

Mar 31 21:59:25 localhost systemd[1]: Starting RStudio Server...
Mar 31 21:59:25 localhost systemd[1]: Started RStudio Server.

Now, if all goes well, you can navigate to your server’s IP at port 8787, and you should behold something akin to this:

The login interface. Technically, you could simply log in with your root password and root as the username. But just don’t yet.

There are a few more things you wish to install at this point – these are libraries that will help with SSL and other functionality from within R. Head back to the terminal and install the following:

sudo apt-get install libssl-dev libcurl4-openssl-dev libssh2-1-dev

Step 6: Some finishing touches

The login screen will draw on PAM, Unix’s own authentication module, in lieu of a user manager. As such, to create access, you will have to create a new Unix user with adduser, and assign a password to it. This will grant it a directory of its own, and you the ability to fine-tune what they should, and what they shouldn’t, have access to. Win-win! This will allow you to share a single installation among a range of people.

Step 7: To reverse proxy, or to not reverse proxy?

There are diverging opinions as to whether a reverse proxy such as NGINX carries any benefit. In my understanding, they do not, and there’s a not entirely unpleasant degree of security by obscurity in having a hard to guess port (which, by the way, you can change). It also makes uploads, on which you will probably rely quite a bit, more difficult. On the whole, there are more arguments against than in favour of a reverse proxy, but I may add specific guidance on reverse proxying here if there’s interest.

References   [ + ]

1.Using this link gives me a referral bonus of $20 as long as you remain a customer for 90 days. If you do not wish to do so, please use this link. It costs the same either way, and I would be using Linode anyway as their service is superbly reliable.

MedDRA + VAERS: A marriage made in hell?

This post is a Golden DDoS Award winner

So far, this blog was DDoS’d only three times within 24 hours of its publication. That deserves a prize.

Quick: what do a broken femur, Henoch-Schönlein purpura, fainting, an expired vaccine and a healthy childbirth have in common? If your answer was “they’re all valid MedDRA codes”, you’re doing pretty well. If you, from that, deduced that they all can be logged on VAERS as adverse effects of vaccination, you’re exceeding expectations. And if you also realise that the idea that Jane got an expired HPV vaccine, and as a consequence broke her femur, developed Henoch-Schönlein purpura, and suddenly gave birth to a healthy baby boy is completely idiotic and yet can be logged on VAERS, you’re getting where I’m going.

MedDRA is a medical nomenclature specifically developed for the purposes of pharmacovigilance. The idea is, actually, not dreadful – there are some things in a usual medical nomenclature like ICD-10 that are not appropriate for a nomenclature used for pharmacovigilance reporting (V97.33: sucked into jet engine comes to my mind), and then there are things that are specific to pharmacovigilance, such as “oh shoot, that was not supposed to go up his bum!” (MedDRA 10013659: vaccine administered at inappropriate site), “we overdosed little Johnny on the flu vaccine!” (MedDRA 10000381: drug overdose, accidental) and other joys that generally do only happen in the context of pharmacovigilance. So far, so good.

At the same time, MedDRA is non-hierarchical, at least on the coding level. Thus, while the ICD code V97.33 tells you that you’re dealing with an external cause of mortality and morbidity (V and Y codes), specifically air and space transport (V95-97), more specifically ‘other’ specific air transport accidents, specifically getting sucked into a jet engine (V97.33), there’s no way to extract from MedDRA 10000381 what the hell we’re dealing with. Not only do we not know if it’s a test result, a procedure, a test or a disease, we are hopelessly lost as to figuring out what larger categories it belongs to. To make matters worse, MedDRA is proprietary – which in and of itself is offensive to the extreme to the idea of open research on VAERS and other public databases: a public database should not rely on proprietary encoding! -, and it lacks the inherent logic of ICD-10. Consider the encoding of the clinical diagnosis of unilateral headache in both:

SOC: System Organ Class, HLGT: High Level Group Term, HLT: High Level Term, PT: Preferred Term, LLT: Lower Level Term

Attempting to encode the same concept in MedDRA and ICD-10, note that the final code in MedDRA (10067040) does not allow for the predecessors to be reverse engineered, unless a look-up table is used – which is proprietary. ICD-10, at the same time, uses a structure that encodes the entire ‘ancestry’ of an entity in the final code. Not only does this allow for intermediate codes, e.g. G44 for an ‘other’ headache syndrome until it is closer defined, it also allows for structured analysis of the final code and even without a look-up table (which is public, as is the whole ICD-10), the level of kinship between two IDC-10 codes can be ascertained with ease.

We know that an ICD code beginning with F will be something psychiatric and G will be neurological, and from that alone we can get some easy analytical approaches (a popular one is looking at billed codes and drilling down by hierarchical level of ICD-10 codes, something in which the ICD-10 is vastly superior to its predecessor). MedDRA, alas, does not help us such.

Garbage in, garbage out

OK, so we’ve got a nomenclature where the codes for needlestick injury, death, pneumonia, congenital myopathy and a CBC look all the same. That’s already bad enough. It gets worse when you can enter any and all of these into the one single field. Meet VAERS.

The idea of VAERS is to allow physicians, non-physicians and ‘members of the public’ to report incidents. These are then coded by the CDC and depending on seriousness, they may or may not be investigated (all reports that are regarded as ‘serious’ are investigated, according to the CDC). The problem is that this approach is susceptible to three particular vulnerabilities:

  • The single field problem: VAERS has a single field for ‘symptoms’. Everything’s a symptom. This includes pre-existing conditions, new onset conditions, vaccination errors, lab tests (not merely results, just the tests themselves!), interventions (without specifying if they’re before or after the vaccine), and so on. There is also no way to filter out factors that definitely have nothing to do with the vaccine, such as a pre-existing birth defect. The History/Allergies field is not coded.
  • The coding problem: what gets coded and what does not is sometimes imperfect. This being a human process, it’s impossible to expect perfection, but the ramifications to this to certain methods of analysis are immense. For instance. if there are 100 cases of uncontrollable vomiting, that may be a signal. But if half of those are coded as ‘gastrointestinal disorder’ (also an existing code), you have two values of 50, neither of which may end up being a signal.
  • The issue of multiple coding: because MedDRA is non-hierarchical, it is not possible to normalise at a higher level (say, with ICD-10 codes, at chapter or block level), and it is not clear if two codes are hierarchically related. In ICD-10, if a record contains I07 (rheumatic tricuspid valve disease) and I07.2 (tricuspid stenosis with tricuspid insufficiency), one can decide to retain the more specific or the less specific entry, depending on intended purpose of the analysis.

In the following, I will demonstrate each of these based on randomly selected reports from VAERS.

The Single Field Problem (SFP)

The core of the SFP is that there is only one codeable field, ‘symptoms’.

VAERS ID 375693-1 involves a report, in which the patient claims she developed, between the first and second round of Gardasil,

severe stomach pain, cramping, and burning that lasted weeks. Muscle aches and overall feeling of not being well. In August 2009 patient had flu like symptoms, anxiety, depression, fatigue, ulcers, acne, overall feeling of illness or impending death.

Below is the patient’s symptom transposition into MedDRA entities (under Symptoms):

Event details for VAERS report 375693-1

The above example shows the mixture of symptoms, diagnostic procedures and diagnostic entities that are coded in the ‘Symptoms’ field. The principal problem with this is that when considering mass correlations (all drugs vs all symptoms, for instance), this system would treat a blood test just as much as a contributor to a safety signal as anxiety or myalgia, which might be true issues, or depression, which is a true diagnosis. Unfiltered, this makes VAERS effectively useless for market basket analysis based (cooccurrence frequency) analyses.

Consider for instance, that PRR is calculated as

PRR_{V,R} = \frac{\Sigma (R \mid V) \/ \Sigma (V)}{\Sigma (R \mid \neg V) \/ \Sigma (\neg V)} = \frac{\Sigma (R \mid V)}{\Sigma (V)} \cdot \frac{\Sigma (\neg V)}{\Sigma (R \mid \neg V)}

where V denotes the vaccine of interest, R denotes the reaction of interest, and the \Sigma operator denotes the sum of rows or columns that fulfill the requisite criteria (a more detailed, matrix-based version of this equation is presented here). But if \{R\}, the set of all R, contains not merely diagnoses but also various ‘non-diagnoses’, the PRR calculation will be distorted. For constant V and an unduly large R, the values computationally obtained from the VAERS data that ought to be \Sigma(R \mid V) and \Sigma(R \mid \neg V) will both be inaccurately inflated. This will yield inaccurate final results.

Just how bad IS this problem? About 30% bad, if not more. A manual tagging of the top 1,000 symptoms (by N, i.e. by the number of occurrences) was used as an estimate for how many of the diagnostic entities do not disclose an actual problem with the vaccine.

In this exercise, each of the top 1,000 diagnostic codes (by N, i.e. by occurrence) were categorised into a number of categories, which in turn were divided into includable (yellow) and non-includable (blue) categories. An includable category reveals a relevant (=adverse) test result, a symptom, a diagnosis or a particular issue, while non-includables pertain to procedures, diagnostics without results, negative test results and administration, storage & handling defects.

According to the survey of the top 1,000 codes, only a little more than 70% of the codes themselves disclose a relevant issue with the vaccine. In other words, almost a third of disclosed symptoms must be pruned, and these cannot be categorically pruned because unlike ICD-10, MedDRA does not disclose hierarchies based on which such pruning would be possible. As far as the use of MedDRA goes, this alone should be a complete disaster.

Again, for effect: a third of the codes do not disclose an actual side effect of the medication. These are not separate or identifiable in any way other than manually classifying them and seeing whether they disclose an actual side effect or just an ancillary issue. Pharmacovigilance relies on accurate source data, and VAERS is not set up, with its current use of MedDRA, to deliver that.

The coding problem

Once a VAERS report is received, it is MedDRA coded at the CDC. Now, no manual coding is perfect, but that’s not the point here. The problem is that a MedDRA code does not, in and of itself,  indicate the level of detail it holds. For instance, 10025169 and 10021881 look all alike, where in fact the first is a lowest-level entity (an LLT – Lower-Level Term – in MedDRA lingo) representing Lyme disease, while the former is the top-level class (SOC – System Organ Class) corresponding to infectious diseases. What this means is that once we see a MedDRA coded entity as its code, we don’t know what level of specificity we are dealing with.

The problem gets worse with named entities. You see, MedDRA has a ‘leaf’ structure: every branch must terminate in one or more (usually one) LLT. Often enough, LLTs have the same name as their parent PT, so you get PT Lyme disease and LLT Lyme disease. Not that it terrifically matters for most applications, but when you see only the verbose output, as is the case in VAERS, you don’t know if this is a PT, an LLT, or, God forbid, a higher level concept with a similar name.

Finally, to put the cherry on top of the cake, where a PT is also the LLT, they have the same code. So for Lyme disease, the PT and LLT both have the code 10025169. I’m sure this seemed like a good idea at the time.

The issue of multiple coding

As this has been touched upon previously, because MedDRA lacks an inherent hierarchy, a code cannot be converted into its next upper level without using a lookup table, whereas with, say, ICD-10, one can simply normalise to the chapter and block (the ‘part left of the dot’). More problematically, however, the same code may be a PT or an LLT, as is the case for Lyme disease (10025169).

Let’s look at this formally. Let the operator \in^* denote membership under the transitive closure of the set membership relation, so that

  1. if x \in A, then x \in^* A,
  2. if x \in A and A \subseteq B, then x \in^* B.

and so on, recursively, ad infinitum. Let furthermore \in^*_{m} denote the depth of recursion, so that

  1. for x \in A:  x \in^*_{0} A,
  2. for x \in A \mid A \subseteq B:  x \in^*_{1} B,

and, once again, so on, recursively, ad infinitum.

Then let a coding scheme \{S_{1...n}\} exhibit the Definite Degree of Transitiveness (DDoT) property iff (if and only if) for any S_m \mid m \leq n, there exists exactly one p for which it is true that S_m \in^*_{p} S.

Or, in other words, two codes S_q, S_r \mid q, r \leq n, may not be representable identically if p_q \neq p_r. Less formally: two codes on different levels may not be identical. This is clearly violated in MedDRA, as the example below shows.

Violating the Definite Degree of Transitiveness (DDoT) property: PT Botulism and LLT Botulism have different p numbers, i.e. different levels, but have nonetheless the same code. This makes the entity 10006041 indeterminate – is it the PT or the LLT for botulism? This is a result of the ‘leaf node constraint’ of MedDRA’s design, but a bug by design is still a bug, not a feature.

Bonus: the ethical problem

To me as a public health researcher, there is a huge ethical problem with the use of MedDRA in VAERS. I believe very strongly in open data and in the openness of biomedical information. I’m not alone: for better or worse, the wealth – terabytes upon terabytes – of biomedical data, genetics, X-ray crystallography, models, sequences  prove that if I’m a dreamer, I’m not the only one.

Which is why it’s little short of an insult to the public that a pharmacovigilance system is using a proprietary encoding model.

Downloads from VAERS, of course, provide the verbose names of the conditions or symptoms, but not what hierarchical level they are, nor what structure they are on. For that, unless you are a regulatory authority or a ‘non-profit’ or ‘non-commercial’ (which would already exclude a blogger who unlike me has ads on their blog to pay for hosting, or indeed most individual researchers, who by their nature could not provide the documentation to prove they aren’t making any money), you have to shell out some serious money.

MedDRA is one expensive toy.

Worse, the ‘non-profit’ definition does not include a non-profit research institution or an individual non-profit researcher, or any of the research bodies that are not medical libraries or affiliated with educational institutions but are funded by third party non-profit funding:

This just keeps getting worse. Where would a non-profit, non-patient care provider, non-educational, grant-funded research institution go?

There is something rotten with the use of MedDRA, and it’s not just how unsuitable it is for the purpose, it is also the sheer obscenity of a public database of grave public interest being tied to a (vastly unsuitable and flawed, as I hope it has been demonstrated above) nomenclature.

Is VAERS lost?

Resolving the MedDRA issue

Unlike quite a few people in the field, I don’t think VAERS is hopelessly lost. There’s, in fact, great potential in it. But the way it integrates with MedDRA has to be changed. This is both a moral point – a point of commitment to opening up government information – and one of facilitating research.

There are two alternatives at this point for the CDC.

  1. MedDRA has to open up at least the 17% of codes, complete with hierarchy, that are used within VAERS. These should be accessible, complete with the hierarchy, within VAERS, including the CDC WONDER interface.
  2. The CDC has to switch to a more suitable system. ICD-10 alone is not necessarily the best solution, and there are few alternatives, which puts MedDRA into a monopoly position that it seems to mercilessly exploit at the time. This can – and should – change.

Moving past the Single Field Problem

MedDRA apart, it is crucial for VAERS to resolve the Single Field Problem. It is clear that from the issues presented in the first paragraph – a broken femur, Henoch-Schönlein purpura, fainting, an expired vaccine and a healthy childbirth – that there is a range of issues that need to be logged. A good structure would be

  1. pre-existing conditions and risk factors,
  2. symptoms that arose within 6 hours of administration,
  3. symptoms that arose within 48 hours of administration,
  4. symptoms that arose later than 48 hours of administration,
  5. non-symptoms,
  6. clinical tests without results,
  7. clinical tests segmented by positive and negative results, and
  8. ancillary circumstances, esp. circumstances pertaining to vaccination errors such as wrong vaccine administered, expired vaccine, etc.

The use of this segmentation would be able to differentiate not only time of occurrence, but also allow for adequate filtering to identify the correct denominators for the PRR.

A future with (for?) MedDRA

As said, I am not necessarily hostile to MedDRA, even if the closet libertarian in me bristles at the fact that MedDRA is mercilessly exploiting what is an effective monopoly position. But MedDRA can be better, and needs to be better – if not for its own economic interests, then for the interests of those it serves. There are three particular suggestions MedDRA needs to seriously consider.

  1. MedDRA’s entity structure is valuable – arguably, it’s the value in the entire project. If coding can be structured to reflect its internal hierarchy, MedDRA becomes parseable without a LUT,1 and kinship structures become parseable without the extra step of a LUT.
  2. MedDRA needs to open up, especially to researchers not falling within its narrowly defined confines of access. Especially given the inherent public nature of its use – PhV and regulation are quintessentially public functions, and this needs an open system.
  3. MedDRA’s entity structure’s biggest strength is that it comprises a range of different things, from administrative errors through physical injuries to test results and the simple fact of tests.

Conclusion

VAERS is a valuable system with a range of flaws. All of them are avoidable and correctable – but would require the requisite level of will and commitment – both on CDC’s side and that of MedDRA. For any progress in this field, it is imperative that the CDC understand that a public resource maintained in the public interest cannot be driven by a proprietary nomenclature, least of all one that is priced out of the range of the average interested individual: and if they cannot be served, does the entire system even fulfill its governmental function of being of the people and for the people? It is ultimately CDC’s asset, and it has a unique chance to leverage its position to ensure that at least as far as the 17% of MedDRA codes go that are used in VAERS, these are released openly.

In the end, however sophisticated our dissimilarity metrics, when 30% of all entities are non-symptoms and we need to manually prune the key terms to avoid denominator bloat due to non-symptom entities, such as diagnostic tests without results or clearly unconnected causes of morbidity and mortality like motor vehicle accidents, dissimilarity based approaches will suffer from serious flaws. In the absence of detailed administration and symptom tracking at an individual or institutional level, dissimilarity metrics are the cheapest and most feasible ways of creating value out of post marketing passive reports. If VAERS is to be a useful research tool, as I firmly believe it was intended to be, it must evolve to that capability for all.

References   [ + ]

1.Look-up table

SafeGram: visualising drug safety

Update: an RMarkdown notebook explaining the whole process is available here.

Visualising vaccine safety is hard. Doing so from passive (or, as we say it in Britain, ‘spontaneous’!) pharmacovigilance (PhV) sources is even harder. Unlike in active or trial pharmacovigilance, where you are essentially dividing the number of incidents by the person-time or the number of patients in the cohort overall, in passive PhV, only incidents are reported. This makes it quite difficult to figure out their prevalence overall, but fortunately, we have some metrics we can use to better understand the issues with a particular medication or vaccine. The proportional reporting ratio (PRR) is a metric that can operate entirely on spontaneous reporting, and reflect how frequent a particular symptom is for a particular treatment versus all other treatments.

Defining PRR

For convenience’s sake, I will use the subscript * operator to mean a row or column sum of a matrix, so that

N_{i,*} = \displaystyle \sum_{j=1}^{n} N_{i,j}

and

N_{*,j} = \displaystyle \sum_{i=1}^{m} N_{i,j}

and furthermore, I will use the exclusion operator * \neg to mean all entities except the right hand value. So e.g.

N_{i, * \neg k} = \displaystyle \sum_{j=1, j \neq k}^m N_{i,j}

Conventionally, the PRR is often defined to with reference to a 2×2 contingency table that cross-tabulates treatments (m axis) with adverse effects (n axis):

Adverse effect of interest
(i)
All other adverse effects
(\neg i)
TOTAL
Treatment of interest
(j)
a = D_{i,j}b = D_{i, * \neg j}a + b = D_{i, *} = \displaystyle \sum_{j = 1}^{n} D_{i, j}
All other treatments
(\neg j)
c = D_{* \neg i, j}d = D_{* \neg i, * \neg j}c + d = D_{* \neg i, *} = \displaystyle \sum_{k=1, k \neq i}^{m} \sum_{l = 1}^{n} D_{k, l}

 

With reference to the contingency table, the PRR is usually defined as

\frac{a / (a+b)}{c / (c+d)} = \frac{a}{a + b} \cdot \frac{c + d}{c}

However, let’s formally define it over any matrix D.

Definition 1. PRR. Let D be an m \times n matrix that represents the frequency with which each of the m adverse effects occur for each of the n drugs, so that D_{i,j} (i \in m, j \in n) represents the number of times the adverse effect j has occurred with the treatment i.

For convenience’s sake, let D_{*,j} denote \sum_{i=1}^{m} D_{i,j}, let D_{i,*} denote \sum_{j=1}^{n} D_{i,j}, and let D_{*,*} denote \sum_{i=1}^{m} \sum_{j=1}^{n} D_{i,j}. Furthermore, let D_{* \neg i, j} denote \sum_{k \neq i}^{m} D_{k,j} and D_{i, * \neg j} denote \sum_{k \neq j}^{n} D_{i, k}.

Then, PRR can be calculated for each combination D_{i,j} by the following formula:

PRR_{i,j} = \frac{D_{i,j} / D_{i,*}}{D_{* \neg i, j} / D_{* \neg i, *}} = \frac{D_{i,j}}{D_{i,*}} \cdot \frac{D_{*\neg i, *}}{D_{*\neg i, j}}

Expanding this, we get

PRR_{i,j} = \frac{D_{i,j}}{\displaystyle\sum_{q=1}^n D_{i,q}} \cdot \frac{\displaystyle\sum_{r=1, r\ne i}^{m} \displaystyle\sum_{s=1}^{n} D_{r,s}}{\displaystyle\sum_{t=1, t\ne i}^{m} D_{t,j}}

Which looks and sounds awfully convoluted until we start to think of it as a relatively simple query operation: calculate the sum of each row, then calculate the quotient of the ADR of interest associated with the treatment of interest divided by all uses of the treatment of interest on one hand and the ADR of interest associated with all other drugs (j \mid \neg i or c) divided by all ADRs associated with all treatments other than the treatment of interest. Easy peasy!

Beyond PRR

However, the PRR only tells part of the story. It does show whether a particular symptom is disproportionately often reported – but does it show whether that particular symptom is frequent at all? Evans (1998) suggested using a combination of an N-minimum, a PRR value and a chi-square value to identify a signal.1 In order to represent the overall safety profile of a drug, it’s important to show not only the PRR but also the overall incidence of each risk. The design of the SafeGram is to show exactly that, for every known occurred side effect. To show a better estimate, instead of plotting indiviual points (there are several hundreds, or even thousands, of different side effects), the kernel density is plotted.

This SafeGram shows four vaccines – meningococcal, oral and injectable polio and smallpox -, and their safety record based on VAERS data between 2006 and 2016.

The reason why SafeGrams are so intuitive is because they convey two important facts at once. First, the PRR cut-off (set to 3.00 in this case) conclusively excludes statistically insignificant increases of risk.2 Of course, anything above that is not necessarily dangerous or proof of a safety signal. Rather, it allows the clinician to reason about the side effect profile of the particular medication.

  • The meningococcal vaccine (left upper corner) had several side effects that occurred frequently (hence the tall, ‘flame-like’ appearance). However, these were largely side effects that were shared among other vaccines (hence the low PRR). This is the epitome of a safe vaccine, with few surprises likely.
  • The injectable polio vaccine (IPV) has a similar profile, although the wide disseminated ‘margin’ (blue) indicates that ht has a wider range of side effects compared to the meningococcal vaccine, even though virtually all of these were side effects shared among other vaccines to the same extent.
  • The oral polio vaccine (OPV, left bottom corner) shows a flattened pattern typical for vaccines that have a number of ‘peculiar’ side effects. While the disproportionately frequently reported instances are relatively infrequent, the ‘tail-like’ appearance of the OPV SafeGram is a cause for concern. The difference between meningococcal and IPV on one hand and OPV on the other is explained largely by the fact that OPV was a ‘live’ vaccine, and in small susceptible groups (hence the low numbers), they could provoke adverse effects.
  • The smallpox vaccine, another live vaccine, was known to have a range of adverse effects, with a significant part of the population (about 20%) having at least one contraindication. The large area covered indicates that there is a rather astonishing diversity of side effects, and many of these – about half of the orange kernel – lies above the significance boundary of 3.00. The large area covered by the kernel density estimate and the reach into the right upper corner indicates a very probable safety signal worth examining.

Interpretation

A SafeGram for each vaccine shows the two-dimensional density distribution of two things – the frequency and the proportional reporting rate of each vaccine (or drug or device or whatever it is applied to). When considering the safety of a particular product, the most important question is whether a particular adverse effect is serious – a product with a low chance of an irreversible severe side effect is riskier than one with a high probability of a relatively harmless side effect, such as localized soreness after injection. But the relative severity of a side effect is hard to quantify, and a better proxy for that is to assume that in general, most severe side effects will be unique to a particular vaccine. So for instance while injection site reactions and mild pyrexia following inoculation are common to all vaccines and hence the relative reporting rates are relatively low, reflecting roughly the number of inoculations administered, serious adverse effects tend to be more particular to the vaccine (e.g. the association of influenza vaccines with Guillain-Barré syndrome in certain years means that GBS has an elevated PRR, despite the low number of occurrences, for the flu vaccines). Discarding vaccines with a very low number of administered cases, the SafeGram remains robust to differences between the number of vaccines administered. Fig. 1 above shows a number of typical patterns. In general, anything to the left of the vertical significance line can be safely ignored, as they are generally effects shared between most other vaccines in general and exhibit no specific risk signal for the particular vaccine. On the other hand, occurrences to the right of the vertical significance line may – but don’t necessarily do – indicate a safety signal. Of particular concern are right upper quadrant signals – these are frequent and at the same time peculiar to a particular vaccine, suggesting that it is not part of the typical post-inoculation syndrome (fever, fatigue, malaise) arising from immune activation but rather a specific issue created by the antigen or the adjuvant. In rare cases, there is a lower right corner ‘stripe’, such as for the OPV, where a wide range of unique but relatively infrequent effects are produced. These, too, might indicate the need for closer scrutiny. It is crucial to note that merely having a density of signals in the statistically significant range does not automatically mean that there is a PhV concern, but rather that such a concern cannot be excluded. Setting the PRR significance limit is somewhat arbitrary, but Evans et al. (2001) have found a PRR of 2, more than 3 cases over a two year period and a chi-square statistic of 4 or above to be suggestive of a safety signal. Following this lead, the original SafeGram code looks at a PRR of 3.0 and above, and disregards cases with an overall frequency of 3Y, where Y denotes the number of years considered.

Limitations

The SafeGram inherently tries to make the best out of imperfect data. Acknowledging that passive reporting data is subject to imperfections, some caveats need to be kept in mind.

  • The algorithm assigns equal weight to every ‘symptom’ reported. VAERS uses an unfiltered version of MedDRA, a coding system for regulatory activities, and this includes a shocking array of codes that do not suggest any pathology. For instance, the VAERS implementation of MedDRA contains 530 codes for normal non-pathological states (e.g. “abdomen scan normal”), and almost 18,000 (!) events involve at least one of these ‘everything is fine!’ markers. This may be clinically useful because they may assist in differential diagnosis and excluding other causes of symptoms, but since they’re not treated separately from actually pathological symptoms, they corrupt the data to a minor but not insignificant extent. The only solution is manual filtering, and with tens of thousands of MedDRA codes, one would not necessarily be inclined to do so. The consequence is that some symptoms aren’t symptoms at all – they’re the exact opposite. This is not a problem for the PRR because it compares a symptom among those taking a particular medication against the same symptom among those who are not.
  • A lot of VAERS reports are, of course, low quality reports, and there is no way for the SafeGram to differentiate. This is a persistent problem with all passive reporting systems.
  • The SafeGram gives an overall picture of a particular drug’s or vaccine’s safety. It does not differentiate between the relative severity of a particular symptom.
  • As usual, correlation does not equal causation. As such, none of this proves the actual risk or danger of a vaccine, but rather the correlation or, in other words, potential safety signals that are worth examining.
Grouped by pathogen, the safety of a range of vaccines was examined by estimating the density of adverse event occurrence versus adverse event PRR. Note that adverse events reported in VAERS do not show or prove causation, only correlation. This shows that for the overwhelming majority of vaccines, most AEFI reports are below the PRR required to be considered a true safety signal.

SafeGrams are a great way to show the safety of vaccines, and to identify which vaccines have frequently occurring and significantly distinct (high-PRR) AEFIs that may be potential signals. It is important to note that for most common vaccines, including controversial ones like HPV, the centre of the density kernel estimate are below the margin of the PRR signal limit. The SafeGram is a useful and visually appealing proof of the safety of vaccines that can get actionable intelligence out of VAERS passive reporting evidence that is often disregarded as useless.

References   [ + ]

1.Evans, S. J. W. et al. (1998). Proportional reporting ratios: the uses of epidemiological methods for signal generation. Pharmacoepidemiol Drug Saf, 7(Suppl 2), 102.
2.According to Evans et al., the correct figure for PRR exclusion is 2.00, but they also use N-restriction and a minimum chi-square of 4.0.

Out for tapas with Katie and Dad!

Pata Negra is an excellent little tapas place near the riverside, halfway between the hill we live on and Újlak to the north. The food is sometimes a little hit and miss – the red wine chorizo is great, but some staples are not that brilliant and others are outright odd. Still, overall a great place with decent atmosphere and a Sangría that should come with its own weekend safety brief  ⭐️⭐️⭐️⭐️

Taken on Feb 25, 2018 @ 20:16 near Pata Negra Budapest, this photo was originally posted on my Instagram. You can see the original on Instagram by clicking here.

Krak’n Town is crackin’!

Krak’n Town is crackin’! We visited Budapest’s one and only steampunk themed restaurant for a belated Valentine’s Day dinner, and had an absolute blast. Great place, good atmosphere and quirky yet yummy dishes! Highly recommended. ⭐️⭐️⭐️⭐️⭐️

Taken on Feb 15, 2018 @ 17:45 near Krak'n Town, this photo was originally posted on my Instagram. You can see the original on Instagram by clicking here..

Ebola! Graph databases! Contact tracing! Bad puns!

Thanks to the awesome folks at Neo4j Budapest and GraphAware, I will be talking tonight about Ebola, contact tracing, how graph databases help us understand epidemics and maybe prevent them someday. Now, if flying to Budapest on short notice might not work for you, you can listen to a livestream of the whole event here! It starts today, 13 February, at 1830 CET, 1730 GMT or 1230 Eastern Time, and I sincerely hope you will listen to it, live or later from the recording, also accessible here.

An epitaph on a life of service

They say your first boss is your most memorable one – the one that shows you the ropes about what work means and how to do it. This is not about my first boss, but my most memorable one. She wasn’t even my boss for particularly long, but only for the few months of a summer internship at the European Parliament. But she passed on some lessons to me that I might not always have been able to live up to, but I sure as anything never forgot.

It was the summer after my first year at Oxford, and my time in Brussels seemed to start under an ill omen: the turbine cowling of the plane I flew in on detached during landing, leaving the travelers on the starboard side seated around the wing screaming like a class of schoolkids on a rollercoaster. For that reason, we had to land at a divert field, which was out in the middle of nowhere, and my escort, whose phone number I did not know, was waiting for me at the other airport. My entry passes weren’t ready. And, of course, not only was I late for my first day because I inherited my father’s inability to read a map, I also had a whole flock of butterflies in my stomach.

“You’ve… grown!”, I heard her say immediately as she saw me. I hastily tried to recall when we last met – it was, I eventually recollected, more than half decade earlier, at a state dinner during her tenure as ambassador to Vienna, to which my father let me tag along, that we last saw each other. Her husband, a keen yachtsman, quite enjoyed having someone to ‘talk shop’ with, something he certainly did not seem to have had much occasion to in Vienna. And there I was, so many years later, in an almost entirely different world – she was now an MEP,1 I finished my first year in Oxford a few weeks earlier. I have gone from a chubby four-foot-something to a muscular 6’2″, no doubt helped by my physically and mentally intense freshman year. And with that single sentence, she dispelled all my anxiety and apprehension.

In retrospect, it’s silly how something so irrelevant could put me at ease immediately. I would learn, eventually, that this was part of her superpower. She was not a career politician. She was not a career diplomat, though she could be plenty diplomatic. She was far from a career Eurocrat. But she had the skill to put everyone at ease, find a voice with a range of people and say the right thing every time. In my personal diary, I wrote this about her style: “she could talk to butterflies.2 And that included not merely her ability to talk to everyone of all ranks and stripes, but a character that earned a begrudging respect even from her fiercest rivals.

She was not a believer in equal opportunities, but in fairness to all and responsibility according to merit. With my area of responsibility focusing on the Single European Sky Air Traffic Management Research and the Galileo GPS network, not only did she not mind taking advice and asking questions from a barely twentysomething intern, she relished it. The same went for everyone else – she took pleasure in learning new things and she enjoyed how such occasions confirmed that she picked right people for her staff. I would run meetings with experts, talk to stakeholders, write draft legislation and be the last to leave the office. And we had a blast.

On the last day on the job, she took me for a stroll across Parc Léopold, sheltering in the shadow of the European Parliament’s Hémicycle and the Solvay library.3 Others, I suppose, did appraisals – she bought us some French fries from a nearby stall, sat me down on a bench and taught me one of the greatest lessons of my life. I remember every moment like it is seared into my mind, and I hope never to forget her words.

“Service,” she would say, “is not about doing a job for a paycheck. It’s not about doing a job for the money. It’s not doing a job for the perks. Many will wonder why you chose to serve rather than being served. Some will make you feel unappreciated. But if I thought you’re after the easy path or if I thought you couldn’t handle watching others have an easier time, we wouldn’t be having this conversation.”

“You see,” she said, tossing a few French fries to the hungry ducks, “service is about doing what is right, because it’s the right thing and because you’re the right person to do it, and damn the consequences. And more often than not, it will be hard, unpleasant and sometimes even painful. You need to remember why you’re doing what it is you’re doing.”

“Now, I don’t know what you’ll end up doing,” she continued, almost as if she had somehow sensed my incipient misgivings about a career in public office. “But I know that you want to serve. I know that you want to do well, at all costs. I know that you want to see the job through. I know you want to do it with courage and honour, whomever or whatever you serve – your family, your nation, your people, or even humanity. And I know you chose that life out of your own volition, knowing it won’t be easy. But I also know there’s nothing on God’s green Earth that could make you happy other than service.”

“So find where you’re needed, run towards the fire, and don’t ever forget why you’re doing what you do.”

“Don’t you ever forget.”

It’s been many long years since that talk, and much has changed. I’ve climbed mountains and dug my way out of holes to find my way to that service that will sustain me. Her words have echoed with every reveille and with every moment on the verge of falling asleep on the keyboard, with every mile run and with every triumph and every failure. And if I am ever blessed to have children, I hope I will have the chance to pass this simple yet deep wisdom on to them.

As I write this, a cold, chilly, clear February winter night’s winds howl outside. I can hear the soft breathing of my gorgeous wife next to me, and the tiny snores of our kitten at the foot of the bed, occasionally stirring. And tomorrow is another day, another challenges and another set of service.


Dr. Barsi Pataky Etelka ran the course, and ran it with courage, strength and a commitment to service I have not often witnessed. Born in 1941, she trained as an architect and engineer. In 1990, with the fall of the communist terror regime, she would enter the political arena, eventually spending a tenure in Parliament and serving as Hungary’s ambassador to Vienna. In 2003, she was among the first batch of MEPs elected from Hungary, and became responsible for Europe’s ATC research and the development of Galileo, the EU’s global geospatial positioning system. Since 2009, she has been President of the Hungarian Chamber of Engineering and since 2010, a Governmental Commissioner for the Danube Regional Strategy. Dr. Barsi Pataky passed away yesterday, at the age of 77.

She has been married to the love of her life, the engineer, merchant and life-long legendary yachtsman István Barsi, for 44 years, until his death in 2012. Theirs was a kind of love that young men aspire to for their older days; a love that carried both of them through stormy waters.

To those she allowed in her confidence and mentored, however briefly, she will never be gone. She will be there, in our consciences and our thoughts, encouraging us to find where we are supposed to be of service and do so. And as long as we try to do our duty, as the Lord and our skills and talents give us the light to see it, she will live on.

References   [ + ]

1.Member of the European Parliament
2.That’s not to be taken literally. It is from a Kipling poem called True Royalty:

There was never a Queen like Balkis,
From here to the wide world’s end;
But Balkis talked to a butterfly
As you would talk to a friend.

There was never a King like Solomon,
Not since the world began;
But Solomon talked to a butterfly
As a man would talk to a man.

She was Queen of Sabaea—
And he was Asia’s Lord
But they both of ’em talked to butterflies
When they took their walks abroad.

3.Famous for being the scene of the Solvay Conferences, including the Fifth Solvay Conference, commemorated in the immortal photo that has basically all luminaries of interbellum physics.

In extremis

It’s not frequent for a State of the Union address to delve into drug approval policy in any depth. Yet that’s exactly what President Trump did when, for the first time, he spoke publicly about legislation allowing terminally ill patients to access experimental treatments that have passed only FDA Phase I trials, often referred to as ‘right to try’ legislation:

We also believe that patients with terminal conditions should have access to experimental treatments that could potentially save their lives. People who are terminally ill should not have to go from country to country to seek a cure — I want to give them a chance right here at home. It is time for the Congress to give these wonderful Americans the ‘right to try’.

The Right to Try is unsurprisingly controversial. On one hand, patient groups see it as a chance to access treatments that are too far in the pipeline for them. It is hard not to have sympathy with this argument. It is especially hard for me to do so so, because my life was saved by an experimental drug that at the time did not have general approval for my condition, though it was known to be safe. At the same time, FDA Commissioner Scott Gottlieb is right to be skeptical about this policy effectively usurping the FDA’s authority to ensure that pharmaceuticals administered to all patients in the United States are safe and effective. Like all great moral quandaries, both sides are, to an extent, right.

What is ‘right to try’?

Quite simply put, right to try laws allow terminally ill patients access to treatments, medications and devices that have passed FDA Phase I testing, but are not yet approved by the FDA. The libertarian Goldwater Institute, which has been pushing and lobbying for right to try, has created a model legislation, variants of which have by now been accepted by 38 states. It provides, in short, an exception for patients suffering from “advanced illness”, defined as

a progressive disease or medical or surgical condition that entails significant functional impairment, that is not considered by a treating physician to be reversible even with administration of current federal drug administration approved and available treatments, and that, without life-sustaining procedures, will soon result in death.1

Patients that qualify under this definition would then be allowed access to any treatment, pharmaceutical or device as long as it has passed Phase I testing,2 although the manufacturer or provider would be under no obligation to sell or provide that treatment to the patient.

Ethical issues

The ‘right to try’ legislation is far from uncontroversial. @gorskon, whom I greatly respect even when I disagree with him, has gone so far as to call it a ‘cruel sham’ and a libertarian attack on the FDA, and his points merit consideration:

I’ve written many times before over the last three years about how “right-to-try” laws have swept the states. When last I wrote about right-to-try, 37 states had passed such laws over the course of a mere three years, and I observed at the time that it wouldn’t surprise me in the least if most or all of the remaining states were to pass such laws within the next year or two. Basically, the idea behind these laws is that the FDA is killing patients (I’m only exaggerating slightly) through its slow drug approval, overcaution, and bureaucratic inertia, or at least letting them die because life-saving drugs are being held up. So the idea, hatched by the Goldwater Institute was that terminally ill patients should have the “right-to-try” experimental drugs not yet approved by the FDA because they have nothing more to lose. Of course, it’s not true that they have nothing more to lose, but I’ll discuss that more later. Basically, right-to-try laws purport to allow the terminally ill “one last shot” by letting them access experimental therapeutics outside of FDA-sanctioned clinical trials. However, these laws operate under a number of false assumptions, not the least of which is the caricature of the FDA as being slow, inefficient, and unwilling to bend, as you will see. They also strip away a number of protections for patients, as you will also see.

While I am not sure I’m on board with the idea of there being a libertarian conspiracy to curb the FDA’s powers – especially given how limited the ambit of right to try legislations would be -, Orac makes an excellent point.

Much of the Goldwater Institute’s position is premised on the FDA being ‘slow’ and inefficient – as they like to present their case, they merely seek to remedy an instance of the state failing to serve citizens adequately. Speaking from personal experience, when you’re dying, everything is too slow and no approval process can come fast enough. It is hard not to have a lot of sympathy towards the patients who know there may be a promising drug in the pipeline but like Moses of old, they will never get to see the promised land. But realistically, the FDA is not slow – indeed, it is as fast as, or sometimes even faster, than regulatory agencies in many other countries.3

I would also add that the benefits of investigational therapies has rarely been particularly high, with only about 10% yielding a clinical improvement.4. For 90% of patients, then, the right to try would mean putting themselves through another round of torturous treatment instead of spending their last weeks or months focusing on appropriate symptom relief, quality of life and putting their affairs in order. In the end, these might be more important than a forlorn hope of extending one’s life by another few months.

Patients are subject to a high degree of informational asymmetry. When I had to decide between various treatments, I spent days on PubMed, reading every single study, building my own little mini-metaanalysis from my hospital bed. I was lucky – I had access to all the academic literature I could want and I was trained in evaluating that evidence. But most patients aren’t (and there’s no reason why they would be!),5 and what takes the place of sound knowledge is often less healthy. Patients may feel emotional pressure to try every treatment, however modest the chance of success: be it because they would feel that not doing so is ungrateful towards the doctors who ‘fought for them’, or because they feel they owe it to their family, the psychological pressure to try potentially ineffective treatments is immense, and might rob the patient from their chance to exercise some degree of autonomy over the last moments of their life.

The reverse of the medal

At the same time, many note, respect for the patient’s autonomy should extend to allowing treatment that a competent patient wants, even if the physician disagrees. And, in addition, many argue that it would be paradoxical to allow patients to outright request physician-assisted suicide but not the administration of a treatment that may just save their lives. These arguments are not pointless, and any policy needs to justify why paternalism is particularly justified in this case, and while treatment would be inappropriate where suicide would be permitted.

More importantly, it is arguable that the absence of a ‘right to try’ leads to its own set of tragic adverse consequences, by directing patients to ‘try’ treatments in the unregulated sector of outright quackery. I had the distinct misfortune of witnessing one of these.

Jillian Mai Thi Epperly is an unqualified naturopathic healer with no educational background in nutrition who is running what she describes as a large-scale experiment on volunteers (aka marks). Her victims – around 30,000 – joined her Facebook group, which is closer to a cult than anything else, and consume vast quantities of a concoction that contains an unhealthy amount of salt and fermented cabbage juice. This is supposed to rid the body of ‘weaponised mutant candida and parasites’, which she claims is responsible for all or most pathological processes in the human body. Ms Epperly’s Facebook group is replete with images generally for the strong of stomach (including gut lining which her acolytes believe are parasites), but that’s nothing compared to the damage she has done to human lives. None is more tragic than the story of J. (name redacted in the interests of privacy), who is suffering from an unspecified cancer, and who was one of the biggest supporters of the ‘protocol’… until the placebo effect wore off, and she realised it is all a fraud. But valuable time spent on a miserable, painful treatment that bore no benefit, and might well accelerated J.’s disease progression.

There are, as we speak, thousands, if not millions, of Jillian Epperlys, peddling their fraudulent wares to an uninformed public. When the chips are down and conventional treatment options have been exhausted, patients will always turn to alternatives. With Right to Try, they could do so under medical supervision, adequately counseled and with their side effects managed. Moreover, the medications administered would have to adhere to standards of manufacture (GMP) and have a well-understood mechanism of action in most cases. There will always be desperate patients – and a well-designed Right to Try policy may keep them away from quacks and within the traditional medical system that would cater better for their needs and handle the transition from trying salvage/last-ditch treatments to palliative care and ensuring adequate end-of-life care.

Another undesirable aspect is the existence of an informal right to try. Darrow et al. describe the case of Josh Hardy, a 7-year-old boy who received the experimental antiviral drug brincidofovir after the media drew sufficient attention to his case for the manufacturer to ‘add’ Josh to an open-label study.6 Similarly, public sympathy for the aid workers from Samaritan’s Purse, including Kent Brantly, allowed for the use of the chimeric monoclonal antibody ZMapp. From the perspective of health equity, it is concerning that this informal procedure is amenable only to those with the means and connections to launch a massive social media campaign. In this sense, it is eerily reminiscent of the case of Sarah Murnaghan, whose lung transplant ineligibility was supervened by a large public campaign. It is fair to question whether the effects of a discretionary scheme that ultimately favours those with social, political and economic influence would not be better supplanted by a formal, equitable system available to all on equal terms.

The light and the dark

I don’t normally discuss end-of-life policy or bioethics: my days in that field are long gone, and my priority now is to try to avert those situations. However, to me, Right to Try will always be more than an abstract issue. A few years ago, a last-ditch therapy ended up working so well, it saved my life and put me into remission. After failing two different treatment regimens, we were out of conventional options, and things looked bleak – until a dedicated consultant oncologist took on the drug manufacturer, the hospital board and even the government, so as to be allowed to administer a drug still not approved for the particular indication. It was a huge gamble, and it worked. I will forever be grateful for the chance I’ve been given – but I’m also aware that I was the exception, not the rule, and n=1 doth not a good rule make.

I believe that even if the current version of Right to Try is, as Orac says, a ‘cruel sham’, it does not inherently have to be so.

There is enormous potential in Right to Try policies, not only for patients but also for drug development and future patients. Well implemented, it does not have to be a cruel sham. Nor does it necessarily have to be a wholesale ouster of the FDA’s competence.

But if it is to be anything other than that, it has to come with a comprehensive institutional structure that ensures that consent is truly free and adequate. Crucially, an independent physician must be available to honestly explain the odds and assess the patient’s understanding and capacity.7 The process must focus on balancing respect for patient autonomy against a degree of paternalism needed to protect a vulnerable patient. And in the end, it is paramount to have a sensitive understanding of the potential pressures the patient is under. It is not an easy task. But it is not an impossible one.

Many states now speak of ‘death with dignity’ as a euphemism for physician-assisted suicide. Perhaps to some people, that indeed is dignity, and it is a choice that deserves consideration. It is not cowardice or refusal to fight. But what about patients whose concept of dignity would closer encompass ‘staying in the fight’? Whether it is right or wrong, the practice of physician assisted suicide has shown that true consent can be separated from impaired consent in such a difficult scenario. Why, then, would it be impossible to separate instances where the Right to Try would merely engender false hope from those where it might have a small but not unrealistic clinical chance to succeed?

In the end, one needs to be able to separate the present rules from the principle. The present rules, and much of the motivation behind it, are clearly imperfect. But the potential behind Right to Try is significant. Regulated Right to Try can curb quackery and unregulated charlatans preying on the incurably ill by providing more legitimate last-ditch treatments carried out under medical supervision. It can accelerate research without prejudicing patient welfare if the pharmaceutical manufacturer is kept at arm’s length. And maybe, just maybe, it can save lives.

The current legislative framework might not be there yet. But it has the potential to make a difference not just to research but for millions of patients who have exhausted all possibilities, who, like me, might strike gold. Just as the history of science is one of incremental development, procedures and practices should be given the chance to develop over time.

References   [ + ]

1.Right to Try Model Legislation, sec.1(2)(a).
2.Ibid., sec.2(1).
3.Downing, N.S. et al. Regulatory review of novel therapeutics – comparison of three regulatory agencies. N Engl J Med 366:2284-2293.
4.Freireich, E.J. et al. The role of investigational therapy in management of patients with advanced metastatic malignancy. J Clin Oncol 27:304-306.
5.Woloshin S, Schwartz LM, Welch HG. Patients and medical statistics: interest, confidence, and ability. J Gen Intern Med 20:996-1000.
6.Darrow, J.J. et al. Practical, Legal and Ethical Issues in Expanded Access to Investigational Drugs. N Engl J Med 372:279-286.
7.In all honesty, I am not entirely sure that all too many patients in that emotionally and physically difficult situation are lucid enough to comprehend the entirety of what is involved in such a decision!

Five years

Five years ago, the love of my life boarded a plane in DC and landed in London. She didn’t know what she was getting into, and I guess there were times that were beyond what she bargained for. Later that year, we got civilly married. She stood by me as I was recovering from a life threatening illness, she helped me find normalcy, and gave me the five best years of my life. Happy sort of anniversary, Katie. I love you to MACS0647-JD and back.

Taken on Jan 20, 2018 @ 00:00 near Budapest, Hungary, this photo was originally posted on my Instagram. You can see the original on Instagram by clicking here.