Some Thoughts on Using Randomized Controlled Trials in International Development

I recently came across a post on the Council for Foreign Relations’ Development Channel that asked: Are Randomized Controlled Trials a Good Way to Evaluate Development Projects?

This is an incredibly important question because, as the authors note, “International donors have spent well over $2 trillion in development assistance over the past five decades, but there remains significant uncertainty about what works and doesn’t work to reduce poverty and grow economies.” Aid accountability has become an increasingly salient prerogative of many donors, yet showing that aid gets where it is supposed to and achieves desired goals continues to be a challenge because we often propose complex interventions whose outcomes may not be linear (for instance, providing school books or desks should not be equated with learning, but it is much more easily quantifiable) and may be longer-term rather than short-term outcomes.

The randomized controlled trial is a mainstay of biomedical research because it allows us to compare the effects of an intervention – a drug, a therapy, a process of care – in one group (the intervention group) and compare it with the results of another population who receives either nothing (often in the form of a placebo), or receives the usual treatment, which we assume to be inferior to the new intervention. By controlling for a number of variables, we try our best to make sure that the two populations are comparable (equally matched numbers of men and women, in the same age range, with similar burdens of disease, etc.) and that they are representative of the population in which we would like to use this tested intervention in, assuming the results show promise.

Issues of methodological and interventional complexity aside, it’s important that we consider the social contract between researchers and research subjects that allows us to conduct this research in order to advance biomedical science. While the underlying principle of the RCT is to advance medical science, we can’t do so without respect for our patients and our research subjects. That is, we can’t perform experiments on people without respecting their basic human rights, including things like their right to autonomy and their right to not participate in research if they don’t want to.

Taking it one step further, we compare our new interventions against the gold standard in order to show an improvement. If no treatment exists, then providing a placebo may be considered acceptable as a control arm; however, when an effective treatment exists, denying patients that treatment in order to provide a placebo to compare your intervention against, would not. For example, if you want to demonstrate a new formulation of insulin is effective, you would have to compare your new intervention against existing types of insulin. You couldn’t deny patients something that they need or that we know is good for them just to show that your new formulation or treatment protocol works better than nothing.

The reason behind experimentation and the use of randomized controlled trials is that we are answering a question about the intervention, and we are unsure whether the intervention is better than existing treatments, or doing nothing. In short, there is a general uncertainty as to whether a treatment will be beneficial. This is known as clinical equipoise in research ethics.

In international development, we are often uncertain of the models for appropriately scaling up development initiatives, but it would be unfair to say that we are uncertain of the many of the interventions. For instance, we know that funding education or immunizations are important and successful interventions for children or that ensuring access to clean drinking water reduces water-borne illnesses. These are not questions of science, they are questions of political will that allow the absence of these interventions to persist.

What is often unclear is not whether interventions, per se, are effective in a global context, but rather what models best allow them to be scaled up and run sustainably. Testing models of implementation in a randomized controlled trial is notably different from testing the effectiveness of the intervention, itself. So, when we know that the intervention is effective, can we justifiably randomize a population to a model that would result in it being unlikely that they would receive it? I don’t believe that we could.

If we do determine that randomized controlled trials are the way that we should proceed in international development, then I argue that the same safeguards need to be in place for conducting these trials with new medical innovations: they need to be approved by a competent research ethics board, there needs to be an oversight committee capable of determining throughout the study whether the intervention or the control is disproportionately placing participants at risk (and if it is, they need to have the authority to stop the trial/study/intervention), and there needs to be an ethical obligation to publish negative findings. If we are going to apply rigorous scientific methods to international development, we need to apply all of these methods and not only the ones that suit us and our information needs best.

And of course, it’s worth noting that not everything that we know about effective interventions has been proved through a randomized controlled trial. I suspect that the authors of this cheeky paper in the British Medical Journal are still waiting for participants for their randomized controlled trial on parachute use to prevent death and major trauma related to gravitational challenge

Gordon C S Smith, & Jill P Pell (2003). Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials BMJ, 327 DOI: 10.1136/bmj.327.7429.1459

Posted in Global Health, International Development, Population Health, Science | Tagged , , , , , , , , , | Comments Off

Broadening Our Knowledge of Respiratory Therapy and Respiratory Care

While most of my life is currently spent working in global health and population health, I originally trained as a respiratory therapist and have continued to maintain a clinical practice throughout my graduate school years. I have worked in adult critical care, anesthesia, emergency medicine – the list goes on. Recently, I was appointed as the new Editor-in-Chief of the Canadian Journal of Respiratory Therapy (CJRT), a small Canadian journal focusing on the practice and the science of respiratory therapy.

As the Editor, my goal is to transform the journal into an increasingly useful and widely-disseminated journal that focuses on the implementation of best practices for respiratory therapy and respiratory care in Canada and around the world. This means that we want to be publishing high-quality research that tells us how effective interventions can be implemented in major Canadian academic health centres, as well as in village health clinics in places like Malawi. We are equally as interested in publishing reports that investigate community-based treatment of pneumonia in Lusaka as we are in reviews of packages of services to reduce ventilator-associated pneumonia in Toronto. We are working to ensure that the journal maintains a strong online presence and to solidify its status as an open-access journal, so that the results of valuable research continues to reach those who can benefit from it most.

To that end, we are actively seeking submissions for upcoming issues of the journal, so I thought I would post some of the topics we are interested in on here. This is by no means an exhaustive list, but rather just a few thoughts on areas that we are interested in developing:

  • Community-based respiratory therapy/respiratory care for common respiratory illnesses (in adults and children)
  • Respiratory Therapist (RT)-led  clinics for asthma, COPD, or other chronic diseases
  • Models of care for community-based respiratory therapy: home care, chronic disease management, etc.
  • RTs working in community-based interprofessional care teams
  • Respiratory Therapists working with hard-to-reach populations
  • The role of respiratory therapists in urban health centres, including homeless, refugee, and immigrant populations (among others)
  • Considerations for respiratory therapists working with immigrant and refugee populations in Canada
  • I am particularly interested in a respiratory therapy-focused review of the Canadian Guidelines for Immigrant Health and this would be a very good subject for a review for a first-time author with an interest and expertise in the field
  • Organization and delivery of respiratory therapy in rural or remote communities in Canada, including RTs involved in telehealth, medevac, or critical care outreach services, as well as rural respiratory therapy, in general
  • New clinical leadership roles for RTs in acute care settings
  • Evaluations of new RT-led care pathways (ventilator weaning, trach management, acute asthma exacerbations, non-invasive ventilation, etc.)
  • Evaluations of the role of RTs in interprofessional care teams
  • Role of RTs and Anesthesia Assistants in peri-operative care and patient management
  • RTs as cardiac arrest team leaders
  • Evaluations of new devices, modes of ventilation, or adjuncts to respiratory therapy
  • Case studies of unique or challenging patients encountered in your practice

We are interested in structured evaluations, as well as interesting commentaries or reviews of the above topics and more. Writing and publishing a review of an interesting or controversial clinical treatment or role is a great introduction to research and publication, and we are happy to support first-time authors. Back issues of the CJRT are currently being re-developed and will be posted in a more friendly manner shortly. Our instructions to authors have recently been updated and are available online here:

Posted in Global Health, Medical Publishing, Pneumonia, Population Health, Respiratory Therapy, Surgery | Tagged , , , , , , , , , , | Comments Off

Waiting to Act: The Sahel Food Crisis is Already A Humanitarian Emergency

On July 20, 2011 the United Nations announced that in parts of Somalia, food security had deteriorated so significantly that two regions were now considered to be in famine, and more than 3.7 million people were in desperate need of assistance. This was the first major famine to be declared since 2000, and became the most prominent evidence of the rapidly deteriorating humanitarian situation in the Horn of Africa at the time.

To many in the humanitarian aid community, this was confirmation of what had been witnessed on the ground for some time. Most major aid organizations had been operating in this region for years and had described the worsening situation for months. In fact, a whole system dedicated to monitoring food accessibility – the Famine Early Warning Systems Network (FEWS Net) – had sounded warnings of famine based on a number of indicators in the region for over a year.

Fast forward to August 2012, and the cruel cycle is poised to repeat itself again. FEWS Net reports that about 16 million people in Djibouti, Ethiopia, Sudan, South Sudan, Kenya, and Uganda are facing “Stressed (IPC Phase 2) to Emergency (IPC Phase 4) levels of food insecurity,” due to conflict, poor rains, high food prices, and poor humanitarian access in some regions (such as in the Darfur, South Kordofan, and Blue Nile states of Sudan, where humanitarian access is so poor that some agencies – such as Médecins Sans Frontières – have been forced to leave altogether).

In the Sahel Region – an impoverished area of West Africa consisting of Senegal, Mauritania, Burkina Faso, Mali, Niger, and Chad – an estimated 18 million people are affected by a growing food crisis, again the result of the interaction of horrific circumstances of drought, conflict, high food prices, and extreme poverty. A rapid nutrition assessment conducted in Niger by World Vision in early 2012 found rates of Global Acute Malnutrition (GAM) averaging 14.4%, with a high of 17.1% in one region. GAM is an important, though late, indicator of malnutrition and is one of the criteria used in declaring a humanitarian emergency, when the prevalence of GAM in a population reaches or exceeds 15%. If the World Vision assessment is representative of the rest of the region, the Sahel food crisis is dangerously close to tipping into a massive regional humanitarian emergency. Many would argue that breaching the emergency threshold would only be epidemiologic confirmation of what is already happening.

And so, for another summer, the world is left watching and waiting for an already deplorable situation to breach an emergency threshold, triggering a global – though likely muted – humanitarian response. We saw this last year in the Horn of Africa, where the world waited and sought to manage the crisis, rather than the risks. The writing was on the wall, there, too – In Somalia, for instance, diesel prices  increased 45%, while local cereal prices rose 240% in some regions of the country and 100% in Ethiopia over the year leading up to the declaration of famine. This year, food prices in West Africa appear to be relatively stable, though are generally much higher than their seasonal averages, suggesting that high prices are here to stay, a factor influenced by rising fuel prices and pest infestations. Add to this the ongoing conflict in northern Mali, where hundreds of thousands of people have been displaced, and poor rains that have resulted in poor crop yields, and a broader picture of the looming emergency begins to reveal itself.

And yet, the world appears poised once again to sit and wait for the situation to deteriorate significantly before we launch major humanitarian interventions. There are several important factors to note in this, the first being that building awareness is critical for building support for the relief efforts. In the Horn of Africa food crisis last year, donations and media coverage only picked up significantly after the United Nations declared the area to be in a state of famine, at which point the emergency had already been unfolding for months.

A declaration of famine may galvanize support, but waiting for a famine to launch a massive intervention across several countries hardly seems practical nor ethical. A declaration of famine is based on more than just food or malnutrition indicators; There must be evidence of three conditions in a given area, specifically: food shortages with a limited ability to cope, greater than 30% prevalence of global acute malnutrition, and crude mortality rates (deaths) equal to or greater than 2 deaths per 10,000 people per day. In essence, the declaration of famine means that people have died and will continue to die without major humanitarian intervention.

The world is faced with an important opportunity: Intervene now and avert famine in the Sahel region. Acting now must first address the immediate threats to life for the population, then focus on coordinated, sustained, long-term development beginning with agricultural and livestock support in order to benefit from future rains and to maximize the growing season and lead toward self-sufficiency. While western assistance should be welcomed, African countries should be playing the leading role in averting famine in the region and contributing to the development of local and regional knowledge and practices to encourage enhanced food security. Changes to agricultural sectors are desperately needed in order to move toward climate-resilient farming practices and away from export-dominated crops, in favour of crops for local food production. There are options for agricultural reform, and several lessons-learned that should be shared. In the immediate sense, however, what is needed are emergency interventions to ensure that the ongoing crisis in the Sahel region doesn’t tip from an emergency into a famine. And that’s a call to action that all countries should be listening to.

Further reading: A Dangerous Delay, a joint paper between Save the Children and Oxfam.

Posted in Disaster, Global Health, International Development, Population Health | Tagged , , , , , , , | Comments Off

Cuts to the Interim Federal Health Program for Refugees in Canada: Unfair, Mean-Spirited, and Economically Dubious

On June 30, bill C-31, the Immigration and Refugee Protection Act, comes in to effect, drastically changing the access that refugees in Canada have to health care. The short name is a misleading one: rather than enhancing the protections that refugees have in this country, the bill dismantles them for many claimants, reducing their access to essential health services.

With this bill comes sweeping changes to the way Canada treats refugees seeking asylum in this country, the most notable changes being the elimination of something called the Interim Federal Health Program (IFHP), a program that provides temporary health insurance to refugees and their dependents who are not yet covered under a provincial health care plan. Effectively, the program ensured that refugees arriving in Canada were not without access to healthcare, which seems the compassionate, ethical, and all-around right thing to do. It ensured that hospitals and health care professionals had a way of being compensated for providing medically-necessary treatment to refugees and also reduced the barriers to accessing the care that new arrivals to our country needed.

The IFHP provided access to things like primary and secondary care, as well as supplementary care like vision care, dentistry, mobility aids, and pharmaceuticals. However, our government has determined that the system is inherently unfair – not to refugees, but to Canadians who already live here. How is it unfair that a refugee arriving from Somalia be given a dental exam and access to insulin, you ask? The short answer is that Canada’s health care system relies on a mix of public and private insurance schemes, where some health services are covered by provincial health plans, while others are either paid for out-of-pocket, or by a private insurer. Supplementary care – vision care, dentistry, mobility aids, and pharmaceuticals, for example – are not routinely covered by a provincial health care plan for most Canadians, but rather are covered by private insurance, often provided as an employee benefit.

So, the removal of the IFHP for refugees sought to address this inequality. Not by reforming the system to increase access to health services for all Canadians, but rather to remove them from a vulnerable group because, according to Canadian Citizenship, Immigration and Multiculturalism Minister Jason Kenney “…we do not want to ask Canadians to pay for benefits for protected persons and refugee claimants that are more generous than what they are entitled to themselves.”

That narrative has prevailed through virtually all of the government’s messages on the subject, and has served as justification for denying a vulnerable population access to things like insulin and other life-saving medications: it’s unfair to everyone else to be providing health services for these people because other Canadians don’t have access to them through a public system. That’s quite an equalizer.

The point is further driven home by a document released by Citizenship and Immigration Canada, where they provide examples of coverage after the changes to IFHP take effect. To be frank, it’s chilling. Under a heading of “Medical Emergency – Heart Attack” they provide the following scenario:

A woman has chest pain and goes to the nearest hospital emergency room where a doctor finds she has suffered a heart attack. Urgent surgery is done to install a stent in an artery. She recovers well and leaves with a prescription for medication.

Sounds good if you’re a Canadian or a protected person or a refugee from a country that isn’t one of the Designated Countries of Origin (DCO). But if you’re a Roma from Hungary, and are therefore a claimant from a DCO, what services are you entitled to in the above scenario? “None.” How about this scenario:

A doctor confirms that a patient is pregnant. She gets prenatal health services for the pregnancy. There are no complications, she delivers the baby in a hospital and then has post-partum follow-up. Prenatal, delivery and post-partum health services are considered essential services.

Refugee claimant from a DCO? No services unless you pay. Even in any of the scenarios listed on the site, consultation may be provided, but the treatment (medications) are not. Expecting a refugee claimant to be able to pay for essential medicines upon their arrival is wholly unrealistic and flies in the face of compassion. More detailed discussions on the problems with the DCO designation and other concerns with the bill (including the impact it will have on children) are available elsewhere, and are really worth a read including this backgrounder on DCOs from Citizenship and Immigration whose opening line is “Too many tax dollars are spent on asylum claimants who are not in need of protection.”

The response to these cuts has been a vociferous outcry from health professionals who provide care to refugees and immigrants. In a country known for its multiculturalism and humanitarian ethos (whether the latter is factually accurate or not seems debatable), there is a considerable community of health care professionals who have worked diligently to provide high-quality care to these communities, and who have publicly opposed these cuts. This opposition has included protesting against Ministers who have supported these cuts, including emergency room physician Chris Keefer at the Toronto General Hospital who interrupted Minister Joe Oliver announcing:

“Doctors in this country will not remain silent in light of the Conservative government’s cuts to the refugee health program…Refugees who are coming from war-torn countries, fleeing hatred, fleeing crimes against humanity, and your government is about to cut the very essential medicines, the very essential services, that these people require in order to continue living.”

The video is available here, and is really worth watching.

Paradoxically, health services are generally provided in refugee and IDP camps by NGOs or by local governments, free of charge. This in no way should mean that the services are comprehensive or even remotely comparable to what is available in Canada, but the point is this: a refugee leaving a camp in Sudan to come to Canada has access to free health care in Sudan, but not in Canada. Does this not seem backward? When a country accepts refugees, this act is grounded in humanitarianism – people escaping horrific conditions and who cannot return should be afforded asylum, and this should include access to basic health services. Recognizing that there is a need for status as a refugee recognizes the need for special protection, the unraveling of which undermines the entire system.

The second argument advanced for the removal of the IFHP is economic: it’s too expensive, costing $84.6 million in 2010-2011. While it may be true that $84.6 million is a large sum of money, it is also true that removing preventative care only shifts the burden of disease further down the pipeline. It is far cheaper to provide insulin to a diabetic than it is to treat ketoacidosis in an emergency room, or diabetic retinopathy. It is far cheaper to treat hypertension in a primary care setting (including drugs) than it is to perform open-heart surgery or treat an aneurysm. The list goes on, and anyone with a modicum of understanding of public health could tell you that the economics of prevention are sound: preventing or treating disease early on is far cheaper than dealing with its consequences.

Yet, this is exactly what removing the IFHP will do: the burden will be shifted downstream to the provinces, either as an acute emergency or as a long-term consequence of untreated disease once refugees become part of a provincial health care scheme. The economic argument is a poor one.

Finally, the argument of abuse comes in to play – people who abuse the immigration system, make bogus refugee claims, or some other assortment of claims to validate the “necessary” reforms to the immigration system. I have worked with refugees and immigrants in Canada and abroad. I have worked in multiple internally displaced persons camps, in slums, and in areas affected by conflict. These are miserable places, and to suggest that people escaping them are somehow abusing Canadian generosity or taking advantage of us is disingenuous, at best, and certainly cold and callous. As UNHCR has so poignantly stated: No one wants to be a refugee.

Fundamentally, this move is mean-spirited and devoid of any rational economic argument. It’s penny-wise and pound foolish. They are irrational and inhumane cuts to vital health services that will without doubt have drastic negative effects on the health of some of our country’s most vulnerable people. As a healthcare professional, I do not and can not support these cuts, and I am not alone.

Posted in Uncategorized | 1 Comment

Mapping Health Facilities in Crises – Reflections and Directions

I am wrapping up my thesis research (finally!), which focuses on the mapping of health facilities in crisis settings. Over the past 3 years, I have spent countless hours looking at datasets, GPS coordinates, and long lists of indicators of the functionality of health facilities. I have been looking to see what effective tools are out there for conducting health facilities censuses, mapping of health facilities, understanding their capacity to deliver health services, and translating what’s going on on the ground into meaningful datasets. In essence – figuring out who is where, doing what, and what capacity they have to do it.

Simple questions, right? Not so fast.

What we have discovered is that these data are notoriously scarce, particularly in crisis settings. Really, what we have found is that usable, accurate data about health facilities like hospitals or foreign medical teams (field hospitals) are hard to come by.

Let’s look at Haiti, where I spent some time in November and December of 2011 looking at health facilities mapping. In the immediate aftermath of the earthquake in 2010, several lists of health facilities emerged: from the Sahana Foundation, to Travax, to an MSPP/PAHO list of health facilities, to OpenStreetMap. All of these initiatives provided something that was sorely missing in Haiti: a list of the functioning health facilities in Port-au-Prince and around the country. Most of them even mapped it, sometimes down to the number of beds that were available. This information was regrettably scarce, and many organizations put considerable time and effort into collecting it and making it available online. Kudos.

The expertise that I have been developing these past few years is very much a health systems in humanitarian crises one, and when I am on the ground I rely on my knowledge as a clinician and as a population health researcher/practitioner to make sense of what I see around me. When I walk in to a hospital or clinic, there are certain things that I want to see or processes that need to be happening. It’s more than data, it’s dynamic interactions. Interestingly, most of the reviews of the health facilities data in Haiti that I have seen have very much focused on the ICT-aspects, rather than the collection or use of the data.

Rather than coming from an ICT perspective, I look at health facilities mapping as being comprised of three discrete, but connected, components:

  1. A data collection tool, including the relevant indicators that will be collected about each health facility, at each health facility
  2. A platform for collecting and analyzing reports from each health facility, whether self-submitted or collected on-site by surveyors
  3. Data visualization and presentation platforms, including maps (like Ushahidi) and humanitarian dashboards, summarizing relevant findings and displaying them in useful ways

The great successes in mapping health facilities in Haiti were, in my opinion, in component #3 – a platform for displaying and communicating important information through unique ICT platforms. There were a number of people who were working effectively to provide very usable data interfaces that could have shared very useful information. However, the merging of datasets and conducting analysis (component #2) were, I think, weak in large part due to the absence of robust and useful data (component #1). Does this amount to a failure of the system? Certainly not, but it highlighted an area that needs to be developed (enter my thesis research).

One of the critical challenges that I have seen with regard to health facilities data is that you can collect innumerable pieces of information about a facility and can quickly become overloaded with data that are not terribly useful. Furthermore, if the data don’t allow for meaningful analysis of key indicators and benchmarks (several of which exist), then we’re not meeting our objectives, and we’re also not likely influencing decision-making at the coordination level.

This raises an important question: who are the data being collected for? If the answer is for NGOs providing short-term relief, then their needs are likely to be much more basic than what the Ministry of Health or the Health Cluster needs for coordination. Short-term relief operations require an assessment of needs and available resources so as to reduce duplication and improve equitable access to essential health services. Similarly, patients requiring care need to know where functional health facilities are so they know where to go for the kind of services they need. These are more straightforward.

From the perspective of health systems strengthening and humanitarian coordination, however, the need for complexity begins to increase and more detailed indicators become necessary. Again, though, there is a balance between being overwhelmed with data and having enough information to make informed decisions. Different time segments of crisis response demand different levels of data complexity –  As part of the initial response, information needs are far simpler – where are the existing health facilities? What capacity do they have? Are they still functioning? What services do they provide? This should be a coordinated part of information management in crises. But, the data need to evolve into more complex data sets and pieces of analysis to allow for detailed inventories of facilities, human resources, and services. Some guidance on frameworks exists, but we need to delve into the issue in greater detail. Furthermore, we need to think about how to bridge in existing databases of health facilities into a crisis response model, because health facilities master lists do exist in some places. We are working on developing this kind of health information system and framework as part of my research and hope to be able to pilot it in the near future, particularly in countries with varying levels of information needs, arising from protracted emergencies.

All of this ties in to the need for there to be structure and clarity to the data being collected. The efforts in Haiti were excellent – but most were ad-hoc and the data collected lacked structure with regard to health services information management, and often didn’t make sense to many of us with health systems expertise. That doesn’t mean that it was bad, it means that there was not enough multilateral information exchange about what information would be most useful to those running the show. We are still developing this science, but we are expanding the understanding and evidence base for health facilities mapping in crisis settings and intend to have some solid recommendations available shortly.

Fundamentally, we need to bridge the gaps between expertise in health systems, public health, health services delivery, and ICTs in crisis settings. The Haiti experience has been a solid proof-of-concept, in my mind, that if we build collaboration amongst us, we can provide useful, timely, and accurate information on health facilities even in austere settings, through the use of novel technologies (like SMS and web-based platforms). But, we need to ensure that these data are in usable forms, using standard measurement indicators, and collected, analyzed, and displayed in a way that makes sense to those who need them.

My time spent working in this area has allowed me to have fantastic conversations with brilliant minds about these problems – this is precisely what being a scholar/practitioner is all about. I look forward to continuing these conversations and practice.

Posted in Disaster, Epidemiology, Global Health, Humanitarian, International Development, Population Health | Tagged , , , , , , , , , , , , , , | Comments Off