Benefits and Risks of Insulin Pumps and Closed-Loop Delivery Systems
A case study in reverse causality and moral hazards
In the first article in this series, “Why Controlling Glucose is so Tricky,” I sympathize with those who look forward to fully automated closed-loop insulin pumps for the modest reason of “living a normal life.” Managing T1D is a difficult and mentally taxing disease. It’d be great to eat without worrying about what’s going to happen next.
I understand–I really love glazed donuts. But alas, I stay away.
Technically, such a system is a highly ambitious goal because, as the article explains, the metabolic system is so complex, even the natural human body struggles keeping glucose levels low and stable.
For context, even among those who get full pancreatic islet transplants, where they are able to have fully-functional beta-cell capacity to create their own insulin, these people still have an exceedingly difficult time maintaining glycemic control. According to the 2023 literature review article from the NIH titled, “Pancreatic islet transplantation in type 1 diabetes: Current state and future perspectives,” only 73.7% of islet transplant patients were able to achieve A1c levels <7% after the first year. And this is about as close to a “cure” as you can get.
To expect an external insulin pump–which can only deliver insulin–to come close to human performance may remain wishful thinking.
Nevertheless, these systems are in a state of rapid development, and there is a near universal perception that today’s technology helps T1Ds optimize glucose management pretty well.
But epidemiological studies over time show that T1Ds as a population are no healthier than they were ten years ago. The T1D Exchange reports that, even though pump use increased from 57% in 2010–2012 to 63% in 2016–2018, “the average A1C overall has risen from 7.8% to 8.4%.”
And yet, other reports show that pump users are able to achieve A1c levels better than these averages. How can this be?
As this article shows through analysis of peer-reviewed studies published in medical literature, the illusion of improved pump performance is a case of reverse causality, where healthier people are using automated pumps more, and these same study subjects perform the same when using MDI (multiple daily injections).
Other studies show that pump effectiveness is neutral, at best, and at worst, pumps may actually impose an artificial upper limit on how well one could manage glucose versus other management techniques.
Relatedly, it may also be that closed-loop systems present a moral hazard, where faith and trust in such systems can actually lead to behaviors that result in worse outcomes for some.
Advocates argue that, effectiveness notwithstanding, their convenience alleviates the daily burden of T1D, which makes them worthwhile for that reason alone.
But again, it’s possible this may be more of a self-induced placebo effect: Studies of larger populations show that pumps themselves do not relieve stress, but there are, nevertheless, many psychological factors at play.
The aim of this article is to tease out these details and get a more nuanced understanding of when, and under what circumstances, today’s pumps and closed-loop insulin systems may be beneficial, when they’re not, and why such perceptions persist.
Today’s Automated Delivery Systems (AID)
Closed loop systems today rely on continuous glucose monitors (CGMs) for glucose readings, which software algorithms use to calculate how much insulin to deliver through an insulin pump. In theory, one would expect that these algorithms improve, but there are other hurdles to consider. CGMs have their own issues, such as accuracy being a serious concern. They can also fall out of calibration, giving insulin pumps faulty data for calculating insulin dosing.
Similarly, pumps themselves can also fail mechanically, often without the patient even knowing it, along with other technical issues. More problematic is that, even under ideal conditions, pumps only deliver insulin, which is only one of many hormones the natural body needs for full glycemic control. Furthermore, insulin is delivered through interstitial tissue, which can delay absorption for varying and unpredictable periods of time, which the dosing algorithms cannot detect.
In the end, determining how much insulin the pump should administer is often erroneous, leading to unhealthy glycemic levels. Overcoming all these challenges has been the focus of manufacturers, but technical barriers are much higher than people think (which I cover in the next article). This means that today’s pumps–automated or not–are about as good as we’ll see for quite a while.
This doesn’t mean that today’s pumps are ineffective–indeed, there are happy users. Data presented by manufacturers, such as those presented at the October, 2023 convention of the European Association for the Study of Diabetes, suggests that these systems work quite well. The following figure was produced by Medtronic for their MiniMed 780G system, which included data from over 100,000 users, showing reduced incidents of hypoglycemia and hyperglycemia, while also improving time-in-range (TIR) to the sought-after threshold of >70%.
Other manufacturers showed similar results, but the combined data also hints at an unappreciated nuance: The greatest beneficiaries are people below their mid-20s, and those who are either unable or unwilling to manage T1D on their own (elderly, cognitive impairment, etc.). For these subgroups, fully- or semi-automated pumps helped them manage glucose levels better than they were able to do on their own. While it’s great that such patients were kept alive–an accomplishment to celebrate–their A1c’s exceeded 8-9%, far outside of “healthy” guidelines.
Note that, the highest A1c level one can have while also reducing the risk of long term complications is 7%, and studies show that only 23% of all T1Ds are able to achieve that level. (See the article, “HbA1c Tests and T1D: The Good, The Bad and the Ugly.”) Even pump manufacturers do not show any statistically significant number of people achieving <7%.
That said, there are those who achieved A1c levels between 7-8% with pumps, and this is because these users manually input data for food and exercise, as well as other manual adjustments, just as if they were manually managing their glucose levels as if they were on multiple daily injections (MDI). (This is called a hybrid closed-loop system.)
In other words, pump efficacy is directly associated with the degree in which the user personally engages with the system. And it may be that it’s that very engagement that is producing better outcomes, not the pump itself. To what degree the pump is actually providing benefit, we look to randomized control trials (RTCs) that are subject to peer review and published in medical literature.
An example is the paper, “Glycemic Outcomes in Adults With T1D Are Impacted More by Continuous Glucose Monitoring Than by Insulin Delivery Method: 3 Years of Follow-Up From the COMISAIR Study.” The authors of this study showed that insulin pumps – both automated and not – have little effect on overall glycemic control than MDI. But here’s the catch–the better outcomes were restricted to those who wore CGMs. Over the past five years, the rate of CGM option has risen precipitously, leading to more and more studies confirming that CGM use is the strongest contributor of improved glycemic control over any other technology or method of insulin intake.
Pump advocates point to other published studies that show pump users outperforming non-pump users, but one must read these papers closely: Most of those studies included non-pump users who didn’t use CGMs. This wasn’t done out of bad faith–it’s more because these studies were done at a time when most CGM users were also pump users, and it was not yet clear that it was the CGM doing the real work, not the pump.
A more robust study that teases out these differences CGM use, pumps and MDI can be found in this 2018 study by the T1D Exchange, where researchers engaged over 21,000 subjects who used a combination of these. The results are differentiated, along with separations for age groups, as shown in the following figure:
The researchers wrote that “there isn’t a substantial difference between A1c outcomes for those who manually take insulin injections compared to those who use pumps” when the users are over the age of 26 and use CGMs. In fact, CGM+MDI users slightly outperformed CGM+pump users.
Once again, there’s nuance to appreciate: As with many other studies, the T1D Exchange study showed that CGM+pump use for people under 26 years old benefitted more than MDI. And this is important because this age group is less capable of managing their own diabetes very well on their own: They lack years of experience living with T1D, their bodies are undergoing rapid changes, and they lack the maturity to understand a difficult disease. Yes, automated systems did better for them than they could do on their own, but their A1c levels were still dangerously high. Teaching them how to manage themselves is the real goal.
A case of reverse causality
Perhaps the most important detail to consider about pump users is demographics. Given the high expense of pumps and CGMs (and the education and medical infrastructure to deliver them), pump users tend to be one or more of the following: They’re well-insured, come from higher socio-economic households, have greater access to medical care, have higher income levels, and/or are typically more educated. Perhaps most importantly, they come from families that promote healthy lifestyles. The more any of those conditions apply, the more likely that person is to be engaged and informed about managing T1D, and as we know, more informed and engaged users yield better health outcomes. The following paper, “Racial, ethnic, socioeconomic disparities in insulin pump use have persisted over 20 years,” breaks down this data accordingly:
“insulin pump use was 67% among non-Hispanic whites, 41% among Hispanics, 29% among Blacks, and 46% among other racial and ethnic groups. In addition, 70% of people with bachelor’s degrees or higher used the pumps, compared with 56% among those with some college, 40% among holders of high school degrees, and 18% among those with no high school education. By income level, 74% of those with household incomes of $75,000 or more, 66% with $50,000 to 74,999, 51% with $25,000 to $49,999, and 41% with less than $25,000, used the pumps.
Many attribute this as being the inequality of our healthcare system, giving white, rich, and educated people better access to technologies like automated insulin pumps. But it may actually be a reverse causality, where most pump users are already primed for being healthier because of those other “privilege” factors. It’s not the pump doing the work, but the people themselves.
Another way to test that is by measuring A1c levels before and after introducing people to pumps and measuring their outcomes afterwards (while including subjects from diverse social, racial and economic subpopulations). This metaanalysis (literature review) of dozens of random controlled trials (RTCs) had these aims in mind, and also included studies involving T1Ds who exercise. The authors concluded that pump users were only able to achieve a 1.07% improvement over control groups who didn’t use an an insulin pumps at all (and did use CGMs), irrespective of their background.
One can get even more granular still: In a 2021 study titled, “Predictors of the effectiveness of insulin pumps in patients with type 1 diabetes mellitus,” the authors conclude that the subjects whose A1c levels were >8% going into the study saw an improvement of 0.9 ± 1.2% when started on insulin pumps, whereas those whose A1c levels were <8% saw no benefit to pumps at all, irrespective of age, race, and income levels. It should be noted that a 1% improvement may well be statistically significant, but not clinically meaningful because detrimental health effects begin with A1c levels over 7%.
All this brings us to those who claim that, even if pumps offer no clinical benefit, the lifestyle benefits make them worthwhile. Don’t take this the wrong way, but are you really having a better lifestyle with pumps? Fortunately, there are RCTs that study that too.
The moral hazards of convenience
An interesting paper is “The DIAMOND Randomized Trial,” which initially aimed to assess the relative difference in costs associated with pumps versus MDI therapy. The authors found that pump users’ lifetime costs are $112,045 greater than those on MDI.
But researchers unexpectedly found that pump users’ quality-of-life factors decreased by 0.71 compared to MDI, and (even more unexpectedly) life expectancy dropped by 0.48 years (based on higher risk of DKA, among other risk factors that rose). Minimal differences from MDI to be sure, but these numbers certainly didn’t improve, as pump advocates would like to believe.
The reduction of efficacy and higher costs between pumps and MDI are two primary reasons why insurance companies are reluctant to cover insulin pumps.
Similar conclusions are found in multiple studies compiled in the paper, “Systematic literature review: quality of life associated with insulin pump use in type 1 diabetes.” By assessing a much larger set of trials, the authors concluded that there was little change in quality-of-life outcomes for most users due to a subtle psychological bias: Pump users really wanted them to work better, and in so doing, they believed they were. This makes “actual” satisfaction hard to assess, which brings us to the realm of psychology, especially around technology.
Satisfaction of high tech devices, even when they don’t deliver on their promises, is a phenomenon known as the Christmas Tree Effect, where users’ attraction to (and satisfaction of) new technology is compounded by the illusion that it is better than the older model/method because of new features (“bells and whistles”). A Christmas tree is more fun if you just add more lights. And make them blink. Even better if they blink to Taylor Swift music.
Companies amplify this emotion in their marketing campaigns; the Dexcom G7 is an example: Dexcom promotes it as being more ‘accurate’ than the G6, and early adopters praised it for its vastly improved physical design, user experience and its new smartphone app. But the data itself, which is (and should be) the most important factor, produced worse outcomes for users on automated pumps and for those who manage to attain tight control on MDI. (See this article, “Continuous Glucose Monitors: Does Better Accuracy Mean Better Glycemic Control?”)
The Christmas Tree effect leads to another moral hazard: People overlook the drawbacks when technology fails, which can be rather serious for T1Ds. For example, diabetic ketoacidosis (DKA), a condition that occurs when the body doesn’t receive enough insulin, sometimes causing death, is more strongly associated with insulin pumps than those on MDI. In the BMJ paper, “Incidence and prevalence of diabetic ketoacidosis (DKA) among adults with type 1 diabetes mellitus (T1D): a systematic literature review,” the authors aimed to summarize incidence and prevalence of T1Ds who were admitted to hospitals around the world for DKA, and sort them into different subgroups (age, sex, geographical region, ethnicity and type of insulin administration). The paper is excessively long, where reviewers started with 1082 separate articles in order to characterize each of the criteria. Among the many citations, a clinic in Colorado stood out, where all the patients admitted with DKA used pumps–none were on MDI. Other citations are listed below:
In the paper, “Insulin pump therapy is associated with higher rates of mild diabetic ketoacidosis compared to injection therapy: A 2-year Swedish national survey of children and adolescents with type 1 diabetes,” the authors found that 85% of admissions were pump users vs. MDI.
In the paper, “Severe Hypoglycemia and Diabetic Ketoacidosis in Adults With Type 1 Diabetes: Results From the T1D Exchange Clinic Registry,” researchers performed a cross-sectional analysis from the T1D Exchange clinic registry at 70 U.S. endocrinology centers, and found that 2972 incidents of DKA were associated with pump use, versus 2034 for those on MDI (roughly 47% greater risk for pump users).
In the paper, “Causes of diabetic ketoacidosis among adults with type 1 diabetes mellitus: insulin pump users and non-users,” researchers conducted a prospective observational study between January and June 2019 at the Cleveland Clinic Fairview Hospital, and found that, among patients admitted with DKA, 55% of the cases were pump users.
The list of similar findings is extensive.
Many people assume that DKA from insulin pumps is largely due to device malfunction, but this isn’t always the case. A more prevalent reason for DKA is user error. In the paper, “Risk and Relevance of Insulin Pump Therapy in the Aetiology of Ketoacidosis in People with Type 1 Diabetes,” the authors found that “an overwhelming majority were those on insulin pump treatment (93%)”, and added, “Overall, patient errors caused more DKA cases than device malfunctions.”
Imagine that: user error on a device that should be either automated or semi-automated.
Authors of another paper were more blunt. In “DKA Prevention and Insulin Pumps: Lessons Learned From a Large Pediatric Pump Practice,” the authors stated that “most events could have been avoided if users followed standard troubleshooting guidelines.”
And yet, T1Ds are willing to overlook this due to the Christmas Tree Effect. And that brings us to the ultimate moral hazard: automation.
The perils of automation
Anything that disengages the user from a task also takes attention away from the system as a whole, while also depriving them of learning the nuances of how the system works and why. When the stakes are low, automation can very well be convenient: a robotic vacuum cleaner, a dishwasher, or the devices that control your thermostat. If something goes wrong, very little harm is done, and you can then go back and read the manual.
But when the stakes are high, such as automated cars or insulin pumps, the results can be catastrophic. Automation robs people of the experience and expertise necessary to manage the same task, which are essential when the consequences are high.
This sentiment is similarly expressed by the authors in a 2022 review of closed-loop systems in the Diabetes Journal article, “Diabetes Technology: Standards of Medical Care in Diabetes—2022,” where the authors wrote:
“The most important component in all of these systems is the patient. [...] Simply having a device or application does not change outcomes unless the human being engages with it to create positive health benefits. [...] Expectations must be tempered by reality—we do not yet have technology that completely eliminates the self-care tasks necessary for treating diabetes.”
When it comes to engagement, this is one of the primary advantages of MDI. The physical act of injecting insulin makes one keenly aware of what the patient is doing. This is an example of The Hawthorne Effect, a phenomenon where people modify their behavior (usually positively) when they know they’re being observed. Observing yourself is the base case of the Hawthorne Effect.
In a study titled, “The Importance of the Hawthorne Effect on Psychological Outcomes Unveiled in a Randomized Controlled Trial of Diabetes Technology,” study subjects were exposed to a series of tasks associated with T1D management. The authors found that T1Ds who exhibited fear of hypoglycemia (FOH) saw a significant reduction in FOH, simply because they were being closely watched, even though there was no intervention whatsoever in how they managed their disease. They simply did what they knew they should do on their own. (All the other studies cited in this article do not have subjects “closely watched.”)
Similar outcomes have been shown for T1Ds who are told to log carbohydrates, exercise, and yes, manually take insulin. When you have to stop and think about calculating insulin and carbs, the act of what you’re about to do becomes more present. You proactively think to yourself, “Do I log 40g or 60g for this glazed donut that I know I shouldn’t be eating?” Believe it or not, the physical act of logging those carbs and taking that insulin causes many to be more engaged and deliberative in their T1D choices. Many people–including me–often choose not to eat that glazed donut. Sigh.
Summary and Commentary
I have lived with T1D for over fifty years, so I can attest to the fact that the disease is grippingly difficult to live with, let alone self-manage. Any technology that can ease the burden–even in the slightest–is a blessing. And this is why I struggle with the concept of automated insulin pumps: They are essential for those who are either unable or unwilling to manage their own care beyond a certain threshold, especially the young or those with cognitive impairment. But for those who are able to take care of themselves, one must remain as engaged as possible with the nuances of daily T1D management. One of the risks of automation is disengagement–failing to learn self-management.
The state of closed-loop technology today is, sadly, as good as it can possibly get, unless and until there are better methods of sensing glucose and delivering insulin in real time (not through interstitial tissue), while also delivering other essential regulatory hormones that are also part of the process.
Without solving the technical barriers that pumps face, today’s closed-loop systems are like a self-driving car equipped with only a hazy rear-view camera (the CGM), a gas pedal (insulin) and a faulty fuel line that may or may not deliver the gas to the engine in a timely manner. No forward view, no side view, no brakes and no reliable navigation system. In automatic mode, there’s not even a driver. The most powerful AI-driven algorithms that try to calculate proper insulin dosing cannot possibly compensate for these mechanical dysfunctions. At best, the car will go on a very straight road at 5mph, and get pretty banged up along the way. You know, like having an A1c of >9%.
I certainly don’t have any problem with people using these systems, so long as they are acutely aware of what they are doing, and are not under the illusion that it’s the pump making things better. It’s you. Of course, pump manufacturers are eagerly trying to make these systems better, but the hurdles they face are monumental. I’ll cover that in my next article.
Once again, you've written an extremely thought-provoking article. Thank you for putting this together!
I've had T1D for 45 years. I use Control IQ and I (1) love it and (2) don't disagree with your conclusions in the least. I'm meticulous about bolusing for food and high blood sugars. Without that effort, the pump wouldn't be effective at all.
Why do I love it? For me, it's COMPLETELY eliminated overnight lows since the day I switched to CIQ. I used to run high at night out of fear of those overnight lows. No longer! I wake up every morning at 110 mg/dl. This is the one area where closed loop tech is (understandably) excellent. When a diabetic is asleep, the pump can simply release basal insulin, adding more or giving less as needed. Without meals or exercise or stress to worry about when the patient is asleep, it's pretty hard for CIQ to mess up. That alone is worth it for me. But nothing I'm saying contradicts your conclusions.
Thanks again. I look forward to reading each of your posts.
Proofreading Police! 😉
“Technically, such a system a highly“
You’re missing a verb, perhaps “is”?
“Another way to test that is by measuring A1c levels before and after introducing people to pumps and measuring their outcomes afterwards (while including subjects from diverse social, racial and economic subpopulations). This metaanalysis (literature review) of dozens of RTCs had these aims in mind, and also included”
Please, sir, what are RTCs?
Policing over. Again, thank you for an interesting article. I always learn something from your posts. I'm a Metronic user and I got a new pump over a year ago. They gave me a box of sensors and a chance to try out this new 780G system. Well, I have a problem with rising blood sugar after I get out of bed. I like to eat right away, so first thing I do is check my numbers and enter carbs for breakfast. However, auto mode thought that my rising blood sugars were from eating, but I knew they weren't. So I had to exit auto mode, bolus in manual mode, and then go back to auto mode a few hours later for the rest of the day. I learned from a forum that I could "phantom bolus" to correct, but it just seemed a hassle. It was way easier to set my Basals where they needed to be. Plus, I would have to pay out of pocket for the sensors, when the Libre 2 was covered by the government.
I missed my old pump, because the new one was specifically designed to be used in auto mode and manual mode requires seemingly endless button pressing. Unfortunately, I forgot to take that old pump off when I went swimming in the sea. "Critical Error!" ;-) So, back to my "back up", newer pump.
My 2 months on the Medtronic "closed loop" system was very frustrating and my time in range numbers were worse. I can see where it would be advantageous for someone who was hypo unaware, especially at night, but it just wasn't working for me.
I look forward to your next article.