The Institute for Clinical and Economic Review (ICER) recently assessed two treatments for spinal muscular atrophy (SMA), a rare and deadly genetic condition impacting 1 in 11,000 newborns each year.[1] In the absence of therapy intervention, death or the need for constant ventilation to breathe before the age of two years is the outcome for more than 90% of individuals diagnosed with SMA Type 1. SMA is the number one genetic cause of death for infants.
ICER’s recently-released final report was disappointing in several ways, including the lack of sufficient data, disregard for outcomes that matter to patients, lack of understanding and oversimplification of the condition, and the inherit weaknesses of the quality-adjusted-life-year (QALY) measure.
This is particularly concerning given the real-world consequences we are observing in the U.K. and other countries, which have utilized QALYs to limit patient access. As one mother of a child with SMA living in the U.K. shared: “It’s an abomination of human rights that the drug isn’t available… I've seen children in America who had it when they were first diagnosed and they're walking. To be put in a position where you have to consider moving abroad to keep your son alive is dreadful.”[2] With such clear consequences for children and their families, it is alarming to see ICER ignore patients’ voices and produce such a flawed and short-sighted assessment of these important, breakthrough therapies.
Lack of Sufficient Data for SMA Treatments Does Not Prevent ICER from Passing Judgement
There is limited evidence on the relative effectiveness or long-term impact of treatments newly-approved or not yet approved by the U.S. Food and Drug Administration (FDA). In the case of treatments for SMA, the lack of data due to timing of the review is magnified by the challenges of assessing a treatment for a rare disease, where there is already an extremely limited population to inform data collection.
Because the assessment was conducted with insufficient evidence, the model is heavily laden with assumptions regarding longer-term outcomes, like the assumption that repeated lumbar punctures will become problematic for patients, despite the fact they were tolerated during the trials, thereby considerably limiting the validity and acceptability of any results. While all models have limitations, ICER’s results are based on these assumptions, and payers who rely on ICER’s assessments to make coverage decisions are therefore making the same assumptions. ICER’s findings hinge on the accuracy of their assumptions, a significant limitation that is not clearly conveyed when providing their results.
The framing of outcomes for pre-symptomatic patients treated with Spinraza is also seriously flawed. Trial data demonstrate that most infants treated proactively, when they are pre-symptomatic, achieve major milestones of walking and standing. To date, no pre-symptomatic infant with SMA treated with Spinraza has died or required a permanent ventilator. The ICER study minimizes this significant positive gain for SMA patients by comparing these results against the development of healthy children versus pre-existing studies of individuals with SMA. ICER seems to have made all of the assumptions they can to give insurers an excuse to deny or limit access to this breakthrough therapy.
ICER Does Not Incorporate Outcomes that Matter to SMA Patients
In the SMA review, ICER chooses to include and exclude certain elements of value based on whether they consider them to be important to a health system. ICER determined that caregiver burden is not within the “scope” of their assessment of treatments for children who have SMA, although it is clearly within the scope of concern for patients and their families. In their letter in response to the ICER review, CureSMA highlighted that ICER assigns benefit to the patient only if the drug allows for obtaining specific milestones, like sitting or walking. This is in direct contradiction to reports from patients and the FDA’s recognition of the great value of functional abilities that allow for more independence and the activities of daily living, even if they do not render a patient ambulatory.
ICER made no attempt to calculate the costs and quality of life impacts associated with caregivers (parents, guardians). This is in spite of the comments submitted on ICER’s draft evidence report from advocates and families that specifically called for the inclusion of these factors. One parent of a child with SMA repeatedly emphasized the need to take time off of work for doctor’s appointments and to support her child at school functions. She also noted that the ICER study neglected to consider the significance of even small amounts of increased strength for the quality of life for patients and caregivers, saying “until you live with SMA, you do not realize how every little bit of strength maintained or gained helps improve quality of life.”[3] Lack of data certainly wasn’t a barrier. In addition to the information submitted to ICER by parents of children with SMA and patient advocates, there is a large body of evidence quantifying the impact of caregiving on individuals’ health, career, and economic stability. For example, a recent study found intensive caregivers were 13 percentage points less likely to be employed, 6 percentage points more likely to take unpaid time off work, and 12 percentage points more likely to quit their job. [4]
A Thorough Understanding of SMA Is Not Displayed
ICER indicated that there is a severe degree of uncertainty due to a limited sample size, which “raises concerns about generalizability of results to the wider population of patients with SMA.” As CureSMA noted in their comment letter, this displays a clear lack of understanding of SMA. The disease is genetically homogeneous, so the mechanism for action of both Sprinraza and Zolgensma is the same across the disease spectrum.
As part of the calculation of the QALY in the SMA assessment, ICER categorizes outcomes into three possible health states: (1) motor skills achieved (only sitting and walking considered), (2) need for permanent ventilation, and (3) time to death. For a disease as complex and severe as SMA, having only three health states does not reflect the spectrum and differences of outcomes patients and their families experience. CureSMA highlights this as a major concern in their comment letter, pointing out that “the process of determining QALYs for patients permanently on ventilators and those who are non-sitters appears arbitrary.” They go on to point on that this does not account for the huge range of patient outcomes that exist between these two states. The use of this oversimplified metric means important data on the scope and variance of patient are not captured.
Additionally, generic patient-reported outcome (PRO) tools, which ICER uses to determine the quality-of-life utilities for SMA health, suffer from similar shortcomings and are not an appropriate method to capture patient experience. As one study states, “Generic tools may not adequately capture QOL changes in SMA, especially given the age group affected by the disease.” [5] The Muscular Dystrophy Association highlights that the ICER report is lacking SMA-specific patient preference in their letter, saying: “Improvements in mobility, however small they may be deemed, often represent major improvements in quality of life and the value of these gains cannot be discounted.”
The QALY Forces Patients and People With Disabilities to Choose Between Life-Extension and Functional Improvements – A Cruel Trap for Those Who Need Both
QALYs presume that people with disabilities value their lives less than those of people without disabilities. The measure depends on a “disability weight” system designed to calculate the exact percentage to “adjust” the value of a year in the life of a disabled person relative to a hypothetical non-disabled person. This does not match the reality of how most people with disabilities perceive their lives. Individuals with disabilities, especially those disabled from birth (like those with SMA), tend to see their quality of life as equal to that of the general population. People with disabilities believe that interventions designed to extend their lives should be given the same or greater priority as those designed to extend the lives of non-disabled people. [6][7]
Unfortunately, a QALY-based system forces people with disabilities into an unfair trap. Because QALYs use disability weights both to assess the value of a treatment for extending life and for improving function (both needs that people with SMA have), these two important needs are placed at cross-purposes. If ICER and other value assessment entities that use the QALY rely on relatively low disability weights, and thus value extending the life of a person with SMA almost as much as extending the life of a non-disabled person, then the QALY will under-value improvements in functional skills, like sitting or walking, that treatments like Spinraza offer people with SMA. If value assessment entities use high disability weights, thus giving greater value to improvements in functional skills, then the vital life-extending properties of Spinraza and other SMA breakthrough therapies are undervalued, since the QALY will de-prioritize extending the lives of people with disabilities relative to a hypothetical non-disabled person.
This is a cruel and shortsighted way to assess value, borne out of the QALY’s simplistic and unnecessary combination of both life-extension and functional improvements into a single arbitrary measure. Instead, value assessment entities should rely on metrics designed specifically to quantify the priorities that matter most to the patient or disability population impacted by a particular drug or other medical intervention. As long as ICER relies on the QALY, a discriminatory metric, people with SMA and many other conditions will be undervalued and underserved.
Nowhere is this clearer than in reading the comment letters submitted to ICER by parents of children with SMA. One parent, who has three children with SMA, shared that her third child, who began Spinraza at 11 days old, is developing on target for her age and currently needs no medical intervention. She ends her letter saying all of her children, including her 7-year-old who cannot breathe or swallow independently, are thriving due to advancements in SMA research.[8] We cannot accept an approach to health care that would turn back the clock on these children’s chances to thrive.
Conclusion
As long as ICER is conducting reports with flawed conclusions that inherently harm patients when referenced by decision-makers, we will do our best to raise awareness and instead urge innovative researchers to work with us to develop novel ways to measure the value of medicines and other health care interventions in ways that improve care for patients and people with disabilities.
[1] CureSMA. About SMA. Available at: http://www.curesma.org/sma/about-sma/
[2]http://healthmedicinet.com/i/parents-urge-officials-to-approve-drugs-to-treat-the-condition-crippling-their-five-year-old-son/
[3]https://icer-review.org/wp-content/uploads/2018/07/ICER_SMA_Public_Comments_022219.pdf
[4] Jacobs JC, et al. Economic Spillover Effects of Intensive Unpaid Caregiving. PharmacoEconomics, 2019.
[5] Vaidya S. Measuring quality of life in children with spinal muscular atrophy: a systematic literature review. Quality of Life Research. 2018 Dec 1:1-8.
[6] Brock DW. Preventing genetically transmitted disabilities while respecting persons with disabilities. Quality of life and human difference: Genetic testing, health care, and disability. 2005 May 9:67-100.
[7] Williams A. Intergenerational equity: an exploration of the ‘fair innings’ argument. Health economics. 1997 Mar;6(2):117-32.
[8] https://icer-review.org/wp-content/uploads/2018/07/ICER_SMA_Public_Comments_022219.pdf
This is particularly concerning given the real-world consequences we are observing in the U.K. and other countries, which have utilized QALYs to limit patient access. As one mother of a child with SMA living in the U.K. shared: “It’s an abomination of human rights that the drug isn’t available… I've seen children in America who had it when they were first diagnosed and they're walking. To be put in a position where you have to consider moving abroad to keep your son alive is dreadful.”[2] With such clear consequences for children and their families, it is alarming to see ICER ignore patients’ voices and produce such a flawed and short-sighted assessment of these important, breakthrough therapies.
Lack of Sufficient Data for SMA Treatments Does Not Prevent ICER from Passing Judgement
There is limited evidence on the relative effectiveness or long-term impact of treatments newly-approved or not yet approved by the U.S. Food and Drug Administration (FDA). In the case of treatments for SMA, the lack of data due to timing of the review is magnified by the challenges of assessing a treatment for a rare disease, where there is already an extremely limited population to inform data collection.
Because the assessment was conducted with insufficient evidence, the model is heavily laden with assumptions regarding longer-term outcomes, like the assumption that repeated lumbar punctures will become problematic for patients, despite the fact they were tolerated during the trials, thereby considerably limiting the validity and acceptability of any results. While all models have limitations, ICER’s results are based on these assumptions, and payers who rely on ICER’s assessments to make coverage decisions are therefore making the same assumptions. ICER’s findings hinge on the accuracy of their assumptions, a significant limitation that is not clearly conveyed when providing their results.
The framing of outcomes for pre-symptomatic patients treated with Spinraza is also seriously flawed. Trial data demonstrate that most infants treated proactively, when they are pre-symptomatic, achieve major milestones of walking and standing. To date, no pre-symptomatic infant with SMA treated with Spinraza has died or required a permanent ventilator. The ICER study minimizes this significant positive gain for SMA patients by comparing these results against the development of healthy children versus pre-existing studies of individuals with SMA. ICER seems to have made all of the assumptions they can to give insurers an excuse to deny or limit access to this breakthrough therapy.
ICER Does Not Incorporate Outcomes that Matter to SMA Patients
In the SMA review, ICER chooses to include and exclude certain elements of value based on whether they consider them to be important to a health system. ICER determined that caregiver burden is not within the “scope” of their assessment of treatments for children who have SMA, although it is clearly within the scope of concern for patients and their families. In their letter in response to the ICER review, CureSMA highlighted that ICER assigns benefit to the patient only if the drug allows for obtaining specific milestones, like sitting or walking. This is in direct contradiction to reports from patients and the FDA’s recognition of the great value of functional abilities that allow for more independence and the activities of daily living, even if they do not render a patient ambulatory.
ICER made no attempt to calculate the costs and quality of life impacts associated with caregivers (parents, guardians). This is in spite of the comments submitted on ICER’s draft evidence report from advocates and families that specifically called for the inclusion of these factors. One parent of a child with SMA repeatedly emphasized the need to take time off of work for doctor’s appointments and to support her child at school functions. She also noted that the ICER study neglected to consider the significance of even small amounts of increased strength for the quality of life for patients and caregivers, saying “until you live with SMA, you do not realize how every little bit of strength maintained or gained helps improve quality of life.”[3] Lack of data certainly wasn’t a barrier. In addition to the information submitted to ICER by parents of children with SMA and patient advocates, there is a large body of evidence quantifying the impact of caregiving on individuals’ health, career, and economic stability. For example, a recent study found intensive caregivers were 13 percentage points less likely to be employed, 6 percentage points more likely to take unpaid time off work, and 12 percentage points more likely to quit their job. [4]
A Thorough Understanding of SMA Is Not Displayed
ICER indicated that there is a severe degree of uncertainty due to a limited sample size, which “raises concerns about generalizability of results to the wider population of patients with SMA.” As CureSMA noted in their comment letter, this displays a clear lack of understanding of SMA. The disease is genetically homogeneous, so the mechanism for action of both Sprinraza and Zolgensma is the same across the disease spectrum.
As part of the calculation of the QALY in the SMA assessment, ICER categorizes outcomes into three possible health states: (1) motor skills achieved (only sitting and walking considered), (2) need for permanent ventilation, and (3) time to death. For a disease as complex and severe as SMA, having only three health states does not reflect the spectrum and differences of outcomes patients and their families experience. CureSMA highlights this as a major concern in their comment letter, pointing out that “the process of determining QALYs for patients permanently on ventilators and those who are non-sitters appears arbitrary.” They go on to point on that this does not account for the huge range of patient outcomes that exist between these two states. The use of this oversimplified metric means important data on the scope and variance of patient are not captured.
Additionally, generic patient-reported outcome (PRO) tools, which ICER uses to determine the quality-of-life utilities for SMA health, suffer from similar shortcomings and are not an appropriate method to capture patient experience. As one study states, “Generic tools may not adequately capture QOL changes in SMA, especially given the age group affected by the disease.” [5] The Muscular Dystrophy Association highlights that the ICER report is lacking SMA-specific patient preference in their letter, saying: “Improvements in mobility, however small they may be deemed, often represent major improvements in quality of life and the value of these gains cannot be discounted.”
The QALY Forces Patients and People With Disabilities to Choose Between Life-Extension and Functional Improvements – A Cruel Trap for Those Who Need Both
QALYs presume that people with disabilities value their lives less than those of people without disabilities. The measure depends on a “disability weight” system designed to calculate the exact percentage to “adjust” the value of a year in the life of a disabled person relative to a hypothetical non-disabled person. This does not match the reality of how most people with disabilities perceive their lives. Individuals with disabilities, especially those disabled from birth (like those with SMA), tend to see their quality of life as equal to that of the general population. People with disabilities believe that interventions designed to extend their lives should be given the same or greater priority as those designed to extend the lives of non-disabled people. [6][7]
Unfortunately, a QALY-based system forces people with disabilities into an unfair trap. Because QALYs use disability weights both to assess the value of a treatment for extending life and for improving function (both needs that people with SMA have), these two important needs are placed at cross-purposes. If ICER and other value assessment entities that use the QALY rely on relatively low disability weights, and thus value extending the life of a person with SMA almost as much as extending the life of a non-disabled person, then the QALY will under-value improvements in functional skills, like sitting or walking, that treatments like Spinraza offer people with SMA. If value assessment entities use high disability weights, thus giving greater value to improvements in functional skills, then the vital life-extending properties of Spinraza and other SMA breakthrough therapies are undervalued, since the QALY will de-prioritize extending the lives of people with disabilities relative to a hypothetical non-disabled person.
This is a cruel and shortsighted way to assess value, borne out of the QALY’s simplistic and unnecessary combination of both life-extension and functional improvements into a single arbitrary measure. Instead, value assessment entities should rely on metrics designed specifically to quantify the priorities that matter most to the patient or disability population impacted by a particular drug or other medical intervention. As long as ICER relies on the QALY, a discriminatory metric, people with SMA and many other conditions will be undervalued and underserved.
Nowhere is this clearer than in reading the comment letters submitted to ICER by parents of children with SMA. One parent, who has three children with SMA, shared that her third child, who began Spinraza at 11 days old, is developing on target for her age and currently needs no medical intervention. She ends her letter saying all of her children, including her 7-year-old who cannot breathe or swallow independently, are thriving due to advancements in SMA research.[8] We cannot accept an approach to health care that would turn back the clock on these children’s chances to thrive.
Conclusion
As long as ICER is conducting reports with flawed conclusions that inherently harm patients when referenced by decision-makers, we will do our best to raise awareness and instead urge innovative researchers to work with us to develop novel ways to measure the value of medicines and other health care interventions in ways that improve care for patients and people with disabilities.
[1] CureSMA. About SMA. Available at: http://www.curesma.org/sma/about-sma/
[2]http://healthmedicinet.com/i/parents-urge-officials-to-approve-drugs-to-treat-the-condition-crippling-their-five-year-old-son/
[3]https://icer-review.org/wp-content/uploads/2018/07/ICER_SMA_Public_Comments_022219.pdf
[4] Jacobs JC, et al. Economic Spillover Effects of Intensive Unpaid Caregiving. PharmacoEconomics, 2019.
[5] Vaidya S. Measuring quality of life in children with spinal muscular atrophy: a systematic literature review. Quality of Life Research. 2018 Dec 1:1-8.
[6] Brock DW. Preventing genetically transmitted disabilities while respecting persons with disabilities. Quality of life and human difference: Genetic testing, health care, and disability. 2005 May 9:67-100.
[7] Williams A. Intergenerational equity: an exploration of the ‘fair innings’ argument. Health economics. 1997 Mar;6(2):117-32.
[8] https://icer-review.org/wp-content/uploads/2018/07/ICER_SMA_Public_Comments_022219.pdf
| the_data_mine_icers_sma_study_falls_short_on_data_accuracy.pdf |
