Clinical Trial Report Card

Clinical trials results are harder to understand than they could and should be because they are messy.

There are no report cards for oncology clinical trials. Sponsors can pick whatever order they want to use for most of their outcomes. Molecular testing and surveys can each add dozens of outcomes that are not directly related to key treatment responses. With www.clinicaltrials.gov percent serious adverse events are always in the same place. However, the rest of the outcomes can differ widely in terms of placement and terminology.

                This is not how patients, loved ones or oncologists need to look at potential clinical trials. There are four key questions about what a treatment is likely to do to and for a patient. Those key questions are “What are the odds that this will 1) kill me 2) cure me 3) keep me alive longer and 4) keep me in the hospital longer?” They are answered by the toxic death rate, complete response rate, median survival (and range) and percent serious adverse events respectively.

                I used this kind of report card to help translate some of the clinical trial offerings my mom received so that she could make informed decision. I remember her reaction when she suddenly realized that a proposed trial was more likely to kill her than cure her (i.e., toxic death rate > complete response rate).

                Most treatments do not even carry a significant risk of toxic death, but it is still pretty rare to see it reported even when it would be a zero. Sometimes researchers even play a trick to hide those deaths by combining grade 4 and grade 5 toxicities. These should not be combined. To quote The Princess Bride “there’s a big difference between mostly dead and all dead.[1]” A grade 4 toxicity can almost kill you, but a grade 5 toxicity will kill you.

                Median survival and the range answer the question what are the odds that this will keep me alive longer. Sometimes it will not be possible for a clinical trial sponsor to report the maximal survival time. That is good news because it means at least one of the patients was still alive when the trial ended. It is not good news when all of the patients die within the two to five-year time frame of a clinical trial. However, the percent still alive could and should be a reported statistic. There are sponsors who report what percentage of the patients were alive at the last follow-up time.  All other things being equal the treatment with 40% still alive at 2 years looks much better than the ones with 4% or 0.4%.

                However, anytime a median is provided, the minimum is known. That minimum also provides information about the survival and its range. If the minimum is a day, then that would point towards a toxic death as odds are against the patient getting killed in a car wreck on the way home from the clinic. Having a report card style report can have the advantage of not just highlighting the need to report median survival but to also use the same units. Survival should be reported in months but sometimes is reported in weeks. This has the effect of making some treatments seem more effective than they deserve.

                Percent serious events is the outcome that answers the question “What are the odds that this will keep me in the hospital longer? Just knowing how many serious events occurred does not answer this question if that number is not zero. As an example, two Phase I clinical trials with ten patients each could have five adverse events each. However, if in the first trial one very sick patient had all five serious events while in the second trial five patients each have a single serious adverse event, they would have the very different percent serious adverse events. The first trial would suggest a 1 in 10 chance of a serious adverse event while the second one would suggest a 1 in 2 chance of a serious adverse event. All other things being equal most people would probably choose the first trial.                 But report cards are rarely used in oncology. This is unfortunate because as the following tables show it can be hard to find the key outcomes without using a report card style. Both tables show the results from the same trial. The first table shows only the key results. The second table is how they are presented on www.clinicaltrials.gov. The results are still there they are just buried among the other results.

Table 1. NCT00556712 report card  

Key Outcome Placebo Erlotinib
Toxic Death Rate (%) NG NG‡
Complete Response Rate (%) 0.7% 0.9%
Overall Survival in Months (median and range) 11.0 months 12.4 months
Percent Serious Adverse Events (%) 7.6% 11.3%

NG = Not Given. ‡ one case of sudden death was reported as a serious adverse event. However, sudden deaths are associated with both the condition (i.e., lung cancer) and the treatment (i.e., erlotinib). A meta-analysis reported a 0.8% toxic death rate for erlotinib. Source: Ding et.al., 2016

Table 2. Original NCT00556712 clinical trial report

ID Outcome description Placebo Erlotinib
1 Percentage of Participants With PD According to Response Evaluation Criteria in Solid Tumors (RECIST) or Death 89.5 %  79.9 %
2 PFS in All Participants 11.1 weeks 12.3 weeks
3 Probable Percentage of Participants Remaining Alive and Free of Disease Progression at 6 Months  15.0% 25.0%
4 Percentage of Epidermal Growth Factor Receptor (EGFR) Immunohistochemistry (IHC) Positive Participants With PD or Death  89.4%  79.5%
5 PFS in EGFR IHC Positive Population 11.1 weeks 12.3 weeks
6 Probable Percentage of Participants in the EGFR IHC Positive Population Remaining Alive and Progression Free at 6 Months  16.0% 27.0%
7 Percentage of All Participants Who Died 87.1% 82.0%
8 Overall Survival (OS) in All Participants 11.0 months 12.4 months
9 Probable Percentage of Participants Remaining Alive at 1 Year 45.0% 50.0%
10 Percentage of EGFR IHC Positive Participants Who Died 87.5% 80.5%
11 OS in EGFR IHC Positive Population 11.0 months 12.8 months
12 Probable Percentage of Participants in the EGFR IHC Positive Population Remaining Alive at 1 Year 47.0% 52.0%
13 Percentage of EGFR IHC Negative Participants With PD or Death 89.8% 77.4%
14 PFS in EGFR IHC Negative Participants 9.0 weeks 11.0 weeks
15 Probable Percentage of Participants in the EGFR IHC Negative Population Remaining Alive and Free of Disease Progression at 6 Months 11.0% 22.0%
16 Percentage of EGFR IHC Negative Participants Who Died 50.8% 41.9%
17 OS in EGFR IHC Negative Participants 10.2 months 8.6 months
18 Probable Percentage of Participants in the EGFR IHC Negative Population Remaining Alive at 1 Year 20.0% 42.0%
19 Time to Progression 11.3 weeks 12.3 weeks
20 Probable Percentage of Participants Remaining Progression-Free in the TTP Analysis at 6 Months 15.0% 26.0%
21 Percentage of Participants With a Best Overall Response (BOR) of Confirmed Complete Response (CR) or Partial Response (PR) According to RECIST 5.4% 11.9%
22A Percentage of Participants With a CR, PR, Stable Disease (SD), or PD According to RECIST (CR) 0.7% 0.9%
22B Percentage of Participants With a CR, PR, Stable Disease (SD), or PD According to RECIST (PR) 4.7% 11.8%
22C Percentage of Participants With a CR, PR, Stable Disease (SD), or PD According to RECIST (SD) 45.4% 48.6%
22D Percentage of Participants With a CR, PR, Stable Disease (SD), or PD According to RECIST (PD) 47.6% 35.6%
23 Percentage of Participants With a Response Upgrade From BL According to RECIST 1.3% 5.5%
24A Percentage of Participants With a Change of PR to CR or SD to PR or CR From BL to End of Treatment According to RECIST (PR to CR) 0.4% 0.5%
24B Percentage of Participants With a Change of PR to CR or SD to PR or CR From BL to End of Treatment According to RECIST (SD to PR) 0.9% 4.8%
24C Percentage of Participants With a Change of PR to CR or SD to PR or CR From BL to End of Treatment According to RECIST (SD to CR) 0.0% 0.2%
25A Percentage of Participants With CR, PR, or SD or With SD [Maintained For Greater Than (>) 12 Weeks] or CR or PR (CR+PR+CD) 50.8% 60.6%
25B Percentage of Participants With CR, PR, or SD or With SD [Maintained For Greater Than (>) 12 Weeks] or CR or PR (CR+PR+CD)>12 Weeks 27.4% 40.8%
26 Percentage of Participants With Symptom Progression Assessed Using the Lung Cancer Subscale (LCS) 44.1% 47.7%
27 Time to Symptom Progression 17.6 weeks 18.3 weeks
28 Probable Percentage of Participants Remaining Without Symptom Progression at 6 Months 35.0% 41.0%
29 Percentage of Participants With Deterioration Assessed Using the Trial Outcome Index 43.1% 50.9%
30 Time to Deterioration in TOI 18.9 weeks 18.1 weeks
31 Probable Percentage of Participants Remaining Without Deterioration in TOI at 6 Months 41.0% 39.0%
32 Percentage of Participants With Deterioration in Quality of Life Assessed Using TOI, SWB, and EWB 51.7% 55.3%
33 Time to Deterioration in QoL 12.3 weeks 12.6 weeks
34 Probable Percentage of Participants Remaining Without Deterioration in QoL at 6 Months 34.0% 32.0%
35DM* Functional Assessment of Chronic Illness Therapy – Lung (FACT-L) Scores (Off Trtmt:Total FACT-L Score)  92.14 (19.324)   91.89 (18.738) 
36DC† Change From BL in FACT-L Scores  -6.26 (15.146)  -7.10 (16.194)
SAE Serious Adverse Events 7.6% 11.3%

*other 116 rows left out for sake of brevity.

†other 106 rows left out for sake of brevity.