Validation and Reliability Testing of the EORTC QLQ-NMIBC24 Questionnaire Module to Assess Patient-reported Outcomes in Non–Muscle-invasive Bladder Cancer

2.1. Questionnaires

Patients completed the QLQ-C30 questionnaire and the QLQ-BLS24 module before treatment in a clinic and at regular intervals thereafter. In the high-risk groups, questionnaires were completed at time point 0 and at 2, 3, 6, and 12 mo. In the intermediate-risk group, assessments were completed at time point 0 (before randomisation) and at 12 mo ( Fig. 1 ). Missing data were imputed according to the EORTC guidelines, and questionnaires were considered as missing if >50% of the items were missing [4] . Using this approach, some missing items still could not be imputed, but the other data from these questionnaires were still used. Response rates (based on entirely missing questionnaires or unusable questionnaire) at each time point were examined and reasons for missing questionnaires documented. Response rates to the sexual items were calculated based on whether patients reported being at least “a little” sexually active (item 48).

Fig. 1 Trial schema with timing of assessments using the EORTC QLQ-C30 and QLQ-NMIBC24. BOXIT = Bladder COX-2 Inhibition Trial; NMIBC = non–muscle-invasive bladder cancer; PRO = patient-reported outcome.

The module was developed according to standard EORTC Quality of Life Group guidelines, and translations followed standard procedures [4] . The module has 24 items originally hypothesised to form multi-item scales assessing urinary symptoms (items 1–7), intravesical treatment issues (items 10 and 11), future perspective (items 12–14), fever and feeling ill (items 8 and 9), and abdominal bloating and flatulence (BAF) (items 15 and 16), along with single items addressing different aspects of sexual functioning (items 17–24). All responses are linearly transformed from 0 to 100, with a high score indicating more symptoms or problems or better function for the functional scales. Ethics committee approval and written informed consent were obtained. The sample was determined by the patients within the BOXIT study up until November 2012.

2.2. Defining the scales within the module

Multitrait scaling analyses with data from each of the time points examined whether the individual items may be grouped into the hypothesised scales. The items assessing sexual functioning included two items to be completed by all patients, two items for completion by men only, and one item for completion by women only; also, there were three items completed by men or women reporting to be sexually active in the past 4 wk. Given the conditional nature of many of these items, it was not possible to analyse them as one scale.

Statistical evidence of item convergent validity was defined as a correlation of ≥0.40 between an item and its own scale (corrected for overlap) [10] . Item discriminant validity was defined as a correlation of <0.40 between an item and other scales in the questionnaire. An item was considered to be a scaling success when the correlation between the item and its own scale was greater than its correlation with any other scale. For each scale, the ceiling and floor effects were examined. After finalising the scale structure, other tests were performed.

2.3. Evaluating the reliability and validity of the module

The internal consistency was assessed by the Cronbach α coefficient, with >0.70 considered acceptable for group comparisons being examined within each scale at each assessment point [11] .

Known group comparisons evaluated whether the module was able to discriminate between subgroups of patients differing in clinical status [11] . Known groups used for this comparison were baseline differences in QLQ-C30 physical function scores, with <90 or >90 representing relatively high (better) or relatively low (worse) scores, respectively. It was hypothesised that the scale scores of the QLQ-BLS24 would be higher (show more problems) in patients with lower physical function. Additional exploratory known groups validity testing was performed comparing data from men versus women. The independent student t test was used to examine differences in mean scores. Effect sizes were expressed as the mean difference divided by the pooled standard deviation (SD). Effect sizes were interpreted using the Cohen rule of thumb that a change of 0.5 SD represents a moderate effect, and a change >0.8 SD is a large effect [12] .

To assess validity, correlations between the scales of the QLQ-BLS24 module and the scales of the QLQ-C30 were made using baseline data. Polychoric correlations were calculated, as is appropriate for items with four response categories. The responsiveness of the module to changes in health over time was examined in high-risk patients who underwent intensive treatments. It was hypothesised that during treatment, patients would report increased urinary symptoms and decreased generic aspects of health. Pairwise comparisons of changes in mean scores from baseline to 2, 3, 6, and 12 mo were evaluated using t tests for correlated samples. Because multiple comparisons were performed, a cautious but uncorrected p value of <0.01 was considered to be statistically significant.

All analyses were performed using Stata/IC statistical software (release 12, 2009; StataCorp LP, College Station, TX, USA).

3.1. Patient characteristics, response rates, and missing data

At the time of data analyses, 472 patients were randomised, 433 patients consented to the quality-of-life study, and 410 of them completed a baseline questionnaire. Of these patients, 401 (97.8%) had complete baseline PRO data sets. The majority (79.3%) were men, and more than two-thirds had high-risk tumours (n = 306, 74.6%) ( Table 1 ). The number of questionnaires returned at each time point and completion rates were 282 (92.2%), 288 (94.1%), 263 (85.9%), and 298 (94.3%) at 2, 3, 6, and 12 mo, respectively, for the high-risk group; at 12 mo, 81 questionnaires (77.9%) were returned for the intermediate-risk group. There were therefore 1532 questionnaires in total, with a completion rate for the five assessment points of 88.2%. At baseline, 48% of patients reported at least a little sexual activity (item 48), meaning that completion rates for the sexual scales and items were generally good (>75%). Sociodemographic and clinical details and questionnaire response rates are shown in Table 1 .

3.2. Defining the scales in the module

Final results of the multitrait scaling analyses are shown in Table 2 . Item within scale correlations in the original hypothesised urinary symptom, fever and malaise, and sexual function scales were all ≥0.40, and therefore these scales were maintained. Items 40 and 41, addressing intravesical treatment issues, showed many scaling errors. Discussion within the trial management group therefore led to agreement to include item 40 as a single item assessing intravesical treatment issues. Item 41 correlated well with the future perspectives scale, and therefore this scale was expanded to a four-item future worries scale (items 41–44). The two items assessing abdominal BAF (items 45 and 46) demonstrated satisfactory scaling properties when combined and thus formed a scale. The scale concerning sexual problems in men, items 49 and 50, functioned well and was retained. The remaining items in the original sexual function scale about sexual intimacy (item 51), risk of contamination of partner (item 52), sexual enjoyment (item 53), and an item for sexual function in women only (item 54) remained as individual items.

At baseline, the revised scales showed some floor effects (as expected), because side-effects of treatment would be limited at that stage (scales for malaise, intravesical treatment >72% reported no problems at all). At all time points, few ceiling effects were noted (<2.5% for each scale; data not shown).

The original hypothesised scales in the EORTC QLQ-BLS24 and the confirmed scales in the EORTC QLQ-NMIBC24 are shown in Table 3 .

3.3. Reliability

The internal consistency of the scales at each time point were good (>0.70) for the urinary symptoms, future worries, sexual function, and sexual function in men scales. The fever and malaise scale had coefficients of >0.57 and 0.76; in the abdominal bloating scale, this ranged between 0.49 and 0.62 ( Table 2 ).

3.4. Clinical validity

Patients with high scores (>90) on the physical function scale of the QLQ-C30 reported significantly better functional scores and fewer symptoms on all QLQ-C30 and three module scales (urinary symptoms, malaise, and sexual function) and on a single item (sexual enjoyment) than patients with poorer physical functioning (p < 0.001, Table 4 ). The effect sizes for these differences were moderate to large. Most scales and items were similar between men and women, except that men reported significantly more problems with sexual function (p = 0.005) than women ( Table 4 ).

3.5. Criterion validity

The correlations between the majority of the scales in the core questionnaire and module (n = 44, 88%) were relatively low (r < 0.40, Table 5 ), indicating that the module is not overlapping in content with the QLQ-C30. Correlations >0.4 were observed between the malaise scale in the new module and role and social function scales, global quality of life, and the pain, fatigue, and nausea and vomiting scales in the QLQ-C30. The urinary symptoms scale was moderately associated with role (0.41) and social function (0.43) and the pain scales (0.44) in the QLQ-C30, and the future worries scale in the module showed a moderate association with the emotional function scale (0.50).

3.6. Responsiveness to changes over time

Table 6 shows change in scores before and after treatment. Although little increase in urinary symptoms was observed during the follow-up period, several aspects of health measured by both the QLQ-C30 and the module did deteriorate during the first year of treatment. Significantly poorer physical, role, and cognitive function scores and worse nausea and vomiting and dyspnoea were seen at all time points. These findings were reflected in worse global quality-of-life scores at most assessments. Problems with malaise and abdominal bloating were observed at most follow-up assessments.

The final module was renamed the EORTC QLQ-NMIBC24 in keeping with current terminology ( Fig. 2 ).

Fig. 2 The European Organization for Research and Treatment of Cancer module for non–muscle-invasive bladder cancer.

2.1. Questionnaires

Patients completed the QLQ-C30 questionnaire and the QLQ-BLS24 module before treatment in a clinic and at regular intervals thereafter. In the high-risk groups, questionnaires were completed at time point 0 and at 2, 3, 6, and 12 mo. In the intermediate-risk group, assessments were completed at time point 0 (before randomisation) and at 12 mo ( Fig. 1 ). Missing data were imputed according to the EORTC guidelines, and questionnaires were considered as missing if >50% of the items were missing [4] . Using this approach, some missing items still could not be imputed, but the other data from these questionnaires were still used. Response rates (based on entirely missing questionnaires or unusable questionnaire) at each time point were examined and reasons for missing questionnaires documented. Response rates to the sexual items were calculated based on whether patients reported being at least “a little” sexually active (item 48).

Fig. 1 Trial schema with timing of assessments using the EORTC QLQ-C30 and QLQ-NMIBC24. BOXIT = Bladder COX-2 Inhibition Trial; NMIBC = non–muscle-invasive bladder cancer; PRO = patient-reported outcome.

The module was developed according to standard EORTC Quality of Life Group guidelines, and translations followed standard procedures [4] . The module has 24 items originally hypothesised to form multi-item scales assessing urinary symptoms (items 1–7), intravesical treatment issues (items 10 and 11), future perspective (items 12–14), fever and feeling ill (items 8 and 9), and abdominal bloating and flatulence (BAF) (items 15 and 16), along with single items addressing different aspects of sexual functioning (items 17–24). All responses are linearly transformed from 0 to 100, with a high score indicating more symptoms or problems or better function for the functional scales. Ethics committee approval and written informed consent were obtained. The sample was determined by the patients within the BOXIT study up until November 2012.

2.2. Defining the scales within the module

Multitrait scaling analyses with data from each of the time points examined whether the individual items may be grouped into the hypothesised scales. The items assessing sexual functioning included two items to be completed by all patients, two items for completion by men only, and one item for completion by women only; also, there were three items completed by men or women reporting to be sexually active in the past 4 wk. Given the conditional nature of many of these items, it was not possible to analyse them as one scale.

Statistical evidence of item convergent validity was defined as a correlation of ≥0.40 between an item and its own scale (corrected for overlap) [10] . Item discriminant validity was defined as a correlation of <0.40 between an item and other scales in the questionnaire. An item was considered to be a scaling success when the correlation between the item and its own scale was greater than its correlation with any other scale. For each scale, the ceiling and floor effects were examined. After finalising the scale structure, other tests were performed.

2.3. Evaluating the reliability and validity of the module

The internal consistency was assessed by the Cronbach α coefficient, with >0.70 considered acceptable for group comparisons being examined within each scale at each assessment point [11] .

Known group comparisons evaluated whether the module was able to discriminate between subgroups of patients differing in clinical status [11] . Known groups used for this comparison were baseline differences in QLQ-C30 physical function scores, with <90 or >90 representing relatively high (better) or relatively low (worse) scores, respectively. It was hypothesised that the scale scores of the QLQ-BLS24 would be higher (show more problems) in patients with lower physical function. Additional exploratory known groups validity testing was performed comparing data from men versus women. The independent student t test was used to examine differences in mean scores. Effect sizes were expressed as the mean difference divided by the pooled standard deviation (SD). Effect sizes were interpreted using the Cohen rule of thumb that a change of 0.5 SD represents a moderate effect, and a change >0.8 SD is a large effect [12] .

To assess validity, correlations between the scales of the QLQ-BLS24 module and the scales of the QLQ-C30 were made using baseline data. Polychoric correlations were calculated, as is appropriate for items with four response categories. The responsiveness of the module to changes in health over time was examined in high-risk patients who underwent intensive treatments. It was hypothesised that during treatment, patients would report increased urinary symptoms and decreased generic aspects of health. Pairwise comparisons of changes in mean scores from baseline to 2, 3, 6, and 12 mo were evaluated using t tests for correlated samples. Because multiple comparisons were performed, a cautious but uncorrected p value of <0.01 was considered to be statistically significant.

All analyses were performed using Stata/IC statistical software (release 12, 2009; StataCorp LP, College Station, TX, USA).

3.1. Patient characteristics, response rates, and missing data

At the time of data analyses, 472 patients were randomised, 433 patients consented to the quality-of-life study, and 410 of them completed a baseline questionnaire. Of these patients, 401 (97.8%) had complete baseline PRO data sets. The majority (79.3%) were men, and more than two-thirds had high-risk tumours (n = 306, 74.6%) ( Table 1 ). The number of questionnaires returned at each time point and completion rates were 282 (92.2%), 288 (94.1%), 263 (85.9%), and 298 (94.3%) at 2, 3, 6, and 12 mo, respectively, for the high-risk group; at 12 mo, 81 questionnaires (77.9%) were returned for the intermediate-risk group. There were therefore 1532 questionnaires in total, with a completion rate for the five assessment points of 88.2%. At baseline, 48% of patients reported at least a little sexual activity (item 48), meaning that completion rates for the sexual scales and items were generally good (>75%). Sociodemographic and clinical details and questionnaire response rates are shown in Table 1 .

3.2. Defining the scales in the module

Final results of the multitrait scaling analyses are shown in Table 2 . Item within scale correlations in the original hypothesised urinary symptom, fever and malaise, and sexual function scales were all ≥0.40, and therefore these scales were maintained. Items 40 and 41, addressing intravesical treatment issues, showed many scaling errors. Discussion within the trial management group therefore led to agreement to include item 40 as a single item assessing intravesical treatment issues. Item 41 correlated well with the future perspectives scale, and therefore this scale was expanded to a four-item future worries scale (items 41–44). The two items assessing abdominal BAF (items 45 and 46) demonstrated satisfactory scaling properties when combined and thus formed a scale. The scale concerning sexual problems in men, items 49 and 50, functioned well and was retained. The remaining items in the original sexual function scale about sexual intimacy (item 51), risk of contamination of partner (item 52), sexual enjoyment (item 53), and an item for sexual function in women only (item 54) remained as individual items.

At baseline, the revised scales showed some floor effects (as expected), because side-effects of treatment would be limited at that stage (scales for malaise, intravesical treatment >72% reported no problems at all). At all time points, few ceiling effects were noted (<2.5% for each scale; data not shown).

The original hypothesised scales in the EORTC QLQ-BLS24 and the confirmed scales in the EORTC QLQ-NMIBC24 are shown in Table 3 .

3.3. Reliability

The internal consistency of the scales at each time point were good (>0.70) for the urinary symptoms, future worries, sexual function, and sexual function in men scales. The fever and malaise scale had coefficients of >0.57 and 0.76; in the abdominal bloating scale, this ranged between 0.49 and 0.62 ( Table 2 ).

3.4. Clinical validity

Patients with high scores (>90) on the physical function scale of the QLQ-C30 reported significantly better functional scores and fewer symptoms on all QLQ-C30 and three module scales (urinary symptoms, malaise, and sexual function) and on a single item (sexual enjoyment) than patients with poorer physical functioning (p < 0.001, Table 4 ). The effect sizes for these differences were moderate to large. Most scales and items were similar between men and women, except that men reported significantly more problems with sexual function (p = 0.005) than women ( Table 4 ).

3.5. Criterion validity

The correlations between the majority of the scales in the core questionnaire and module (n = 44, 88%) were relatively low (r < 0.40, Table 5 ), indicating that the module is not overlapping in content with the QLQ-C30. Correlations >0.4 were observed between the malaise scale in the new module and role and social function scales, global quality of life, and the pain, fatigue, and nausea and vomiting scales in the QLQ-C30. The urinary symptoms scale was moderately associated with role (0.41) and social function (0.43) and the pain scales (0.44) in the QLQ-C30, and the future worries scale in the module showed a moderate association with the emotional function scale (0.50).

3.6. Responsiveness to changes over time

Table 6 shows change in scores before and after treatment. Although little increase in urinary symptoms was observed during the follow-up period, several aspects of health measured by both the QLQ-C30 and the module did deteriorate during the first year of treatment. Significantly poorer physical, role, and cognitive function scores and worse nausea and vomiting and dyspnoea were seen at all time points. These findings were reflected in worse global quality-of-life scores at most assessments. Problems with malaise and abdominal bloating were observed at most follow-up assessments.

The final module was renamed the EORTC QLQ-NMIBC24 in keeping with current terminology ( Fig. 2 ).

Fig. 2 The European Organization for Research and Treatment of Cancer module for non–muscle-invasive bladder cancer.

2.1. Questionnaires

Patients completed the QLQ-C30 questionnaire and the QLQ-BLS24 module before treatment in a clinic and at regular intervals thereafter. In the high-risk groups, questionnaires were completed at time point 0 and at 2, 3, 6, and 12 mo. In the intermediate-risk group, assessments were completed at time point 0 (before randomisation) and at 12 mo ( Fig. 1 ). Missing data were imputed according to the EORTC guidelines, and questionnaires were considered as missing if >50% of the items were missing [4] . Using this approach, some missing items still could not be imputed, but the other data from these questionnaires were still used. Response rates (based on entirely missing questionnaires or unusable questionnaire) at each time point were examined and reasons for missing questionnaires documented. Response rates to the sexual items were calculated based on whether patients reported being at least “a little” sexually active (item 48).

Fig. 1 Trial schema with timing of assessments using the EORTC QLQ-C30 and QLQ-NMIBC24. BOXIT = Bladder COX-2 Inhibition Trial; NMIBC = non–muscle-invasive bladder cancer; PRO = patient-reported outcome.

The module was developed according to standard EORTC Quality of Life Group guidelines, and translations followed standard procedures [4] . The module has 24 items originally hypothesised to form multi-item scales assessing urinary symptoms (items 1–7), intravesical treatment issues (items 10 and 11), future perspective (items 12–14), fever and feeling ill (items 8 and 9), and abdominal bloating and flatulence (BAF) (items 15 and 16), along with single items addressing different aspects of sexual functioning (items 17–24). All responses are linearly transformed from 0 to 100, with a high score indicating more symptoms or problems or better function for the functional scales. Ethics committee approval and written informed consent were obtained. The sample was determined by the patients within the BOXIT study up until November 2012.

2.2. Defining the scales within the module

Multitrait scaling analyses with data from each of the time points examined whether the individual items may be grouped into the hypothesised scales. The items assessing sexual functioning included two items to be completed by all patients, two items for completion by men only, and one item for completion by women only; also, there were three items completed by men or women reporting to be sexually active in the past 4 wk. Given the conditional nature of many of these items, it was not possible to analyse them as one scale.

Statistical evidence of item convergent validity was defined as a correlation of ≥0.40 between an item and its own scale (corrected for overlap) [10] . Item discriminant validity was defined as a correlation of <0.40 between an item and other scales in the questionnaire. An item was considered to be a scaling success when the correlation between the item and its own scale was greater than its correlation with any other scale. For each scale, the ceiling and floor effects were examined. After finalising the scale structure, other tests were performed.

2.3. Evaluating the reliability and validity of the module

The internal consistency was assessed by the Cronbach α coefficient, with >0.70 considered acceptable for group comparisons being examined within each scale at each assessment point [11] .

Known group comparisons evaluated whether the module was able to discriminate between subgroups of patients differing in clinical status [11] . Known groups used for this comparison were baseline differences in QLQ-C30 physical function scores, with <90 or >90 representing relatively high (better) or relatively low (worse) scores, respectively. It was hypothesised that the scale scores of the QLQ-BLS24 would be higher (show more problems) in patients with lower physical function. Additional exploratory known groups validity testing was performed comparing data from men versus women. The independent student t test was used to examine differences in mean scores. Effect sizes were expressed as the mean difference divided by the pooled standard deviation (SD). Effect sizes were interpreted using the Cohen rule of thumb that a change of 0.5 SD represents a moderate effect, and a change >0.8 SD is a large effect [12] .

To assess validity, correlations between the scales of the QLQ-BLS24 module and the scales of the QLQ-C30 were made using baseline data. Polychoric correlations were calculated, as is appropriate for items with four response categories. The responsiveness of the module to changes in health over time was examined in high-risk patients who underwent intensive treatments. It was hypothesised that during treatment, patients would report increased urinary symptoms and decreased generic aspects of health. Pairwise comparisons of changes in mean scores from baseline to 2, 3, 6, and 12 mo were evaluated using t tests for correlated samples. Because multiple comparisons were performed, a cautious but uncorrected p value of <0.01 was considered to be statistically significant.

All analyses were performed using Stata/IC statistical software (release 12, 2009; StataCorp LP, College Station, TX, USA).

3.1. Patient characteristics, response rates, and missing data

At the time of data analyses, 472 patients were randomised, 433 patients consented to the quality-of-life study, and 410 of them completed a baseline questionnaire. Of these patients, 401 (97.8%) had complete baseline PRO data sets. The majority (79.3%) were men, and more than two-thirds had high-risk tumours (n = 306, 74.6%) ( Table 1 ). The number of questionnaires returned at each time point and completion rates were 282 (92.2%), 288 (94.1%), 263 (85.9%), and 298 (94.3%) at 2, 3, 6, and 12 mo, respectively, for the high-risk group; at 12 mo, 81 questionnaires (77.9%) were returned for the intermediate-risk group. There were therefore 1532 questionnaires in total, with a completion rate for the five assessment points of 88.2%. At baseline, 48% of patients reported at least a little sexual activity (item 48), meaning that completion rates for the sexual scales and items were generally good (>75%). Sociodemographic and clinical details and questionnaire response rates are shown in Table 1 .

3.2. Defining the scales in the module

Final results of the multitrait scaling analyses are shown in Table 2 . Item within scale correlations in the original hypothesised urinary symptom, fever and malaise, and sexual function scales were all ≥0.40, and therefore these scales were maintained. Items 40 and 41, addressing intravesical treatment issues, showed many scaling errors. Discussion within the trial management group therefore led to agreement to include item 40 as a single item assessing intravesical treatment issues. Item 41 correlated well with the future perspectives scale, and therefore this scale was expanded to a four-item future worries scale (items 41–44). The two items assessing abdominal BAF (items 45 and 46) demonstrated satisfactory scaling properties when combined and thus formed a scale. The scale concerning sexual problems in men, items 49 and 50, functioned well and was retained. The remaining items in the original sexual function scale about sexual intimacy (item 51), risk of contamination of partner (item 52), sexual enjoyment (item 53), and an item for sexual function in women only (item 54) remained as individual items.

At baseline, the revised scales showed some floor effects (as expected), because side-effects of treatment would be limited at that stage (scales for malaise, intravesical treatment >72% reported no problems at all). At all time points, few ceiling effects were noted (<2.5% for each scale; data not shown).

The original hypothesised scales in the EORTC QLQ-BLS24 and the confirmed scales in the EORTC QLQ-NMIBC24 are shown in Table 3 .

3.3. Reliability

The internal consistency of the scales at each time point were good (>0.70) for the urinary symptoms, future worries, sexual function, and sexual function in men scales. The fever and malaise scale had coefficients of >0.57 and 0.76; in the abdominal bloating scale, this ranged between 0.49 and 0.62 ( Table 2 ).

3.4. Clinical validity

Patients with high scores (>90) on the physical function scale of the QLQ-C30 reported significantly better functional scores and fewer symptoms on all QLQ-C30 and three module scales (urinary symptoms, malaise, and sexual function) and on a single item (sexual enjoyment) than patients with poorer physical functioning (p < 0.001, Table 4 ). The effect sizes for these differences were moderate to large. Most scales and items were similar between men and women, except that men reported significantly more problems with sexual function (p = 0.005) than women ( Table 4 ).

3.5. Criterion validity

The correlations between the majority of the scales in the core questionnaire and module (n = 44, 88%) were relatively low (r < 0.40, Table 5 ), indicating that the module is not overlapping in content with the QLQ-C30. Correlations >0.4 were observed between the malaise scale in the new module and role and social function scales, global quality of life, and the pain, fatigue, and nausea and vomiting scales in the QLQ-C30. The urinary symptoms scale was moderately associated with role (0.41) and social function (0.43) and the pain scales (0.44) in the QLQ-C30, and the future worries scale in the module showed a moderate association with the emotional function scale (0.50).

3.6. Responsiveness to changes over time

Table 6 shows change in scores before and after treatment. Although little increase in urinary symptoms was observed during the follow-up period, several aspects of health measured by both the QLQ-C30 and the module did deteriorate during the first year of treatment. Significantly poorer physical, role, and cognitive function scores and worse nausea and vomiting and dyspnoea were seen at all time points. These findings were reflected in worse global quality-of-life scores at most assessments. Problems with malaise and abdominal bloating were observed at most follow-up assessments.

The final module was renamed the EORTC QLQ-NMIBC24 in keeping with current terminology ( Fig. 2 ).

Fig. 2 The European Organization for Research and Treatment of Cancer module for non–muscle-invasive bladder cancer.

2.1. Questionnaires

Patients completed the QLQ-C30 questionnaire and the QLQ-BLS24 module before treatment in a clinic and at regular intervals thereafter. In the high-risk groups, questionnaires were completed at time point 0 and at 2, 3, 6, and 12 mo. In the intermediate-risk group, assessments were completed at time point 0 (before randomisation) and at 12 mo ( Fig. 1 ). Missing data were imputed according to the EORTC guidelines, and questionnaires were considered as missing if >50% of the items were missing [4] . Using this approach, some missing items still could not be imputed, but the other data from these questionnaires were still used. Response rates (based on entirely missing questionnaires or unusable questionnaire) at each time point were examined and reasons for missing questionnaires documented. Response rates to the sexual items were calculated based on whether patients reported being at least “a little” sexually active (item 48).

Fig. 1 Trial schema with timing of assessments using the EORTC QLQ-C30 and QLQ-NMIBC24. BOXIT = Bladder COX-2 Inhibition Trial; NMIBC = non–muscle-invasive bladder cancer; PRO = patient-reported outcome.

The module was developed according to standard EORTC Quality of Life Group guidelines, and translations followed standard procedures [4] . The module has 24 items originally hypothesised to form multi-item scales assessing urinary symptoms (items 1–7), intravesical treatment issues (items 10 and 11), future perspective (items 12–14), fever and feeling ill (items 8 and 9), and abdominal bloating and flatulence (BAF) (items 15 and 16), along with single items addressing different aspects of sexual functioning (items 17–24). All responses are linearly transformed from 0 to 100, with a high score indicating more symptoms or problems or better function for the functional scales. Ethics committee approval and written informed consent were obtained. The sample was determined by the patients within the BOXIT study up until November 2012.

2.2. Defining the scales within the module

Multitrait scaling analyses with data from each of the time points examined whether the individual items may be grouped into the hypothesised scales. The items assessing sexual functioning included two items to be completed by all patients, two items for completion by men only, and one item for completion by women only; also, there were three items completed by men or women reporting to be sexually active in the past 4 wk. Given the conditional nature of many of these items, it was not possible to analyse them as one scale.

Statistical evidence of item convergent validity was defined as a correlation of ≥0.40 between an item and its own scale (corrected for overlap) [10] . Item discriminant validity was defined as a correlation of <0.40 between an item and other scales in the questionnaire. An item was considered to be a scaling success when the correlation between the item and its own scale was greater than its correlation with any other scale. For each scale, the ceiling and floor effects were examined. After finalising the scale structure, other tests were performed.

2.3. Evaluating the reliability and validity of the module

The internal consistency was assessed by the Cronbach α coefficient, with >0.70 considered acceptable for group comparisons being examined within each scale at each assessment point [11] .

Known group comparisons evaluated whether the module was able to discriminate between subgroups of patients differing in clinical status [11] . Known groups used for this comparison were baseline differences in QLQ-C30 physical function scores, with <90 or >90 representing relatively high (better) or relatively low (worse) scores, respectively. It was hypothesised that the scale scores of the QLQ-BLS24 would be higher (show more problems) in patients with lower physical function. Additional exploratory known groups validity testing was performed comparing data from men versus women. The independent student t test was used to examine differences in mean scores. Effect sizes were expressed as the mean difference divided by the pooled standard deviation (SD). Effect sizes were interpreted using the Cohen rule of thumb that a change of 0.5 SD represents a moderate effect, and a change >0.8 SD is a large effect [12] .

To assess validity, correlations between the scales of the QLQ-BLS24 module and the scales of the QLQ-C30 were made using baseline data. Polychoric correlations were calculated, as is appropriate for items with four response categories. The responsiveness of the module to changes in health over time was examined in high-risk patients who underwent intensive treatments. It was hypothesised that during treatment, patients would report increased urinary symptoms and decreased generic aspects of health. Pairwise comparisons of changes in mean scores from baseline to 2, 3, 6, and 12 mo were evaluated using t tests for correlated samples. Because multiple comparisons were performed, a cautious but uncorrected p value of <0.01 was considered to be statistically significant.

All analyses were performed using Stata/IC statistical software (release 12, 2009; StataCorp LP, College Station, TX, USA).

3.1. Patient characteristics, response rates, and missing data

At the time of data analyses, 472 patients were randomised, 433 patients consented to the quality-of-life study, and 410 of them completed a baseline questionnaire. Of these patients, 401 (97.8%) had complete baseline PRO data sets. The majority (79.3%) were men, and more than two-thirds had high-risk tumours (n = 306, 74.6%) ( Table 1 ). The number of questionnaires returned at each time point and completion rates were 282 (92.2%), 288 (94.1%), 263 (85.9%), and 298 (94.3%) at 2, 3, 6, and 12 mo, respectively, for the high-risk group; at 12 mo, 81 questionnaires (77.9%) were returned for the intermediate-risk group. There were therefore 1532 questionnaires in total, with a completion rate for the five assessment points of 88.2%. At baseline, 48% of patients reported at least a little sexual activity (item 48), meaning that completion rates for the sexual scales and items were generally good (>75%). Sociodemographic and clinical details and questionnaire response rates are shown in Table 1 .

3.2. Defining the scales in the module

Final results of the multitrait scaling analyses are shown in Table 2 . Item within scale correlations in the original hypothesised urinary symptom, fever and malaise, and sexual function scales were all ≥0.40, and therefore these scales were maintained. Items 40 and 41, addressing intravesical treatment issues, showed many scaling errors. Discussion within the trial management group therefore led to agreement to include item 40 as a single item assessing intravesical treatment issues. Item 41 correlated well with the future perspectives scale, and therefore this scale was expanded to a four-item future worries scale (items 41–44). The two items assessing abdominal BAF (items 45 and 46) demonstrated satisfactory scaling properties when combined and thus formed a scale. The scale concerning sexual problems in men, items 49 and 50, functioned well and was retained. The remaining items in the original sexual function scale about sexual intimacy (item 51), risk of contamination of partner (item 52), sexual enjoyment (item 53), and an item for sexual function in women only (item 54) remained as individual items.

At baseline, the revised scales showed some floor effects (as expected), because side-effects of treatment would be limited at that stage (scales for malaise, intravesical treatment >72% reported no problems at all). At all time points, few ceiling effects were noted (<2.5% for each scale; data not shown).

The original hypothesised scales in the EORTC QLQ-BLS24 and the confirmed scales in the EORTC QLQ-NMIBC24 are shown in Table 3 .

3.3. Reliability

The internal consistency of the scales at each time point were good (>0.70) for the urinary symptoms, future worries, sexual function, and sexual function in men scales. The fever and malaise scale had coefficients of >0.57 and 0.76; in the abdominal bloating scale, this ranged between 0.49 and 0.62 ( Table 2 ).

3.4. Clinical validity

Patients with high scores (>90) on the physical function scale of the QLQ-C30 reported significantly better functional scores and fewer symptoms on all QLQ-C30 and three module scales (urinary symptoms, malaise, and sexual function) and on a single item (sexual enjoyment) than patients with poorer physical functioning (p < 0.001, Table 4 ). The effect sizes for these differences were moderate to large. Most scales and items were similar between men and women, except that men reported significantly more problems with sexual function (p = 0.005) than women ( Table 4 ).

3.5. Criterion validity

The correlations between the majority of the scales in the core questionnaire and module (n = 44, 88%) were relatively low (r < 0.40, Table 5 ), indicating that the module is not overlapping in content with the QLQ-C30. Correlations >0.4 were observed between the malaise scale in the new module and role and social function scales, global quality of life, and the pain, fatigue, and nausea and vomiting scales in the QLQ-C30. The urinary symptoms scale was moderately associated with role (0.41) and social function (0.43) and the pain scales (0.44) in the QLQ-C30, and the future worries scale in the module showed a moderate association with the emotional function scale (0.50).

3.6. Responsiveness to changes over time

Table 6 shows change in scores before and after treatment. Although little increase in urinary symptoms was observed during the follow-up period, several aspects of health measured by both the QLQ-C30 and the module did deteriorate during the first year of treatment. Significantly poorer physical, role, and cognitive function scores and worse nausea and vomiting and dyspnoea were seen at all time points. These findings were reflected in worse global quality-of-life scores at most assessments. Problems with malaise and abdominal bloating were observed at most follow-up assessments.

The final module was renamed the EORTC QLQ-NMIBC24 in keeping with current terminology ( Fig. 2 ).

Fig. 2 The European Organization for Research and Treatment of Cancer module for non–muscle-invasive bladder cancer.

2.1. Questionnaires

Patients completed the QLQ-C30 questionnaire and the QLQ-BLS24 module before treatment in a clinic and at regular intervals thereafter. In the high-risk groups, questionnaires were completed at time point 0 and at 2, 3, 6, and 12 mo. In the intermediate-risk group, assessments were completed at time point 0 (before randomisation) and at 12 mo ( Fig. 1 ). Missing data were imputed according to the EORTC guidelines, and questionnaires were considered as missing if >50% of the items were missing [4] . Using this approach, some missing items still could not be imputed, but the other data from these questionnaires were still used. Response rates (based on entirely missing questionnaires or unusable questionnaire) at each time point were examined and reasons for missing questionnaires documented. Response rates to the sexual items were calculated based on whether patients reported being at least “a little” sexually active (item 48).

Fig. 1 Trial schema with timing of assessments using the EORTC QLQ-C30 and QLQ-NMIBC24. BOXIT = Bladder COX-2 Inhibition Trial; NMIBC = non–muscle-invasive bladder cancer; PRO = patient-reported outcome.

The module was developed according to standard EORTC Quality of Life Group guidelines, and translations followed standard procedures [4] . The module has 24 items originally hypothesised to form multi-item scales assessing urinary symptoms (items 1–7), intravesical treatment issues (items 10 and 11), future perspective (items 12–14), fever and feeling ill (items 8 and 9), and abdominal bloating and flatulence (BAF) (items 15 and 16), along with single items addressing different aspects of sexual functioning (items 17–24). All responses are linearly transformed from 0 to 100, with a high score indicating more symptoms or problems or better function for the functional scales. Ethics committee approval and written informed consent were obtained. The sample was determined by the patients within the BOXIT study up until November 2012.

2.2. Defining the scales within the module

Multitrait scaling analyses with data from each of the time points examined whether the individual items may be grouped into the hypothesised scales. The items assessing sexual functioning included two items to be completed by all patients, two items for completion by men only, and one item for completion by women only; also, there were three items completed by men or women reporting to be sexually active in the past 4 wk. Given the conditional nature of many of these items, it was not possible to analyse them as one scale.

Statistical evidence of item convergent validity was defined as a correlation of ≥0.40 between an item and its own scale (corrected for overlap) [10] . Item discriminant validity was defined as a correlation of <0.40 between an item and other scales in the questionnaire. An item was considered to be a scaling success when the correlation between the item and its own scale was greater than its correlation with any other scale. For each scale, the ceiling and floor effects were examined. After finalising the scale structure, other tests were performed.

2.3. Evaluating the reliability and validity of the module

The internal consistency was assessed by the Cronbach α coefficient, with >0.70 considered acceptable for group comparisons being examined within each scale at each assessment point [11] .

Known group comparisons evaluated whether the module was able to discriminate between subgroups of patients differing in clinical status [11] . Known groups used for this comparison were baseline differences in QLQ-C30 physical function scores, with <90 or >90 representing relatively high (better) or relatively low (worse) scores, respectively. It was hypothesised that the scale scores of the QLQ-BLS24 would be higher (show more problems) in patients with lower physical function. Additional exploratory known groups validity testing was performed comparing data from men versus women. The independent student t test was used to examine differences in mean scores. Effect sizes were expressed as the mean difference divided by the pooled standard deviation (SD). Effect sizes were interpreted using the Cohen rule of thumb that a change of 0.5 SD represents a moderate effect, and a change >0.8 SD is a large effect [12] .

To assess validity, correlations between the scales of the QLQ-BLS24 module and the scales of the QLQ-C30 were made using baseline data. Polychoric correlations were calculated, as is appropriate for items with four response categories. The responsiveness of the module to changes in health over time was examined in high-risk patients who underwent intensive treatments. It was hypothesised that during treatment, patients would report increased urinary symptoms and decreased generic aspects of health. Pairwise comparisons of changes in mean scores from baseline to 2, 3, 6, and 12 mo were evaluated using t tests for correlated samples. Because multiple comparisons were performed, a cautious but uncorrected p value of <0.01 was considered to be statistically significant.

All analyses were performed using Stata/IC statistical software (release 12, 2009; StataCorp LP, College Station, TX, USA).

3.1. Patient characteristics, response rates, and missing data

At the time of data analyses, 472 patients were randomised, 433 patients consented to the quality-of-life study, and 410 of them completed a baseline questionnaire. Of these patients, 401 (97.8%) had complete baseline PRO data sets. The majority (79.3%) were men, and more than two-thirds had high-risk tumours (n = 306, 74.6%) ( Table 1 ). The number of questionnaires returned at each time point and completion rates were 282 (92.2%), 288 (94.1%), 263 (85.9%), and 298 (94.3%) at 2, 3, 6, and 12 mo, respectively, for the high-risk group; at 12 mo, 81 questionnaires (77.9%) were returned for the intermediate-risk group. There were therefore 1532 questionnaires in total, with a completion rate for the five assessment points of 88.2%. At baseline, 48% of patients reported at least a little sexual activity (item 48), meaning that completion rates for the sexual scales and items were generally good (>75%). Sociodemographic and clinical details and questionnaire response rates are shown in Table 1 .

3.2. Defining the scales in the module

Final results of the multitrait scaling analyses are shown in Table 2 . Item within scale correlations in the original hypothesised urinary symptom, fever and malaise, and sexual function scales were all ≥0.40, and therefore these scales were maintained. Items 40 and 41, addressing intravesical treatment issues, showed many scaling errors. Discussion within the trial management group therefore led to agreement to include item 40 as a single item assessing intravesical treatment issues. Item 41 correlated well with the future perspectives scale, and therefore this scale was expanded to a four-item future worries scale (items 41–44). The two items assessing abdominal BAF (items 45 and 46) demonstrated satisfactory scaling properties when combined and thus formed a scale. The scale concerning sexual problems in men, items 49 and 50, functioned well and was retained. The remaining items in the original sexual function scale about sexual intimacy (item 51), risk of contamination of partner (item 52), sexual enjoyment (item 53), and an item for sexual function in women only (item 54) remained as individual items.

At baseline, the revised scales showed some floor effects (as expected), because side-effects of treatment would be limited at that stage (scales for malaise, intravesical treatment >72% reported no problems at all). At all time points, few ceiling effects were noted (<2.5% for each scale; data not shown).

The original hypothesised scales in the EORTC QLQ-BLS24 and the confirmed scales in the EORTC QLQ-NMIBC24 are shown in Table 3 .

3.3. Reliability

The internal consistency of the scales at each time point were good (>0.70) for the urinary symptoms, future worries, sexual function, and sexual function in men scales. The fever and malaise scale had coefficients of >0.57 and 0.76; in the abdominal bloating scale, this ranged between 0.49 and 0.62 ( Table 2 ).

3.4. Clinical validity

Patients with high scores (>90) on the physical function scale of the QLQ-C30 reported significantly better functional scores and fewer symptoms on all QLQ-C30 and three module scales (urinary symptoms, malaise, and sexual function) and on a single item (sexual enjoyment) than patients with poorer physical functioning (p < 0.001, Table 4 ). The effect sizes for these differences were moderate to large. Most scales and items were similar between men and women, except that men reported significantly more problems with sexual function (p = 0.005) than women ( Table 4 ).

3.5. Criterion validity

The correlations between the majority of the scales in the core questionnaire and module (n = 44, 88%) were relatively low (r < 0.40, Table 5 ), indicating that the module is not overlapping in content with the QLQ-C30. Correlations >0.4 were observed between the malaise scale in the new module and role and social function scales, global quality of life, and the pain, fatigue, and nausea and vomiting scales in the QLQ-C30. The urinary symptoms scale was moderately associated with role (0.41) and social function (0.43) and the pain scales (0.44) in the QLQ-C30, and the future worries scale in the module showed a moderate association with the emotional function scale (0.50).

3.6. Responsiveness to changes over time

Table 6 shows change in scores before and after treatment. Although little increase in urinary symptoms was observed during the follow-up period, several aspects of health measured by both the QLQ-C30 and the module did deteriorate during the first year of treatment. Significantly poorer physical, role, and cognitive function scores and worse nausea and vomiting and dyspnoea were seen at all time points. These findings were reflected in worse global quality-of-life scores at most assessments. Problems with malaise and abdominal bloating were observed at most follow-up assessments.

The final module was renamed the EORTC QLQ-NMIBC24 in keeping with current terminology ( Fig. 2 ).

Fig. 2 The European Organization for Research and Treatment of Cancer module for non–muscle-invasive bladder cancer.

2.1. Questionnaires

Patients completed the QLQ-C30 questionnaire and the QLQ-BLS24 module before treatment in a clinic and at regular intervals thereafter. In the high-risk groups, questionnaires were completed at time point 0 and at 2, 3, 6, and 12 mo. In the intermediate-risk group, assessments were completed at time point 0 (before randomisation) and at 12 mo ( Fig. 1 ). Missing data were imputed according to the EORTC guidelines, and questionnaires were considered as missing if >50% of the items were missing [4] . Using this approach, some missing items still could not be imputed, but the other data from these questionnaires were still used. Response rates (based on entirely missing questionnaires or unusable questionnaire) at each time point were examined and reasons for missing questionnaires documented. Response rates to the sexual items were calculated based on whether patients reported being at least “a little” sexually active (item 48).

Fig. 1 Trial schema with timing of assessments using the EORTC QLQ-C30 and QLQ-NMIBC24. BOXIT = Bladder COX-2 Inhibition Trial; NMIBC = non–muscle-invasive bladder cancer; PRO = patient-reported outcome.

The module was developed according to standard EORTC Quality of Life Group guidelines, and translations followed standard procedures [4] . The module has 24 items originally hypothesised to form multi-item scales assessing urinary symptoms (items 1–7), intravesical treatment issues (items 10 and 11), future perspective (items 12–14), fever and feeling ill (items 8 and 9), and abdominal bloating and flatulence (BAF) (items 15 and 16), along with single items addressing different aspects of sexual functioning (items 17–24). All responses are linearly transformed from 0 to 100, with a high score indicating more symptoms or problems or better function for the functional scales. Ethics committee approval and written informed consent were obtained. The sample was determined by the patients within the BOXIT study up until November 2012.

2.2. Defining the scales within the module

Multitrait scaling analyses with data from each of the time points examined whether the individual items may be grouped into the hypothesised scales. The items assessing sexual functioning included two items to be completed by all patients, two items for completion by men only, and one item for completion by women only; also, there were three items completed by men or women reporting to be sexually active in the past 4 wk. Given the conditional nature of many of these items, it was not possible to analyse them as one scale.

Statistical evidence of item convergent validity was defined as a correlation of ≥0.40 between an item and its own scale (corrected for overlap) [10] . Item discriminant validity was defined as a correlation of <0.40 between an item and other scales in the questionnaire. An item was considered to be a scaling success when the correlation between the item and its own scale was greater than its correlation with any other scale. For each scale, the ceiling and floor effects were examined. After finalising the scale structure, other tests were performed.

2.3. Evaluating the reliability and validity of the module

The internal consistency was assessed by the Cronbach α coefficient, with >0.70 considered acceptable for group comparisons being examined within each scale at each assessment point [11] .

Known group comparisons evaluated whether the module was able to discriminate between subgroups of patients differing in clinical status [11] . Known groups used for this comparison were baseline differences in QLQ-C30 physical function scores, with <90 or >90 representing relatively high (better) or relatively low (worse) scores, respectively. It was hypothesised that the scale scores of the QLQ-BLS24 would be higher (show more problems) in patients with lower physical function. Additional exploratory known groups validity testing was performed comparing data from men versus women. The independent student t test was used to examine differences in mean scores. Effect sizes were expressed as the mean difference divided by the pooled standard deviation (SD). Effect sizes were interpreted using the Cohen rule of thumb that a change of 0.5 SD represents a moderate effect, and a change >0.8 SD is a large effect [12] .

To assess validity, correlations between the scales of the QLQ-BLS24 module and the scales of the QLQ-C30 were made using baseline data. Polychoric correlations were calculated, as is appropriate for items with four response categories. The responsiveness of the module to changes in health over time was examined in high-risk patients who underwent intensive treatments. It was hypothesised that during treatment, patients would report increased urinary symptoms and decreased generic aspects of health. Pairwise comparisons of changes in mean scores from baseline to 2, 3, 6, and 12 mo were evaluated using t tests for correlated samples. Because multiple comparisons were performed, a cautious but uncorrected p value of <0.01 was considered to be statistically significant.

All analyses were performed using Stata/IC statistical software (release 12, 2009; StataCorp LP, College Station, TX, USA).

3.1. Patient characteristics, response rates, and missing data

At the time of data analyses, 472 patients were randomised, 433 patients consented to the quality-of-life study, and 410 of them completed a baseline questionnaire. Of these patients, 401 (97.8%) had complete baseline PRO data sets. The majority (79.3%) were men, and more than two-thirds had high-risk tumours (n = 306, 74.6%) ( Table 1 ). The number of questionnaires returned at each time point and completion rates were 282 (92.2%), 288 (94.1%), 263 (85.9%), and 298 (94.3%) at 2, 3, 6, and 12 mo, respectively, for the high-risk group; at 12 mo, 81 questionnaires (77.9%) were returned for the intermediate-risk group. There were therefore 1532 questionnaires in total, with a completion rate for the five assessment points of 88.2%. At baseline, 48% of patients reported at least a little sexual activity (item 48), meaning that completion rates for the sexual scales and items were generally good (>75%). Sociodemographic and clinical details and questionnaire response rates are shown in Table 1 .