Statistical Assessment of Validity

There are two broad approaches to establishing validity between test and reference measures: comparison of mean values and correlation. The use of mean values is appropriate where group intakes are to be determined or where an absolute measure of intake is required. This is especially important where a threshold value of intake will be used to make recommendations (e.g., recommending an increase in potassium intake because of its association with lower blood pressure and reduced risk of myocardial infarction—there would be no point in recommending additional consumption of potassium for individuals who were identified as having intakes already above the levels that were seen to be protective). The correlation technique (plotting the observed measure against the reference measure) is appropriate where it is important to classify subjects according to high or low intakes because differences in intake are associated with different levels of disease risk. In relation to disease risk, a steeper or shallower slope of observed intake in relation to true intake will have a profound effect on the relative risk estimates in relation to disease outcome. Moreover, the correlation coefficient describes only one aspect of agreement. In practice, both ranking and absolute levels are important in establishing correct diet-disease relationships.

Mathematically, disease risk can be modeled by the expression

where R(DIT) is the risk of disease D, T is the unobservable true long-term habitual intake of a given food or nutrient relevant to disease risk, a0 is the underlying risk of disease in the population independent of dietary exposure, and a1 is the log relative risk (RR) that may be positive (for predisposing factors) or negative (for protective factors).

Because we cannot measure the true dietary exposure, we approximate it using the dietary test measure that is the focus of the validation process. The expression for describing disease risk then becomes

where Q is the observed intake, and a* is the observed log relative risk. In most circumstances, it can be argued that a* = Aa1, where A is known as the 'attenuation' factor and is equal to the slope of the regression line of T plotted against Q. In most dietary studies, the value for A is between 0 and 1, but in cases of differential misclassification it may also be negative. The consequence is that the estimate of disease risk in relation to diet is likely to be different from the true risk (usually tending toward a relative risk of unity).

For a given individual i, the observed measure Qi will be given by the expression

Qi = (Tj )B + a + 6j + ej where Ti is the true measure, and the attenuation factor A is a function of B (proportional bias), a (constant bias), ei (random error within a subject, such that the mean of the random error across all subjects is equal to zero), and ei (bias in the ith subject, such that the mean of the individual biases across all subjects is not equal to zero—this is the consequence of differential misclassification). In practical terms, the aim of a validation study is to quantify these sources of error (see Table 3 for the main likely sources) and to estimate the value for A so that the true relative risk a1 can be estimated using the expression a1 = a*/A.

It is probable that both the test measure and the reference measure are positively correlated with the truth. This is represented by tqT and rRT in Figure 5. There will also be a relationship between the test and reference measures, rQR, given by the expression rQR=rQT x rR

The relationship between the test measure and the truth can be estimated by solving for rQT:

Assuming that the reference measures are unbiased, rRT can be estimated by knowing the relationship between within- and between-subject variance in a group of subjects whose records or measures are assumed to be valid (e.g., from whom

TRUTH

TRUTH

24 h recall)

Figure 5 The relationship between test and reference measure and the truth. (Source: Nelson M (1997) The validation of dietary assessments. In: Margetts BM and Nelson M (eds.) Design Concepts in Nutritional Epidemiology, 2nd edn. Oxford: Oxford University Press.)

24 h recall)

Figure 5 The relationship between test and reference measure and the truth. (Source: Nelson M (1997) The validation of dietary assessments. In: Margetts BM and Nelson M (eds.) Design Concepts in Nutritional Epidemiology, 2nd edn. Oxford: Oxford University Press.)

likely under- or overreporters have been excluded) using the expression yjn/(n +(si/sD)

where n is the number of repeat observations within one subject, and sW and s^ are the within- and between-subject variances, respectively. In this way, the likely relationship between the test measure and the truth can be estimated, and the relative risk can be adjusted to account for misclassification of subjects based on the test measure alone.

This approach has two weaknesses. First, if the reference measure is a dietary measure, it does not address the problem of correlation of errors (the tendency for an individual to misreport diet in the same way using the test and reference measures). If errors are correlated between methods, then the observed rRQ is likely to overestimate rRT x tqt; it will appear that the test method is performing better than it actually is performing. If the errors are correlated within methods (e.g., if the same types of within-person bias are occurring from day to day using repeat 24-h recall), then the observed rRQ is likely to underestimate rRT x rQT. The second weakness of this approach is that it does not address the problem of differential misclassification.

A similar technique is the method of triads (Figure 6), in which no assumption need be made about the relationships between reference measures and the truth. Instead, the relationships between three measures can be used to estimate values for p, which in theory approximate the correlation between each of the measures and the truth. As in the technique described previously, valid estimates of p are based on the assumption that the errors in the methods are uncorrelated and that the errors are random and not differentially biased between subjects.

Because the validation process helps to identify subjects who are likely to be misreporting their diet, the temptation may be to exclude from analysis those subjects who have misreported their diet. It may be, however, that the very subjects who are most likely to misreport their food consumption (e.g., people who are overweight) are also those who are at increased risk of disease (e.g., hypertension, heart disease, and colon cancer). In estimating disease risk, therefore, the aim must be to retain all of the subjects in the analysis.

Estimating the components of error and finding appropriate values for A is the best way to address this issue. A special case is to adjust nutrient intakes to allow for misreporting in some subjects by assuming that true energy intake and true nutrient intake in all subjects are well correlated. Thus, if a subject underreports energy intake, it is assumed that other nutrients will be underreported to a similar extent. By estimating nutrient intake in relation to reported energy intake, subjects can be ranked according to whether, for a given level of energy intake, their nutrient intake was above or below the average. This is known as energy adjustment. To find the energy-adjusted estimate of nutrient intake, the nutrient intakes should be plotted against energy intakes and the regression line and the residual values derived (Figure 7). Energy-adjusted nutrient intakes are then computed by adding the residual to the mean nutrient intake. This approach allows all subjects to be included in an analysis, and it provides realistic estimates of intake (unlike computations of nutrient density in which each subject's nutrient intake is divided by his or her energy intake). Like doubly labeled water, however, the

METHOD A

METHOD C

METHOD A

THE TRUTH

METHOD B

Figure 6 Graphic representation of the method of triads. (Source: Ocke M and Kaaks R (1997) Biochemical markers as additional measurements in dietary validity studies: Application of the method of triads with examples from the European Prospective Investigation into Cancer and Nutrition. American Journal of Clinical Nutrition 65:1240S-1245S.)

Energy intake

Figure 7 Energy-adjusted nutrient intake for the i th individual = ai + b. (Adapted from Willett WD, Howe GR and Kushi LH (1997) Adjustment for total energy intake in epidemiologic studies. American Journal of Clinical Nutrition 65:1220S-1228S.)

Energy intake

Figure 7 Energy-adjusted nutrient intake for the i th individual = ai + b. (Adapted from Willett WD, Howe GR and Kushi LH (1997) Adjustment for total energy intake in epidemiologic studies. American Journal of Clinical Nutrition 65:1220S-1228S.)

weakness of energy adjustment lies in the fact that not all nutrient intakes are well correlated with energy intake. Overreporting of fruits and vegetables consumption leading to an overestimate of vitamin C intake, for example, would not be appropriately compensated for using energy adjustment if the person was at the same time underreporting his or her fat consumption.

Diet Tweak System

Diet Tweak System

Trying To Lose Weight Can Be Tough. But... Not Losing Weight and Gaining What You Lost Back, Sucks. If you've ever felt that no matter what you do to lose weight nothing seems to work. If you've ever felt that there has got to be some kind of a system or way to lose weight...but just have not found it yet.

Get My Free Ebook


Post a comment