|
|
|
|
3 Departments of Medicine, Clinical and Social Sciences in Psychology, and 4 Psychiatry, University of Rochester, Rochester, NY 14642; 5 Cancer Prevention and Control Program, Department of Epidemiology and Biostatistics, Arnold School of Public Health, University of South Carolina, Columbia, SC 29208; 6 Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD 20892-7344; 7 Biometry Program, Division of Cancer Prevention, National Cancer Institute, Bethesda, MD 20892-7344; 8 Behavioral Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD 20892-7344; 9 Health Behavior and Health Education, School of Public Health, University of Michigan, Ann Arbor, MI 48109-2029; 10 Oregon Research Institute, Eugene, OR 97403; 11 Department of Nutrition and Food Sciences, University of Rhode Island, Kingston, RI 02881; 12 Department of Food and Nutrition, Harvard School of Public Health, Cambridge, MA 02115; and 13 Center for Health Services and Policy Research, Department of Epidemiology and Biostatistics, University of South Carolina, Columbia, SC 29208
* To whom correspondence should be addressed. E-mail: geoffrey_williams{at}urmc.rochester.edu.
| ABSTRACT |
|---|
|
|
|---|
| Introduction |
|---|
|
|
|---|
Self-reports of diet are typically used to assess dietary change in intervention studies because few biomarkers that accurately reflect intake are available and, for those few that do exist, such measures are often prohibitively expensive. The interviewer-administered 24-h dietary recall (24HR)14 provides relatively accurate and complete self-reported information about the individual's diet for a given day and is feasible in diverse populations (9–11). However, because the interviewer-administered 24HR is costly and burdensome for both investigators and participants, FFQ or brief screener instruments with a more narrow focus on whole foods or a particular class of nutrient (e.g., fat) are typically employed (12). Several short fat screeners have been developed and evaluated in cross-sectional settings (13,14). In addition, a 13-item fat screener developed for the Prostate Cancer Prevention Trial was evaluated in male participants at baseline; this screener had a significantly lower correlation with the criterion 24HR than did an FFQ [r = 0.51 vs. r = 0.71 (15)]. Based on this finding, the investigators in that study recommended against the use of a short screener as the sole assessment instrument in intervention studies (15). Nonetheless, because of resource constraints, many behavioral dietary interventions continue to employ various short screeners to assess change.
A National Cancer Institute (NCI) short screener assessing percentage of energy from fat (PFat) but not the amount or type of fat has been developed and tested cross-sectionally at baseline in a cohort setting [the NIH-AARP Diet and Health Study (16)] and cross-sectionally at baseline in these Behavior Change Consortium (BCC) sites (17). In these analyses, we sought to evaluate the ability of the PFat to assess change in percentage energy from fat intake in comparison to change in percentage energy from fat intake estimated from multiple 24HR across 4 intervention sites participating in the BCC. This is the first study to evaluate a short percentage energy from fat screener in an intervention setting.
| Methods |
|---|
|
|
|---|
Four sites [University of Rhode Island (URI), Harvard School of Public Health (HSPH), Emory University, and University of Rochester (ROC)] administered both the 16-item PFat screener and multiple 24HR at baseline and follow-up. Additional information on study sites, the PFat tool, and the 24HR protocol are provided in Yaroch et al. (18). Briefly, in 3 of the 4 sites (not Emory), the baseline PFat was administered after enrollment into the intervention study but before randomization to usual care versus experimental care group. At the Emory site, the baseline PFat was administered after randomization. PFat was self-administered in 2 sites (Emory, ROC) and interviewer administered in 2 sites (URI, HSPH) at baseline and at follow-up. Up to 3 nonconsecutive 24HR were administered by telephone to each participant at baseline and at follow-up (total of 6 possible 24HR). Duration of follow-up was 12 mo at URI, HSPH, and Emory and 6 mo at ROC. Subjects (n = 278) who completed the PFat and 24HR at both the baseline and follow-up time points were included in the analysis for this report.
PFat. PFat is composed of 16 questions, which ask about usual consumption practices in the past 12 mo. Frequency of intake is asked for 15 food groups, selected in earlier analyses to optimally predict intake of percentage energy from fat (16). One question asks about use of reduced-fat margarine. Portion size is not explicitly queried; however, the scoring algorithms assign gender- and age-specific median portion sizes in grams to each food group asked. Then gender-specific regression coefficients relating intake of each food group to percentage energy from fat are applied. Complete scoring information is available on line (19). Screener data were coded at each center and sent to URI for central processing.
24HR. Multiple nonconsecutive 24HR were administered by telephone to respondents by trained interviewers. Data collection and processing for Emory and HSPH were performed by the Diet Assessment Unit of the Cancer Prevention and Control Program at the University of South Carolina; for ROC by the Diet Assessment Center at Pennsylvania State University; and for URI by their staff. All sites followed the same protocol, including a preinterview mailing of a 2-dimensional food portion guide (20) and 3 nonconsecutive unannounced 24HR over a 3-wk period, including 1 weekend day. Of the 278 participants, 62% had 3 24HR at both baseline and follow-up, and over 92% had at least 2 at both times (n = 255).
The University of Minnesota Nutrient Data System for Research (NDS-R), versions 4.05_33 and 4.06_34, was used to conduct, code, and process 24HR. Interviews were conducted by trained interviewers using the multipass approach interface of NDS-R. Interviews were reviewed by supervisors, and foods reported that were not on the nutrient database were added in consultation from the Nutrition Coordinating Center (NCC) at the University of Minnesota. Coding quality was checked with built-in systems that flag extreme values. All individual recalls defined as unreliable by the interviewer were reviewed and exclusion confirmed by a dietitian with experience in conducting/supervising NDS-R recalls and an author with similar credentials (G. W. Greene).
Intervention condition
A variable indicating whether or not dietary fat intake was a target of the intervention was created. Decreasing intake of fat was a primary target of the dietary intervention at 2 sites (HSPH and ROC). For the other 2 sites, Emory and URI, the intervention targeted FV intake, and because FV intake is inversely associated with fat intake (21), the intervention also encouraged a reduction in fat intake. Thus, these 2 types of interventions were considered fat interventions. At all sites, alternative, nondiet intervention targets (e.g., physical activity, smoking, usual care) or no-treatment controls were included in the design. URI, HSPH, and Emory had a no-treatment control group; URI also had an exercise-only control; and ROC had both usual care (received results from a cholesterol screening, but no active diet or other counseling) and smoking cessation control groups. For the purposes of these analyses, controls were defined as those participants who received no intervention or a nondiet intervention.
Analytical procedure
Demographic data for the sample of 278 were computed by gender. The distributions of both PFat- and 24HR-derived estimates of percentage energy from fat were approximately normal. Therefore, data are presented as the mean and its standard error by gender and treatment group at both baseline and follow-up, controlling for site. The mean difference between instruments (PFat – 24HR) was tested at each time point by gender and treatment group, controlling for site, using a paired t-test.
Although true usual intake is not observable in free-living populations (21), we can estimate the distribution of true intake in the population by use of appropriate reference data and statistical methods. Our reference instrument is multiple nonconsecutive 24HR. We used a measurement error (ME) model, described in Freedman et al. (22), to estimate the relation between true intake (a latent construct derived from 24HR) and the PFat, assuming that the reference instrument (24HR) was unbiased at the individual level and contained only within-person error (23). Adjusting for ME, we estimated the overall level of agreement as the correlation (
) between the screener and estimated true intake from 24HR. The ME model was fit at baseline and at follow-up in separate analyses. The correlation coefficients were compared using Fisher's Z-transformation to test whether the correlation varied over time (24).
Before fitting the final ME models, multiple ME models were fit to evaluate the influence of site and gender, and these models were compared using a likelihood ratio test. Although correlations varied by site and gender, our primary interest was to evaluate PFat, by gender, in the context of an intervention. As a result, site was fit as a control variable in all ME models. Before analysis, values of percentage energy from fat from each instrument were examined to identify outliers (i.e., for each gender, values deviating either more than 3 times the equivalent of the interquartile range below quartile 1 or above quartile 3 of that variable's distribution were defined to be outliers); none were found. Subsequent to fitting cross-sectional ME models, diagnostic statistics were used to identify influential observations both as baseline and at follow-up. Three observations were identified in the baseline stratum for women that had a significant impact on the estimate of
. When these 3 subjects were excluded,
changed by 45%; ME results are presented excluding these 3 subjects (n = 275).
One way to look at the ability of an instrument to measure change is to see if it captures the treatment group effect as well as the referent measure does. To evaluate this capability, an ANCOVA approach that is commonly employed in a pretest posttest setting was used (25). Models were run separately for each instrument, regressing the follow-up measure of fat intake on a design variable for treatment group while controlling for baseline levels of the outcome variable and site.
| Results |
|---|
|
|
|---|
|
|
|
|
| Discussion |
|---|
|
|
|---|
Looking at the data cross-sectionally at baseline and at follow-up, we found that PFat systematically underestimated fat intake relative to 24HR, ranging from 1.5% to 2%. Although the underestimate was slightly smaller among men than among women, small sample size and accordingly large standard deviations among men limited our ability to examine gender differences. Among women, the underestimate increased at follow-up in both intervention and control groups. The statistically significant underestimate of PFat-derived percentage of energy from fat relative to 24HR observed at both baseline and follow-up in women is consistent with the social desirability bias that was observed in these same data (31). Among men, the underestimate increased slightly at follow-up, but only for controls. Moreover, even though the underestimate was greater in women in the intervention group than in the control group at both time points, for men there was a larger underestimate at follow-up in the controls than in the intervention group. The measures were moderately correlated and of similar magnitude at both times. However, for both men and women, deattenuated correlations between PFat and 24HR were higher at follow-up than at baseline for the intervention groups, whereas no difference was observed for the control groups. This pattern of intervention subjects being more likely than controls to report positive changes has been identified in previous studies of brief dietary instruments (26,32–34). It suggests that the act of participating in a dietary intervention program may increase awareness, knowledge, or salience of foods consumed, thus enhancing the quality of 1 or both dietary reports. Regardless of the explanation, this difference in agreement between treatment and control groups could bias estimates of dietary change in intervention studies and calls for further study. If the bias occurs because the intervention subjects report their intake more accurately, it suggests that to detect whether an intervention creates real change in dietary intake, all trial participants should "learn" to use the measures before randomization. Thus, the performance of PFat varied by intervention status and indicates further need to understand and control for this bias. The performance also varied to some extent by gender, but the much smaller sample size for men may account for this difference and indicates that larger sample sizes are needed to confirm this bias.
The 4 BCC sites in this investigation did not produce significant change in the percentage of energy from fat as measured by 24HR; this finding was consistent with PFat, which also showed no change in percentage of energy from fat. Confirmation of a null intervention effect is important because scarce resources can then be more efficiently managed with regard to future program designs. However, more definitive testing is needed in intervention studies where significant changes in percentage energy from fat are produced and in studies with larger numbers of participants and from diverse samples.
Several limitations, both general and specific to this study, need to be acknowledged. Because of the lack of an objective biomarker for percentage energy from fat, researchers usually rely on self-reported dietary intake to characterize an individual's usual fat intake. Although the reference measure of multiple nonconsecutive 24HR may be considered a gold standard in self-report dietary assessment, it contains measurement error. In addition, error in 24HR may be correlated with error in PFat, potentially leading to an overestimate of the correlation between instruments (35). Other biases (e.g., selection bias, response set bias, and training effects) also could produce misestimates of true correlation coefficients (30,31,36). Specific to our study, 24HR were administered over a much briefer time period than the 12 mo assessed by PFat; ideally, 24HR would be administered at several time points over the same 1-y interval assessed by the screener. Additionally, the 6-mo follow-up at 1 site creates overlapping periods of dietary assessment using the screener, which looks back over the past year. We examined this by rerunning the analyses excluding that site and found that the results were not appreciably different from those run on the entire sample. Another limitation is the small number of subjects completing both baseline and follow-up measures, reducing the analytical sample, especially among males. A concern with assessing the effects of a behavioral intervention is unexpected change in the control group. In this study, the control groups from several studies included subjects undergoing a tobacco cessation and physical activity. Change in these behaviors may occur in parallel with change in diet, thus reducing our ability to see an intervention effect. Finally, the influence of study site on the results could not be definitively analyzed because of the small sample size and the strong relationships between site and other sociodemographic variables such as age, gender, race/ethnicity, region, and education.
A major strength of the current study was its ability to administer both PFat and multiple 24HR in participants in 4 diverse intervention sites and at 2 different time points. This particular assessment design is needed for intervention studies, as information derived from cross-sectional or baseline data is inadequate to assess the performance of self-report diet assessment instruments to measure change in dietary behaviors.
In summary, PFat was consistent with the reference 24HR, showing no significant change in percentage energy from fat from baseline to follow-up. However, further studies are needed to assess PFat's ability to reflect change in interventions where percentage energy from fat does change in response to an intervention. Because of possible performance differences across samples and between intervention and control groups, these interventions should include adequate numbers in diverse study groups. Although PFat and other short instruments may offer a tempting resource to intervention researchers, caution must be exercised in their use. In dietary intervention studies focused on reducing fat components of the diet, more extensive assessment is still warranted, if feasible. An automated self-administered 24HR is currently being developed for public use (37), which may prove useful in intervention studies with limited resources.
| FOOTNOTES |
|---|
2 Author disclosures: G C. Williams, T. G. Hurley, F. E. Thompson, D. Midthune, A. L. Yaroch, K. Resnicow, D. J. Toobert, G. W. Greene, K. Peterson, L. Nebeling, H. Patrick, J. W. Hardin, and J. R. Hebert, no conflicts of interest. ![]()
14 Abbreviations used: 24HR, 24-h dietary recalls; BCC, Behavioral Change Consortium; HSPH, Harvard School of Public Health; ME, measurement error; NCC, Nutrition Coordinating Center; NCI, National Cancer Institute; NDS-R, Nutrient Data System for Research; NWG, Nutrition Working Group; PFat, National Cancer Institute short screener assessing percentage of energy from fat; ROC, University of Rochester; URI, University of Rhode Island. ![]()
| LITERATURE CITED |
|---|
|
|
|---|
1. Tracking Healthy People 2010. Washington, DC: US Department of Health and Human Services; 2000.
2. Diabetes Prevention Program Research Group. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002;346:393–403.
3. Tuomilehto J, Lindstrom J, Eriksson JG, Valle TT, Hamalainen H, Ilanne-Parikka P, Keinanen-Kiukaanniemi S, Laakso M, Louheranta A, et al. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med. 2001;344:1343–50.
4. Swinburn BA, Metcalf PA, Ley SJ. Long-term (5-year) effects of a reduced-fat diet intervention in individuals with glucose intolerance. Diabetes Care. 2001;24:619–24.
5. Prentice RL, Caan B, Chlebowski RT, Patterson R, Kuller LH, Ockene JK, Margolis KL, Limacher MC, Manson JE, et al. Low-fat dietary pattern and risk of invasive breast cancer: the Women's Health Initiative Randomized Controlled Dietary Modification Trial. JAMA. 2006;295:629–42.
6. Chlebowski RT, Blackburn GL, Thomson CA, Nixon DW, Shapiro A, Hoy MK, Goodman MT, Giuliano AE, Karanja N, et al. Dietary fat reduction and breast cancer outcome: interim efficacy results from the Women's Intervention Nutrition Study. J Natl Cancer Inst. 2006;98:1767–76.
7. Buzdar AU. Dietary modification and risk of breast cancer. JAMA. 2006;295:691–2.
8. Dietary Guidelines for Americans 2005, 6th edition. Washington, DC: U.S. Department of Health and Human Services and U.S. Department of Agriculture; 2005.
9. Thompson FE, Subar AF. Dietary assessment methodology. In: Coulston AM, Rock CL, Monsen ER, editors. Nutrition in the prevention and treatment of disease. San Diego: Academic Press; 2001.
10. Buzzard IM, Faucett CL, Jeffery RW, McBane L, McGovern P, Baxter JS, Shapiro AC, Blackburn GL, Chlebowski RT, et al. Monitoring dietary change in a low-fat diet intervention study: advantages of using 24-hour dietary recalls vs food records. J Am Diet Assoc. 1996;96:574–9.[Medline]
11. Hebert JR, Hurley TG, Chiriboga DE, Barone J. A comparison of selected nutrient intakes derived from three diet assessment methods used in a low-fat maintenance trial. Public Health Nutr. 1998;1:207–14.[Medline]
12. Willett WC. Nutritional epidemiology. 2nd edition. Oxford: Oxford University Press; 1998.
13. Block G, Gillespie C, Rosenbaum EH, Jenson C. A rapid food screener to assess fat and fruit and vegetable intake. Am J Prev Med. 2000;18:284–8.[Medline]
14. Coates RJ, Serdula MK, Byers T, Mokdad A, Jewell S, Leonard SB, Ritenbaugh C, Newcomb P, Mares-Perlman J, et al. A brief, telephone-administered food frequency questionnaire can be useful for surveillance of dietary fat intakes. J Nutr. 1995;125:1473–83.
15. Neuhouser ML, Kristal AR, McLerran D, Patterson RE, Atkinson J. Validity of short food frequency questionnaires used in cancer chemoprevention trials: results from the Prostate Cancer Prevention Trial. Cancer Epidemiol Biomarkers Prev. 1999;8:721–5.
16. Thompson FE, Midthune D, Subar AF, Kipnis V, Kahle LL, Schatzkin A. Development and evaluation of a short instrument to estimate usual dietary intake of percentage energy from fat. J Am Diet Assoc. 2007;107:760–7.[Medline]
17. Thompson FE, Midthune D, Williams GC, Yaroch AL, Hurley TG, Resnicow K, Hebert JR, Toobert DJ, Greene GW, et al. Evaluation of a short dietary assessment instrument for percentage energy from fat in an intervention study. J Nutr. 2008;138:193S–199S.
18. Yaroch AL, Nebeling L, Thompson FE, Hurley TG, Hebert JR, Resnicow K, Greene GW, et al. Baseline design elements and sample characteristics for seven sites participating in the Nutrition Working Group of the Behavior Change Consortium. J Nutr. 2008;138:185S–192S.
19. Applied Research Program, National Cancer Institute, National Institutes of Health. Percentage Energy from Fat Screener. Available from: www.riskfactor.cancer.gov/diet/screeners/fat [accessed July 4, 2007].
20. Posner BM, Smigelski C, Duggal A, Morgan JL, Cobb J, Cupples LA. Validation of two-dimensional models for estimation of portion size in nutrition research. J Am Diet Assoc. 1992;92:738–41.[Medline]
21. Calvert C, Cade J, Barrett JH, Woodhouse A. Using cross-check questions to address the problem of mis-reporting of specific food groups on Food Frequency Questionnaires. UKWCS Steering Group. United Kingdom Women's Cohort Study Steering Group. Eur J Clin Nutr. 1997;51:708–12.[Medline]
22. Freedman LS, Carroll RJ, Wax Y. Estimating the relation between dietary intake obtained from a food frequency questionnaire and true average intake. Am J Epidemiol. 1991;134:310–20.
23. Kipnis V, Subar AF, Midthune D, Freedman LS, Ballard-Barbash R, Troiano RP, Bingham S, Schoeller DA, Schatzkin A, et al. Structure of dietary measurement error: results of the OPEN biomarker study. Am J Epidemiol. 2003;158:14–21.
24. Kleinbaum DG, Kupper LL, Muller KE. Applied regression analysis and other multivariable methods. Second Edition. Boston: PWS-KENT Publishing Company; 1988.
25. Bonate PL. Analysis of pretest-posttest designs. Boca Raton: Chapman & Hall/CRC; 2000.
26. Coates RJ, Bowen DJ, Kristal AR, Feng Z, Oberman A, Hall WD, George V, Lewis CE, Kestin M, et al. The Women's Health Trial Feasibility Study in Minority Populations: changes in dietary intakes. Am J Epidemiol. 1999;149:1104–12.
27. Thomson CA, Giuliano A, Rock CL, Ritenbaugh CK, Flatt SW, Faerber S, Newman V, Caan B, Graver E, et al. Measuring dietary change in a diet intervention trial: comparing food frequency questionnaire and dietary recalls. Am J Epidemiol. 2003;157:754–62.
28. Howard BV, Van Horn L, Hsia J, Manson JE, Stefanick ML, Wassertheil-Smoller S, Kuller LH, LaCroix AZ, Langer RD, et al. Low-fat dietary pattern and risk of cardiovascular disease: the Women's Health Initiative Randomized Controlled Dietary Modification Trial. JAMA. 2006;295:655–66.
29. Beresford SA, Johnson KC, Ritenbaugh C, Lasser NL, Snetselaar LG, Black HR, Anderson GL, Assaf AR, Bassford T, et al. Low-fat dietary pattern and risk of colorectal cancer: the Women's Health Initiative Randomized Controlled Dietary Modification Trial. JAMA. 2006;295:643–54.
30. Kristal AR, Andrilla CH, Koepsell TD, Diehr PH, Cheadle A. Dietary assessment instruments are susceptible to intervention-associated response set bias. J Am Diet Assoc. 1998;98:40–3.[Medline]
31. Hebert JR, Hurley TG, Peterson KE, Resnicow K, Thompson FE, Yaroch AL, Ehlers M, Midthune D, Williams GC, et al. Social desirability trait influences on self-reported dietary measures among diverse participants in a multicenter multiple risk factor trial. J Nutr. 2008;138:226S–234S.
32. Beresford SA, Farmer EM, Feingold L, Graves KL, Sumner SK, Baker RM. Evaluation of a self-help dietary intervention in a primary care setting. Am J Public Health. 1992;82:79–84.
33. Kristal AR, Curry SJ, Shattuck AL, Feng Z, Li S. A randomized trial of a tailored, self-help dietary intervention: the Puget Sound Eating Patterns study. Prev Med. 2000;31:380–9.[Medline]
34. Little P, Barnett J, Margetts B, Kinmonth AL, Gabbay J, Thompson R, Warm D, Warwick H, Wooton S. The validity of dietary assessment in general practice. J Epidemiol Community Health. 1999;53:165–72.[Abstract]
35. Kipnis V, Midthune D, Freedman LS, Bingham S, Schatzkin A, Subar A, Carroll RJ. Empirical evidence of correlated biases in dietary assessment instruments and its implications. Am J Epidemiol. 2001;153:394–403.
36. Hebert JR, Peterson KE, Hurley TG, Stoddard AM, Cohen N, Field AE, Sorensen G. The effect of social desirability trait on self-reported dietary measures among multi-ethnic female health center employees. Ann Epidemiol. 2001;11:417–27.[Medline]
37. Subar AF, Thompson FE, Potischman N, Forsyth BH, Buday R, Richards D, McNutt S, Hull SG, Guenther PM, et al. Formative research of a quick list for an automated self-administered 24-hour dietary recall. J Am Diet Assoc. 2007;107:1002–7.[Medline]
This article has been cited by other articles:
![]() |
S. A. A. Beresford, L. M. Klesges, and H. R. H. Rockett Introduction J. Nutr., January 1, 2008; 138(1): 183S - 184S. [Full Text] [PDF] |
||||
![]() |
A. L. Yaroch, L. Nebeling, F. E. Thompson, T. G. Hurley, J. R. Hebert, D. J. Toobert, K. Resnicow, G. W. Greene, G. C. Williams, D. L. Elliot, et al. Baseline Design Elements and Sample Characteristics for Seven Sites Participating in the Nutrition Working Group of the Behavior Change Consortium J. Nutr., January 1, 2008; 138(1): 185S - 192S. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. E. Thompson, D. Midthune, G. C. Williams, A. L. Yaroch, T. G. Hurley, K. Resnicow, J. R. Hebert, D. J. Toobert, G. W. Greene, K. Peterson, et al. Evaluation of a Short Dietary Assessment Instrument for Percentage Energy from Fat in an Intervention Study J. Nutr., January 1, 2008; 138(1): 193S - 199S. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. R. Hebert, T. G. Hurley, K. E. Peterson, K. Resnicow, F. E. Thompson, A. L. Yaroch, M. Ehlers, D. Midthune, G. C. Williams, G. W. Greene, et al. Social Desirability Trait Influences on Self-Reported Dietary Measures among Diverse Participants in a Multicenter Multiple Risk Factor Trial J. Nutr., January 1, 2008; 138(1): 226S - 234S. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||