Health Education Research, Vol. 17, No. 2, 211-220,
April 2002
© 2002 Oxford University Press
Development and validation of the Day in the Life Questionnaire (DILQ) as a measure of fruit and vegetable questionnaire for 79 year olds
BHF Health Promotion Research Group, Department of Public Health and
1 ICRF General Practice Research Group, Department of Primary Health Care, Instituteof Health Sciences, Oxford OX3 7LF, UK
| Abstract |
|---|
|
|
|---|
The Day in the Life Questionnaire (DILQ) was developed as a supervised classroom exercise to measure children's consumption of fruit and vegetables. The DILQ uses words and pictures to encourage the child to recall and describe a range of activities from the previous day, including their entire food intake. This study tested the validity and reliability of the DILQ for children aged 79 years (n = 255) in four English schools. Reliability, validity and sensitivity to change were assessed through repeated rounds of data collection. Comparisons were made of observations during school breaks and classroom completion of the DILQ. Children enjoyed completing the DILQ and teachers thought it appropriate for the age group. The questionnaire performed either well or acceptably on all validity, reliability and sensitivity tests. The DILQ can be recommended as a method of collecting data for fruit and vegetable consumption from children aged 79 in the classroom. The validation study included comparison of schools with and without `fruit only' breaktime policies, and sensitivity to a brief intervention in which free fruit was distributed at morning break. The results suggest that it would be a sensitive measure for descriptive studies, before and after studies and controlled trials.
| Introduction |
|---|
|
|
|---|
Nutritional recommendations include increasing consumption of fruit and vegetables to about five portions per day (Williams, 1995
However, the evaluation of school-based interventions for primary school children is hampered by the dearth of valid, reliable and acceptable methods for gathering self-reports of dietary intake in a school setting (Rockett and Colditz, 1997
). This is particularly true for children between the ages of 7 and 9 years, who are thought too young to complete their own food frequency questionnaires (Baranowski and Domel, 1994
), yet parents may be unaware of what the child is eating during the school day.
Measurement of habitual dietary intake has a long history and a variety of techniques have been employed. These include diet diaries, where the whole diet rather than a specific food type is assessed over several days or weeks. This tends to be time consuming, and demands a certain competence of literacy and considerable motivation on the part of the subject which makes them unsuitable for most 79 year olds. The 24-h dietary recall method is a snapshot of an individual's food intake. Baranowski and Domel suggested that food information is likely to be stored in the memory along with physical activity as part of the day's events, and reported that recall was improved if an instrument was structured and had sequential questions in a segmented day (Baranowski and Domel, 1994
). Food frequency questionnaires estimate typical intake of specific foods over a particular period of time. A list of foods or food groups and a response set indicating the frequency with which the individual consumes the food (each day, week or month) is provided, and are most effective when food lists were tailored for the study population (Paisley et al., 1996
). One food frequency questionnaire for 511 year olds was identified for epidemiological use in the National Study of Health and Growth, but did not focus on fruit and vegetables (Hammond et al., 1993
). There is evidence that the 24-h recall method provides more accurate data compared with the food frequency method (Rocket and Colditz, 1997).
Validation of self-report measures with children have included comparisons with diet records/diaries and with observations. Comparing one from of self-report with another may not provide a rigorous validation, whereas comparison with adult observations utilizes independent methods. The following studies examined the capacity of 811 year olds to provide accurate data compared with observations. Lytle et al. validated 24-h recall interviews against researcher observations and concluded that 24-h recall was a valid method to assess dietary intake in 8 year olds for the purposes of group comparisons (Lytle et al., 1993
). In a subsequent study to validate fruit and vegetable intake, Lytle et al. compared observations with 24-h recalls (Lytle et al., 1998
). Overall, the correlations between observed fruit and vegetables and recall were moderate (0.420.65); however, correlational analyses do not permit estimates of agreement at the individual level.
Weighed records are suitable for use with adults (Bingham et al., 1994
), but are rarely used in validation studies with children as they are time consuming and require even more skills than maintaining a diet diary. Observations can provide accurate data and do not require children to do anything extra. Observing school meals is recognized as relatively straight-forward (Domel, 1997
), even when the children are hurried through the lunch queue (Turner et al., 1995
). Packed lunches, on the other hand, are more challenging. Their contents are variable, and often involve containers and packaging that may impair observations. In an attempt to quantify the accuracy of researchers to observe packed lunches, Simons-Morton et al. compared the contents of children's (aged 811 years) lunches and the agreement between observers (Simons-Morton et al., 1992
). They found 83.7% agreement between observers on both the contents and portion sizes for school meals. Although packed lunches are more problematic for the observer, these observations still provide acceptable data.
To aid recall in children, creative methods which use cues and prompts to add context may provide data of sufficient accuracy to be used to assess a specific aspect of children's diet. The write and draw technique (Pridmore and Bendelow, 1995
), for example, allows children to enhance their reporting by drawing their meals and presenting a pictorial memory in addition to a written or verbal one.
A recent review of dietary assessment methods has considered 47 validity and reliability studies of dietary assessment methods with school-age children (McPherson et al., 2000
). The shortage of acceptable methods for assessing fruit and vegetable consumption has encouraged the development of the Day in the Life Questionnaire (DILQ), for 79 year olds, an early draft of which was used in a pilot study of the Artie Beat 5-a-day fridge charts (Boaz and Ziebland, 1998
; Boaz et al., 1998
). The authors of the pilot study concluded that the questionnaire showed promise as a method for assessing fruit and vegetable consumption. The development and validation study, reported in this paper, was funded to assess the performance of the questionnaire as a classroom exercise to collect reliable self-reports of fruit and vegetable consumption. It was also thought important to ensure that the DILQ was both interesting and educational to complete so that teachers and pupils would be willing to take part in studies that might require repeated assessments.
| Methods |
|---|
|
|
|---|
Development and design
The DILQ has been developed as a supervised classroom exercise to assess one aspect of children's diet: the number of servings of fruit and vegetables consumed. The DILQ uses words and pictures to encourage the child to recall and describe a range of activities from the previous day, including everything they had to eat and drink (see Figure 1
|
During the development phase of the study six focus groups were conducted with children to discuss the language and question format of the questionnaire. Most children said they enjoyed completing it. The DILQ was well received by teachers who thought it appropriate for the age group. Further work with primary school teachers and designers has led to the development of additional resources for follow-up class work to support the educational component of the DILQ. The final design was produced by a professional graphic artist.
Validation study methods
A purposive sample of four schools was selected to cover different school environments and socio-economic catchment areas. Two schools were city-based and two were village-based primary schools. All children in the Year 3 and 4 cohort of classes (age group 79 years) were included in the study and the mixed ability classes ranged in size from 17 to 33 children. This range of school environments and catchment areas was included to ensure that the DILQ could be completed in a variety of settings. The same children were assessed in both rounds of data collection. Those children with data for only one of the rounds were excluded from the analyses.
At the beginning of the study, in February 1999, a workshop was organized with leading researchers in the field of fruit and vegetable research in the UK to discuss validation methods. The absence of a suitable `gold standard' for comparison presents some difficulties for a validation study. This issue was explored thoroughly in the methods workshop, and it was agreed that the research teams' detailed observations of what individual children were seen to eat and drink during school lunch and break times would be the primary reference point for the comparisons with the DILQ.
Ethics committee approval was gained for the study (N98.048) from the Central Oxford Research Ethics Committee. A letter and information sheet was sent to the parents approximately 1 week to 10 days before data collection was due to commence. None of the parents took the option of excluding their child from the observations or the DILQ completion. The observation methods were selected after a pilot study, in which different approaches were tested. Children in the class being observed were all given name badges to wear during morning break and lunchtimes. Children were aware that they were being observed, but did not know that the researchers were recording what they were seen to eat. The research team (L. E. and three other researchers) positioned themselves around the playground or canteen with clipboards on which the names of children were noted and anything they were seen to be eating or drinking was described. Because the observations were being used as the reference point for the DILQ validation, only positive sightings of foods being consumed were noted by the observers. This method seemed to work well for school lunches, but was a little more difficult for packed lunches where the contents of children's lunches could be hard to ascertain without questioning. Observers avoided getting into conversations with children about their lunches in case talking about what they were eating made the meal easier to recall when they completed the DILQ on the following day.
Data were collected in the Summer and Autumn terms of 1999. Observations were made on a Tuesday, Wednesday or Thursday and the DILQ was completed the following day in the classroom, usually after lunch to avoid interfering with literacy and numeracy hours. This procedure was carried out for each class separately and formed the first round of data collection. With one exception (school D, n = 17), each class involved in the study repeated data collection on the same week day, 2 weeks later.
Three researchers and the class teacher were present in the classroom. One researcher read out each question in turn and the class was encouraged to keep together while completing each item. Children who finished any section within the DILQ before the rest of the class had caught up were invited to illustrate their answers. The researchers roamed round the classroom, responding to requests for help with spellings, annotating drawings and, where necessary, offering carefully worded prompts to help the child to remember what they had been doing the previous day. All but a few of the children needed help with some spellings, although very few had trouble remembering what they had done and eaten the day before. The DILQ was not completed in `exam conditions' and the children were able to talk to each other while completing the questionnaire. Children were sometimes reminded by their neighbours about what they had been doing at break or where they had gone after school.
In the analysis
coefficients were used to compare counts of fruit and vegetables from children's self-reports from the DILQ and the researchers' observations, as well as the inter-rater reliability of the DILQ coding. The
coefficient is a measure of agreement between variables where the coefficient represents the number of agreements beyond chance. A
value of 1 indicates perfect agreement, a value of 0 indicates no agreement better than chance and a negative value indicates agreement that is worse than chance.
values can be interpreted using the following guidelines (Altman, 1991
). This test was used to measure the agreement between children's self-reports of their fruit and vegetable consumption from the DILQ and the researchers' observations, as well as the inter-rater reliability of the DILQ coding. Differences were examined by analyses appropriate for discrete numerical data that may be skewed. These included the WilcoxonMannWhitney tests (non-parametric equivalent of the t-test to compare means of two groups) and the KruskalWallace test (non-parametric equivalent of the ANOVA to compare means of more than two groups
Quantifying fruit and vegetable consumption
An important aspect of the validation process was to establish what counts as an instance of fruit and vegetable consumption. Epidemiological studies usually require that food intake is described in sufficient detail to identify the proportions of different nutrients in the diet. If consumption of composite foods (such as ketchup, jam, pizza, pies) is recorded, typical recipes and computer databases can be used to estimate the contribution of fruits and vegetables to the diet. However, interventions to encourage an increase in fruit and vegetable consumption do not usually include composite foods. Within the context of a fruit and vegetable intervention, these foods would contribute little to, for example, a 5-a-day goal (Cullen et al., 1999
). For the purposes of the DILQ validation the narrower definition, excluding composite foods, was adopted, similar to the approach used on the 5-a-day Power Plus Program (Eldridge et al., 1998
). No attempt was made to estimate the size of the portions.
Sensitivity to change
The DILQ's sensitivity to change was regarded as a crucial feature since it is intended for use as an outcome measure in interventions to detect changes in fruit and vegetable consumption. This presented a problem in that an intervention trial might offer the most suitable circumstance in which to detect change, but no suitable trials were running during the course of the project. Also, if an intervention was new and no change was detected, it would be uncertain whether the intervention was ineffective or the DILQ was not sensitive to change. We therefore decided to conduct a small before-and-after study with the DILQ to see if the questionnaire picked up an increase in reported fruit as a result of providing each child in a school with a piece of fruit at break. This school was not part of the main study and was in an Education Action Zone (n = 264 in school with 50% receiving free school lunches). The DILQ was administered twice, to classes in Year 4 (n = 49), on the Tuesday and Friday of one school week in the summer term. Monday was a normal school day, but on Tuesday, Wednesday and Thursday, boxes of free fruit (donated by a local supermarket) were distributed by the school administrator. While the distribution of fruit at break time might make the day unusual and more easily recalled, we hoped to reduce this effect by offering fruit on three consecutive days and repeating the DILQ after the last of these.
| Results |
|---|
|
|
|---|
In total 255 children aged 79 years from four schools took part in the validation study. All children were in mixed classes of Years 3 and 4. Table I
|
The results from each round of data collection have been presented separately, since they are repeated measures with the same children.
Comparisons were made between the 195 children for whom observations were recorded and the 80 who were not. There were few differences between the groups but less of the children who described themselves on the DILQ as having spent their break time `running around' were observed to eat anything at break. This may represent a real difference in break time eating habits or reflect the difficulty of making reliable observations of more active children
Validity of the DILQ
Convergent validity was examined by comparing the DILQ and the research teams' observations. Table II
presents comparisons for the school day since observation data was available only during school time. Children's DILQ reports of their fruit and vegetable intake approached 70% agreement with the observations.
|
Reliability of the DILQ
Testre-test reliability
Two rounds of completion for the DILQ were needed to assess reliability. In the absence of an intervention it was envisaged that each class's fruit and vegetable consumption would not differ significantly. Table III
|
Inter-rater reliability
Inter-rater reliability assessed the level of agreement between different coders, using the same strategy to count the numbers of fruit and vegetables the children reported in the DILQ. All data were compared from the first visit to School C. This school provided the largest number in children aged 79 years old and the widest range of consumption patterns. Two researchers independently identified the numbers of fruit, vegetables and totals for a sample of the DILQs (School C, Round 1; n = 83). Comparisons for fruit, vegetables and total consumption and their observation equivalents were assessed using
coefficients. Agreement ranged from 0.85 for fruit to 0.92 for vegetables, indicating very close agreement between coders.
Observed differences in consumption
Girls were consistently observed eating more fruit than boys, although this only reached statistical significance in Round 1 data. These findings appear to differ from those of the National Diet and Nutrition Survey (Gregory and Lowe, 2000
), where gender differences at 710 years showed girls eating significantly more raw and salad vegetables, but not more fruit. However, the inclusion of 10 year olds may affect comparisons as may basing the NDNS results on weighed intake rather than numbers of fruit and vegetables. Vegetable and snack consumption were not significantly different between genders.
`Fruit only' break time policy
Two of the schools (A and C) in our study had a school policy that only fruit could be consumed at break time. Analyses between the Schools (A and C) and (B and D) showed few differences between overall fruit and vegetable consumption, but showed significant differences in snack consumption (P < 0.000 for both rounds) with children in Schools A and C eating far fewer snacks. In the absence of any attempt to actively promote fruit consumption at break time, fruit only policies may simply result in very few snacks of any kind being consumed at break.
Sensitivity to change
In the single school in which the free fruit intervention was offered the children's DILQ reports indicate a baseline mean fruit consumption of 0.96 for the whole day at the Tuesday (baseline) completion and 1.43 on the Friday (follow-up) completion (P = 0.04). The change in the morning break section of the DILQ, when a piece of fruit had been distributed to each of the children, was highly significant with a change from 0.31 to 0.96 between Tuesday and Friday (P < 0.000).
| Discussion |
|---|
|
|
|---|
The development of the DILQ was designed following consultation with researchers in the field. We aimed to develop the questionnaire as a 24-h recall method that could be completed as a classroom exercise; would include draw and write elements to be attractive to children and teachers; would cover all eating and drinking during the previous day; would include a sequential and broad range of activities to structure and prompt the children's recall, while disguising the focus of the questionnaire; and be supplemented with worksheets and exercises to encourage cooperation from schools.
In this study no parents chose to exclude their children from completing the DILQ, which was conducted as a classroom exercise, with helpers available to assist with spellings and provide careful prompts. In all of the classes, regardless of size, the DILQ was always completed within a single 3040-min period. Our observations, and interviews with teachers and children, confirm that the final version of the DILQ is thought to be attractive, interesting and appropriate for children at this stage of the curriculum. The structure of the questionnaire helps children to recall what they had done the day before and only a very few were apparently unable to remember what they had eaten. The questionnaire covers a broad range of activities including TV viewing, method of travel to school, attendance at clubs, playing out of doors after school and evening activities, so it should be possible for researchers to collect information about what the children had to eat without the children being alerted to the precise focus of the study. Worksheets have been prepared to help teachers use the data in class work and are available from the researchers.
We are aware that our study has certain limitations that may cast doubt on the reliability of the questionnaire. The DILQ is intended to provide reliable counts of the number of times children have eaten fruit or vegetables during the previous day. In a validation study comparisons would usually be made against some form of `gold standard'. In the absence of a widely accepted reference point, the selection of an alternative was a central concern at the expert workshop held at the beginning of the study. The recommendation from that meeting was that the research team's observations of what individual children were seen to consume during school morning breaks and lunchtimes would provide the most appropriate comparisons for the DILQ.
Due to the pressures of the National Curriculum, children were asked to complete the questionnaire after lunch, whereas completion in the morning may have enhanced recall, particularly with children as young as 7 years (Domel, 1997
). This may partially explain their accuracy rate of around 70%. Also, the finding that many children reported consuming fruit or vegetables only once or not at all may have influenced the validation process. This is because the narrow numerical range of consumption allows less opportunity for error in the validation and reliability comparisons. The accuracy of the DILQ for populations with a high fruit and vegetable consumption remains to be established.
We would also be cautious about using the DILQ to record fruit juice consumption since children were often unsure whether their drink was a squash, a `fruit drink' or a pure fruit juice. It was also often hard for the research team to be confident about what they were observing the child drink, since many packed lunches included unidentifiable liquids transported in opaque plastic flasks. It should also be noted that, due to marketing practices, parents may be unsure whether they are giving their child a pure fruit juice. We decided that disentangling the fruit juice issue was probably beyond the scope of this project and excluded drinks from our counts.
We do not recommend that the DILQ is used as an entirely self-completion questionnaire for 79 year olds because most classes will include children who need considerable help with writing, spelling and, if they choose to draw their food, annotating the drawings.
There are advantages to be gained from taking the whole class through the questionnaire together and giving appropriate reminders and prompts (e.g. `Don't forget to describe everything you ate; Did you have any vegetables or salad with your meal; Did you have anything for pudding?') while they are all completing the same section. The provision of spaces on the questionnaire for the children to draw pictures helped to ensure that children who finished sections quickly did not get bored waiting for others to catch up. Modifications to the language used in the DILQ may be needed for non-British English.
The sensitivity study was carried out in a separate school in which a large proportion of the children reported that they had no fruit at all at the baseline. At the follow-up, after 3 days of free fruit distribution at break, only 10% of the children reported that they had had no fruit on the previous day. Further evidence of the sensitivity of the DILQ can be seen in the significant drop in portions reported on the second visit to School A, which coincided with a day when the school lunch did not include a side vegetable. We were also able to detect a difference in fruit consumption between the schools with fruit only break policies and those without. The ability to detect changes of this sort, following interventions to increase children's access to fruit and vegetables in schools, is highly relevant to current initiatives. The data provided in these tables, and in greater detail in the manual (see below), can be used for power calculations. However, researchers wanting to randomize schools to different interventions will need to take account of intra-cluster correlations and would be well advised to consult a statistician at an early stage when designing their study.
The DILQ performed either well or acceptably on all of the validity, reliability and sensitivity tests. We therefore believe that it can be recommended as a method of collecting data about children's fruit and vegetable consumption. The involvement of teachers in the development of the DILQ and supporting classroom materials has helped us to produce a 24-h recall method that even control group schools are likely to be willing to use more than once. We believe this is an important, and often overlooked, characteristic for a data collection tool for primary schools.
| Note |
|---|
Detailed advice about the process of using the DILQ and further tables are included in the DILQ manual, which is available from the authors. There is no charge for the use of the DILQ, but we would like to hear from any researchers who want to use the questionnaire to evaluate intervention studies or are considering validating it for other purposes.
| Acknowledgments |
|---|
The authors would like to thank the children and staff of the four schools that took part in the study. Many thanks to colleagues who helped with data collection: Pamela Baker, Charlie Foster, Alice Fuller, Jean Harcourt, Jeyanthi John, Sharmila Kaduskar, Sarah Norris and Sophie Petersen. We are also grateful to the workshop participants, especially members of the steering group, Annette Boaz and Sally Wyke, who were involved in the early development of the DILQ and to Hugh Ribbans who designed the DILQ. The Oxford and Anglia Region NHS Executive Research and Development Fund are also gratefully acknowledged for funding the study (grant HSR/080798/04; COREC N98.048). L. E. works in the BHF Health Promotion Research Group and S. Z. is funded by the ICRF General Practice Research Group.
| References |
|---|
|
|
|---|
Altman D. (1991) Practical Statistics for Medical Research. Chapman & Hall, London.
Baranowski, T. and Domel, S. B. (1994) A cognitive model of children's reporting of food intake. American Journal of Clinical Nutrition, 59 (Suppl.), 212S217S.
Bingham, S. A., Gill, C., Welch, A., Day, K., Cassidy, A., Khaw, K. T., Sneyd, M. J., Key, T. J. A., Roe, L. and Day, N. E. (1994) Comparison of dietary assessment methods in nutritional epidemiology: weighed records v. 24-h recalls, food-frequency questionnaires and estimated diet records. British Journal of Nutrition, 72, 619643.[ISI][Medline]
Boaz, A. and Ziebland, S. (1998) A `five-a-day' fruit and vegetable pack for primary school children. Part I: development and pre-testing. Health Education Journal, 57, 97104.
Boaz, A., Ziebland, S., Wyke, S. and Walker, J. (1998) A `five-a-day' fruit and vegetable pack for primary school children. Part II: controlled evaluation in two Scottish schools. Health Education Journal, 57, 105116.
Cullen, K. W., Baranowski, T., Baranowski, J., Hebert, D. and de-Moor, C. (1999) Behavioral or epidemiologic coding of fruit and vegetable consumption from 24-hour dietary recalls: research question guides choice. Journal of the American Dietetic Association, 99, 849518.[ISI][Medline]
Domel, S. B. (1997) Self-reports of diets: how children remember what they have eaten. American Journal of Clinical Nutrition, 65 (Suppl), 1148S1152S.
Eldridge, A. L, Smith-Warner, S. A, Lytle, L. A. and Murray, D. M. (1998) Comparison of 3 methods for counting fruits and vegetables for fourth-grade students in the Minnesota 5 a day power plus program. Journal of the American Dietetic Association, 98, 777782.[ISI][Medline]
Hammond, J., Nelson, M., Chinn, S. and Rona, R. J. (1993) Validation of a food frequency questionnaire for assessing dietary intake in a study of coronary heart disease risk factors in children. European Journal of Clinical Nutrition, 47, 242250.[ISI][Medline]
Lytle, L. A., Nichaman, M. Z., Obarzanek, E., Glovsky, E., Montgomery, D., Nicklas, T., Zive, M. and Feldman, H. (1993) Validation of 24-hour recalls assisted by food records in third-grade children. Journal of the American Dietetic Association, 93, 14311436.[ISI][Medline]
Lytle L. A., Murray D. M., Perry C. L. and Eldridge A. L. (1998) Validating fourth-grade students' self-report of dietary intake: results from the 5 a day power plus program. Journal of the American Dietetic Association, 98, 570572.[ISI][Medline]
McPherson, R. S., Hoelscher, D. M., Alexander, M., Scanlon, K. S. and Serdula M. K. (2000) Dietary assessment methods among school-aged children: validity and reliability. Preventive Medicine, 31 (Suppl. 2), S11S33.[ISI]
Gregory, J. and Lowe, S. (2000) National Diet and Nutrition Survey: Young People Aged 4 to 18 Years. The Stationary Office, London.
Paisley, C. M., Lloyd, H. M., Brown, W. and Mela, D. J. (1996) Production and validation of a food frequency questionnaire to estimate dietary fat intake. Journal of Human Nutrition and Dietetics, 9, 189206.
Pridmore P. and Bendelow G. (1995) Health images: the use of the write and draw technique with children. Health Education Journal, 54, 473488.
Rockett, H. R. H. and Colditz, G. A. (1997) Assessing diets of children and adolescents. American Journal of Clinical Nutrition, 65 (Suppl), 1116S 1122S.
Simons-Morton, B. G., Forthofer, R., Huang, I. W., Baranowski, T., Reed, D. B. and Fleishman, R. (1992) Reliability of direct observation of schoolchildren's consumption of bag lunches. Journal of the American Dietetic Association, 92, 219221.[ISI][Medline]
Turner, S., Mayall, B. and Mauthner, M. (1995) One big rush: dinner time at school. Health Education Journal, 54: 1827.
Williams C. (1995) Healthy eating: clarifying advice about fruit and vegetables. British Medical Journal, 310, 14531455.
Received on February 13, 2001; accepted on November 11, 2001
![]()
CiteULike
Connotea
Del.icio.us What's this?
This article has been cited by other articles:
![]() |
L Moore and K Tapper The impact of school fruit tuck shops and school food policies on children's fruit consumption: a cluster randomised trial of schools in deprived areas J. Epidemiol. Community Health, October 1, 2008; 62(10): 926 - 931. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

