Highlights

• Research reports were identified by searching library databases, the Internet, scientific journals, and reference lists.

• The 66 reports included in the Research Guide met at least one of the following criteria: Conducted in Canada, conducted outside Canada and the US, conducted by an acknowledged research team, studied at least 100 children, studied effects on school achievements.

• The child care factors represented in the reports were identified: type of care, age of child care entry, time spent in care, stability of care, quality of care, teacher education, adult-child ratio.

• Demographic characteristics of the studies, characteristics of the sample, and research methods were extracted and coded from each study.

• Results were represented as the relationship of each child care factor to scores on tests of children's cognition, language, and behaviour.

• Across the 66 reports there were 563 tests of the relationship of child care factors to children's development.

• The results were quantified, summed, and displayed as positive, null, or negative based on statistical significance.

The first step in the construction of CCKM’s Research Guide for Child Care Decision Making was to find and retrieve possibly-relevant research publications. Possibly-relevant reports were defined as those that empirically studied the relationship of child care conditions to children’s development. Most reports were found in peer-reviewed journals but some were found in commissioned government reports. Rather than limiting ourselves to peer-reviewed database searching, the CCKM review team cast a wider net with a more efficient and targeted strategy of iterative Reference searching.

Research reports were first identified through a search of the most prominent and relevant database of early childhood research, ERIC (Educational Resources Information Center), using the Cambridge Scientific Abstracts database platform. Resultant citations were then subjected to the Web of Science database search engine to obtain collateral reports. Internet database search engines were used to obtain national and international government reports and other position and opinion papers. Although these documents were rarely in themselves empirical studies, their lists of reference citations provided more opportunities for discovering possibly-relevant research reports. Canadian governmental and non-governmental documents helped locate research conducted in Canada but external to ERIC’s database. Using these procedures, a working reference list was created for the Research Guide.

Database searching was followed by reference searching. The reference list of each article retrieved from the databases was examined for citations. As the Research Guide’s working reference list grew, it was possible to identify two research journals that published a significant number of child care articles. They were: Child Development and Early Childhood Research Quarterly. The review team searched the Tables of Contents of 45 issues of these journals and located only 13 articles that had not already been identified. Finally, six reviews of research literature including meta-analyses and other reports were found and retrieved. The citation lists of the reviews were examined for possibly-relevant articles. Sixteen additional articles were identified and added to the Research Guide’s working reference list.

The iterative process of reference searching was continued. Each new article was retrieved and examined for additional citations. By the end of the search process, possibilities for the discovering new articles were essentially exhausted, and the search for research reports was considered complete.

Details of the steps taken to identify research reports for possible inclusion in the Research Guide are presented below.

The inclusion of citations in the working reference list was based on information contained in the titles and abstracts of the articles. The inclusion of research reports in CCKM’s Research Guide for Child Care Decision Making was determined by examining the full-text of the 146 citations.

Articles were selected if they included data on children’s development and met one or more of the following criteria.

The study was conducted in Canada.
The study was conducted outside Canada and the US.
The study was conducted by a frequently cited child-care research team.
The sample size was ≥ 100.
The design included follow up assessments at school age.

The objectives that guided these criteria included interest in data collected in Canada, interest in international research, interest in studies that had been recognised by governments and other child care researchers, studies of significant magnitude to suggest reliability of data, and studies that addressed long-term impact. These criteria reflected the idea that national context influences the research goals, and they minimized the possible influence of the large number of small-sample idiosyncratic studies conducted in the United States. Such studies by individual university researchers are subject to possible publication-bias due to the fact that many journals in which child care research reports typically reside (e.g., Child Development, Developmental Psychology, etc.) are owned and operated in the United States, and may be more likely to publish small-scale US reports. We sought an international perspective of research on child care factors and their impact on children’s development. Our strategies were consonant with the Research Guide’s objective of providing a landscape of research findings for child care decision making in Canada. 

Details of the steps taken to include research reports in the Research Guide are presented below.

The 66 research articles were coded in terms of one or more of the seven child care factors they addressed: teacher education, adult-child ratio, time spent in care, age of entry, type of care, stability of care, and quality of care. An article could contain more than one study and the research could address more than one factor.
 

Child Care Factors
Teacher Education	Adult-Child Ratio	Time Spent in Care	Age of Entry
Caregivers’ years of education and/or amount of specific child care training.	Adult-child ratio and class size results are  grouped under the same factor heading. Higher adult-child ratio and smaller class size are on the same end of the continuum.	Number of hours, days, weeks, months, or years in a child care setting.	Age at which children entered child care.
Type of Care	Stability of Care	Quality of Care
Primary child care setting: parental in-home care, non-parental child care in home setting, or centre setting, etc.	Number of different caregiving arrangements a child experienced.	Definitions of quality of child care differed across studies. For example, quality of care was defined as sensitive caregiving, and/or as physical and structural aspects of care. Researchers used various standard assessment scales of child care quality or created their own measurement scales.

Information from each article was systematically extracted and entered in a Microsoft Excel database by two trained research assistants. The following information was extracted from each study in each article.
 

Demographic Characteristics of Studies
Citation	Year	Country	Age	Age Group
Author(s), year, title, journal, volume pages	Publication year	Location of data collection	Years & months of children’s ages at time of assessment	Infancy, Preschool, or School-age at time of assessment

 

Characteristics of Sample
Jurisdiction of Sample	Sample Size	Databases & Measures	Type of Child Care
Municipality, provincial, state, inter-provincial/state, national, international	Number of children who participated in the study.	Database source if applicable; measures of children’s development	Parental home care Care in non-parental family home setting with non-relative, relative, other Non-parental centre care Other

 

Characteristics of Methods
Time of Child Care Data Collection	Design of Study	Time of Assessments	Control Variables
Prospective or retrospective collection of data concerning factors of child care and children's development	Observational or  experimental (including method of participant assignment and experimental variables)	Contemporary with child care (No Follow-up); longitudinal (Follow-up)	Characteristics of family and child that were measured and accounted for either by selection of groups (e.g., low-income vs high income) or by statistical procedures

Results relating to the association of child care factors with child development outcomes were counted as positive, negative, or zero relative to their direction and whether they met the statistical convention of p ≤.05. A result was given a value of 1 if the effect was found for the composite measure. For example: If a positive relationship of Teacher Education to the summary score of the Peabody Picture Vocabulary Test was reported, a count of 1 was entered as a ‘+’ in the Language Category cell of the row for the article on the Teacher Education spreadsheet.

Each positive, negative, or zero result found for a subscale of an assessment tool was assigned a fractional (proportion of 1) value of the assessment scale. For example: If a positive result for Teacher Education and the Woodcock-Johnson Tests of Cognition on the letter-identification subscale was reported and a null results was found for the memory sentences subscale, .5 was added to the ‘+’ column, and .5 is added to the ‘0’ column. 

Subscales of tests were given fractional scores for two reasons. Studies in which children happened to be measured in finer detail (many subscales) would not overly influence the summary of results for the factor studied. After all, any one report might be unique in other ways (country in which it was conducted, regulations concerning parental leave, characteristics and quality of research methods). By using fractional scores for subtests and confining the overall test to a value of 1, multiple testing procedures did not give any one report undue sway over the conclusions. Secondly, differences in results across subscales of the same test should not be ignored. Fractional scores allowed the results of the subtests to be accurately credited as +, 0, or -. The disadvantage of this procedure was cosmetic in that the final tally of test results was not always an integer (see results for the impact of type of care on tests of language development).

CCKM prepared the Research Guide as a quantitative and qualitative, easy to understand summary of findings within and across the 66 reports. The Research Guide is meant for parents, practitioners, policy analysts, and others who want to consider accumulated research findings. The preferences of stakeholders, as well as the complexities and vagaries of the study designs and analyses, did not advocate for meta-analyses of the data.

Nevertheless, the methods used in the Research Guide were not seen as a substitute for meta-analyses. Indeed, the citations and where possible full-text of reports are available in the Research Guide for interested readers who want to conduct meta-analyses for the seven factors and three categories of child development. Such analyses will require their own compromises in calculating and synthesising effect sizes, but will add to the compendium of knowledge that can inform decisions about the work-time care of young children. Confidence in conclusions was provided by the large sample sizes and the many test results obtained in the reports.

Citations and descriptions of all database surveys of children’s development relating to child care (e.g., NLSCY) identified in the reports were recorded. Assessment scales and other tools used in the 66 articles to measure the characteristics, development, and behaviour of children and families were identified and described. Assessment scales and other tools used to measure the construct of child care quality were also identified and described. Citations of the 66 articles reviewed in the Research Guide and citations from which the included articles were drawn as well as  other references cited in the Background text of the Research Guide were listed.

The spreadsheets were used to construct CCKM's Research Guide for Child Care Decision Making. Information from the spreadsheets and the full text of the reports was drawn to complete the Summary of Evidence.

Scorecards for each child care factors were constructed by compiling data derived from the Microsoft Excel file. The citations for each research reports that address each factor were annotated with tables that listed  the country of data collection, the sample size and age group of the children in the study, the large-scale database survey from which the data were drawn if applicable, and the methods of the study. Published abstracts were added to the annotated citations, and full-text files were linked. Finally, the Resources sections of the Research Guide were compiled from the spreadsheets.