PROBING STATISTICAL EDUCATION POLICY RESEARCH DATA

: This article argues for a particular view of advanced statistical data analysis within an eclectic methodology for Education Policy research and is predicated on the notion that the function of such analysis is to identify higher-order relationships in quantitative research data. The article outlines the application of advanced statistical analysis in two less advanced countries with the aim of promoting such analysis among researchers and decision-makers in those countries.


Introduction
When as a researcher in Education, I began to reflect on the nature of the policy process in Education, I found myself questioning whether research (in Education) probed enough the quantitative research data for Education in less advanced countries.

Policy Research
The purpose of policy research is to give policy-makers relevant information that can guide them in taking difficult decisions even if, as reported by Puryear (1995), the mentality of some government officials in less advanced countries is that "you do not need statistics or research for policy"! Also, Aid agencies need information as it plays a role in confirming and reinforcing their investment strategies (King, 1990) and users of educational services (such as parents, employers and students) are probably less willing than in the past to accept a lack of information.
Two models of the interface of policy with research are well known and are both vulnerable to much criticism (Trowler, 1998); they are: a) the Engineering model; b) the Enlightenment model.
Studies in summary form. The aim is to enhance understanding about Policy Research and Educational statistics among researchers and decision-makers (in Education).

The Research Studies
The topics for the two Research Studies that I undertook some years ago and that are summarised in this article are: • Study A: Polytechnic Teachers' Attitudes to Curriculum Innovation in South India. • Study B: Parent-Teacher Associations at Lower Basic (Primary) Schools in The Gambia. Each summary begins with the Context, the Problem of the study, and the Aim, followed by the Methodology, and the Results.
Study A illustrates the application of Factor Analysis and Study B, the application Discriminant Analysis.

The Context
The present study arose from my secondment to the Technical Teachers Training Institute (TTTI) in Madras in 1973, when I was assigned the consultancy for researching the reactions of Engineering Drawing teachers in the Polytechnics of Tamil Nadu (a state of South India) to curriculum innovation in Engineering Drawing for technician students.

The Problem Statement
From a theoretical perspective, the "problem" of the study was to account for attitudes to innovation in "formal organisations" (Blau, 1957) such as the Polytechnics of Tamil Nadu; for, in such organisations the decision to implement innovation was forced upon subordinates (such as the Polytechnic teachers) by someone in a superordinate power position (such as the Director of Technical Education through the Polytechnics' Principals). In this study, the research question then was, what were the factors that contributed to variations in the Polytechnic teachers' attitudes to curriculum innovation? When I took a theoretical view of the problem, I realised that it could be considered within the broad outline of the stimulus-response paradigm whereby a person's response in a given stimulating situation was a function of his/her Personality and of the Stimulus itself (Cattell, 1965).

The Aim
My aim then became to determine the relative 'importance' of Personality and that of relevant stimulating situational variables: that is, the relative quantitative contributions which each variable made to the variations in the teachers' attitudes to curriculum innovation. An appropriate statistical approach for this study was therefore a combination of Factor Analysis and Multiple Correlation Analysis.

The Methodology
Having conceptualised curriculum innovation as a stimulus -object (Hull et al., 1973) it followed that the perception of curriculum innovation could be considered as the process of classifying, recognising, and interpreting the diverse stimulus events, objects, persons and concepts associated with the innovation.

Further Considerations
According to Berlyne (1960), a novelty in unfamiliar stimuli could induce conflict (both perceptual and conceptual), and conflict generated uncertainty. Berlyne identified a number of reasons for conflict; among these were complexity, incongruity and ambiguity. A detailed study of the literature on curriculum innovation in schools in England in the 1970s showed that many innovatory curriculum projects tended to be characterised by complexity, incongruity and ambiguity, and the inference was that these characteristics probably gave rise to conflict and uncertainty (Owen, 1973). Taking 'complexity', for example, an OECD report (1975) had identified 'project complexity' as one of the 'internal difficulties' that loomed largest in curriculum development projects.

a. The Personality Variable
Moreover, it was asserted that each time a teacher was uncertain about what faced him/her, the teacher was "properly cautious" and such caution either "looked like resistance' or 'transformed itself into purposeful resistance" (Owen, 1973). It seemed therefore that the dynamics of teachers' reactions to curriculum innovation required that psychological mechanisms be considered. Of these, Dogmatism came first to mind because, according to Rokeach (1960), the dogmatic mind was "extremely resistant to change". I assumed that Dogmatism was a personality pattern (Vacchiano, 1968) and included it as the Personality variable in the explanatory framework for a Multiple Correlation Analysis of interrelationships between the "dependent" variable (the Teachers' attitudes to Curriculum innovation) and the "independent", explanatory Situational and Personality variables (see Figure 1).

b. The Proposition
Broadly speaking, then, the proposition was that the teachers' attitudes to curriculum innovation in Tamil Nadu would correlate significantly with Dogmatism and with each of a number of situational variables. However, to express this proposition in statistical terms required that teachers' attitudes to curriculum innovation be defined operationally.

c. Teachers' Attitudes and Perceptions
To begin with, a distinction was made between perceptions and attitudes. An individual's perception of a specific aspect of the innovation was taken to be the score that he obtained for his response to a specific statement (or to a questionnaire item) about that particular aspect. It was then assumed that his perceptions of various aspects of the innovation were mediated by different attitudes. From this point of view, an attitude was a "latent variable" (Green, 1954) or a factor that accounted for the co-variation of a number of different perceptions. Basically then, a teacher's 'perceptions' or responses concerning a number of aspects of curriculum innovation might in fact be based on a small set of mediating factors. Thus, Factor Analysis was the analytical technique used to develop measures of attitude to curriculum innovation. The important point was that measures thus derived were reliable, in the sense of being internally consistent.

d. The Situational Variables
As for the situational variables, their inclusion in the Multiple Correlation Analysis was based partly on the findings from a number of relevant preliminary studies (including a review of the theoretical issues from the relevant literature) and partly on consultations with Indian colleagues at the Madras TTTI. The variables which were finally selected as regards the Polytechnic teachers are shown in Figure 1:  The inclusion of the key variables was well justified. Thus, Crash Courses aimed at explaining the theoretical rationale for the curriculum innovation and gave practice in acquiring skills such as writing objectives, and constructing objective-type tests. Importantly too, the Polytechnics in Tamil Nadu were bureaucratic in various degrees. A scale developed at Bradford University to measure the 'experience of bureaucracy' (Musgrove, 1968) was used.

e. The 'Quasi -Illuminative Study'
It was not to be expected that all of the variations in the teachers' attitudes could be explained by the variables included in the explanatory framework. And, as Parlett and Hamilton (1972) among others, had recommended an 'illuminative' methodology for the evaluation of innovations (Trow, 1970), I decided to make what I called a 'Quasi -Illuminative study' consisting of on-site interviews with the Polytechnic teachers and of observations of their classroom behaviours.

The Data Collection Procedures
Statements obtained from various sources during the preliminary studies were gathered together and their contents studied. This study yielded some 300 statements but a number were redundant and only 74 were left to map the universe of content for the teachers' attitudes to curriculum innovation. These 74 statements made up the items of a four-part questionnaire about curriculum innovation. Subsequently, the questionnaire was split into three Sections about, respectively, (a) Curriculum innovation in general (b) the Traditional Curriculum for Engineering Drawing (c) the new TTTI Curriculum.
Access to the population of Polytechnic teachers in Tamil Nadu was through meeting them at four regional centres where it was administratively convenient for them to assemble. Eighty teachers completed the questionnaire and their responses were factor analysed at two levels (see Figure 2).
As for the 'Quasi -Illuminative' research, it was only possible to visit nine of the twenty -four Polytechnics in Tamil Nadu, these being within travelling distance of the four regional centres where the teachers were assembled.

The Data Analysis Procedures
• The Two Levels of Factor Analysis At the first level of the factor analysis, seven factors were identified, and each was represented by a cluster of salient items. Then the intercorrelations of these clusters were themselves factor analysed, and at this second level, two major orthogonal factors of teachers' attitudes to curriculum innovation were extracted (see Figure 2).

The Results
• The Results of the Two Levels of Factor Analysis Figure 3 shows the descriptive statements for the seven clusters derived from the first level of Factor Analysis.   Table 1 shows the results of the second level of the Factor Analysis. Taking FA first, judging from the high factor loadings (for C4, C5, C6), this factor seemed to be a dimension of beliefs about the new curriculum materials themselves. Teachers who scored positively on FA were likely to be those who, on the whole, favoured the new subject matter, the new techniques and so on. In addition, large classes and insufficient time at work for lesson preparation did not present problems to them. FA was a factor of 'Support for the design, content and teaching requirements of the new curriculum'. As for FB, its high loadings for C1 and C7 showed that teachers who believed that curriculum innovation should be the responsibility of Polytechnic instructors/teachers themselves and not that of other bodies (such as the TTTI) tended to believe that they needed little guidance in the preparation of teaching aids in order to implement the new curriculum materials and that the new materials gave them confidence in their teaching. Teachers who took the opposite point of view were likely to think that it was a waste of time for them to try new ideas (unless their Heads of Department had approved of them) and that they had no time for curriculum innovation. FB was a factor of 'Belief in the professional competence of teachers for initiating and implementing curriculum innovation'. Table 2 gives the results of the Multiple Correlation Analysis. It was unmistakably clear that Dogmatism was a correlate of both factors of attitude to curriculum innovation (FA and FB, respectively). The respective amounts of variance in these factors (that is, the semi-partial correlations squared) explained by Dogmatism were small: approximately 10.5 per cent for FA and 18 per cent for FB. Dogmatism -0.4214** * Significant at the five per cent level ** Significant at the one per cent level Note: No other independent variables were significantly related to either FA or FB.

• The Results of the Quasi -Illuminative Study
The study showed that probably the main source of conflict was in the management of the innovation at the institutional level. Some of the difficulties reported by the teachers in implementing the innovation were that: a) timetables had not been re-organised to allow for the change in curriculum content; b) there was not a nucleus of staff fully committed to the innovation; moreover, the bureaucratic procedures for purchasing materials for model-making were so complex and time-consuming that they tended to become a disincentive for creative thinking. An important conclusion was that there was now a need to discern the way in which resistance to change amongst teachers might be overcome, given that Dogmatism accounted for a relatively small percentage of the variation in teachers' attitudes; and a personal proposition was that a specific innovation in the curriculum could in time become attractive to teachers if they were allowed to gain a sound theoretical knowledge of the innovative ideas and practices associated with the innovation.

The Context
At the beginning of this century there were concerns in the Lower Basic (Primary) Schools (LBS) sector of the Education system in The Gambia there about the high drop-out rates in schools, particularly among girls (due to early marriage and pregnancy), about disparities in the regional distribution of schools, and about the quality of the educational programmes. However, steps had been taken to allay these concerns. Thus, there was a revision of the National Education Policy in The Gambia in the late 1980s, and, since 1992, there had been consistently a pattern of support by Donors for Primary School education as a priority. For instance, 65 percent of the externally financed Development expenditure was allocated to the construction of primary schools, the expansion of facilities for teacher training, and the in-service training of unqualified teachers (UNDP, 2000). By 1998/99, the GER had risen to 71.7 percent, and was thus slightly above the average of 69 percent for Sub-Saharan Africa. This increase notwithstanding, there remained significant differences between the enrolment of girls and that of boys: only 61 percent of girls of primary school-going age were enrolled in primary schools, as against 79 percent of boys of the same age (RoG, 1998).
Moreover, many of the documents that reviewed education in less advanced countries had highlighted several issues in school management and administration. At the same time, the empowerment of local communities was emerging as one of the most important objectives of education in the rural areas of less advanced countries (Phunyal et al, 1998). Thus, there was a growing emphasis on more community-level participation in the activities of local schools, through community involvement in, for example, local educational planning. In point of fact, an important development for the empowerment of local communities had been that Parent-Teacher Associations (PTAs) were expected to be engaged in school activities. In this connection, ACTION AID in The Gambia, as an external partner of local communities, had been acting as a catalyst for their development (AATG, 1999).

The Problem Statement
However, the establishment of PTAs for the Lower Basic Schools was not incorporated in the Education Act (RoG, 1997) -there was a question about the effectiveness of the PTAs' involvement in school affairs given that the administrative responsibility for Education was devolved to the Local Government Authorities and that District School Committees would be established as an outcome of the policy of decentralisation for educational provision. One "problem" was that there was an issue about what authority would be given to the PTAs, and to whom would they be accountable if they were to take educational decisions about, for example, the school budget, and the professional development of teachers.

The Aim
In view of the above, the aim of the present study was to examine PTAs' involvement in matters pertaining to schooling at the Lower Basic Schools level. Specifically, the aim was to scrutinise the set of relationships between PTAs' involvement in school affairs, School Management, and School Performance. Figure 1 shows diagrammatically the relationships which underpinned the researchers' conceptualisation of the study and indicate at the same time its focus.

The Methodology
Much has been written worldwide about school management and school effectiveness. Indeed, with regard to Africa, a conceptual framework had been developed by African educators and World Bank staff to assist policy-makers in Africa when addressing school practice (ADEA, 1995). The framework had identified a set of factors that are characteristic of effective schools and the interactions among the factors, but the present study concentrated on a sub-set of these interactions (namely, that between PTAs and School Management) and on how these impinged on the performance of schools. For each school, its Performance was measured by its School Achievement Rate in the Primary School Leaving Certificate (PSLC) examination in recent years, and by the Level of its pupils' achievements in the examination. However, it was acknowledged that "the crude comparison" of examination results, by the school, can lead to "hasty conclusions" (Sauvageot, 1997). b) the dotted lines indicate the relationships on which the study focused. c) School Performance was measured in this study by two indicators, namely: • the School Achievement Rate for each sampled school in the PSLC examination in recent years; • the Level of the pupils' achievements in the same examination. • level of pupils' academic achievements as indicated by the estimate of the Average Examination Mark for each school in the PSLC examinations over two years (1998 and 1999). The next step was to proceed as follows: o the public schools within each region were ranked in descending order on the basis of their Average Examination Marks. o The five schools with the highest Average Examination Mark were labelled "Top Grade schools" or simply "Top" schools, and the five schools with the lowest Average Mark were labelled "Bottom Grade schools" or simply "Bottom" schools. Importantly, however, the terms "Top" and "Bottom" were contextualised as the Average Examination Mark varied between regions. o subsequently, from each of these two groups of schools, one or two schools were selected, at random, for close study (as explained below) depending on the prevailing circumstances at the time of the research. o the same procedure was followed to select two Private schools for close study from the Urban Region of Banjul and KSMD combined.

b. The Two-way Classification of the Sampled Schools
In order to undertake the analysis of differences between the schools in the way that they were managed and that the PTAs operated, the performance of each school on both indicators was displayed together in a Two-Way Cross-Classification table. To this end, each school was classified not only as "Top" or "Bottom", but also as "High" or "Low" depending on whether its Pass Rate (averaged over the years 1998 and 1999) was above or below the National Pass Rate (which was also averaged over these two years); and the categories of the two indicators ("Top-Bottom" and "High-Low") were then crossed with each other. The cross-tabulation yielded four Second-order Performance Indicators for schools, namely, "Top-High", "Top-Low", "Bottom-High" and "Bottom-Low".

c. The Close Study at the Selected Schools
Eight researchers collected relevant data at the selected schools (from Headteachers, Teachers, Parents and Grade 6 Pupils), using self-completion Questionnaires, Observation schedules and Interview guides. Also, officials of the DoSE and of the DoLG were interviewed individually.

The Data Analysis Procedures
The data analysis consisted in: (a) comparing the PTA organisations within each Region and, similarly, comparing the management of schools (b) comparing School Performances both within Regions and between Regions (c) deriving a composite set of School and PTA variables that made a significant difference to School Performance as explained below.

a. The Analysis of Differences in School Performance between the "Top-High" and "Bottom-Low" Groups of Schools
To begin with, the combination of School Background variables, School Management variables, and PTA variables (that was important for distinguishing between the "Top-High" and the "Bottom-Low" groups of schools) was identified. For this purpose, a data matrix was constructed and labelled "derived matrix" to convey the idea that the variables included in the matrix were not only those drawn from the various research instruments used but that some were derived from arithmetical transformations on the raw data for each school (such as the Pupil-Teacher Ratio, the average National Pass Rates over the two years 1998 and 1999, and the Girl to Boy ratio in School Enrolment over the same years).

b. The Discriminant Analysis
A Discriminant Analysis was undertaken as it is a statistical technique that attempts to maximise differences between groups. The analysis yielded a "Discriminant score" (for each school) which was a linear function of all the variables within the composite set; that is, this "Discriminant score" was obtained from a weighted combination of the scores on each variable, and the weight (or coefficient) for each variable was estimated using the SPSS computer program for Discriminant Analysis.

The Results a. The Parent-Teacher Associations
Most (78%) of the PTAs were established some five years ago or so, largely through the initiatives of parents and/or school teachers. The responsibility for the organisation and the activities of the PTAs fell to their Executive Committees (ExCOs); these averaged 13 in size, were male-dominated, and the term of office for most was more than 3 years. In the rural regions, most of the parents on the ExCOs were illiterate, whilst in the urban regions, they were mostly Primary School educated. Most PTAs had their own funds as a result of parents' contributions, donations, and their own income-generating activities. Their programmes of activities were prescribed jointly by their ExCOs and their schools, and were wide-ranging, including supporting the infrastructural development of the schools. There were guidelines for PTAs in three regions but these guidelines were not legally binding.

b. School Performance (1997-2000)
During this period the regional Achievement Rate (i.e. the proportion of examination candidates in each region, each year, who obtained the "Pass" mark or higher) had increased in all the regions, whilst the differences in Achievement Rates between the regions had narrowed; however: a) the regional difference in Achievement Rate (between the urban/sub-urban regions, and the rural regions) was fairly stable over time -with the urban/suburban regions performing significantly better (statistically) than the rural regions. b) the Lower River Region performed significantly (statistically) and consistently better than the other three rural regions (i.e. the North Bank, Central River and Upper River regions).

c. The relationship between PTAs, School Management and School Performance
A set of PTA and School variables was found to make a significant difference (statistically) to School Performance. Thus, the contrast between an "Effective" school and a "Less Effective" school in terms of their performance was that the latter was likely to be characterised by: • a PTA ExCO with an above-average male membership that was involved in the maintenance of the physical plant of the school (rather than in its academic work) • a Headteacher who spent much of his/her time on official business, liaising with people in the local community • its large size (especially if the school was in an urban area) Figure 2 shows the very clear identified separation between the "Top -High" schools and the "Bottom -Low" schools. The horizontal line through zero highlights the effective separation between the "Top-High" schools and the "Bottom-Low" schools. The "Top High" schools were those above the horizontal line.
A strong message from the research findings was that, as far as the parents in the rural and urban economically poor districts were concerned, the combination of poverty and educational disadvantage was potentially a barrier to their empowerment as partners in the business of managing the Lower Basic Schools. The apparent opposition of many Headteachers to such empowerment served only to maintain the existence and pervasiveness of this barrier. Figure 3 was an attempt to portray the complexity of the set of relationships (between PTAs' operational involvement in school affairs, School Management and School Performance) which emerged from the present, empirical study. Hence, there was a challenge to determine how to galvanise the Lower Basic Schools and their PTAs so that they can respond as authentic partners to the country's need for Lower Basic Schools that are "Effective".
It was recommended that the DoSE should state clearly its policy with regard to the establishment of PTAs and appoint a Working Group to advise on the empowerment of PTAs through developing effective School-PTA partnerships in a decentralised Education System.

Conclusion
These two studies have led me to the conclusion that it is potentially worthwhile to apply such advanced statistical techniques as the above in Education policy research at the lower tiers of the Policy process because they can contribute to a deeper understanding of empirical educational research data than that obtained from so-called "social bookkeeping".

Conflict of Interest Statement
The author declares no conflicts of interest.

About the Author
Yves Benett was born and educated in Mauritius. He started his school teaching career in 1952. Benett studied in England and obtained the BSc (London) degree in the physical sciences and the Postgraduate Certificate in Education. He taught for nine years in his home country as an Education Officer. After obtaining the MA (London) degree in Education, he next worked as Lecturer, Senior Lecturer, and Principal Lecturer at the Huddersfield College of Education (Technical), and later, with a PhD in Education, got the Readership in the School of Education at the University of Huddersfield -a position he held until he retired in 1996. As an International Education consultant Dr. Benett has undertaken sponsored research projects in England and in less advanced countries and has been for many years the Scientific Adviser to the Educational Research Network for West and Central Africa (ERNWACA, Gambia). Dr. Benett has published several journal articles and books about Education.  a) the arrowheads indicate the likely directions of influence; b) the dotted lines indicate the relationships on which the study focused (see Figure 1); c) the measurement of School Performance was based on the results of the PSLC examination in recent years; d) the words in italics (within the boxes) are the likely descriptors (of the contents of the boxes) for the "Less Effective" schools the term "school resources" refers to the human, material and financial resources of the schools.

School resources (inadequate)
Headteachers' attitude to PTAs' empowerment (negative) Headteachers' time taken up with extra-curricular, school-related activities (much) PTAs' participation in the repairs of school buildings and furniture (marked)

School Performance
PTAs' involvement in schools' academic work (constrained) Educational background of parents (many are illiterate) PTAs' income-generating activities (marked) PTAs' empowerment (insufficient)