A look at questionnaire development and administration...
Introduction and Design
A. This study aims to discover the perception of administrators, teachers, and Grade 11 students of Technical and Vocational Education. It also seeks to ascertain the importance and adequacy of Technical and Vocational Education in preparing Grade 11
Students for trade extension and for the job world.
The research makes three assumptions:-
1. That the ultimate aim of Technical and Vocational course is to produce skilled workers to perform effectively in their chosen field.
2. That quality training , practical examples, interactive strategies are important if the Grade 11 students are to become competent professionals and
3. That measuring the perceptions of administrators, teachers, and grade 11 students of Technical and Vocational Education (TaVE) is a means of determining how these respondents think of TaVE in providing for individual development in ensuring that all students acquire the basic competencies necessary for later employment and for further studies.
B. The review of literature will focus on research related to attitudes to Technical and Vocational Educational and academic subjects.
Attitude Towards Technical and Vocational Education and Academic Subjects
Many people in Jamaica are of the view that the only good form of Secondary education is one that focuses mainly on the so-called "academic subjects". Many parents and wards in Jamaica especially in the urban areas have expressed their feelings of disappointment; some have even vented their feelings of anger when their own children or children entrusted to their care earned a place in the Technical or Comprehensive High School. It is the perception of some of these parents and some administrators that Technical and Vocational Education is less prestigious than the academics. As a result they insist that children whose talent lies mainly in technical or practical work should abandon training in these pursuits, substituting purely academic education for which they are unsuitable. UNESCO REPORT (1992)
The UNSECO Report (1992) highlights the division between the academic and the Technical and Vocational Education, with the former being perceived to be "better and more prestigious than the latter". In many high schools in Jamaica, many children who are labelled "bright" are placed in academic streams and "dull" children in the Technical and Vocational streams. According to the UNESCO Report (1992) such attitudes create real problems and are essentially divisive and unproductive. The perception that Technical and Vocational Education and training is for weak students, give rise to negative attitudes amongst parents, students, teachers and administrators.
Peters 1990, Professor of Philosophy at the University of London, posits that a key requirement for development in modern technological age is a population that is well educated and trained in Science and Technology, and capable of being readily mobilized to meet the changes in technology. Thus, the prerequisite for training in Technical an Vocational Education should be one which helps students to develop their capacity to learn, to think critically, to adjust to rapid changes in Technology, and to gain some understanding of their later working environment, UNESCO Report (1992).
Despite the UNESCO Report of the perquisite for training in Technical and Vocational Education many educators and administrators still hold firmly to their perception that only the academically weak students should be put in Technical and Vocational Education programs. It is worthwhile to note that lack of the basic skills of literacy and numeracy and not being able to think critically raised serious concerns for the Technical Vocational Education training programs as in any other education program.
Findings from research indicate that many candidates for Technical and Vocational Education and training are unable to read and perform elementary mathematical operations, yet in many cases in the Jamaica school system this is the principal rationale for their being placed in Technical and Vocational Education and training programs.
Powell (1998) & Peters (1990).
According to Peters (1990), entry requirements to access Technical and Vocational Education should be, in addition to basic skills on literacy and numeracy continuing education at the Primary level such as concepts, principles, skills and attitude etc.
In developing countries such as Jamaica, the critical importance of education at the Primary level cannot be over emphasized as education at this level provides the adequate foundation for activities at the Secondary level which in most instances is paid scant regard, UNESCO Report (1992). It is therefore, the responsibility of the relevant authorities such as the Ministry of education to pay urgent attention to the specific analyses and interventions to improve and strengthen education provisions at this level i.e. the Primary level.
A. The writer developed the survey to determine what the respondents perceive or
feel about the theme: Technical and Vocational Education. The Likert scale was then selected as the most appropriate scale for the participants to respond to a series of statements by indicating, whether they strongly agree (SA) agree (A) undecided (U) disagree (D) or strongly disagree (SD) with each statement. Each response is associated with a point value and an individual's score is determined by summing the point values of each statement. For example, the following point values are typically applied to statements: SA = 5, A = 4, U =3, D = 2, SD = 1.
Relevant literature on the theme was then reviewed, The writer then looked at what was to be measured and the methods most suitable. Items were then formulated and letters were drafted for the instrument. This was sent then sent to the instructor for a review. The instructor advised, for items to be grouped under each construct. Changes were then made to the survey plan after which it was then given to two content experts to conduct validity testing before it was pilot tested
Two Instruments were used: One for grade 11 students and the other for Teachers and Administrators. The instrument for Grade 11 students has 16 items and uses the 5 point Likert scale and the level interval measurement. The instrument for Teachers and Administrators has 20 items and uses the 5 point Likert scale and the interval level measurement.
The items that the writer included were designed both for students and for teachers and administrators. For students the writer used constructs such as: field of choice, items 1-5, curriculum, items 6 - 9, students perceptions of the teachers in the field, items 10 - 13, and availability of materials and equipment, items 14 - 16.
For teachers and administrators, the writer used constructs such as: personal data, items 1 -5, impression of the students, items 6 - 7, impression of the teachers and administrators, items 8 - 12, the program, items 13 - 17, and availability of materials and equipment, items 18 - 20.
B. Instrument Validity
Instrument validity is the degree to which the instrument measures what it is suppose to measure. The type of validity used by the writer was content validity. Two experts in the content area were given the instrument for them to give expert judgement. The two experts are curriculum developers in Technical and Vocational Education. They are senior advisers in the field. Specific questions were asked of the experts related to wording of the items, readability, the understanding, clarity, appropriateness and organization of the instrument. The experts were also requested to give any criticism relating to validity of both the items relating to validity of both items and the cover letter.
The experts gave some useful comments as to the wording of the cover letter to yield better effect and for the letter to be more persuasive. One expert disagreed with the wording of item 21 on the instrument, for teachers and administrators. Hence the writer omitted the item. However both experts accepted the constructs.
Another method of validity testing that could be used to measure the validity of the instruments for this survey is criterion - validity. The writer would administer two tests using the same constructs but different items. The second test is the criterion against which the validity of the first test is judged. This method is chosen against the background that it involves an explicit standard against which claims about the test can be made.
Values undoubtedly play a very important role in any survey development. Test scores pose value - laden consequences. These consequences need to be checked to determine if the inferences and the way scores are used to generalize or make decisions are valid for that particular use. The scores can have positive as well as negative impact.
C. Pilot Test
The instruments were pilot tested after the content experts had checked the validity of the instruments. The pilot was done with three Grade 11 students for the Student Survey Instrument and with three teachers for the Teachers and Administrators Survey Instrument.
The writer visited two institutions and got permission from the Heads of Department of the Technical and Vocational Sections. A letter was given to a teacher of Technical and Vocational Education at each school. The writer also got permission from the teachers to ask Grade 11 students in the field of Technical Education to volunteer. A permission slip was sent to the parents of the students selected. The writer also sent a cover letter and instrument to an administrator in the field of Technical and Vocational Education. The cover letter in each case outlined ethical issues. Respondents were asked to complete the questionnaire and make criticisms and /or recommendations for improving the instrument. The respondents returned the completed instrument and the writer thanked each respondent for volunteering. One volunteer called the writer seeking clarification on two of the items. That showed how important it is to be careful with the wording of the items. Hence the usefulness of pilot testing an instrument.
In summarizing the feedback of the pilot survey it can be stated that all the respondents returned the questionnaires. Changes were made to the questions that needed clarification. Changes were made relating to the wording of certain items.
Reliability is the degree to which a test consistently measures whatever it suppose to measuring. The more reliable a test is the more confidence we can have that the scores obtained are essentially the same scores that would be obtained if the tests were re-administered to the same test takers. The first step for this approach is to administer the test to the sample then, the same test will be given to the sample again. The next step is to correlate the scores, and calculate the correlation coefficient.
Two methods which could be used to test instrument reliability are: stability and coefficient of Equivalence. These two methods could be used with the survey instruments.
Stability (test - retest) is widely used in educational surveys. This is the stability of the scores over time. Here the writer could give a group the same instrument at two different times and correlate the two scores.
The coefficient of equivalence could also be used as it facilitates two forms of the test to the sample and then the scores are correlated. This method would be appropriate, as it would show how researchers arrive at similar results using the same procedure in two forms. Thus the close relationship between individual scores on two parallel versions of the test. This makes it a form that could be used to assess the reliability of the survey.
Analysis and Conclusions
The writer would use descriptive statistics to analyse the data. Descriptive statistics would permit the writer to meaningfully describe many pieces of data with a few indices. The major types of descriptive statistics are measures of central tendency, measures of variability, and measures of relationships. Measures of central tendency would be used to determine the typical or average score of the set of scores, for example, mean, median, and mode. The measures of variability would be used to indicate how the set of scores is spread out. The measures of relationship would be used to indicate the degree to which two sets of scores are related.
For all the items in the Students Survey, items 1-16, the writer would use frequency distribution as this will supply the number of times each variable occurs. Central tendency would also be employed so as to determine the average score. For the Teachers/Administrators Survey, the writer would use frequency distribution on items 1-20 so as to determine the number of times each variable occurs. For items 6-20, the writer would use independent T-test to be used in determining whether there is a difference between means for males and females. The ANOVA would be used for items 6-20, to show the significant difference that might exist at the various levels of: age group, years of service or qualification.
Limitations in data analysis imposed by the student survey instrument design, was the absence of items to gather more demographic information as to: age, sex, and years spent in specialized field.
Strengths and Weaknesses of the Survey
This study has contributed to a better understanding of how to develop survey items. The feedback was quite helpful for further assessment. The arrangement of constructs helped to simplify the aforementioned analysis. One of the underlying weaknesses was that when the items was being developed no consideration was given to the analysis plan. Hence an item with a statement in favor of Technical Education may have a point-value of five, whereas an item not in favor of Technical Education may have a point-value of five. The main strength of the survey was that the items were short and simple. There was at most five items under each construct. If the survey instrument were to be developed again the writer would design an analysis plan. All the items on the upper level of the scale would be in favor of the theme. The writer would remove the construct and see how well the items stood by themselves. This would provide for a better set of items. The lesson learnt is, it is quite time consuming to write an excellent instrument without much practice.
Peters, R. E. (1990). Skills on literacy and numeracy. Technical and vocational education.Torch, 20 (2), 22-28.
Powell, J., & Peters, R. (1998). Technical and vocational education training programs. Countries and future challenges in prospects. 19 (2), 205-224.
UNESCO, (1992). World education report 1992. Technical and vocational education.