Evaluation2

CAREER CONNECTIONS TO TEACHING WITH TECHNOLOGY (CCTT)
EXTERNAL EVALUATION, YEAR 2
Karen C. Cohen
Karen C. Cohen and Associates
9 Cliff Road
Weston, MA 02493

The research reported herein was supported by the U.S. Department of Education, Grant Award #R303A970001. The opinions and findings are those of the evaluator and do not necessarily reflect those of the U. S. Department of Education.

Acknowledgements

This report is the result of the input of over a thousand students, hundreds of teachers, and scores of staff members and administrators involved with the CCTT Project (Career Connection to Teaching with Technology) in the states of California, Florida, Georgia, Kansas, Nebraska, and Nevada, each state containing a Hub Site and participating in the consortium of curriculum and technological developers and implementers. The specific schools involved are: A-Tech Academy, Las Vegas, Nevada; Sprayberry High School, Marietta, GA; Mainland High School, Volusia County, FL; Omaha North High School, Omaha, NE; and Fort Leavenworth Unified School District 207, Fort Leavenworth, KS. Several people helped with the evaluation data analysis and reporting, and I am specifically appreciative for their efforts and results: Marcus Lieberman, Cathy Thibeault, and Al Liberman. Finally, the conceptual and financial support of the US Department of Education Technology Innovation Challenge Grant Program has been crucial to the development and implementation of the entire program including the evaluation and research efforts related to improving and documenting its attainments.

This ambitious project will require continuing input from all of the people mentioned above, and more. We look forward to continuing work both to provide formative feedback to the project and to document attainments and results in the next three years.

May, 1999

Summary

In its second year, the CCTT (Career Connection to Teaching with Technology) Project has developed both a framework and process for creating standards-based, electronically delivered curriculum materials as well as a number of lessons or units at all grade levels and in all subject matter areas of the project. This cohesive consortium draws on the talents of six geographically dispersed Hub Sites (California, Florida, Georgia, Kansas, Nebraska, and Nevada) as well as nationally recognized consultants in education. Sharing a mutually developed set of operational beliefs and a mutually created development and review process, the standards-based, electronically delivered curriculum units under development are tested first at each originating hub site and are then available for review and testing by the other sites as well.

The evaluation of the CCTT Project has focused on:

(1) summative evaluation or research strategies for pre and post project assessment
(2) formative feedback to the project.

Summative evaluation activities include student and teacher surveys, the baseline results of which are reported here, and plans for testing achievement at the end of the project related to the standards-based lessons it develops.

The student survey covers:

- experience with and current use of computers in school,
- experience with and current use of computers at home,
- amount of time spent with computers,
- nature of the time spent,
- nature of computer-related activities in general,
- nature of educational activities involved,
- subject-matter areas where such time is spent,
- software and related hardware considerations, and
- attitudes toward computers.

We also developed and implemented a teacher survey (administered near the very start of the project and which will be re-administered as the project matures and disseminates. The survey derived many components from instruments recommended by the US Department of Education. More specifically, it covers:

I. Background Information
II. Use of Technology
III. Availability of Hardware and Resources
IV. Effects of the Project
V. Demographics

At the end of the project we will also attempt to study impacts of the project materials on achievement scores. Comparisons of these data with other general state or national norms will be done when possible.

Formative evaluation and feedback to the project has depended primarily on participant observation of annual and semi-annual project meetings, focus groups, Hub Site visits, telephone interviews and on-line data collection from different levels of participants. Summaries of these activities are also presented.

The current (Year 2) report presents full baseline data from both the student and teacher surveys collected as well as the instruments developed and implemented for the project. We present here:

1.0 Introduction and Methodology--Evaluation of Project Objectives
2.0 Results of Student Survey (Detailed Findings)
3.0 Results of Teacher Survey (Detailed Findings)
High School
Elementary and Middle School
4.0 Results of Participant Observation, Site Visits, and Focus Group Findings
5.0 Conclusions and Plans for the Future

The Appendix includes samples of the student and teacher surveys used.

1.0 Introduction and Methodology--Evaluation of Project Objectives

The original Grant Proposal for the CCTT (Career Connection to Teaching with Technology) Project focused on four primary evaluation objectives. We list them briefly here and summarize the work to date and work planned for the project. The reader should note that in our preliminary report, (May 15, 1998), developed near the end of the first year of project operation, we modified and expanded assessment of many of these objectives in view of the resources available, the project�s national geographic dispersion, and the five-year time-frame involved. We update these plans, activities and results here.

1. Identify and incorporate into curriculum development the national reading, writing, mathematics, and science achievement standards, benchmarks and accountability measures for students.

The Project has developed an elaborate process for standards-based curriculum development and review, utilizing subject-matter content and standards experts in the country. This effort is organized and coordinated by TIES, an educational consulting firm in Minnesota, but involves a national network of consultants to curriculum or materials development. The process is iterative, supportive, and continuing in each subject-matter or content area arid by definition, each designated hub site. The annual project report contains a schematic of this process as does the Web Site.

Actual measurement of students� gain in these subject matter areas will involve using standardized student scores of participating students at the end of year four and beginning of year five of the project in specific subject matter areas where such data collection is possible. At this point the Hub Sites and content specialty areas are:

Using a "medical model�, where each student serves as his/her own control, we will assess content or achievement in subject-matter areas and compare that with achievement in years 4 and 5 of the project against scores recorded three-four years prior where school records allow and exposure warrants such work. Taking such data now, with turnover as high as over 90 percent in some participating school districts annually could prove to be a waste of time and effort. In addition, we will track group results and means to have data (baseline collected this year) about student use of technology (primarily computers and related software) in a variety of contexts and for a variety of educational purposes to assess changes in the nature of such activities over a similar time span. The results of the baseline data are reported and discussed below. It will be possible to use the instruments developed and used for baseline information in these school settings with county and state-wide samples of students in some or all of the six participating states to help inform national norms as well.

2. Create a career connection to integrated core academic studies that increases the relevance and authenticity of learning.

This second evaluative objective, again, involves collecting baseline data regarding the number of faculty and students involved, and the diffusion and dissemination of the project results as the project develops. Each Hub Site is responsible for tracking and maintaining this information under the general direction of the Project Leadership

3. Train teachers and students to access and utilize existing technologies, to create original instructional materials, and to collaborate with business partners in co-development of resources.

The external evaluator is working with the project to document involvement and training of teachers (see survey findings) and to review developing curriculum products. In addition to the general survey, there are also site-specific evaluation reactions collected through interviews, focus groups, and on-site surveys that provide a continuing stream of formative feedback to the Project Director and Hub Site managers.

With six Hub Sites spread across the entire country, it is the responsibility of the Project Director and of the Hub Site Managers to co-develop resources with business partners in their own areas. The Volusia County, FL leadership has formally convened a national consortium which meets regularly to work with the project to attain these goals not only in Florida but in areas served by the entire project.

4. Contribute to national educational net-works using telecommunications to disseminate products and best practice.

The evaluator(s) feel that there are two major impact areas for this objective where evaluation efforts will contribute to the CCTT project and, in turn, augment the entire TICG US Department of Education effort:

(1) National dissemination through documenting project impacts and results and publicizing these efforts is a vital activity. New efforts, possibly not envisioned at the start of this program, make publicizing and delivering the products of this grant using the technological resources of the Internet, intranets, and other modern technological advances quite easy. The project has worked long and hard to develop, code, and work with a system to locate and identify electronically standards-based lessons.

(2) Documenting and publicizing the results. impacts. and effectiveness of these materials involves additional effort--an effort which can in part be done by the consortium members as they implement and utilize each others� lessons and curriculum units. It is hoped, here, that careful studies of the use of the technologies in the subject matter areas covered by the project and used by thousands of students in the involved sites in comparison with those of students without benefit of such materials may demonstrate not only the effectiveness of this project per se, but the impact of this entire approach to development and dissemination of technologically based and delivered curricula across the country in general. Presentations at national meetings, publications in appropriate Journals, and collaboration with other evaluators across projects will help attain these dissemination goals.

As the project has evolved, the evaluation has evolved better to serve its needs as well as those of the US Department of Education. Assessing the attainment of the goals and objectives of the CCTT Grant requires measuring the delivery of curriculum materials to the educational community through integrated technologies. To do so, however, involves developing baseline data or information about the current use of those technologies by students both in school and at home and in a variety of subject matter or curriculum areas. To understand change and the rate of change and implementation, it was important to develop some kind of yardstick to assess teacher and student use of technology in the curriculum and in general.

We, therefore, developed a student survey and a teacher survey. The student survey was piloted and then used with a sample of the students in the six sites in close collaboration with the Hub Site managers and a variety of consultants. This survey covers many of the areas of concern selected for possible national assessment by consultants to the the Department of Education TICG program, but goes into greater depth. The survey covers:

- experience with and current use of computers in school,
- experience with and current use of computers at home,
- amount of time spent with computers,
- nature of the time spent,
- nature of computer-related activities in general,
- nature of educational activities involved,
- subject-matter areas where such time is spent,
- software and related hardware considerations, and
- attitudes toward computers.

The survey takes 3-8 minutes and was easily handled by some students as young as first and third grade and by all middle and high school students. Students of teachers involved in the project during the third period class of the first week in March, 1999 were tested. This sample was selected to attain maximum attendance in school and avoid replication of survey-taking by students throughout the day as they worked with different teachers. The specific directions are in the Appendix along with the two- page survey form. The findings are presented below. Beyond survey efforts, there was participant observation of mid-year and annual project meetings, site visits to each Hub Site, and two focus groups with participating teachers early in the project. Findings from these activities are used in the interpretation of the project results and summaries.

We also developed and implemented a teacher survey (administered near the very start of the project and which will he administered again at the end.) The survey derived many components from recommended instruments in the sourcebook* developed for the US Department of Education.

More specifically, it covers:

I. Academic/Career Background Information
II. Use of Technology
III. Availability of Hardware and Resources
IV. Effects of the Project
V. Demographics

We have administered this survey to all teachers involved at this point with the CCTT Challenge Grant as well as with the entire group of teachers in the same schools who are not involved with the program for comparison purposes, although at the inception of the project all of the teachers in each Hub Site were considered as not yet affiliated with the project or program.

It is anticipated that comparing current findings or results from the student and teacher surveys with those from students and teachers 4-5 years later in the project will yield valuable information and documentation of results. For the participating teachers, impacts of participation as well as curriculum unit results will be measured. For the students, not only attainments, but possible comparisons with students in their respective counties and states may yield valuable information not only for each site, but when aggregated for the US Department of Education through creating and/or informing norms. We could document general effects and work toward sorting out which effects may be due to the project specifically and those which may simply be due to the general proliferation of computational resources over a four to five-year time span.

• Bodilly. S. and Mitchell, K.J., �Evaluating Challenge Grants for Technology in Education: A Sourcebook,� (1997) RAND. Santa Monica. CA

2.0 Results of Student Survey

The results of the student survey are presented in this report. For each question there is a table showing the counts, percentages and valid percentages. The valid percentage is the percentage of those who answered the question. The percentages may not add to exactly one hundred due to rounding.

The Sample

Table 1 gives the number of surveys returned by site. It should be noted here that the findings are based on results from the six schools in the project. These schools, selected for their computational excellence, are not necessarily representative of the counties or states in which they are located. The sample, however, is appropriate for the CCTT project.

copy starts here

end copy here

The distribution of students by grade is given in Table 2. Eighty-five percent of the respondents are in high school. Five of the six Hub Sites are high schools. One represents K-8.

The frequency distribution of student respondents by age is given in Table 3. The mean age of the students was 15.3 years.

The gender of the responding students is shown in Table 4. There is an even split between boys and girls.

The remaining tables in this section present responses divided by three grade levels: elementary (grades one through six), junior high school (grades seven through nine) and high school (grades ten through twelve). The majority of students were in grades 9-12.

Table 5 gives the frequency distribution of students who have ever used a computer. A very high percentage of students at all three levels have used a computer.

Table 5
Have you ever used a computer?
Grade Count Percent Yes
Elementary 35 97.2
Junior High 398 96.8
High School 518 98.9

Table 6 gives the average number of years the students have used a computer. The number of years increases with age, but not in a linear fashion. There is comparatively more computer use for younger students, even discounting potentially a-typical results at the elementary level. This finding appears in other studies using this instrument.

Table 6
If yes., for how many years?
Grade Count Average
Elementary 35 5.400
Junior High 349 5.690
High School 437 6.780

Table 7 shows the distribution of students who wrote that they use a computer at school. A very high percentage at each level report using a computer at school. The six Hub Site Schools are individually recognized for their leadership in implementing computational resources for education. It would be interesting to compare these figures with national or state norms, resources permitting.

Table 7
Do you use a computer at school?
Grade Count Percent Yes
Elementary 34 94.4
Junior High 397 92.4
High School 520 -.- 91.0

Table 8 gives the average number of hours per week that the students reported using the computer at school. Junior high students may he learning how to use the computer, while high school students are actually using it for work, when required. This finding may also parallel the trend noted in Table 6 above.

Table 8
If yes, how many hours per week on average?
Grade Count Average
Elementary 33 1.939
Junior High 328 6.082
High School 422 4.374

Table 9 shows the distribution of students who report using the computer in various courses. At the elementary level, most students used computers in English, Earth Science and Mathematics. In junior high and high school, most students use computers in English class. At the-high school level, we find English, Biology, and Foreign Languages to be the predominant subject matter in-school uses.

Table 9
In which classes do you use a computer?
Counts (and percentages)
Elementary Junior High High School
Biology I ( 2.7) 102 (24.5) 149 (28.3)
Physics 0 ( 0.0) 20 ( 4.8) 93 (17.7)
English 34 (91.9) 279 (67.1) 311 (59.1)
For. Lang. 2 ( 5.4) 90 (21.6) 100 (19.0)
Business 0 ( 0.0) 50 (12.0) 90 (17.0)
Chemistry 1 ( 2.7) 6 ( 1 .4) 66 (12.5)
Earth SCI. 33 (89.2) 71(17.1) 52 ( 9.9)
Math 34 (91.9) 109 (26.2) 84 (16.0)
Soc. Stud. 1 ( 2.7) 95 (22.8) 85 (16.2)
Voc. Educ. 1 ( 2.7) 70 (16.8) 97 (18.4)

Table 10 gives the distribution of students who use a computer at home. There appears to be an inverse relationship between age and use of computers at home, although the percentages at all grade levels are high.

Table 10
Do you use a computer at home?
Grade Count Percent Yes
Elementary 35 97.2
Junior High 320 80.2
High School 394 75.5

Table 11 shows the average number of hours per week students use a computer at home. Junior high students use the computer at home slightly more hours per week than high school students do, and elementary students use the computer the least number of hours per week.

Table 11
If yes, how many hours per week on average?
Grade Count Average
Elementary 32 4.031
Junior high 301 10.342
High School 364 8.698

Table 12 shows the number (and percentage) of students who describe how using the computer affects the difficulty of homework. In all three grade levels, at least half feel that the computer has made doing homework easier, again with greater impact at the middle or junior high school level than the high school level.

Table 12
Has the computer made doing homework...?
Easier About Harder
The Same
Elementary 11 (50.0) 8 (36.4) 3 (13.6)
Junior High 250 (72.3) 85 (24.6) 11 ( 3.2)
High School 282 (62.7) 164 (36.4) 4 ( 0.9)

Table 13 shows the distribution of students who like using the computer. Here, as well as in the previous questions very high proportions of students enjoy using the computer.

Table 13
Do you LIKE using the computer?
Grade Count Percent Yes
Elementary 35 97.2
Junior High 355 91.7
High School 466 91.7

Table 14 gives the distribution of students who receive help during computer use. At the elementary level, most students receive help from a parent. At the junior high arid high school levels, most students get help from a teacher.

Table 14
Who helps you use the computer?
Parent Sibling Teacher Adult Friend Other
Elem. 26 (70.3) 14 (37.8) 18 (48.6) 11(29.7) 11(29.7) 5 (13.5)
J.H. 175 (42.1) 83 (20.0) 239 (57,5) 68 (16.3) 187 (45.0) 56 (13.5)
H.S. 167 (31.7) 139 (2&4) 283 (53.8) 110 (20.9) 240 (456) 94 (17.9)

Table 15 shows the distribution of students who have used a computer in a variety of ways. Most students at all levels currently use the computer for games, the World Wide Web and research or information.

Table 15
Check all the ways you have used a computer.
Counts (and percentages)
Elementary Junior High High School
Games 35 (94.6) 379 (91.1) 489 (93.0)
Word Proc. 31 (83.8) 330 (79.3) 467 (88.8)
E-Mail 26 (70.3) 294 (70.7) 339 (64.4)
WWW/Int. 35 (94.6) 351 (84.4) 466 (88.6)
Res./Info 34 (91.9) 352 (84.6) 464 (88.2)
Educ. Soft. 30 (81.1) 204 (49.0) 266 (50.6)
ChatRoom 17 (45.9) 236 (56.7) 315 (59.9)
Spreadsheet 2 ( 5.4) 189 (45.4) 243 (46.2)
Art/Graph 29 (78.4) 296 (71.2) 369 (70.2)
Program 15 (40.5) 147 (35.3) 155 (29.5)
Graph.Calc. 8 (21.6) 111 (26.7) 182 (34.6)
Caic. Labs 2 ( 5.4) 78 (18.8) 135 (25.7)
Cmp.Probes 1 ( 2.7) 26 ( 6.3) 58 (11.0)
VirtualReal 7 (18.9) 78 (18.8) 83 (15.8)
GIS Model 1 ( 2.7) 25 ( 6.0) 25 ( 4.8)

Table 1 6 gives the distribution of students who have used various devices with a computer. Printers and CD-ROM devices were used by the largest proportions of students.

Table 16
Which of the following devices have you used with a computer?
Counts (and percentages)
Elementary Junior High High School

Table 17 gives the distribution of students� ease of use fbr two platforms. All three levels of students report that PC�s are easier to use than Macintosh computers.

Table 17
Which type of computer do you find easier to use? Elementary Junior High High School
Macintosh
PC Clone
Neither/DK
1 ( 2.8)
19 (52.8)
16 (44.4)
62 (16.3)
164 (43.0)
155 (40.7)
139 (27.3)
205 (40.3)
165 (32.4)
Printer
36
(97.3)
379
(91.1)
508
(96.6)
Scanner
18
(48.6)
228
(54.8)
296
(56.3)
Projector
17
(45.9)
101
(24.3)
132
(25.1)
CD-ROM
34
(91.9)
353
(84.9)
466
(88.6)

Table 18 shows the distribution of students who started on the two platforms. Slightly more elementary students began on PC�s and more junior high and high school students began using the Macintosh.

Table 18
When you started to use a computer in school did you use...?
Elementary Junior High High School
Macintosh 10 (27.8) 173 (45.4) 245 (47.9)
PC Clone 14 (38.9) 55(14.4) 98(19.2)
Neither/DK 12 (33.3) 153 (40.2) 168 (32.9)

Open Ended Questions

There were three open-ended questions in the survey. The first asked if students liked using computers (only one did not) and why. Although a variety of reasons were given, a content analysis indicated that most students liked computers for games. They also liked computers to make things neater and help with their homework.

The things they liked best about the computer were:

- games,
- access to the Internet, and
- easy to use.

The things they liked worst about computers were:

- having to type,
- crashes or freezes, and
- loss of work (�doesn�t save).

3.0 Results of Teacher Survey

3.1 High School Level

Teachers in the Career Connection to Teaching with Technology Project responded to a survey regarding their use of technology both in the classroom and in the home. Five of the six hub sites are high schools, and we received a completed total of 363 surveys from these five sites. These surveys were administered to most of the teachers in the five hub site schools represented, not only those currently involved with creating materials for the Project (a far smaller number). Administration varied from hub site to hub site during this earliest organizational phase of the project, but the baseline interests and findings were sufficiently covered to allow for later comparison. In some sites, all teachers at an initial Orientation Meeting were given surveys and expected to return them at the same meeting. At others, surveys were left in mailboxes with expectations for voluntary returns. Although there were varying rates of response from one hub site to another, there were few if any significant differences between the sites, and the results in general are reported here. An analysis was done searching for significant differences between or among sites, and few appeared. They were, however, reported internally to the project.

For each of the questions, a summary statement will be presented which will include the substance of the question. A copy of the survey is included in the appendices. Question numbers refer to survey question numbers. In addition, the actual data tables, sub-divided by site, hut including totals in the margin are also available for site-specific feedback.

I. Academic/Professional Background and Training

Q1. 53% of the respondents indicated that they had Masters degrees or higher.

Q2. The average year that respondents received their highest degree was 1986.

Q3. 35% of the teachers had their degrees in a single subject, 23% had their degree in general secondary education and 22% had their degree in multiple subjects.

Q4. The average number of years teaching was 14.2 years.

Q5. 35% of the teachers had already participated in the project for one year.

Q6. 77% of the teachers had less than 31 hours of technology-related professional development.

II. Use of Technology

Q7. The average number of hours per week teachers spent in the tech center was 2.7.

Q8. 60% indicated that they had video equipment available to them as part of their regular classroom program. Of those who did, the average numbers of hours per week they used that video equipment was 7.7. 63% wrote that they had computer equipment available to them as part of their regular classroom program. Of those who did, the average number of hours per week they used the computer was 11.4. 20% checked that they had laser disc equipment available to them as part of their regular classroom program. Of those who did, the average number of hours per week using that equipment was 7.4.7% indicated that they had other technology equipment available to them as part of their regular classroom program. Of those who did, the average number of hours per week that the equipment was used was 12.9.

Q9. The average number of hours per month participating in project meetings and in-service was 3.8. The average number of hours per month planning or developing instructional activities or curriculum units was 12.8. The average number of hours per month becoming familiar with technology on their own was 9.1. The average number of hours per month helping students in the use of� equipment during non-class time was 4.1. The average number of hours per month doing other project-related activities was 5.2.

Q10. The most cited use of computers in the classroom or tech center was the World Wide Web (53%), followed by drill and practice programs (42%). The least mentioned use was for advanced authoring applications (19%).

Q11. 62% of the teachers reported being a frequent user of technology over 3 years.

Q12. 35% of the teachers indicated that they are provided with regular out-of-class preparation time for learning and integrating technology into their classroom.

Q13. 66% of the teachers felt they were either at the Entry and Adoption (28�/a) or Adaptation (39%) level of technology skill.

Q14. The teachers were evenly divided between the upper two levels and the lower two levels in their classroom�s use of technology, 43% at the Irregular, Individual level and 8% at the Regular, Individual level. Similarly, 22% were at the Irregular Group Use level and 27% were at the Regular, Individual and Group Use level.

Q15. The average percentage of students who use a computer daily in school was estimated at 55.0. The average percentage of students who use a computer daily outside of school was estimated at 48.3.

Q16. The average percentage of teachers who use a computer daily in school was 70.2. The average percentage of teachers who use a computer daily outside of school was 56.6. The percentage of time that was un-reimbursed, non-school time was 60.0.

III. Availability of Hardware and Resources

Q17. 3 1% of the teachers report a student-to-computer ratio lower than 5:1, and 25% reported a ratio greater than 25:1.

Q18. 72% reported having an inkjet or laser printer in most classrooms.

Q19. 53% indicated that they had full-time in-school technical support.

Q20. 33% wrote that their Internet connection was in the library or computer lab, 34% indicated that less than 50% had classroom connections and 33% checked that more than 50% had classroom connection to the Internet.

IV. Effects of the Project (primarily for post-project use)

Q21. On a scale where �1� = �low� and �5� = �high�. teachers felt that they had been most effectively trained in; use of equipment (3.21/5), next in use of software (2.1 8/5) and least in curriculum development (2.95/5).

Q22. Teachers wrote that the greatest amount of follow-up and assistance was in: becoming familiar with and learning about technology (3.02/5) and integrating the use of technology with instruction (3.0 1/5). The least amount of follow-up and assistance was thought to be in; training students in equipment usage (2.76/5) and planning and developing curriculum units and activities (283/5). The use of technology in instruction fell in the middle (2.92/5).

Q23. Only 2% of the teachers have shared their products and ideas with others at conferences, 4�/a at in-service training sessions and 3% at informal presentations.

Q25. The area where the teachers felt that participating in the project had the most influence on students� school-related attitudes and behaviors was increased interest in technology (3.4 1/5). The areas where the teachers felt the project had the least influence were improved confidence as a learner (3.2 1/5) and improved student-teacher rapport (3.22/5).

Q26. Other areas where the teachers wrote that the project was most effective were motivating teachers to use technology in their classrooms (3.37/5) and making technology available in their classrooms (3.34/5). Areas where the teachers felt the project was least effective were communication with team members (2.95/5) and involving teachers in project decision making (2.98/5).

Q27. Teachers also rated the extent to which their involvement with the project would affect several areas. The area judged most affected was improving overall teaching effectiveness (3.77/5) followed by changing instructional methods (3.69). The area least affected by project involvement was changing their perceptions about their students� learning abilities (3.42/5).

Q28. Teachers rated the extent to which they agreed or disagreed with a number of statements about their teaching. A low rating (1-2) indicates agreement arid a high rating (4-5) indicates disagreement. The most negative ratings were given for �The project is helping me develop professionally� (2.88/5), followed by �I feel a lot of stress at work.� (2.80/5). The items with the most positive ratings were �My principal provides strong support for my teaching� (2.30/5) and �Administrators at my school generally support the project� (2.40/5).

3.2 Elementary and Middle School

One participating Hub Site has no high school students, but represents all subject matter areas K-8. Rather than combining their teachers� responses with those from the other five hub sites (all of whom were represented by high schools), their responses are presented separately here. 58 surveys are included.

1. Academic/Professional Background and Training

Q1. Degree
The responding sample is about evenly divided between those with bachelor�s degrees and master�s degrees.

Q2. Year of Degree
Half of the respondents received their degrees since 1990.

Q3. Credentials
Just over half of the respondents had their teaching credentials in elementary education and just under a fifth were in secondary education.

Q4. Experience
Just over half had ten or more years of teaching experience. Nearly a fifth had twenty or more years.

Q5. Project Participation
All but one of the teachers had participated in the project for one year.

Q6. Technology related professional development.
One third of the teachers had more than 50 hours of professional development in technology-related subjects.

II. Use of Technology

Q7. Hours per week in the tech center this past year
Three quarters of the teachers had two or less hours per week in the tech center.

Q8. Available technology
Sixty percent of the teachers had video equipment available to them and three-quarters of them used that equipment one to two hours per week. Over ninety percent had computer equipment available to them and half of those used the equipment five or more hours per week. Fourteen percent had laser disc equipment available to them and half of those used the equipment four or more hours per week.

Q9. Hours per month in project-related activities
Half of the teachers spent four or more hours per month participating in project meetings and in-services. Half of the teachers spent ten or more hours per month planning and developing instructional activities or curriculum units. Just over half spent five or more hours per month becoming familiar with technology on their own. Just under half spent two or more hours per month helping students in the use of equipment during non-class time.

Q10. As part of the curriculum what do most students use?
Just over half of the teachers reported most students use drill and practice programs. About seventy percent of the teachers reported most students use basic authoring applications. About one third of the teachers answered that most students used advanced authoring applications. Just over forty percent of the teachers indicated that most students use simulation software. About sixty percent of the teachers reported most students used CDROM research resources. About sixty percent of the teachers indicated that most students used the World Wide Web. About forty percent of the teachers answered that most students used network communications.

Q11. How long a frequent user of technology?
Almost seventy percent have been a frequent user of technology for more than three years.

Q12. Out of class preparation time available for learning and integrating technology into the curriculum? About half reported that preparation time was available to them.

Q13. Level of technology skill
About sixty percent of the teachers are at the Appropriation or Invention level.

Q14. Pattern of student technology skill
About three quarters of the teachers report their students at the higher two levels.

Q 15. Percent of students who use a computer daily
Half of the teachers report that more than one-third of their students use a computer in school daily. Half of the teachers report than more than forty percent of their students use a computer outside school daily.

Q16. Percent of teachers who use a computer daily
Half of the teachers reported that more than -sixty percent of the teachers in their school use a computer in school daily. Just over half the teachers reported that more than forty percent of the teachers in their school use a computer outside of school daily. Over half reported that ninety or more percent of this time was un-reimbursed non-school time.

III. Availability of Hardware and Resources

Q17. Student to computer ratio in the classroom
Thirty percent reported a student to computer ratio less than 10 to 1.

Q18. Printers available -
Four fifths have inkjet or lasers available to them.

Q19. Technical maintenance
Just under half have full-time, in-school technical support.

Q20. Connection to the Internet
half of the teachers report that Internet access is available in the library or computer lab.

IV. Effects of the Project (primarily for later comparative use)

Q21. Perception of training effectiveness
The teachers felt that they were trained most effectively in curriculum development, less effectively in use of equipment and least effectively in use of software, although the mean ratings ranged from 3.03 down to 2.55 out of 5.

Q22. Follow-up and assistance
Teachers reported receiving the most follow-up and assistance in planning and developing curriculum units and activities and the least assistance in the area of training students in equipment usage. The mean ratings ranged from 3.10 to 2.57.

Q23. Shared your products and ideas with others.
About a fifth reported sharing products and ideas at conferences one to four times. A quarter of the teachers reported sharing products and ideas at in- service training sessions from one to six times. Thirty percent reported sharing products and ideas at informal presentations from one to ten times.

Q25. Impact on students� school-related attitudes and behaviors
All of the mean ratings were more than 3.0, ranging from 3.0 to 3.44, indicating some impact on students� attitudes and behaviors. The most impact was for increased interest in technology and the least impact was for increased liking of school.

Q26. Effectiveness of project
All the areas but one were rated with means higher than 3.0, ranging from 4.24 down to 3.53. The highest rated area was communication with team members, followed by motivating teachers to use technology in their classrooms. The lowest rated area was operation of your tech center, the only mean rating less than 3.0, 2.64.

Q27. Impact of project involvement
Teachers rated these items very high, with mean ratings ranging from 3.71 to 4.39. The highest mean rating was for improving overall teaching effectiveness and the �lowest� mean rating was for changing perceptions of students� learning abilities.

Q28. General perceptions
The mean ratings here ranged from 2.49 to 3.00, indicating a slightly positive response to the various questions. One question was worded negatively, 28d, "I feel a lot of stress at work.� That item was inversely recoded resulting in a mean score of 3.07, indicating a very slight disagreement with the statement. The highest mean rating (2.49) was for enthusiasm about participation in the project and the lowest rating (3.00) was for three items, administrative support, principal support and satisfaction with work environment.

4.0 Participant Observation Findings--Project Meetings and Site Visits Since the last Evaluation Report (September, 1998)

erence
to its levels and stages. This process will be available to others on the CCTT Website.

Incorporate standards-based considerations, subject-matter expertise, in-house review and revision, the CCTT process will be a template for not only this project, but for assessing results of other curriculum development efforts as well. The ability of the group to test and revise materials in six different sites or locations throughout the country may prove to be a tremendous advantage in the ultimate adoption and use of these units. The �not invented here� syndrome may be diminished, and the general and generalizable utility of the materials established.

The Hub Site visits took place at various points in the Project�s development. Focus groups with teachers were conducted at two sites, hut the timing was so early in the project�s history, that the results were meager in terms of specific expectations for the project or reactions to it. What was of a great deal of interest, however, was the individual and very high caliber of all of the six sites involved. We learned late this year that the project deliberately selected six disparate �Schools of Excellence� -- schools receiving national recognition and/or awards -- for this venture to help assure its quality and success. And this selection probably will. Attempting to generalize from the experience of these six schools or Hub Sites and extrapolate from the lessons developed may be more of a challenge.

It is hoped that funding will allow comparisons of teacher and student survey data in these sites as well as effects of the materials developed on achievement (in-class and in-unit tests as well as standardized subject- matter scores) with appropriate samples of students not only in the same counties as these singular academic institutions, but--perhaps--in the six states involved: Florida, Georgia, Kansas, California, Nebraska, and Nevada. The cohesive CCTT group is fully capable of undertaking such research and is ideally situated to do so. Not to do so would probably be an ultimate disservice to the special program underway.

5.0 Conclusions and Plans for the Future

The CCTT Project has made major organizational headway in the past year, and lessons and units are under development and review in a collaborative process. Its products include not only curriculum units in a variety of subject matter areas, but a documented development arid review process for assuring quality and control of these products. The process could be of benefit in establishing criteria for acceptability for units not developed by the project itself. It is anticipated that the evaluation will continue to track student and teacher involvement and impact, focus on teacher enhancement, and commence tracking outreach efforts in this coming third year. the Baseline data are collected (with the thankful help of all of the Hub Site managers), participating teachers, and students, and in Years 4 and 5 we will start to collect impact data using the same instruments and developing new approaches appropriate to providing formative and summative feedback to the project as well. Subject-matter achievement data will be collected, when possible, at the end of the project. Site visits, focus groups, on-line surveys, participant observation, interviews, and on-line querying will all be used to further the evaluation goals and objectives of the project and provide both formative and summative service to it.

We also anticipate at least two research study reports relating to the general impact of technology on the students and teachers in this program with appropriate dissemination at professional society meetings and through juried journals.