Vol 5 . . . No 4 . . . December, 1995
"Did anybody learn anything?"
The Sad and Sorry State of Technology Program Assessment
In September of 1992, From Now On explored this topic in considerable detail and depth. Go to fulltext of that issue. At that time it was clear that little was being done to measure or report student gains. That finding holds true three years later.Failure to Make the Connection:
Before turning specifically to the status of program assessment, we need to confront the evidence that fifteen years after the attempted infusion of technology into schools, we have failed to make much progress. The federal OTA (Office of Technology Assessment) recently published a disheartening report describing the huge gap between the promise and the reality of technology use by teachers. Key findings are quoted below:
Integrating Technologies into Classroom Learning
(Go by Web for full text of report - http://otabbs.ota.gov/T128/)
TEACHERS AND TECHNOLOGY: MAKING THE CONNECTION
SUMMARY OF KEY FINDINGS
o Projections suggest that by spring 1995, schools in the United States will have 5.8 million computers for use in instruction--about one for every nine students. Almost every school in the country has at least one television and videocassette recorder, and 41 percent of teachers have a TV in their classrooms. Only one teacher in eight has a telephone in class and less than 1 percent have access to voice mail. Classroom access to newer technologies like CD- ROM and networking capabilities are also limited. While 75 percent of public schools have access to some kind of computer network, and 35 percent of public schools have access to the Internet, only 3 percent of instructional rooms (classrooms, labs, and media centers) are connected to the Internet.o Despite technologies available in schools, a substantial number of teachers report little or no use of computers for instruction. Their use of other technologies also varies considerably.
o While technology is not a panacea for all educational ills, today's technologies are essential tools of the teaching trade. To use these tools well, teachers need visions of the technologies' potential, opportunities to apply them, training and just-in-time support, and time to experiment. Only then can teachers be informed and fearless in their use of new technologies.
o Using technology can change the way teachers teach. Some teachers use technology in traditional "teacher-centered" ways, such as drill and practice for mastery of basic skills, or to supplement teacher-controlled activities. On the other hand, some teachers use technology to support more student-centered approaches to instruction, so that students can conduct their own scientific inquiries and engage in collaborative activities while the teacher assumes the role of facilitator or coach. Teachers who fall into the latter group are among the most enthusiastic technology users, because technology is particularly suited to support this kind of instruction.
Lack of Actual Evaluation Reports
As I discovered when I did the research for the 1992 issue on this topic, very little actual evaluation of technology is being reported each year. ERIC has published fewer than 40 articles each year since 1980, and the trend has persisted since 1992.Number of Evaluation of K-12
Educational Technology Articles
Reported by ERIC Each Year 1980-92
1980 - 8Perhaps 10 per cent of the reports cited share actual student performance data. Most are anecdotal or testimonial program evaluations.
1981 - 16
1982 - 10
1983 - 29
1984 - 25
1985 - 22
1986 - 32
1987 - 27
1988 - 27
1989 - 27
1990 - 29
1991 - 36
1992 - 13 - ERIC search results
1993 - 9 - ERIC search results
1994 - 11 - ERIC search results
1995 - 2 (incomplete year)
The abstracts from two 1994 exceptions to this tendency are quoted below. Both New York City and Austin, Texas report no significant gains from their (expensive) ILS systems.
ED379305
Instructional Technology in AISD, 1993-94. Publication Number 93.06.
Curry, Janice; Sabatino, Melissa
-ABSTRACT- During the 1993-94 school year, the Office of Research and Evaluation of the Austin Independent School District (AISD) (Texas) conducted a districtwide evaluation of instructional technology. The evaluation consisted first of an accurate count of all computers in AISD schools, and then of an in-depth evaluation of the integrated learning systems of the Computer Curriculum Corporation (CCC) and Jostens Learning. The over 11,000 computers in the Austin schools are more than twice the amount present 3 years ago. Of these, 39% are considered old. This amounts to six students for every one computer in the district. Gains in student achievement have not been significant enough to declare either of the integrated learning systems effective, but the gains made at some schools warrant their continued use.ED381133
Educational Systems Integrators/Integrated Learning System Project: Titan Schools 1993-94. OER Report.
-ABSTRACT-
The 1993-94 Integrated Learning System (ILS) project, a means of delivering individualized instruction through a computer network, involved approximately 70 schools from New York City school districts. To help schools learn about and operate the technology in an ILS, districts were given the option of hiring one of the following companies (referred to as education systems integrators): Instructional Systems Inc., Jostens, the Waterford Institute, and Titan. Of the four integrators, Titan elected to have the Office of Educational Research (OER) evaluate its program. Titan, who was chosen as integrator by six schools, contracted with Computer Networking Specialists (CNS) on Long Island to perform the integration services, and with the Waterford Institute to provide teacher training. Two of the six schools were part of the grantback phase and the other four were in the capital phase of the project. Problems resulting from the asbestos crisis in New York City public schools and delayed deliveries and installations affected both phases of the project, but especially the capital phase. Half of the schools were very satisfied with the teacher training they received, while the other half voiced dissatisfaction with the initial training. Opinions about the software programs were mixed; one area of dissatisfaction was the schools' involvement in decision making about the ILS project. Student achievement scores showed no significant differences in reading between program participants and the rest-of-school population. Recommendations include: reexamine teacher training; clarify the roles of CNS and Waterford; and consider how the program expects schools to integrate the use of the ILS lab.
Do we have a case of the Emperor's New Clothes?
The Sad and Sorry State of Technology Program Assessment---Hypotheses for the Sad and Sorry State---Why Bother? What's the Pay-Off?---The Centrality of Clear Goals and Outcome Statements---Assessment for Navigation---Self-Assessment Instruments---Performance Assessment Instruments---When all is said and done---Resources
Copyright Policy: Materials published in From Now On may be duplicated only in hard copy format for educational, non-profit school district use only. All other uses, transmissions and duplications are prohibited unless permission is granted expressly. Showing these pages remotely through frames is not permitted.
FNO is applying for formal copyright registration for articles.