Please feel free to e-mail this article to a friend, a principal, a parent, a colleague, a teacher librarian, a college professor, a poet, a magician, a vendor, an artist, a juggler, a student, a news reporter or to anyone else you think might enjoy it.
From Now On
|Vol 10|No 2|October|2000
Four teachers and two administrators have flown two thousand miles to attend a national conference on educational uses of new information technologies. They boarded the plane confident that they would return home armed with research findings to guide decision-making. They are united in their faith that decisions should be guided by data.
Sadly, they are disillusioned by the lack of significant research being conducted by most educational communities rushing to network schools.
"There's a big gap here," complains Sally, the veteran technology enthusiast of the team. So committed is she to the vision that she is spending evenings, weekends and summers picking up an advanced degree in educational technology. "There is so little research to help us think about our choices."
The team is camped out in an espresso Internet cafe several blocks from the Convention Center. It is the final morning of the final day, and they are having trouble mobilizing for the closing keynote.
"I'm tired of all these promises and visions and the whole 'new new thing,'" adds Gregory, the team's hardened skeptic. "I haven't found a single session where they reported sound research."
At this point, one of the administrators joins in. "Well, I did find one session - the one by Hank Becker from UCI - and his reports are a good start, but I agree that there were slim pickings, unless, of course, we include the corporation funded reports."
"I tried a few of those," laughed Sally, "and they failed to meet any of the research design standards I am expected to observe in my graduate classes. They were quite biased. Seems like we're going home empty-handed."
Answering Key QuestionsEducational research is meant to help us to explore the most important questions we have at the end of each day. In the best of worlds, research would also provide guidance for educators to make wise choices from the sometimes bewildering menu being thrust toward them by eager vendors.
Given dozens of ways to use laptops or desktop computers once they arrive in our classrooms, which uses are most productive and which professional development experiences are needed to bring about the most return on investment?
If a district hoped to improve student scores on a demanding state writing test, for example, planners would want to know which of the following would have the greatest payoffs . . .
A properly designed study of these six strategies would prove enlightening, but no credible, comprehensive, comparative studies exist on writing or any other major educational question related to the impact of new technologies.
The design of reliable educational research is beyond the scope of this article, but there are classic books, like Campbell and Stanley Experimental and Quasi-Experimental Designs for Research (1966, Houghton Mifflin College; ISBN: 0395307872) that outline the traits of a credible study. Unfortunately, many of the reports gaining national attention these days do not meet those specifications.
A recent example would be an Educational Testing Service study, "Does it Compute? The Relationship Between Educational Technology and Student Achievement in Mathematics," by Harold Wenglinsky, exploring relationships between computer experiences and math performance on the NAEP tests (National Assessment of Educational Progress). (http://www.ets.org/research/pic/technolog.html )
This study found that certain teacher strategies were associated with lower math performance while others were associated with higher performance (but only at some grade levels).
The researchers did what is called ex post facto research. Rather than setting up a study to test hypotheses with all confounding conditions controlled, they take data from a broader study and "mine" the data to find links between various events. This kind of research shows association but does not show cause.
We can see a correlation between phone ownership in African countries and lower infant mortality, but that doesn't mean we could reduce infant mortality by purchasing more phones for everybody. Phone ownership is an indicator of wealth. Wealthy countries have better inoculation programs, better nutrition and better water quality some likely causes of lower infant mortality. Using ex post facto research we could (erroneously) claim that people should buy more phones to save childrens lives!
A standard issue in research is the possibility that researchers might have a vested interest in the outcome of the study. Validity and credibility is undermined when the authors and funders are closely allied and likely to benefit if the study comes out in a manner that makes their product look good. This basic rule is frequently ignored today as companies pay for research without observing the objectivity standards and are able to win national press attention by virtue of their superior public relations resources while more credible academic research fails to gain notice.
What we need is more theory testing like the writing example provided earlier. Schools need well constructed, publicly funded, academically sound research to determine which strategies are most likely to produce the learning outcomes required to meet performance and curriculum goals. Instead, we are seeing an increase in national reports attempting to describe what is happening across the land.
Before a public opinion pollster can claim to describe behaviors and attitudes of the general public, they must discuss how their sample is representative. Most of the recent reports have failed to meet that standard, but are still quick to make generalizations. Their findings about teachers use of the Internet may be based on one district employees speculation, for example.
The New York Cybertimes of December 1, 1999 summarizes a report, "Technology in Education 1999," a Market Data Retrieval about computers in the schools.
One finding . . .
"And 54 percent of schools report that a majority of their teachers are now using the Internet in instruction." This conclusion contrasts dramatically with previous reports by Becker and others that show much smaller percentages of use by teachers in instruction.
". . . a minority of teachers had students use Web browsers during the last school year (36%) . . ." (Becker, "Internet Use by Teachers," p. 6)
"What might be called "regular" use using the World Wide Web to do research on at least 10 occasions was a practice of nearly one-quarter of all teachers with a modem in their classrooms and 30% of those with direct high-speed connections." (Becker, p. 6)
The reporter does an excellent job of adding interviews from skeptics who raise the point that the type of use is more important than the amount of access, but the 54% statistic stands pretty much unchallenged, even though it is highly speculative in its origins, being based on self reports of schools and teachers with sample sizes often quite limited and possibly skewed.
In one Market Data Retrieval report for Technology Counts99, for example, the responses were quite low . . .
An educational researcher would be expected to comment on how respondents might differ from non-respondents, but Market Data Retrieval remains silent on this important issue.
What Research is Needed
We need much more research comparing learning strategies for both teachers and students. We see little that would help us to make choices. So far the research on teachers has done little but count the hours of training. There is little differentiating between software training courses and adult learning strategies such as study groups and mentor programs. They all are lumped together.
We also need to know more about the consequences of different delivery systems and types of equipment deployment. Are we better off spreading computers evenly across all classrooms, for example, than moving them about in strategic clusters?
Most important of all, we need government funding for robust, independent research that will free schools from reliance upon research driven by marketing concerns and interests.
Credits: The photographs were shot by Jamie McKenzie.