Vol 1 . . . No 6 . . . June, 1991
Editor: Jamie McKenzie, Ed.D.


Measuring Results: What happens to student writing with the word processor?"
District Technology Assessment Form

Measuring Results: What happens to student writing with the word processor?

by Jamie McKenzie

A survey of research describing the impact of word processing on student writing often turns up confounding or disappointing results. One Canadian study (Owston, 1990) reported in this month's literature search concluded:

(1) the computer-written work was significantly better in overall quality and better on the competency and mechanics subscales of the evaluation instrument; (2) students produced significantly longer pieces of writing on the computer than off; (3) students reported very positive attitudes toward computer-based editing and writing; (4) there were no macrostructural differences in writing across media; and (5) only one surface feature, spelling, was found to be significantly better in the computer-written work.

In this particular study, the eighth grade students had already been using word processing for a year and a half. We learn that they prefer the computer and do somewhat better work when they use one, but we don't learn how their writing might have changed for the better because of their year and a half.

There are several important issues which deserve attention from those who design writing programs utilizing new technologies, and several evaluation agendas deserve greater attention.

Issue #1 - Has the school, the department or the district identified in explicit terms why and how the word processor might radically alter the quality of student writing?

Like many other new technologies, the word processor is easily bent into the shadow of the old technology by prevailing attitudes and habits. In far too many computer labs, the word processor is employed like a glorified electronic typewriter. In such classrooms, the students are often blocked from composing early drafts and pre-writing on the word processor. They spend tedious moments transferring drafts onto the computer, frequently without the benefit of keyboarding skills.

All too often, the culture of past practice persists even with the new technology. Thus we find that teachers who once restricted their written comments and corrections to issues of grammar and mechanics may well maintain that same practice with the word processor, emphasizing correctness rather than coherence, conviction and clarity.

There was a time, not so long ago, when students in most writing classes first created an outline, then a draft in pencil and finally a draft with pen or typewriter. Students rarely invested in revisions after receiving teacher comments, and it was often considered unwise to vary from the thinking which surfaced in the original outline. That first thinking was to control the flow of ideas much as steel rails are meant to keep a locomotive on track.

During the 1970s, writing as process pointed to a different view of writing, one which permitted a far longer period for incubation of ideas and thoughts, one which emphasized multiple versions, flexibility, audience, non-linear thinking and peer review. A basic tenet of this approach was the possibility that the best route to a good paper was not a straight line. In this approach, a writer was more like a gardener than a railroad engineer.

Combined with the word processor, writing as process offers the prospect of idea processing. The word processor provides greater fluidity and flexibility than other writing technologies. It supports greater word play and association. The writer can try out dozens, even hundreds of variations until the resulting product is just right. The word processor actually makes thinking more powerful, as long as students are taught how to use it in that way.

Unfortunately, the word processor works little magic by itself. If the mechanics-driven approach to writing still dominates a school and department, the word processor will do little more than improve the appearance and mechanics of student writing.

Issue #2 - Is the staff prepared to employ the word processor as an idea processor?

Unless teachers have had personal experience with the ways that word processors may alter the processing of ideas, it is unlikely that they will know how to show the way to their students. Through no fault of their own, they may have grown up with the railroad school of writing instruction and know little about circular thinking. They may know little about fertilizer and cultivation.

If a district expects word processing to work wonders, staff development in some form of writing as process is of paramount importance. That staff development should be conducted with word processors to demonstrate how ideas may be constructed, broken down, reconstructed, elaborated, extended, condensed and amplified. As teachers of writing see their own thinking pass through stages of development in the fluidity of the screen, they will grasp the challenge of today's writing teacher - how best to lead students through similar stages of development.

As with so many other issues facing schools today, idea processing requires a paradigm shift. Strong writing thrives when minds are free to toy with many possibilities. This kind of word play is most likely to emerge when the writer or thinker works with fragments rather than completed sentences. Those taught in the railroad school of writing often feel compelled to spin out sentence after sentence in a logical flow without having to retract words or circle back. This effort placed tremendous pressure upon the thinker and often produced "writer's block," a condition of high anxiety preventing words from coming to mind.

In writing as process, fragments are collected as beautiful beads which might later be strung in some ordered sequences. There is no early pressure for order and logic. The emphasis is upon richness. The goal is to gather as many impressions, thoughts and insights as possible without feeling constrained by critical and analytical judgments.

One excellent resource to guide a school, a district or a department toward the kinds of writing programs which take advantage of the word processor as an idea processor is the "Macintosh Writing Resource Guide" (1990) available from Apple Computer. This publication provides an extensive profile of schools which are conducting such programs as well as a listing of articles, books, consultants and materials which emphasize writing as process.

Issue #3: Do students have sufficient access to computers so they can develop their ideas appropriately over time?

Unfortunately, because access to computer labs can be somewhat limited, the ability to employ the word processor effectively as an idea processor may be all too related to home ownership of a computer. For ideas to gather, pass through an incubation period and then receive the kind of careful attention they deserve, the student really needs many hours spread out over time. It is difficult to perform such writing during 30-40 minute segments once each day or several times each week.

Concern for equity suggests the importance of providing sustained access to computers at times other than English class. Some schools have provided portables for students to take home. Others have kept their computer labs open outside the normal school schedule. Some times this has meant evening hours. Other times it is a matter of adding afternoon hours. In any case, periodic surveys of students will uncover relationships between patterns of use and home ownership. If these surveys show that there is a big difference in who uses the computer for writing, the school has a responsibility to close the gap.

Issue #4: Is form more important than substance? Is less more?

Many studies point to the fluency with which students express themselves on the word processor. After decades of complaining that students seemed almost constipated when it came to writing, one of the first victories of the word processor was the release of student words in torrents which filled page after electronic page.

Unfortunately, verbosity is not a desirable outcome. Expansive expression is fine as an early stage of idea processing, but we should be showing our students how to write concise prose which moves quickly and strongly to the point. We should show them how the word processor can support evaluation of their first thoughts so the young writers may increase clarity and impact. What we need is a major commitment to teaching students how to revise material.

"How long must it be?"

"Write an essay of at least 1000 words . . ."

We must reconsider the messages we have expressed for much of this century. The job of writing is complete when the ideas have been expressed with clarity and impact. We should stop confusing quantity with quality. Too many students take our length requirements as a challenge to fill blank space, not recognizing that excess words provide a kind of blankness not unlike white paper. We would be better off expressing expectations in terms of other criteria. Request strong evidence to bolster arguments. Underline the importance of carefully chosen words. Stress the value of sentences which flow from one to another with a clarity that supports the essay's argument. Communicate your belief that less is often more. Challenge them to distil their thinking. No wine before its time!

The focus of this article is upon expository writing, but other forms would deserve their own series of questions and strategies.

"But I have read through, corrected all the spelling and changed anything which sounded awkward!"

How often our students complain that there is little more to change in their writing. This is most likely a sign that they need an expanded toolkit of editing strategies and questions. To be effective editors of their own work, they must be skilled at self-evaluation. They must be good questioners. They must analyze their first drafts to see where the sentence structure might be improved, where the logic might be strengthened, where the word choice might be more compelling, where the clarity might be sharpened and where the perspective might be deepened. They must learn to employ peers as audience, reflecting upon their reactions to early drafts before turning to a final version.

Analysis and evaluation point the way to synthesis, rearrangement of elements to create a better version. Students can learn to SCAMPER, employ a series of synthesis strategies developed by Robert Eberle. Each letter of SCAMPER stands for a strategy. S = substitute. C = combine. A = adapt. M = modify, minify, magnify. P = put to other uses. E = eliminate. R = reverse. When students are hard-pressed to see how things might be altered, teach them to SCAMPER on the words and sentences of their early drafts.

To reach its full potential, editing must be reconceptualized across the curriculum to emphasize its role in "making meaning." Whether it be social studies, science or math class, there is value in challenging students to make meaning out of the often confusing data which swirls around us during the Age of Information. Just as writing provides a way to order one's thinking, to puzzle things out, to make connections between seemingly unrelated fragments, editing is an essential stage in the development of ideas and in the communication of those ideas. When converting data into information and then into insight as Toffler suggests in his new book, Powershift, editing becomes especially important. Editing permits focus and provides perspective. The puzzle otherwise remains blurred. Nonsense and chaos prevail.

To meet Toffler's challenge, to equip all students with the ability to convert raw data into insight, departments should collaborate to give students experiences with raw data. Writing as a vehicle for thinking should become a school-wide focus and commitment in order to address the national deficit in reasoning abilities documented by the series of Report Cards flowing out of the NAEP (National Assessment of Educational Progress) tests in recent years, tests which have shown that fewer than 10 per cent of 11th graders taking the tests can demonstrate high levels of reasoning, whether it be in writing persuasive essays or performing scientific reasoning.

A school-wide commitment to writing for thinking and writing as process would require agreement among teachers on the stages of development of an idea as a student moves from raw data to insight. Teachers become guides showing young people how to make the journey from uncertainty and curiosity through theory-testing to the development and expression of tentative conclusions. Teachers help students to see the interplay between thinking and word play. Sentence structure and logic become partners rather than opponents. Poetry belongs in the math and science paper as much as the English paper, as metaphor and the flow of language help to express the elegance of scientific and mathematical theories and principles.

Issue # 5 - How do we know that writing is improving?

We need evaluation designs which match our goals, but we must also acknowledge that the most important intervention of all is the writing methodology itself, not the technology. Teaching of writing with the railway approach will make little use of the word processor's special potential. Conversely, writing as process will exploit the technology to its fullest. At the same time, once a writer learns word play with the computer, she/he is likely to transform the use the old technologies of pencil and paper, outlining ideas for writing with word webs, for example, when wielding a pencil. As the Canadian study quoted in the beginning of this article found, "there were no macrostructural differences in writing across media." Good thinking and writing, once learned, surfaces with pencil and paper as well as with the computer. The wise teacher shows students how to perform word play on the screen of one's mind's eye so as to guard against undue reliance upon the new electronic version. Otherwise students might be unduly handicapped in a society which persists in testing students with blue composition booklets rather than computers.

Once we have identified the characteristics of effective writing, we should construct district evaluation procedures to trace the development of those characteristics over time. Student portfolios maintained throughout the K-12 experience provide good research data to explore important questions about program impact. We can look at specific programs and how they have influenced the development of student writing.

Take, for example, the goal of improving student editing skills with regard to word choice. If one group of students has received intensive practice on this skill throughout the year and a second group is serving as a control group, how can we measure the effectiveness of the program? We can collect early writings and count how many words have been "upgraded" between early drafts and the final drafts. To avoid thoughtless use of "big words" and stilted prose, we must teach the importance of words as part of a quilt-like pattern. When counting "upgrades," evaluators must agree upon criteria which reward only changes which fit the context in an appropriate manner. The number of upgrades then can be divided by the total number of words in the final draft to establish a basis for comparison which reflects quality as well as quantity. We can then explore the following questions:

1) Have students in the word choice program increased the frequency of appropriate word replacement to a statistically significant extent?

2) Is the change in the frequency of appropriate word replacement greater or smaller in the control group vs. those in the word choice program?

In a similar fashion we can seek evidence of improved coherence by identifying changes in the sequencing of ideas from early drafts to later drafts. We can look for evidence of improved transitions, the introduction of parallelism and the modification of sentence structure. If we care about macro-structural changes, we must devote attention to evaluation procedures which track these developments.


Even though the word processor promises much, delivering on those promises may require significant program changes which extend far beyond the installation of computers in a lab facility. This article has identified key issues in staff development, program development and program implementation which can intensify the benefits likely to emerge from writing with the computer.

In order to see whether or not the promises are fulfilled, districts would be wise to establish evaluation programs which are matched to the kinds of writing goals established for the students. The purpose of such evaluation, at least in part, should be to provide data which can steer the programs forward with insight rather than blind faith. "Hear no evil, see no evil" is a poor program strategy. Careful program assessment combining qualitative measures as well as quantitative data can inspire program changes which will lead to better results. Assessment can also broaden the base of community and Board support for expanded commitments to technology as a tool to support improved student reasoning and performance.

--------------------- District Technology Self-Assessment Form ---------------------

by Jamie McKenzie

The following survey was designed to assist decision-makers, policy-makers and educational planners in evaluating the strength of their technology programs.

1. Is there a sense of purpose? a plan?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Is there a written district technology plan which clarifies philosophical commitments and directions for district staff? Does this plan focus upon the horizon - the long view? Does it leave room for steering and flexibility as staff learns through experience? Does it address all critical elements of program implementation including staff development as well as hardware purchases? Is the plan research-based? Were all key constituents involved in creating the plan?

2. Is the technology being used?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Does the district have some method to quantify or track the per cent of time that equipment is being used by students or community members? Is there a system to figure out which staff members are making use of the equipment and which are not? Does the administration and the Board take a position with regard to computer and technology usage? How much usage would be desirable? 100% of the school day? 75%? 65%? 35%? Is there a gap between desired and actual? Does anyone know why? Is there a staff plan to narrow the gap?

3. Is technology blended into regular classroom learning?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: If technologies are basic tools for managing information in an Age of Information, they should be used broadly, within the art classrooms, the English classrooms, the vocational classrooms and all others. Studies of schools have shown that technology usage is often centered in special "niches" and departments such as a computer department or the media center and the usage is often concentrated in the hands of a narrow group of pioneers or champions. Are appropriate uses of technologies written into all of the district curriculum documents as mandatory activities to prepare students for the next century? Do all subject teachers make use of on-line databases, CD-ROM discs and word-processing for research projects, for example? Do students learn to create multimedia reports? If your district uses CAI (computer-assisted instruction) or an ILS (integrated learning system) approach, how closely are those learning experiences dove-tailed with their corresponding subject areas? If a child is doing reading practice on the computer, will it be on the same skills as she/he is developing with the rest of the language arts program? Is the new technology an organ transplant which has been accepted, or is it a grafted limb being rejected?

4. Does the use of technology mirror workplace realities?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Has your district explored how adult workers are currently using technologies to do scientific research, writing, planning, designing, evaluating, etc.? Has that exploration been translated into school experiences and programs? Is technology thought of primarily as a teaching tool or as a problem-solving tool of every day life? How well are the school technology experiences preparing students for the Information Age workplace? Is there an explicit district definition of the Information Age? Is staff aware of how the use of information is transforming work and the kinds of skills required by today's workers? Has Toffler's Power Shift been carefully reviewed? Was ASTD's Workplace Basics considered? Are technologically savvy representatives of the profit and non-profit sectors consulted when the district does technology planning?

5. Is the staff adequately prepared to use the technology?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Has the Board funded a comprehensive staff development plan over 3-5 years or more to provide all teachers with sufficient technology skills to implement an appropriate program? Are all teachers required to acquire such skills? Are assessment plans in place to determine what course offerings need to be added in future years? Does staff development take place during the regular work year/day or is it added on in ways that require teachers to subsidize the learning with volunteered time? Is compensation for training/learning reasonable and fair?

6. Does the staff ever visit the workplace?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: What provisions are made for staff members to spend time in the modern workplace seeing how technologies are employed? What percentage of English teachers, for example, have spent a day in a modern newspaper office seeing how technologies support the writing, design, layout and production of a newspaper? How many science teachers have visited a modern science lab to see how computers may be used to conduct experiments and model scientific phenomena? How many media specialists have visited modern libraries offering cutting-edge information systems?

7. Is access to technology equitable?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Does the district monitor technology usage by gender, race, location, and academic track to make certain that access is equitable? What kinds of data are collected and reported to help guarantee equal access? When evidence arises that there are gaps of various kinds, what provision is made to close such gaps?

8. What kinds of relationships should students have with machines?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Has the Board expressed community values in concert with the professional staff with regard to desirable relationships between students and machines? Are these values communicated explicitly in Board policy or in a district technology plan? How much time should students be engaged with various kinds of technologies? Who should be in control, the student or the machine? What are the long term consequences of such relationships? How do they relate to other educational goals such as citizenship and self esteem?

9. What implicit values are taught by technologies?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Is there a system in place to review the implicit values or hidden curriculum taught by various technologies? If computer software rewards students for correct responses by providing game time or opportunities to blow up aliens, for example, is such a reward system consistent with Board policies and community values? Does the technology stress extrinsic or intrinsic rewards? Is responsibility for reviewing such issues clearly identified? Is there a Board policy regarding software piracy and violations of copyright? Do students see ethical behavior modeled by the professional staff? Are ethical issues related to technologies addressed by curriculum areas such as social studies? Are students taught critical thinking and critical viewing skills to equip them to counter propaganda and media distortions? Will they emerge from schooling as thoughtful consumers or impulsive consumers? Will they be passive viewers or active viewers?

10. Does the technology enhance self-esteem, independence and imagination?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Workplace Basics states the need for workers who know how to learn independently, come up with novel solutions to problems and ride through the turbulence of a changing economy and society with self-confidence and adaptability. Are district technology experiences designed to deliver that kind of workforce? How do you assess progress toward such goals? Do you measure student self-esteem? independence? imagination? Is there a program review process to determine which learning experiences are most likely to promote the growth of such qualities?

11. Is the technology more effective than alternative strategies?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Is the technology the only way to provide a particular learning experience such as mastering various reading skills? If not, does the technology deliver results which are superior to corresponding alternatives for a comparable investment? If a district chose to invest in staff development aimed at improving teachers' reading instruction, for example, would students in those classrooms make smaller or larger gains than those in classrooms where an ILS system was installed? Does the district introduce such programs as pilots allowing for compare-and-contrast reviews of costs and benefits? Is the data from such studies made available to the Board as part of the district decision-making process?

12. Is the district evaluating what is happening?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: What kinds of data are gathered to assess the impact of various technologies? Is the data gathered in an objective fashion following accepted principles for experimental design to avoid bias? Does the evaluation design take into account issues such as the "Hawthorne effect" and the differential impact of volunteers as implementers of pilot programs? Is data used formatively , as a guide to future decisions and program modifications?

13. Is the technology efficient, flexible, adaptable and current?

excellent (5) good (4) fair (3) weak (2) poor (1)

Related Questions: Are issues such as processing speed, expandability and connectivity addressed in district planning and purchasing? If students will be using the technology to do graphics or CAD, for example, do cost considerations result in the purchase of low speed technology which will require students to sit and stare at the screen for minutes at time, wasting thousands of hours over the course of a year? Do similar considerations result in the purchase of black and white monitors which limit the ability of students to work in 3-D or work with multiple variables in graphing and statistics programs? When videodisc players, CD-ROM players, computers, video cameras, etc. are purchased, are choices made with a 3-5 year perspective? Are models with maximal expandability and adaptability selected to protect against pre-mature obsolescence? Is obsolescent existing technology maintained far past program usefulness because it has not stopped working or broken down? What planning procedures are in place to provide for timely updating of technologies and the transfer of obsolescent equipment to programs where the shortcomings are irrelevant?

Total Score____ (55-65 = outstanding; 45-55 = strong; 45 or less indicates a need for improvement)

Credits: The background is from Jay Boersma.
Other drawings and graphics are by Jamie McKenzie.

Copyright Policy: Materials published in From Now On may be duplicated in hard copy format educational, non-profit school district use only. All other uses, transmissions and duplications are prohibited unless permission is granted expressly. Showing these pages remotely through frames is not permitted.
FNO is applying for formal copyright registration for articles.

From Now On Index Page