DRAFT (3/15/03)

Judy Arzt

Session Title: “Electronic Portfolios’ Transformative Effects on Assessment”

Panel Title: Technology Transforms Writing Assessment,

College Composition and Communication Conference, 2003, March 21, 200, New York City

Electronic Portfolios’ Transformative Effects on Assessment

Statement of Issue

Two divergent trends have evolved in the linked fields of writing assessment and electronic media, both using the umbrella term electronic portfolios. One is the assessment driven and seeks to explore ways in which new technologies can support large-scale portfolio programs. The other is classroom-based and centers on students creating websites that need to be assessed. These two trends look at the terms “assessment” and “electronic media” from opposite vantage points. Perhaps the trends could be thought of as two horizontal lines intersecting at a point, and although they share some vocabulary, they have distinct characteristics. One effort seeks to use technology to fine-tune an assessment program—to make it more manageable and user-friendly—whereas the other is THE technology in need of an appropriate assessment. The short history of web authoring in the curriculum coupled with the evolving technology asks us to redefine constantly what makes a good website.

Ergo, one movement aims to solve an assessment problem—storage and access to a large body of knowledge—while the other searches for a vocabulary to describe and thereby assess students’ accomplishments. One is a top-down approach, a solution to an administrative problem, and the other is grassroots, emanating from classroom needs. How these fields merge, as well as diverge, demands scrutiny as both are becoming increasingly present on our campuses.

Debate over the Term E-Portfolios

The distinction between the use of technology to serve assessment and the use of technology for students to create products has led to some debate, including what the very term “electronic portfolios” means. The TechRhet and Writing Programs Administrators listservs recently saw some robust discussion on this topic. One discussion thread unfolded after Trent Batson posted on TechRhet a link to his article “The Electronic Portfolio Boom: What’s It All About,” where he states the term “electronic portfolios” should be used only to refer to large-scale, database-driven portfolio models. He calls portfolios of this kind “dynamic,” refers to the kind that students create in courses as “static,” and suggests the course kind be called “webfolios.”

Batson’s article, although helpful for providing a framework for distinguishing between the two kinds of portfolios, upset some who saw their students’ course portfolios as “dynamic.” Steven Krause writes of Batson’s article, “It was a good introduction and interesting … [b]ut there’s one part of it I don’t know if I get or agree with,” and he quotes:

Since the mid-90s, the term “eportfolios” or “electronic portfolios” has been used to describe collections of student work at a Web site. Within the field of composition studies, the term “Webfolio” has also been used. In this article, we are using the current, general meaning of the term, which is a dynamic Web site that interfaces with a database of student work artifacts. Webfolios are static web sites where functionality derives from HTML links. “E-portfolios” therefore now refers to database-driven, dynamic Web sites, not static, HTML-driven sites.

Krause finds the distinction between “dynamic” and “static” misleading, and Rich Rice adds Batson’s distinction between the two kinds of portfolios is “not healthy.” Bill Condon chimes in that historic precedent already has defined a portfolio as involving “collection, reflection, and … selection… and there are other distinctions but none that preclude putting an electronic portfolio (eportfolio) into HTML,” plus “changing the definitions of things that are already fully and firmly defined [is] like playing Tweedle Dum and Tweedee Dee.”

Lee Honeycutt writes: “Webportfolios tend to support greater student ownership of the process and can be much more creative than what you might see on a template-driven system. But they also have the potential to be incredibly unorganized and … individualistic to support matrix competency program assessment.” To clarify, Honeycutt offers the example of a departmental assessment of students’ portfolios across fifteen competencies, remarking: “I don’t know how faculty assessors could go about rating, much less, finding, examples of these [competencies] in a batch of Webfolios that were all over the map in terms of their site architecture and labeling systems.” Honeycutt’s use of the term “webportfolios” suggests he accepts Batson’s nomenclature.

Kathleen Yancey, Carl Whithaus, and Steven Krause assert that the websites that students create in courses are more “dynamic” than database-driven portfolios. Krause proffers: “If you make students responsible for their own websites—even simple ones—they learn another set of skills—I see it as more ‘dynamic’ than [the] database way” even if these ventures have the “downside” of “tremendous inconsistency” that make them more difficult to assess. Whithaus, speaking of first-year composition, stresses the idea that students need exposure to electronic tools for creating hypermedia rather than experience dumping work into an “IT [information technology] designed system.” Students, he seems to think, should do more than fill in boxes; they should practice with real composing tools and not just “interface/database programs.” Yancey offers that although a number of institutions are opting for database-web interfaces, student-created websites provide a richer learning environment. She enumerates:

· form and content are related, and that if students are to exert agency, they need to have some control over the form

· the interface and navigation design are a major part of self-representation (and thus self-construction)

· hyperlinking is related to cognitive thinking

In fact, Yancey suggests that the database model might more aptly deserve the appellation “static.” In referring to this model, she remarks, “…much of what is being called a portfolio is really simply a ‘capture’ of information—no reflection, no student choice, but indeed a lot of stuff. That, to me, seems totally “static.”

Yancey’s comments on the TechRhet listserv echo her earlier comments on the Writing Program Administrators list in December. Returning from the American Association of Higher Education e-portfolio conference held in November, she reports on WPA the emerging use of large-scale database models: These folios as places where students “drag and drop” work into a “template,”; “[i]t is more a response … to a set of prompts than a creation or product or a composition.” Having “reservations” about the value of these experiments, she pines for student-designed portfolios even if that means variations across departments and programs. These portfolios, as she sees it, are “better for student learning” and “more interesting to read.” In short, Yancey advices “more composition people” to stay “involved” in the conversation about e-portfolios on their campuses.

A Sampling of Large-Scale Electronic Portfolios

A review of some of the large-scale models reveals commonalities and a shared vocabulary. One model used by the English Department at Georgia State University in the Rhetoric and Composition Program requires students concentrating in Professional and Technical Writing to complete e-portfolios as an exit requirement. George Pullman notes that “an electronically stored version of the portfolio” does not require students “to know anything about making web pages” and can be made using “Word documents, WordPerfect documents, Rich Text Formatted documents, PDfs, even images like GIFs and JPGs.” He explains that the “electronic portfolio has many advantages over a regular paper portfolio.” Students can quickly and easily revise “any of the documents … from nearly any computer connected to the Internet” and can show them to anyone “without having to print copies or pay postage.” In summary, an e-portfolio is “like a resume” that offers “proof” of a student’s “skill set,” Pullman adds.

In the Alverno College model, which is touted as the “first-of-its-kind, web-based system,” a student “anyplace, anytime” can “follow her learning process throughout her years of study.” This portfolio system, called Diagnostic Digital Portfolio, enables a student to receive feedback from faulty, peers and others, and, as claimed on Alverno’s website, helps a student “look for patterns in her academic work so she can take more control of her development and become a more autonomous learner.” A screen capture at the site shows the gateway into a student’s portfolio as a table. A column on the left lists competencies (e.g., communication, analysis, problem-solving, etc.), and a column at top gives levels: level 1, level 2, level 3, and level 4. Remaining cells on the table have course prefixes and numbers. The implication seems to be that if one clicks in a cell, a portal to viewable evidence of a student’s competency at the level will appear. Although this system appears to manage a large collection of work well and provides a way for others to critique a student’s work, it is difficult to tell as a visitor to the site if the technology engenders student invention and learning or merely accomplishes making competencies observable. In fact, perhaps the only experience students have using hypertext in this environment is filling out forms and clicking on buttons to access feedback. The advantages seem much like those of working in a BlackBoard or WebCT except now the technology is customized to fit the institution’s specific assessment program.

The financial expense of Alverno’s startup of the Diagnostic Digital Portfolio Program is suggested by the list of sponsors who funded the inception of the project, among them the W.K. Kellogg Foundation, the Pew Charitable Trusts, and a U.S. Department of Education Title III Grant. It would appear that this kind of investment bodes that the model will be transportable to other campuses.

A similar model used at the University of Washington provides a space for students to collect and store “work that they have done as part of their academic career.” The website describing the project notes the e-portfolio system enables students to “interact with instructors and advisers” and “present …work on the Web for a variety of audiences.” A screen capture pictures “the main interface that students … use to interact with the tool.” The electronic portfolio is touted as a means “to collect” and “annotate” a variety of “digital ‘artifacts’” in an “organized” way by using the navigational features built into the program.

Stanford University uses a similar system called “E-Folios,” short for Electronic Learning Portfolios. The literature describing the program states: “E-folios are ubiquitous, portable, electronic knowledge databases that are private, personalized and sharable, and are easily accessible via the web.” Screen previews again reveal a form in which students drop work. Although the screens on the Stanford, U. of Washington and Alveno sites look slightly different, the programs seem to serve the same basic purpose: provide a place for students to drop work and access feedback. These programs make it possible for faculty, administrators, and staff to view student work from virtually any computer and provide immediate feedback. The programs also seem to help with monitoring student progress toward achieving specific outcomes and appear similar to those that are being designed on the state level to help with assessing skills for teacher certification programs wherein a range of competencies must be demonstrated.

As we well know, large databases help us manage large amounts of information, but how well do these programs engender student creativity? To what extent do these programs promote creativity and help students learn aside from the guided feedback provided? How much invention and play students have in these environments is not readily apparent by viewing the websites with visitor status. Furthermore, as many of these database e-portfolios are password driven, they do not allow for the kind of accessibility that occurs when students mount their work in a generic way on the World Wide Web. So, herein is another distinction between the two divergent trends in the field of assessment and technology that fall under the umbrella term e-portfolios. One operates in an environment that keeps snoopers out, and the other freely permits anyone in.

Databases versus HTML Sites: What about the Waters between the Two Rivers?

With these two distinct movements unfolding on our college campuses, we have to wonder about students getting caught in the middle. Will students be able to drop an existing website into a newly implemented database portfolio? How will assessors using a grid respond to student creativity? Will students be able to readily extract work created in a database websites and export it to html websites? In effect, how well do the two models co-exist, and can they be successfully wedded? Institutions engaged in both enterprises will provide fertile grounds for answering these questions, but students will also have to be engaged in the conversations to see how they are affected.

If a February 1, 2002 Chronicle of Higher Education article prediction proves true, we will increasingly see database-driven portfolios models on our campuses. John Ittelson, professor of communication science and technology at California State University-Monterey Bay, director of the California State Idea Lab, and lead figure in e-portfolios nationally, notes in Chronicle article: “‘A few years ago the question was, Was every student going to have an e-mail account…[ now]...everyone is going to want to have some kind of Web space that represents their [sic] learning and their assessment’” (as cited in Young). This statement about exponential rate of growth underscores the fact that students do and will continue to create websites regardless of whether the technology is part of the college curriculum.

Those of us at institutions using, or about to use, both flavors of electronic portfolios should be thinking about the interconnectedness of a student’s education. Although both forms of portfolio development share the common principle of being a way for students to demonstrate learning and academic accomplishments in an integrated, seamless environment viewable to many in many locations, will students be able to synthesize these learning experiences and make connections? Perhaps their educations will be enriched by participating in both. But like all other college experiences, we still need to wonder about how all of the pieces fit together to form an integrated, holistic experience as opposed to a frustrating, fragmented education. Further, will students who have labored over the creative design of a website fare well with assessment number crunchers looking to assess students’ skills across a matrix of skills and then punch results into a canned database?

This question gets to the heart of another matter: the difficulty of allowing for creativity of expression in large-scale portfolio programs that are driven by set, clearly defined criteria—an issue that presents itself in both offline and online portfolios, but perhaps more so in online environments where it is so readily possible to present oneself through visuals and multimedia that conventionally have not been considered germane to writing assessment. John O’Connor, who participated in the AAHE conference on electronic portfolios, notes that at George Mason University’s New Century College, communication and technical writing majors self-design portfolios in an environment that encourages “…individuality and creativity while developing some consistent review standards ….”. O’Connor affirms, “We do not want students to … fill in spaces in a standardized, formulaic portfolio template.” Summing up one of the central issues, O’Connor says, we need “guidelines that help students think through imaginative hypermedia composition and linking … while providing enough guidance about reflection that there is a real fire behind the flash ….” We want substance with style, and we want to know there is something worth evaluating beneath the gimmickry.

Evaluating Student Designed Websites

Thus, I turn here to the whole matter of how we assess student-created websites. As we shift our teaching more and more online and work increasingly with more multimedia, how well do we articulate for our students how we will evaluate their final products or course portfolios? How do we address the fact that a large part of our training might have been in analyzing and responding to compositions as a written texts and not in responding to student-authored hypermedia? Just how interdisciplinary and multimodal are we? And, how do we accommodate for the range of skills we find in our classes not only in terms of writing ability but computer dexterity?

In my own teaching with electronic portfolios, which has been evolving over the last five years, I have increasingly turned to theorists for answers, including hypertext theorists like Jay David Bolter as well as composition theorists such as James Sosnoski and Gunther Kress who have melded comp theory with hypertext theory. Sosnoski eloquently reminds us in his metaphorical essay, "Hyper-readers and Their Reading Engines," web readers employ filtering, skimming, and fragmenting strategies that not only alter the reading process but confront web authors with new challenges. Gunther Kress in his aptly titled essay "'English' at the Crossroads: Rethinking Curricula of Communication in the Context of the Turn to the Visual" cautions us that the visual nature of the Internet is revolutionizing communication and linear text exists as just one of many options for communicating. Kress holds that we must train students to produce "culturally valued text" and acknowledge that the visual is rapidly "becoming prominent in the landscape of public communication" (67).

In effect, composition studies needs to be re-contextualized to consider not just the value of the image but the power of the hyperlink. As Jay David Bolter states in Writing Space, "As a global hypertext, the Web has provided the most convincing evidence of the computer's potential to refashion the practice of writing" (xi). Hypertext reading, as Bolter points out, is an associative process, not a sequential one, and viewers assume authorial authority by the path they use to navigate a site. Here, we are reminded of reader-response theory, which gives the reader liberty to find meaning in a text—but now the author is pushed further by multimedia that coax readers in all sorts of directions, including possibly leaving the site altogether by the links the author provides for off-site viewing. A weak hypertext, perhaps, is easier to leave behind than a weak printed text—with all kinds of other distractions so readily available on the Web. However, not to distort Bolter’s meaning, it seems as important as ever to teach students about unity, coherence, and transitions.

The rapid nature of the Web engenders pecking and pawing, challenging writers to compose in ways that capture viewers’ attention and reflection—mental processes that at times seem antithetical to the very pulsating, flashy nature of the Web. In addition to turning to theorists for advice, practical guidebooks such as Miles Kimball’s The Web Portfolio Guide prove valuable, as evidence recently when I created on my own website a PowerPoint for students on tips for revising websites. David K. Farkas and Jean B. Farkas’s Principles of Web Design, according to Shelia Sandapen’s review in Kairos, is another good resource. Sandapen notes that the book describes a Web site as an “unstructured database … [where] text may be viewed as dialog even though the user is not exchanging with the author … [and] how … links are arranged … express meaning.” Furthermore, “…since the Web is a ‘nonsticky’ medium and the possibility of a user entering from a side door is very real, a good designer must strive to give context and guidance throughout the site.” Most important, “functionality … drives the development of good Web sites.” Users expect “to find information quickly and be able to digest it quickly … [so] information must be presented in a way that is visually pleasing, easy to read on the screen and gets to the point.” Farkas and Farkas, according to Sandapen, find that graphics “emphasize aesthetic, help to express the theme of the Web site and help show the logical connection among all the elements that appear on the page while maintaining balance and harmony on the page.” Statements such as this remind us visual literacy must be a part of our curriculum.

Conclusion

The fields of assessment and technology are rapidly changing, changing so rapidly sometimes that the only way to keep current is to check websites daily, scan listservs, and use other online gateways for access to the most up-to-date information. Yet despite our frenzy to keep pace, we also must practice some reserve.

A story comes to mind. An American educator visiting British schools asked about the apparent lack of assessment in British schools and was told, “Me dear fellow … in Britain we are of the belief that, when a child is hungry, he should be fed, not weighed” (Georgi and Crowe). So, are we feeding our students, or are we assessing them? Where are our priorities? In the answer to the assessment and technology dichotomy and questions it raises, we must also examine our own basic beliefs. Ultimately, what do we think is important? The principles guiding our basic philosophical positions are likely to provide the answers. At times, we have to step back and not let all the glitz propel us into instantaneous action. We must mesh learning theory with our adoption of the technology, as well as consider the merits of the assessment programs we design. Regarding the latter, we must ask if the technology supports well-designed assessment programs and assesses skills that students have actually learned and are ready to demonstrate. Underneath the simplicity of the reply of the British schoolmaster’s reply there is some real wisdom.

Works Cited

Alverno’s Diagnostic Digital Portfolio. 2002. http://www.ddp.alverno.edu (20 Jan. 2003).

Batson, Trent. “The Electronic Boom: What’s It All About?” Syllabus. December 1, 2002. http://www.syllabus.com/article.asp?id=6984.

Bolter, Jay David. Writing Space: The Computer, Hypertext, and the History of Writing. Hillside, NJ: Lawrence Erlbaum Associates, 1991.

Condon, Bill. “Electronic Portfolios—an article.” TechRhet Listserv. techrhet@interversity.org. (30 Jan. 2003).

Georgi, David, and Judith Crowe. “Digital Portfolios: A Confluence of Portfolio and Assessment and Technology.” Teacher Education Quarterly, Winter 1998. http://www.csubak.edu/~dgeorgi/projects/digital.htm (7 Feb. 2003).

Honeywell, Lee. “Electronic Portfolios—an article.” TechRhet Listserv. techrhet@interversity.org. (31 Jan. 2003).

Kimball, Miles. The Web Portfolio Guide: Creating Electronic Portfolios for the Web. New York: Longman, 2003.

Krause, Steven. “Electronic Portfolios—an article.” TechRhet Listserv. techrhet@interversity.org. (1 Feb. 2003).

Kress, Gunther. "English at the Crossroads: Rethinking Curricula of Communication in the Context of the Turn to the Visual." Passions, Pedgagies, and 21^st Century Technologies. Eds. Gail Hawisher and Cynthia Selfe. Logan: Utah State P. 66-88

O’Connor, John. American Association of Higher Education. Electronic Portfolios Conference, November 2002. http://aahe.ital.utexas.edu/electronicportfolios (20 Jan. 2003).

Pullman, George. Rhetoric and Composition Portfolio Website. Georgia State University. http://rhetcomp.gsu.edu/portfolios.htm. (27 Feb. 2003).

Rice, Rich. “Electronic Portfolios—an article.” TechRhet Listserv. techrhet@interversity.org. . (31 Jan. 2003).

Sandapen, Shelia. “A Review of Principles of Web Design. Karios 7.3. (Fall 20020). http://english.ttu.edu/kairos/7.3/binder.html?reviews/sandapen/index.htm. (23 Feb. 2003).

Sosnoski, James. "Hyper-readers and Their Reading Engines." Passions, Pedgagies, and 21^st Century Technologies. Eds. Gail Hawisher and Cynthia Selfe. Logan: Utah State P. 161-177.

Stanford Learning Lab. “E-Folio.” http://sll.stanford, edu/consulting/tools/efolios. (20 Jan. 2003).

University of Washington. ed-techdevelopmentgroup. http://depts.washington.edu/edtecdev/portfolio. (20 Jan. 2003).

Whithaus, Carl. “Electronic Portfolios—an article.” TechRhet Listserv. techrhet@interversity.org. (31 Jan. 2003).

Yancey, Kathleen Blake. “Electronic Portfolios—an article.” TechRhet Listserv. techrhet@interversity.org. (31 Jan. 2003).

Yancey, Kathleen Blake. “ eportfolios news: old thread, new sources.” Writing Program List serv. WPA-L@asu.edu. (16 Dec. 2003).

Young, Jeffrey. “Creating Online Portfolios Can Help Students See ‘Big Picture,” Colleges Say.’” The Chronicle of Higher Education. (21 Feb. 2002) online.