Tag Archives: textbook publishers

Alex's Teaching Tidbits, Comment-fishing, Linkfest

ATT8: Alternatives to Textbooks

1 Comment

[It’s potentially dangerous for me to blog about this since I have many ideas on the topic. But thinking about teaching is something I enjoy so these “tidbits” are almost a release…]

I use textbooks in two of my courses: introductory sociology and introductory cultural anthropology. More accurately, I still use textbooks. But I do want to switch away from textbooks in these courses.

In some contexts, such as a discussion of the usefulness of textbooks in anthropology, I’m almost defensive in my description of the role textbooks play in my teaching. It’s a sign of ambivalence, of course. But the fact that I still use textbooks shows that something weighs in their favour, Despite being problematic, textbooks still seem useful to me and I have yet to find an alternative.

But I think a solution is finally on the horizon. It’s a bit difficult to explain and may seem farfetched or even crazy, but I’m getting more comfortable with it. So it may be the right time to put it in writing.

A few words about how I use and choose textbooks…

I perceive textbooks as shared resources, rallying points, conversation starters. study guides, “lecture notes”, and sources for examples. Textbooks aren’t ideal for any of this, but they’re rather unique in fulfilling all of these functions.

Textbooks are also expected to be part of at least some courses. The two main introductory courses I teach are of that type. If I were not to use textbooks at all, I’d have to propose something else.

Which leads me to thinking about existing alternatives to textbooks…

A relatively uncommon practice, but one I’ve experienced as a student, is for instructors to teach from extensive coursenotes, without making use of shared texts. In such a case, the “content” is “delivered” in the classroom. Without being common practice, it wasn’t unknown at Université de Montréal, where I did my B.Sc. in anthro (back in the early 1990s). I suspect that this approach was somewhat common in French-speaking contexts a while before that. It avoids all problems associated with textbooks, but it has none of the advantages textbooks have. In fact, because it puts so much emphasis on the “sage on the stage” model, it’s incompatible with my teaching philosophy.

Apart from my main “intros”, all of my courses are built on “coursepacks”. A coursepack is a custom package of primary texts which weren’t originally connected to one another. I like this approach in any course and have been thinking about ways to shift my intros to something of a coursepack model.

The traditional coursepack model is to send a bunch of texts to a third-party service which will make all the necessary enquiries to ensure copyright compliance and then print coursepacks to be sold at the bookstore. At Concordia, the university bookstore currently handles the whole process. Eastman Systems used to do it. In both cases, the process requires a lot of lead time. It’s thus more flexible than the textbook model but remains quite constraining. As I do most of my reading online, it’s actually far from ideal. But it’s an interesting alternative to publishers’ textbooks.

I have yet to find a way to use coursepacks in my main “intros”. I know some colleagues do it and I could invest time in such a solution. But the convenience of traditional textbooks still seems to me to outweigh the costs of the textbook model. (Yes, this is where I get defensive.)

In some contexts, I was able to use a model which was an alternative to traditional coursepacks, using online reserve instead. In those situations, I could select texts during the semester and have them on online reserve quickly after that. It was close to an ideal model, especially paired with the collaborative syllabus. But it requires three things which I don’t currently have: reasonable copyright laws, access to a service dealing with copyright compliance, and efficient electronic reserve. I can get somewhat close to this approach at Concordia, but some pieces are still missing.

It’s also possible to use “readers”, which are midway between coursepacks and textbooks since they contain primary texts but those are edited together and published like textbooks. I haven’t used readers. To me, they have some of the advantages coursepacks have (multiple voices, intellectual depth) but also most of the disadvantages of textbooks, including a lack of flexibility and all the issues associated with commercial publishing.

Of course, there are people who simply post files online without worrying about legal implications. This is obviously a non-solution, as failure to comply with copyright laws, however arcane these laws may be, can have dire consequences on learning institutions.

Which leads me to the first key issue I identify with textbooks: “the content industry”. As I said two years ago:

If your only business is “content,” now might be a good time to think about diversifying.

(My approach to “content” has long been the basis for diverse discussions, including one which got me cited by tycho garen in such august company as John Gruber and Merlin Mann. Incidentally, I’ve been listening to a number of 5by5 podcasts, including those by Mann and Gruber. Textbooks have been a topic of discussion in those podcasts and part of my thinking may have shifted because of them.)

Got lots to say about content, which is part of the reason why I was worried this post may become overwhelming. But I’ll summarize the situation as much as I can.

For the past ten to fifteen years, an increasing number of people have been made aware of problems with “content-based models”, especially in terms of “publishing”. The context for this increased awareness is the expansion of the Internet. But the process started much earlier, with the transformation of information into “content”, consolidation in diverse industries, and what scholars have long been calling “postmodernism”. As you might notice, I’m specifically not saying that one of these things caused any of the others. But I see connections between all of them. They “go together well”.

The first well-known case of a “content industry” having some difficulty is the one involving music labels, the so-called “Music Industry”. At the very end of the 20th Century, the “Napster Revolution” associated with trading of audio files was a turning point for music labels and the business model on which they relied. Some might say that it was quickly over. And there are many points of continuity in the “political economy of music”. But the current state of music-related business is clearly in a “post-” phase: it’s significantly different from what it was twenty years ago. I wrote extensively about this on a blog which is currently offline. I happen to think that this turning point in the way musical “content” is distributed is the early model of turning points happening elsewhere.

Subsequent “content industries” going through major transformations include films, video games, journalism, and books (possibly in that order, with significant overlap). In each of these cases (including music), “intellectual property” has been used as a core concept in discussions surrounding the “content shift”. I’ve had (and still have) a lot of things to say about any of these. But I’m focusing on textbooks which, I’d argue, are just now going through a major shift.

Most of the problems associated with textbooks have to do with the content model. What textbook publishers do is quite elaborate and rather complex (from dealing with authors and editing texts to obtaining media usage rights and coaxing people into “adopting” textbooks). But their main business model is based on selling access to content. That model isn’t obsolete and some publishers can survive for a little while longer. But it’s clearly not forward looking.

Access to content can take many forms, many of which incur no cost to the person accessing such content. Typically, “content access” and the business model behind the creation of this content are disconnected.

Borrowing books from a public library is a key form of “content access”, and it’s typically paid by taxes. Watching television advertisement can be described as access to content, paid by advertisers. Even glancing at someone’s “freesheet” in the métro is “accessing content”, and it’s also paid by advertisers. Open Access journals, YouTube, WordPress/Moodle/Joomla/Sakai/Drupal, Craigslist, Khan Academy, Wikipedia, and iTunes U all focus on “access to content” but their business models clearly separate production costs from access costs (with the result that access to content is often “free as in beer”). Even a café conversation can be conceived as “content access” and it’s associated with no direct cost in money. (Interestingly, the UnivCafé model does involve some financial costs, which are covered partly by Concordia University’s School of Extended Learning, and partly by donations. I’ve been on the record about my interest in this model.)

The “elephant in the room” is “online piracy”, which contains in its name the notion that it’s illicit or illegitimate access to content. Those who pirate content online typically don’t pay directly for that same content (though, according to some peer-sharing enthusiasts, these same users may be more likely than others to spend money online, including for “content”).

Of course, all of these things are much more than “content access” and reducing all of them to the same model should eventually show the absurdity of said models to those who aren’t in the “content industry”. “Access to content” isn’t really a business model. Were publishers able to get past the “content access” model, we may finally get an improvement over textbooks.

Call me naïve, but I think the shift is finally happening.

Part of it is through the slow transition from physical textbooks for electronic ones.

Which makes this blogpost into a followup to a post I wrote just before Apple’s iBooks Author announcement. In that previous post, I tried to talk about “learning content” in as general a way as possible, making as explicit as possible the notion that “content” is but a small part of learning. It encapsulated something about which I’ve been adamant: those of us who are interested in learning and teaching should not focus so much on content.

But I’ve been so “obsessed” with this issue that I haven’t been heeding my own advice. I keep talking about content despite the fact that I care a whole lot more about learning.

And my focus on content is part of what prevented me from thinking further than textbooks. As I kept going back to those problems with textbooks which are associated with content (cost, medium, distribution, usage rights, rigidity…), I was missing a core point  about textbooks: they intrude on, encroach upon, interfere with, and disrupt the relationships between learners and teachers. Their existence is a form of «ingérence».

See, my teaching philosophy is based on constructivism. Though I still lecture, I’m looking at alternative teaching models, which may put me at odds with some colleagues. I eventually found a way to use textbooks which makes sense to me. Point is, I care more about “building a context in which learning happens” than about “transmission of content”.

My class meetings now have the following structure…

Students come prepared, having worked with some textual material and, in the case of introductory sociology, taken a reading quiz about the class material (teamwork exercises serve part of the same role in my introductory anthropology). So, at the beginning of the class period, we go through a kind of collective Q&A session during which the class comes up with both questions and (partial) answers. Though I often end up providing extended answers to some questions, I try to engage the whole class in the process. In the two sections of introductory sociology with which I work, this semester, many questions are the outcome of reading quizzes. Taking those quizzes, students often find out that they may not have fully understood some core concept or aren’t yet able to apply a given perspective. That Q&A section of each class meeting often takes more than half of the classroom time. As it’s the one part which most directly depends on face-to-face interactions with the whole class, it makes a lot of sense that it’d be the core part of classroom interaction.

Another proportion of class time is devoted to “tangents” which connect the material with diverse issues. These discussions are especially useful for learners who reached the “upper” positions in “William Perry’s Scheme of Intellectual and Ethical Development”. At the same time, they might be less appropriate for students who (still) focus on “what’s the right answer that I need to remember for the exam”. In prior semesters, these tangents took a very significant portion of the class time. The shift from these discussions to Q&A is probably the most significant change in my teaching, this semester. Perhaps due to this shift, online forums in this course have become much more active than they ever were in the past. Parenthetical discussions work quite well online and I’m convinced that they do accomplish a lot in terms of the “higher domains” in Bloom’s Taxonomy.

During the last (and often shortest) portion of each class meeting, I go on “lecturing mode” to prepare students for the material which is coming up. In other words, I spend twenty minutes to half an hour at the end of each class period trying to get students started with a new topic. While my lectures are relatively “classic” in format (in fact, I feel the need to work on this structure, as I often end up running against the clock, at that point), they are mostly oriented toward giving learners their first exposure to some ideas, concepts, and issues. As a compulsive outliners, I base my “lecture mode” on slides with bullet points. But what I may say in relation to these bullet points varies quite a bit. At times, I will teach students concept definitions which are meant to help them understand the material (these are relatively rare and almost choreographed, with artificial pauses)  During “wink, wink, nudge, nudge” moments, I even hint at potentially tricky quiz questions. Otherwise, much of the lecture section is a restructuring of the material in a way which makes sense to me. This is where my outlining is probably the most obvious.

As you might notice, my class meetings devote little time to summary of textbook content. Even those short sections when I restructure that content, emphasis is on the connections between ideas, not with the “content”.

Having used several different textbooks in my teaching, I have a lot of “content” in my mind. In fact, some of the “bullet points” I present in class are carried over from prior teaching experiences which may have relatively little to do with the material for this one course. While I’ll never go to the “text-less teacher-driven content delivery model” I described about my UdeM past, I often muse about teaching without textbooks.

As a partial step in that direction, I started using a Wikibook in introductory sociology, a few years ago. It’s somewhat fragmentary as a textbook so I also use a printed textbook (recently switched from OUP’s “critical” Knuttila to Pearon’s “contested” Hale). However, were I to expand the Wikibook, I may be able to teach without a traditional textbook. If I succeed in doing so and colleagues start following suit, textbook publishers may run into deeper problems than they currently imagine. And it has very little to do with “piracy”, as the “content” used in my courses would be completely legitimate and licit, licensed through Creative Commons or taken from the Public Domain.

What’s more, I could easily complement the “textual content” with all sorts of things, including “interactive material”, collaborative activities, and open discussions. Almost any website could allow for this and I have most of the infrastructure in place to make it doable.

The only thing I’m missing is a bit of “content”. Not much, as the Wikibook is fairly extensive. But enough to make this experience of learning in this environment a clear improvement over textbooks. With just a bit of grant- or fellowship-style funding to devote some person-hours on such a project, it’d be easy to produce most of that content, including a large part of the job which is usually done by publishers.

There are many projects like this, going on. In my mind, it’s only a question of time before publishers are required to pivot their business model.

What I’ve been thinking about is, obviously, a shift away from the “content access” model. I kept thinking that publishers should transform into a service bringing together people who use the same content. Instead of “selling access to textbooks as content” (and offering some services as a way to force people into “adopting” their textbooks), publishers could be paid for services which are truly added value over that content. If access to content were free of charge, a lot could be done around that content. During a recent crisis with journalism, a lot of things have been said about curation and advertising. While advertising models are antithetical to the learning models most of us have in mind, something close to curation makes a lot of sense in learning and teaching. What I have in mind has less to do with “recommendation engines” than with “localization”. The idea, here, is closer to “citizen journalism” than to HuffPo. The added twist is that some people would be paid to adapt the “content” (including “ancillary material”) to specific contexts.

And this is where the hard work and added value can be found. It’s not that difficult to “produce content”, especially if we think about what we do as lecturers or the “content” created by students. As some of my students just created an elaborate study guide to prepare for my midterm (and claimed that it was easy to do), I have a neat example of how effortlessly that may happen.

What takes more effort, though, is creating appropriate material, such as exam questions related out of a classroom discussion, moderating a forum, or editing a text to make it understandable by outsiders. It would make sense to pay people to do some of these things. In some cases, the amounts paid for all sorts of learning-related services could be as much, if not more, as what is currently paid by students for textbooks. Sure, it requires some creativity to make the business model work. But that’s the reason people like Clay Christensen and Horace Dediu are around. True disruption in the market doesn’t happen by merely extending the current business models.

Most of this has been on my mind for quite a while. And though I spent some time explaining it, I still went rather quickly about it.

What has just hit me is the part about learner-teacher relationships. With all the talk about “student engagement”, we may forget the “intermediation effect” textbooks (and standardized examinations!) have had on the relationship between students and teachers. By moving away from textbooks, we can “disintermediate education”.

I had a sort of epiphany while reviewing a textbook manuscript. A significant part of the problem I had with that manuscript is that it was a “voice from nowhere, talking to ‘my’ students”. Not that I’m protecting of my students. But I was imagining a situation in which we were using this text as “content” for a course. As I imagined this, I grew increasingly concerned about the effect such a text could have on the most interesting things happening in my classes. My guess is that a large number of teachers have had an experience of “battling with a text” (because it’s confusing, misleading, inaccurate, inappropriate, outdated, boring, or flat out offensive). Beyond the frustration of being unable to modify the text to be more appropriate is the broader issue that teaching shouldn’t be about transmitting somebody else’s dogma. Especially not university teaching.

Fortunately, I can now choose which textbook I adopt. And I might soon elect not to use any textbook.

 

Series, Tenure

Rethinking Tenure, Part V: Broadening the Scope

1 Comment

Can’t We Cool Off?

Like the child of parents who are committing divorce, I just wish people would stop fighting, talking past each other, and having knee-jerk reactions.

Not that I want people to agree or get along, but it sometimes feel that debates and disagreements focus too much on some specific problems (or on some specific positions and even on some specific people) instead of looking at things in as broad a way as possible. Sure, the issues involved may be “serious” and “important.” But it’s too easy to get caught up in those issues and think that any form of thinking has two sides or that these issues are all that matters. As Michel Rivard sings: «Une toute personnelle fin du monde» (“A Very Personal End of the World”).

What else is there?

Even in the divorce case, it’s not “think of the kids.” And, despite what some teachers say, “higher” education clearly isn’t about “kids.” So, as I talk about universities, it’s not about a specific group of people. It’s about the whole picture, the whole set of alternatives. Not limiting your options to what is being presented, as in the proverbial nine-point “box.”

Call me a holist (it’s allowed in my field), but I can’t help thinking about broader ways to approach things.

Questions about Tenure

More specifically…

How can we think about tenure in a broader way? What are the implications of the tenure system? Can other systems provide something similar to what the tenure system provides? Aren’t there shifts in what’s needed from tenure? Is there more than one alternative option to the tenure system? Aren’t there broader changes related to tenure? What are the constituencies involved when we talk about the tenure system? Is it just something happening between administrators and professors?  In business, the questions about stakeholders are used to open up discussions. So, who are the stakeholders in the tenure system? Would it be enough to add non-tenure employees, students, and “society” or “governments” to administrators and professors? Isn’t the tenure system involved in a much broader set of issues, which include not only tuition fees and costs for textbooks but social expectations in terms of post-secondary education and even the meaning of life outside the Ivory Tower?

A Bit of Context

The trigger for this new post in an already-overextended series and the reason for my long preamble is a post by blogging dean “Dean Dad” (DD) meant as a response to a proposal by the New Faculty Majority (NFM), “The National Coalition for Adjunct and Contingent Equity.”

Confessions of a Community College Dean: A Response to The New Faculty Majority.

“Program for Change: 2010-2030” | A Discussion Covened by NFM.

Avoiding the Debate?

Despite my blogging from this starting point, I would actually prefer not to enter the latent or actual debate between “higher ed” administrators and contract-based faculty members. Previous posts in this series, especially the first one, contain enough about my opinions about the specific situation of non-tenured faculty. And I specifically don’t want to wedge myself between NFM and DD. I haven’t read the full NFM proposal and my reaction to DD’s post isn’t about specific “points of contention.” Personally, I would both disagree and agree with several points made on both “sides.” But, in my mind, it’s not about agreement or disagreement. And it’s not about me.

In fact, I don’t feel that connected to either side. Though I’m officially working as a part-time faculty member and I certainly empathize and sympathize with my colleagues who are unsatisfied with such a position, I have a hard time seeing this as a “side.”

«C’est la lutte finale!»

At Concordia, I’m represented by a union which has as its specific mission to defend the interest of part-time faculty members.  In fact, my union has scored some significant victories in the recent past and it seems that “my side” is adequately represented. But, by analogy with nationalism and patriotism, I wish I could be a “citizen of the World” instead of belonging to just one society.

I don’t have anything against unions. Not only was I raised in union-friendly Quebec but unions have been a relatively significant part of my own environment. Besides, the underlying point of the previous post in this series was that unions are a very appropriate way to solve labour issues, including what “academic freedom” ends up being, in a number of contexts.

Yet I think that union-mindedness can become a bit too focused, in some specific contexts. In my mind, there’s room for thinking outside of the debates and even conflicts between “union and management” (I’d say «syndicat et patronat», so much has union-speech infiltrated my life). So, to come back to my preamble: who’s involved, here? Is it a back-and-forth between employees and employers? Is it all about working conditions?

“Think about Textbook Publishers!”

Textbook publishers aren’t that directly impacted by issues related to tenure (apart, maybe, from longtime textbook adoption commitments by individual professors they bribdo business with). But they do have a stake in the current university system. Not that they contribute significantly to the operational income for “higher learning” institutions. But textbook publishers are part of “the whole picture.”

Why do I mention them here? Several reasons. One is that discussions about tenure often  revolve around money, including the money spent on faculty salaries and money spent as tuition fees. The cost of textbooks is something many (undergraduate) students may need to put in their budgets. In fact, if one is to follow “the money trail” in some educational systems, an important path in that trail probably leads to publishers. Not that textbook publishing is “where the money is.” But, after teaching courses in which students spent more on textbooks than was spent on my salary, I got to think about that part of the money picture.

But part of what leads me to think about textbook publishers has less to do with textbook costs and more to do with transitions in educational systems.

I originally thought about mentioning journalism. The advantage would have been widespread awareness of a current journalism crisis. Not that I know the financials for textbook publishers, but people don’t usually talk about a “crisis” when they talk about textbook publishers. At least, not on the side of publishers.

Yet textbook publishers, like newspaper publishers, have to adapt to new realities. Large-scale corporate concentration, longterm shifts in business practices, the cost of paper and… the Internet. If textbook publishers aren’t adapting, they might end up with significant problems. I happen to think that publishers tend to have too narrow a focus and that they will eventually run into significant problems. But even if I’m wrong, the idea remains that a broad focus can help. (And, yes, I sound like a broken record but it’s not a song which is being heard so clearly, so redundancy may help.)

“The New Realities”

So, going back to the tenure system… Universities will “need to adapt to new realities.” From the trenches, you can hear some people talk about this. But I mostly get the impression that many people in academic contexts are either ignoring the broad isuses or focusing too specifically on a subset of what “the new realities” might be.

In the US especially, people tend to talk about “for-profits.” Usually, people are against “for-profits” (at least, among people and blogs I read). The core idea, though, is that a shift is happening and that “for-profits” are part of the new realities. And what are “for-profits,” you ask? From what I gather (just heard recently about this), they’re accredited institutions of higher learning which deviate from some norm as to what colleges and universities should be like. So, it sounds like a variation on the dichotomy between “private” and “public” institutions with a shift in emphasis on the motives instead of the way these institutions are financed. The way they’re described, “for-profits” sound quite different from private universities. But the implied distinction (between “for-profits” and “not-for-profit?”) resonates in some of the same ways.

Online courses are also among “the new realities.” In this context, rarely do people connect these courses with the history of distance education. What puts online courses as part of “the new realities” isn’t really about the context in which learning happens but about some shifts in roles and goals. Part of this is connected to “for-profits” as there’s an embedded business model in people’s minds when people discuss online courses. In a way, an increase in online courses transforms these courses into products, regardless of pedagogical approaches used.

Then there’s the increase in the number of courses taught by “adjunct and contingent faculty.” It’s probably the reason behind the NFM name: the majority of faculty members  is found on the contract-based side instead of the tenure side. The shift is perceived as being mostly negative, since “adjuncts obviously have worse conditions than people in the tenure system.” Besides, ranking systems clearly state, “courses taught by tenured faculty members are inherently better.” (Yes, there was a bit of sarcasm in here….)

There are obvious connections between the adjunct-teaching “new reality” and the previous issues. For one thing, adjuncts teaching online may get even less protection than with offline courses. And one issue people have with “for-profits” is that they mostly hire people through contracts.

What these connections tell me is that we’re talking about a complex system, with numerous parts which are interacting in non-obvious ways. It doesn’t sound like a complicated causal model with a clear set of causes and a clear set of results. Especially if we consider a number of issues which aren’t frequently discussed within the walls of institutions of “higher learning.”

Alternative Learning

For an example of something infrequently discussed within the Ivory Tower: how about learning outside formal institutions of higher learning?

When a similar issue is raised (outside of the Ivory Tower), it’s often in rather provocative terms.

Bill Gates: The Internet Will Replace Universities – PSFK.

With the “money quote”:

Five years from now on the web for free you’ll be able to find the best lectures in the world. It will be better than any single university.

Haven’t listened to Gates’s whole speech but this specific quote, reproduced in a number of places, seems to imply a rather narrow perspective on learning.

Problem is, the counterpoint from universities isn’t that well-composed. “What’s the role of the university when you can get content online? The campus experience! Grades! Credentials!” I love improvisation but these reactions sound like someone trying to catch up with chord changes.

And catching up is perhaps not the most efficient way to adapt and cope to “new realities.”

Not only is the Web making it possible to attend lecture and to get learning “content,” but the world in which we live offers some new opportunities for “learning outside the school”: homeschooling, self-learning, informal learning, personal learning networks, etc. Sure, steady jobs still tend to be associated with degrees that people get, and even grades matter in some contexts. But there also seems to be an increase in contractual hiring outside the “CV and interview” system. No, getting jobs without presenting a CV isn’t likely to become an appropriate solution for everyone. But happy lives achieved without seals of approval from formal institutions represent a challenge many a university system.

Through all of this, what needs to be rethunk about tenure isn’t just tenure itself. It’s how the whole university system fits in a new context. Tenure is just a sore thumb, sticking out in a mangled hand.

Alex's Teaching Tidbits, Comment-fishing, Placeholders

ATT5: EZ Test Online

Leave a comment

Got a number of things to say about testbanks, including a sort of “justification” for my use of them and several comments about how I use them. But I thought I’d post a “tidbit” about a specific testbank system. In a way, it’s more “practical” than other things I have to say about testbanks and practical things fit more in the “tidbit” concept I’m using here. I even thought about doing a tutorial. But tutorials are time-consuming and I’m not sure it’d be more useful or useful to more people than this less-structured post.

EZ Test Online (EZTO) is, as you might expect, an online system to facilitate test creation. More specifically, it’s the online version of the EZ Test program developed by McGraw-Hill to distribute the testbanks which accompany its textbooks as “anciliary material.” Almost all of the features of the standalone version of EZ Test are available in EZTO and the overall user experience is quite similar. Haven’t used the offline version in a while but, if I remember correctly, it also used a Web browser (though it could be used without an Internet connection; basically, it was using the local machine as a simplified webhost).

Obvious advantages of the online version over the offline one:

  • You can work from any machine.
  • You can share testbanks.
  • You can have students take tests directly in EZTO.

Obvious disadvantages:

  • You need an Internet connection.
  • In some circumstances, it might be slightly less secure than an offline program.
  • In some ways, you’re sharing data with McGraw-Hill (though their privacy policy doesn’t really specify much about this).

In my case, EZTO makes a good deal of sense. I tend to use “the cloud” as much as possible, these days, and EZTO is convenient in that sense.

In a way, I wish McGraw-Hill were to go further with EZTO. As with Learning Management Systems, I can easily imagine some connection with the “social Web.” Of course, most people would prefer testbank items to remain protected, hidden, and private. But I can still imagine the sharing of testbanks between instructors to become similar to sharing Web content generally.

In fact, I’m slowly coming to terms with a notion of “transparent testbanks.” Basically, it would mean that all potential questions would be available to the wide public, instead of being restricted to instructors. It’s an extreme case, and one which I think most instructors would think is crazy. But I think it might actually make it easier to control such things as cheating, as counterintuitive as it may sound. Even though I’m typically in favour of radical transparency, this is really an edge case and I’m not that attached to it. But knowing that many exam questions are already shared among students, putting all questions in the open (and, one would suspect, making sure that some very similar questions would be part of those testbanks, to prevent mere memorization of the answers) could have actually make this practice of private sharing of questions more impractical.

After all, there are already some sample questions available in diverse places, including on open access websites created by textbook publishers. These questions are typically meant to help students prepare for exams and quizzes. Using the same questions on exams may have the effect of encorcing rote learning. But if there are enough questions in a centralised testbank which serves both as a practice and as a source for exam questions, the result may actually be to avoid using questions which were already seen by individual students.

But, again, I’m not pushing this model. I’m just wondering about the effects. Part of the reason is because of things like information about the “Rorschach test” having been publicised. Haven’t had to deal with this issue directly and what I’ve heard about it was rather minimal, but the fact that test items can now be shared easily means that we need to think about the effects of sharing. After all, even if one is to limit testbank access to instructors of a given course (with all the necessary processes of verification and validation), it only takes one hacked account to make the whole thing insecure. And even the most careful (and even paranoid) of instructors can be the victim of some form of hacking, including “social engineering.” So the assumption that testbank items can be protected clashes with the ease of “leaking content” online. A transparent testbank is a way to flood potential cheaters with an overwhelming number of potential questions, which is similar to disinformation strategies currently used to protect some private information, including trade secrets.

Even in the case of a “transparent testbank,” I imagine much metadata to remain private. In this I include success rates for each question, answers given by individual students, and comments about use of the question. Other pieces of metadata could potentially be shared too, including “source information” about each question (chapter or page number in a textbook), question type, and even aggregate ratings. These can make testbanks look like so much “user-generated content” online. Which could have some advantages.

I didn’t originally plan to make this tidbit about those issues, but I’ve been thinking about them a bit and it’s probably better to put them “out there.”

What I did plan to talk about is how I use EZTO.

So, in some courses I teach, I do use McGraw-Hill textbooks which are accompanied by testbanks in EZ Test format. In fact, I just finished creating an exam using such testbanks.

This is for an online course using Moodle. Students take the actual exam through Moodle. Students have also been taking reading quizzes, with questions from the same testbanks. The midterm exam focuses on understanding. Students have a full week to take the exam (i.e., there’s no set schedule for everyone to take the exam at the same time). In this context, it seems necessary for each student to receive a different set of questions as other students in the class (otherwise, shared questions would guarantee a near-perfect grade for every student who sees them). With individual versions of the exam, risks related to questions being shared are still a matter of concern, but the effects are much less damaging. (In fact, I haven’t noticed any significant effect from questions being shared.) All questions are multiple-choice and they are graded automatically in Moodle.

The key thing, here, is that I want to have a big pool of questions for the exam (to make sure that very little overlap exists between two versions of the exam) yet I want to make sure the questions are as appropriate as possible. The testbanks I use have many questions which are really about rote learning, and that’s not the kind of exam I want to give. I’m also trying to get a nice balance in terms of the types of questions asked. Otherwise, I’d just dump all the testbanks in Moodle and select the appropriate number of questions from the whole set of testbanks.

I’ve used several methods in the past but I think I found the most useful one in this context. There are likely other methods which work well for other people and/or other contexts. But it’s the one I’m likely to use, for the time being.

Here’s what I did… (This is where a tutorial would make sense!)

  • In EZTO:
    • Uploaded all the testbanks to EZTO (they’re separated by chapter).
    • Created new tests for each question category (according to the testbank): definitions, perspectives, etc.
    • For each test by question category, added questions of that question category from all the chapters. In other words, collected all the questions about definitions from each chapter by using “Select Questions from Banks” with each chapter-based testbank.
    • In each question category testbank, created a new category to select questions.
    • Assigned that new category to questions which seemed appropriate for the exam.
    • Created new testbanks for the selected questions from each question category (again, using the “Select Questions from Banks,” but using the new category to select the questions).
    • Checked and edited each testbank of selected questions by question category.
    • Exported each bank of selected questions by question category to Blackboard format.
  • In Moodle:
    • Created new question categories and new folders to reflect the exported EZTO testbanks (e.g., a folder and question category for questions about “definitions”).
    • Uploaded the files exported by EZTO (“bboard.zip”) into the new folders.
    • Unzipped these Blackboard export files to get multiple files which are all called “res00001.dat” (that’s why it was so important to separate in folders).
    • Imported questions from each “res00001.dat,” making sure the appropriate category was selected.
    • Created a “quiz” activity, with a description of the exam, some information about the process, and some settings (can only be taken once, time is limited to 60 minutes, etc.).
    • Editing the exam, chose each question category and added a number of random questions from that category.
    • Changed the total on which grades will be calculated (60 points total for 60 questions).
    • Previewed the exam.
    • Made the exam visible to students.
    • Announced the exam.

A fairly convoluted process, but one which provides me with remarkably appropriate results. Actually, other attempts I’ve made, in the past, were even more convoluted. For instance, noticing that several questions revolved around the same concept or issue, I would create subcategories from which only one question was randomly picked. It was very time-consuming and the rule of only having one question per issue or concept was self-imposed. Even with a lot of cautious work, it’s almost impossible to get each version of the exam cover exactly the same topics in exactly the same depth as every other version. Given a sufficient number of questions, one can do as if these things even out. Not ideal. But reasonable in this context.

Selecting questions is neither difficult nor that time-consuming. A lot of questions are easy to eliminate, using a few “rules” I have about exam questions (including “no question about factoids from the text”). There are enough questions in the testbanks I use that I need not worry about having enough for the exam. For this midterm, I had more than 800 questions in the relevant testbanks. I selected a bit less than half of these questions as being appropriate for the exam. The exam contains 60 questions. So, basically, I threw away more than half of the testbank questions and ended up with six times the number of questions for each version of the exam. Unless I’m mistaken, the chances that two students have almost identical versions of the exam are low enough to be insignificant (in a group of 25 students). And, though the questions come from a testbank, they’re appropriate for this specific group.

Much of this could be done through another testbank platform. And, of course, the questions could come from a testbank I’ve created. But, in this case, the quality of the questions I selected is sufficient for me to be rather satisfied. I’ve created other exams through similar methods in similar contexts and the results seemed rather appropriate. If problems arise, it’s still possible to work through issues without losing my sanity.

If I sound like I’m justifying myself, it’s probably because I am. There’s something strange about using testbanks, especially when exam versions are randomised. I wouldn’t use the same strategy in other contexts but this is a course for non-major given purely online. It’s also a course for which the textbook really is the main source. And it so happens that the testbanks are rather decent, containing a good number of questions that I find very appropriate.

The parts which are specific to EZTO mostly have to do with how questions are selected and exported. This is a situation in which I wish I had access to ratings for each question, and some way to bulk process the kind of question I don’t want. (For instance, I exclude all questions which have “All of the above” or “None of the above” as the correct answer.) I’d also want to get some way to assess the overall difficulty of the exam so that I could tweak it up or down depending on what seems to be needed.

I also feel like I’m getting too close to standardized tests. In fact, I keep thinking about the GRE and TOEFL (the two standardized tests I’ve taken). I used to react strongly against them but it sounds as if I were getting closer to them, in this context. Exams in my other courses differ significantly from that model, thanks in part to open questions. And I could technically apply some of the same principles in this case, though it’d make things more difficult for everyone. Yet the reasons I use randomized questions from testbanks have to do with the conditions in which I teach this course, much beyond convenience and time commitment.

In such a context, McGraw-Hill’s EZ Test Online makes some sense. It’s “good enough” as a solution to a given task in a very specific set of conditions.

I wouldn’t use it too extensively, though.