Webster's defines
evaluation as follows: To determine the significance,
worth, or condition of usually by careful appraisal and study.
The words "careful appraisal and study" serve to remind
us that evaluation is a complex process requiring mental effort
in order to establish worth or value.
Evaluation of
scholarly writing has always been a central activity in the
academic enterprise. Evaluation of student work is an important
duty of teaching faculty; M.A. and PhD theses are "defended"
before a group of experts whose role is to evaluate what the
candidate has written; scholarly manuscripts are "reviewed"
by experts in the field before publication in the form of books
or articles. Evaluation of scholarly publications on the Web
is just as central and essential as any of these forms of evaluation.
The Web presents
scholars with the opportunity to do research and to publish
at unprecedented speed. Scholars have the potential to send
each other their research results and have comments and reactions
back in a matter of days or even hours. In some disciplines
web based discussion groups have replaced the academic journal
as the medium of scholarly communication. Within small homogenous
groups of scholars who know each other and are engaged in similar
work, review and evaluation take a back seat to speed and convenience.
However, the Web is also home to a growing amount of research
publication of a more traditional sort - journals, reports,
monographs. It is this web publication that presents some challenges
in the area of evaluation and quality control.
Because there
are no expert editors operating peer review systems controlling
much of what gets published on the Web, the responsibility for
evaluation of scholarly material is thrown back upon the user
to a much greater extent than is the case with research results
published in the traditional manner.
Scholars must
approach research material on the web with the same healthy
scepticism they exercise when using research published anywhere
else. In using this new medium, however, we must also develop
new expertise and apply new criteria. Web evaluation is a subject
that is covered fairly well on the Web, and I have included
links to several sites that deal with this topic. Also, although
there is no control over who can put up a web site, expert editors
are emerging who are attempting to create high quality sub sets
of the Web in the various academic disciplines, including Anthropology.
Their sites, some of which I have listed below, can legitimately
be placed under the heading of evaluation, since they include
links based on sets of criteria. Lastly, I have included some
information about listservs and newsgroups, in the belief that
an important source of information about high quality research
sites is other scholars who are using the web. Messages posted
to listservs often include tips about good sites and mini reviews
of sites the author liked or didn't like.
Evaluation
Criteria:
A standard list
of criteria for evaluating web sites can be found at
this address:
http://www.vuw.ac.nz/~agsmith/evaln/index.htm
It is the work
of Alistair Smith who teaches Library and Information Studies
at Victoria University of Wellington New Zealand. It provides
a basic, common sense list of issues to consider when assessing
a scholarly publication or site on the Web. Like many other
lists of criteria, Smith's identifies the core concerns of the
scope of the work, the timeliness and accuracy of contents,
and the authority of the creator of the site. These sorts of
measures are the same ones we would apply to work published
in other formats. Then he includes issues specific to the web
site - such things as user friendliness and the hardware and
software needed to access the site.
Gateway
Sites On Evaluation:
There are several
gateway or guide sites which attempt to bring together web based
materials on the topic of evaluation. One of them is:
Alistair
Smith maintains such a page on the World Wide Web Virtual
Library site. The address for this site is:
http://www.vuw.ac.nz/~agsmith/evaln/evaln.htm
Anthropology
Gateway Sites:
Also called guide
sites, hub sites, index sites, and other names, these sites
collect and organize links in a subject area. When you start
your search at a guide or gateway site, you are benefitting
from the evaluation work that someone else has done in choosing
the links in the site. A few examples are:
1. Louisiana
State University Library Anthro site - links are organized
in categories: College Departments, Government Resources, Listservs,
Organizations, Periodicals, Projects, Reference Materials, Other
Indexes
http://www.lib.lsu.edu/soc/anthro.html
2. Anthropology
Resources on the Net - an excellent example of a comprehensive
index or hub site with many useful categories, including one
for free software available to anthropologists, a listing of
on line bibliographies and a "jobs" link.
http://www.anthropologie.net/
3. Academic
Info - Anthropology is, to quote the "about us"
blurb: "...an
independent Internet subject directory compiled and maintained
by Mike Madin with the assistance of a volunteer group of subject
specialists... Academic Info aims to be the premier educational
gateway to online college and research level Internet resources."
Links are added
on the basis of a collection policy, which includes factors
like: "The primary focus of the site must be academic",
"The site must be kept current. The weeding of broken and
outdated links is as important as adding new ones.", and
"Do not include support groups, advocacy or tribute pages."
The Anthropology
section is found at this address:
http://www.academicinfo.net/anth.html
4. WWW
Virtual Library: Anthropology. Again I'll quote from
their "about us" blurb:
"The WWWVL
is the oldest catalog of the web, started by Tim Berners-Lee,
the creator of the web itself. Unlike commercial catalogs, it
is run by a loose confederation of volunteers, who compile pages
of key links for particular areas in which they are expert;
even though it isn't the biggest index of the web, the VL pages
are widely recognized as being amongst the highest-quality guides
to particular sections of the web."
The editor for
the Anthropology pages is Eliot Lee, a young man who combines
knowledge of the technology with an M.A. in Anthropology. The
Anthropology pages for the WWWVL are at:
http://vlib.anthrotech.com/
Google
- a search engine attempts to address quality
The use of a
set of criteria is an attempt to make evaluation a rational
process, but in their application the process inevitably becomes
subjective - you or I have to make a decision about how well
a site conforms to the criterion in question. There are examples
of attempts to make evaluation less subjective. An interesting
case, which is undoubtedly only the first of many, is the search
engine Google which uses PageRank software developed by company
founders Larry Page and Sergey Brin. To quote from the Google
site:
"PageRank
relies on the uniquely democratic nature of the web by using
its vast link structure as an indicator of an individual page's
value. In essence, Google interprets a link from page A to page
B as a vote, by page A, for page B. But, Google looks at more
than the sheer volume of votes, or links a page receives; it
also analyzes the page that casts the vote. Votes cast by pages
that are themselves "important" weigh more heavily
and help to make other pages "important."
Important, high-quality
sites receive a higher PageRank, which Google remembers each
time it conducts a search. Of course, important pages mean nothing
to you if they don't match your query. So, Google combines PageRank
with sophisticated text-matching techniques to find pages that
are both important and relevant to your search. Google goes
far beyond the number of times a term appears on a page and
examines all aspects of the page's content (and the content
of the pages linking to it) to determine if it's a good match
for your query."
http://www.google.com/technology/index.html
So, while many
search engines are offering to give web pages a higher ranking
if the creators pay for it, Google's software attempts to find
high quality sites that match the search terms.
Listservs:
Are loose associations
of people interested in the same thing. You must join a listserv
by registering your email and then you can post messages and
receive messages posted by others. Some are carefully moderated
by dedicated souls who do such things as group messages under
topics, which can be a big time saver for members. Scholarly
listservs can be ENORMOUS time savers in another way by giving
you a community of people to ask questions of. People have sent
me complete bibliographies on topics, including citations I
had not found anywhere else, in response to questions to listservs.
Newsgroups are similar to listservs but they are unmoderated
sites very like a chat room where people can post messages and
have discussions with one another. One way to find listservs
and newsgroups in Anthropology and Archaeology is by looking
at the following site:
Tile.net:
a database containing links to listservs and newsgroups. It
has a search engine which allows you to search for the term
"anthropology" and this will result in a list of listservs
and newsgroups dealing with your topic, including links.
http://www.tile.net/
Examples
for Evaluation: Kennewick Man
On July 28, 1996,
a skeleton was found in the bank of the Columbia River near
Kennewick in Washington State. The bones were carbon dated to
8410 years before the present. Subsequent carbon dating has
confirmed their age as being between 5700 and 8400 years old.
Under the 1990
Native American Graves Protection and Repatriation Act (NAGPRA),
any person living in North America before the arrival of Columbus
is considered to be Native American. On the basis of this legislation,
several first nations groups in the area of Kennewick claimed
that the bones were those of their ancestors and should be turned
over to them for a traditional burial.
A group of anthropologists
who wanted a chance to study this ancient and rare find filed
a law suit arguing that the application of NAGPRA in this case
was unscientific and legally flawed. Earlier this year the court
asked the National Park Service to reach a conclusion about
the cultural origins of Kennewick Man, as the skeleton came
to be called, on the basis of various kinds of tests, including
DNA testing. While it proved impossible to secure enough material
to do DNA testing, the Parks Service declared, in September
of this year, that the bones were culturally affiliated to the
local aboriginal groups. The lawsuit brought by the anthropologists
was re activated at the end of October.
This topic seemed
like a good one to use as an example for web site evaluation.
If we are looking for anthropological information on the skeleton,
we are likely to find it, but we are also likely to find a lot
of non-scholarly material, because of the attention this incident
has attracted as a "news story". Here are some sites
to evaluate. Look at them and apply the criteria we have discussed
above.
1.
Laura Lee Show Coverage of Kennewick Man
http://www.lauralee.com/glover2.htm
A Web "Show",
formerly a syndicated radio show. Laura Lee's guests tend to
be experts in the paranormal. Her mission statement says that:
"This
show serves as a forum to meet the researchers of tomorrow today,
to get on the cutting edge while it's still sharp. Worldviews
are meant to be examined, explored and expanded."
2.
National Park Service Site
3.
Burke Museum of History and Culture at the University of Washington
Kennewick Man Site
