From: Forrest Bishop (forrestb@ix.netcom.com)
Date: Thu Jul 25 2002 - 11:59:02 MDT
----- Original Message -----
From: Anders Sandberg <asa@nada.kth.se>
To: <extropians@extropy.org>
Sent: Wednesday, July 24, 2002 11:02 PM
Subject: Re: Scientific output
> On Tue, Jul 23, 2002 at 08:08:54PM +0200, Joao Magalhaes wrote:
>
> > Counting publications is imperfect
> > too. After all, only a small percentage of scientific papers leads to
> > scientific revolutions; in other words, the great majority of papers being
> > published are of little importance, so why count them?
>
> I just got another idea about dealing with this, based on the Way of
> Google. If you count the number of citations of a paper you get a
> rough measure of its importance, and presumably the breakthroughs are
> sets of linked widely cited papers. The problem with this is of course
> that some breakthroughs take time to be recognized, so ideally they
> should be evaluated frim time=infinity. So an approximation would be to
> look for the number of widely cited papers after a set time, like two
> or five years. If you count the number of papers with citation levels a
> few quartiles above median, then you might get a kind of breakthrough
> measure. Still hardly perfect, but calculable.
The utility of mathematical data reduction and analysis of highly complex systems runs into several severe restrictions.
Scientific "output" (I would not use this word) is part of the much larger world of human affairs, which is in turn an aspect of the
natural world. Attempts at rigorous quantification of a single aspect immediately run up against the sampling problem:
a) what data should be included or excluded?
b) what relative weighting should be assigned to what?
c) How should the natural world be divided into catagories?
d) who shall judge the judges?
The phrase "scientific output" implies (imo) a model similar to the input/output econometric models. In this model the
individual, subjective value of an artifact, be it an eating utensil or a scientific theory, is submerged in aggregate indexes of
the author's own design. The aggregate index of patents issued or papers published cannot accurately measure either progress or
degeneracy. Crank theories such as Freudianism, Keynesianism, Idealism, Dialectical Materialism, or Critical Analysis can have
massive numbers of supporting cites without adding the slightest validity to the concepts. One could just as easily use this as a
measure of anti-scientific progress, by simply re-framing the division of the world (item c) above).
There are a few books and papers extant on the Citation Index game: publishing some quackery or ad hominem attack simply to induce
others to cite the paper, republishing essentially the same material in different journals, attaching Dr. Maximum Dude's name to
grad student work, various granstmanship games, and so forth. This hardly qualifies as progress.
Forrest
-- Forrest Bishop Chairman, Institute of Atomic-Scale Engineering www.iase.cc
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:15:40 MST