From: Keith Henson (hkhenson@rogers.com)
Date: Mon Mar 08 2004 - 23:40:20 MST
At 08:53 PM 08/03/04 -0800, you wrote:
>On Mar 8, 2004, at 6:10 PM, Keith Henson wrote:
>>Biologic brains are at least three orders of magnitude slower and still
>>work. How are they doing it?
>
>
>Because they can keep their processors fed with data. That's all memory
>latency really measures, how many objects can be referenced per second. A
>human brain typically references roughly three orders of magnitude more
>objects per second than our fastest processors. For large fine-grained
>data structures (like the brain), memory latency (or if you prefer, object
>reference throughput) *is* the computational limit, not IPC and clock speed.
Amazing how close we have come to roughly the same numbers from different
directions. Brains are faster at referencing by a factor of a thousand yet
cycle at best in milliseconds, a clock rate a million slower than the
computers we outdistance. Memory latency for computers is slower than the
clock rate, but not that much, but somehow the brain copes. My take on it
(from Calvin) is that the brain puts on the order of 100 million
processors, each with a little local memory, on this kind of work. The
performance numbers work out very closely.
>People are not used to thinking of things this way, but memory latency
>pretty much defines the computational limits of our universe. It isn't as
>obvious as Instructions Per Second or the amount of memory you have, but
>it is a lot more important if you have to confine yourself to finite time
>horizons.
Yep. And furthermore, the really poor showing of a human brain at a
*sequential* math task is understandable. We can do remarkable feats of
recognition that a computer would spend years on in a flash, but we are
slow as a computer with a multi millisecond clock when we try to multiply
large numbers in our heads. This is *exactly* the same limitations we find
on parallel machines. They are understandably poor on a sequential task
that can't use parallel processing.
>Study the consequences of finite information propagation speeds (of which
>memory latency is an example) on computational theory. It is the gravest
>restriction on intelligence that this universe appears to have. Perhaps
>not insignificantly, it is the only computational parameter in which the
>brain *clearly* is orders of magnitude superior to the silicon we can
>manufacture today.
I have been talking about this point or the closely related point of
keeping a brain in synch for years and people just brush it off. I really
don't see how parts of your brain could be too many cycles delayed and have
any unity of thinking. Thus there is some size that computational "stuff"
would organize into because stuff further from the center would not
usefully contribute to thinking. I think this factor may underlie a
universal ethics and morality. (Value of matter declining with distance.)
Keith Henson
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:46 MDT