From: Metaqualia (metaqualia@mynichi.com)
Date: Thu Oct 30 2003 - 17:00:45 MST
I was thinking of a universal pattern recognition machine, something that
will work to recognize whatever, no matter the stimulus. it will recognize
automatically configurations with low entropy and high organization in the
sensory input. The same machine could learn to do audio or video analysis or
solve problems, whatever.
I asked myself, in the simplest case, what would it take for a basic layer
of neurons that are activated with light intensity, to make the next layer
wire itself to detect edges? It would be great if there was a way to let
white noise filter itself out and for low entropy configurations to emerge
spontaneously! This way layer by layer the system would adapt to the
external reality, modeling anything in it.
So this is how it would work (!!)...
1
let's say we have two adjacent neurons a and b.
most times the difference between their input will be little since they are
very close. but at times the difference will be large because there is an
edge there. so the trick would be, let the next layer rearrange itself so
that only this unlikely configuration can get through.
2
let's say we have two adjacent bits of the image, one has an edge detection,
the other also has an edge detection. This configuration is quite uncommon,
because if you take a random point just around an edge detector that fired,
unless you pick a point that is also on the edge line, this second point
won't be firing (assuming the edge is sharp). So locally, for a bit of line
to exist, this is an unlikely situation.
3.
let's say we have a spot at which a bit of a line was detected. It is very
unlikely that from the center point another line will be going in another
direction. In other words, the neighborhood will contain more empty space
than space containing bits of lines. Locally, the occurrence of an edge is
not likely. So an edge filter is what comes out of this stage
4.
When we have detected a corner, or a feature, then right next to it there
will be features and non-features (think of the edge of a cube; in three
directions we will find more edges, but not in the other directions). Very
unlikely that in the proximity of a feature there will be another feature,
so features are grouped together forming superfeatures (imagine the corner
of a cube plus the three adjacent edges).
Of course levels can communicate with each other so that if a 3d corner + a
bit of line for some reason form a low entropy configuration then the system
will detect this pattern in some way.
So.... would this algorithm work? Provided there was a way to physically
filter out the locally commonly occurring configurations and only keep the
rare ones, of course.
What if neuron connections were actually weakened by activation? Then,
during sleep, all connections were strengthened. In this case, rare stuff
would build strong connections because it wouldn't be so depleted during the
day time. But very frequently occurring patterns (such as two adjacent
pixels of nearly the same color) would get weaker and weaker. This would
explain the need for sleep by the way.
When we see something every day we stop noticing it; could this mechanism be
the reason? I am not pushing too hard the neuron weakening hypothesis, but
the important part is, if there was a way to filter out locally common
percepts, could a universal pattern detector be built? With many layers,
would it automatically learn to detect 3d shapes from 2d pictures, to
account for shadows, occlusions etc?
This same algorithm could be used to create a mind, I haven't had enough
time to think about how the layer structure would work, but it would
probably get its first input from the high level 3d percepts which include
objects, actions, actors, transformations.... patterns would be detected in
time between actions and objects, between transformations of objects and
serotonin release, so that actions probably leading to pleasure would be
linked all the way down to the steps needed to achieve the pleasure, and
that's visually based human-style planning.
So things occurring less often *locally*, are the most important, such as
any two random things being cause and effect. Things occurring more often,
such as any two random things being completely unrelated, these are white
noise and cancel out.
I will give it some more thought....
curzio
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:43 MDT