From: Aaron Davidson (ajd@ualberta.ca)
Date: Sun Sep 12 1999 - 00:04:35 MDT
>On Fri, 10 Sep 1999, J. R. Molloy wrote:
>
>The problem isn't "wiring them together" but having them "do
>something" once wired. Nobody has ever had a 70 million cell
>neural net before. How deep do you make it? How broad do you
>make it? How many inputs & outputs should each neuron have?
>
>I'm not sure how flexible the de Garis architecture really is
>but if the depth can be up to 10 deep, and the net has
>1000 inputs and 100 outputs, and each neuron can interact with 500
>others, the possible number of configurations is huge. [I'm not an
>expert on neural nets, so if someone can explain this better
>please do so.]
It gets even worse than this -- and what you mention above still is indeed
an active researche problem. There isn't even a good formula for choosing
depth as a function of say, input size. Once you have the physical layout
of the NN, there are dozens of different ways to connect the neurons. There
are timing issues, and there are a kazillion ways to train the net. The
learning algorithm is very important because in solving different problems,
some algorithms which work great for one type of domain get easily stuck at
local maxima or oscillate wildly, in another domain.
Some attempts at solving these problems involve dynamically reconfiguring
nets, that change their structure as they learn. Neurons are killed off if
they are deemed to be hindering or useless. New layers can be removed if
the net is too large to generalize a problem (and instead turns into a
giant look-up table), or added if the net is underfitting the data.
Genetic Algorithms have been used to try and find optimal organizations.
The list goes on and on. No one has yet found a way to build a general
purpose neural net system. We still need to custom design NN's to work well
for different domains.
--Aaron
+-------------------------------------------------------------------------+
| Aaron Davidson <ajd@ualberta.ca> http://ugweb.cs.ualberta.ca/~davidson/ |
+-------------------------------------------------------------------------+
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:05:07 MST