Inhibitory Nodes

    I've long conjectured that the reason we have an inhibitory network in our brains is to prevent an epileptic burst and it seems that this thought has some truth. In my early simulations of neurons, I found quickly that excitatory networks would get to a tipping point where they would just start firing uncontrollably; i.e. every neuron (node) would be firing as fast as it could and the output would end up being a jumbled mess of motor neurons all firing, all the time. I even continue to experience this sometimes in more true emulations due to the timing or the number of neurons aren't in a correct balance so that the inhibitory neurons aren't in a great enough strength to overcome the excitation. 

In AI/Deep Learning (et al), inhibitory networks don't exist. The excitatory network is refined by the error factors and outcomes are created by numeric voting. DL tunes an excitatory network by weighted factors. 

There is a delicate balance between the excitatory networks and inhibitory networks. Too little inhibition creates runaway networks (epileptic bursts), and too much dampens the excitation to a point where you get extremely little or no output. Combine that with your computer and network capabilities makes this process very difficult to tune. 

What makes inhibition operate so well is that it allows the excitatory network to operate fully and comes to play as the excitation continues. If Neuron E is excitatory and Neuron I is inhibitory, Neuron E excites Neuron E2 and Neuron I at the same time, but Neuron E2 could meet its threshold and fire before Neuron I starts to inhibit Neuron E.  

This animated image shows how excitation occurs even though inhibition is also at play

In the mammal neocortex, there are different types of inhibition. Some inhibitory neurons are direct inhibitors whereas others are part of the network pathways. There are roughly three (3) types of neural inhibitors in the neocortex:

Source: Brain Computations by Edmund T. Rolls

PV/BC (Parvalibumin) neurons which are capable of firing rapidly and with high temporal precision. There are two types which either target the soma and proximal dendrites, or those that target the axon at the initial segment. To use a more colloquial paraphrase, these neurons mean business.

SOM (Somatostatin) neurons target the tuft dendrites of principal cells, They are implicated in behavior-dependent control. SOMs also inhibit other inhibitory neurons.

NG/VIP (5HT3a-receptor expressing interneurons) are the most numerous and thought to be involved in control of higher order cortical circuits. Note that VIP (vasoactive intestinal peptide) neurons also inhibit inhibitory neurons (SOM).

When we discuss AGI, we must include the fact that for AGI to happen, it must be continuous and the governor that keeps the AGI network under control is an inhibitory network. I will never be able to envision an AGI system using the current AI techniques because these are purely excitatory networks constantly tuning over steps in time. Over fitting and underfitting become exponentially greater problems as the network is expanded. It all works well with a few hundred of thousand nodes but when we talk about billions with trillions of connections, good luck!

I can't pretend to be a Neuro-symbolic AI expert but even with this approach, it seems to me to be caught in the non-continuous methodology so an inhibitory network would have no place in its development. 

Inhibition is extremely important for continuous (learning) systems. In order to control who's on first, we must have the means to filter that position. Excitatory networks alone cannot pull this off.