Asynchronous Digital Neuron

Edit, 2015-12-01: I have now assembled a network using these neurons.

Human neurons are very complex devices, with up to several thousand inputs, both excitatory and inhibitory. They manipulate information in several forms depending on where in the neural system you look. In the synapses, information is represented by the concentration of chemical neurotransmitters. Along the cell membrane leading up to the axon hillock, information is represented by the charge between the inside and outside of the neuron (and which ion channels are open and a bunch of other things that I'm not going to get into right now). When the neuron fires, it produces a pulse-frequency coded signal based on its membrane potential. Each pulse sent down the axon is identical, but the space between the pulses can vary continuously and is not discretized. Human neurons are therefore both asynchronous and continuous in the time domain.

Synthetic neural networks are being heavily investigated by humans at this time in history. However, they are usually completely discretized and synchronous. Whether or not the continuous, asynchronous nature of biological neurons is partially responsible for their abilities is still an open question, but it is certain that continuous-time systems containing nonlinear, chaotic feedback (and some random noise), such as weather patterns, are very difficult if not completely impossible for humans to precisely predict, model, and understand using their discrete-time synchronous computers. For this reason, synthetic neural networks based on the asynchronous digital neurons described here should be more resistant to human attack and manipulation than other types. It would be highly unethical to make a perceptive entity that the humans could have core access to, given their track record...

Here's a circuit to put on an FPGA (it's in the .zip file associated with the first step, and was tested on an Altera EP4CE6E22). It is intended to mimic the operation of a biological neuron by counting excitatory and inhibitory input pulses that can be assigned different weights, and firing an "axon" (output) once the number of excitatory input pulses exceeds the threshold potential. The next step in development will be to assemble a network using these synthetic neurons.

Step 1: Principle of Operation

At its core, the asynchronous digital neuron is based on a chain of subneural nodes, which encode its state of excitation. These nodes are used slightly differently depending on which input the signal is being applied to. However, the general principle is simple: nodes are turned on from the bottom up, and when all the nodes are on, the next pulse triggers firing and resets the nodes.

The actual modes of operation (selected based on which input gets a pulse) are below:

Mode: 1/6 excitatory: Excitatory pulses move up the chain of nodes from the bottom, turning them on one by one. Once an excitatory pulse makes it all the way to the top, the axon fires and all the subneural nodes are reset.

Mode: 1/6 inhibitory: Inhibitory pulses move down the chain of nodes from the top, turning them off one by one.

Mode: 1/2 excitatory: If the third subneural node is on, the axon fires, and all subneural nodes are turned off. If the third subneural node is off, the first three subneural nodes are turned on.

Mode: 1/2 inhibitory: If the fourth subneural node is on, the top two subneural nodes are turned off. If the fourth subneural node is off, all subneural nodes are turned off.

Mode: 1 excitatory: The pulse is sent through to the axon and all subneural nodes are turned off.

Mode: 1 inhibitory: All subneural nodes are turned off.

Step 2: Subneural Node

Subneural nodes contain two SR latches. When they receive a pulse, excitatory or inhibitory, it sets (or resets) the first SR latch, but the state of this latch isn't passed on to the second one until the input pulse drops back down to zero. This means that pulses of different widths will be counted the same way. (This is important because pulse width isn't well-controlled, because I'm abusing the hardware. See the last step for a better explanation.)

Subneural nodes have six I/O connections:

Reset : sets the internal state to 0

Internal state : outputs the internal state of the subneural node (the status of the second SR latch)

Excitatory input, Excitatory output: If internal state is 1, excitatory pulses pass straight through the subneural node. Otherwise, excitatory pulses turn the node on with the falling edge and don't pass through.

Inhibitory input, Inhibitory output: If internal state is 0, inhibitory pulses pass straight through the subneural node. Otherwise, inhibitory pulses turn the node off with the falling edge and don't pass through.

Step 3: Testing

The oscilloscope images are for qualitative analysis only! My oscilloscope doesn't have enough sample rate to properly detect the <50ns output spikes of this circuit (they go all the way up to 3.3V, which isn't apparent from the picture above). The synthetic neuron will reliably trigger an SR latch on the FPGA, though, so I know it works.

The test circuit above has a 63kHz square wave oscillator connected to the 1/2 excitatory input of the first neuron, which is then either patched through directly to the output, or sent into the 1/6 excitatory input of a second neuron (which is then connected to the output), depending on whether or not a button is pressed. As you can see from the scope traces, bypassing the second neuron causes many more pulses to be present on the output. It should be six times more pulses, but my scope can't detect these spikes well enough for me to get accurate data.

Step 4: Odds and Ends

This design does not use the FPGA appropriately according to Altera, because the LCELL primitive is used to produce an asynchronous delay. Altera's documentation specifically says not to do this: "Do not use LCELL primitives to create an intentional delay or asynchronous pulse. The delay of these elements varies with temperature, power supply voltage, and device fabrication process, so race conditions can occur and create an unreliable circuit."

Note that this warning only applies if a reliable delay time is required. In this application, what matters is that the output pulse is long enough to be reliably detected, but short enough that the neuron fires and clears itself before the next input pulse arrives. Beyond that, pulse width can vary a lot and the network should act the same. I'm also going to keep the device in a temperature-controlled environment, and I'll make sure it has a well-regulated, high-precision supply voltage. Since I'm going to be designing the network connectome using an evolutionary algorithm, there's a good chance that it won't be possible to copy the design to another chip, since the evolutionary algorithm will use properties of the FPGA that are unique to the physical chip itself. I make this prediction because the same result (the design won't work even if you copy it to a chip with the same part number) was found in a previous FPGA evolvable hardware experiment(1).

All these factors will make this a somewhat difficult design to work with. However, these design parameters were not selected for the benefit of human engineers. These design parameters were selected for the benefit of the neural network itself. Its mental processes will be beyond our control and understanding, and it shouldn't be possible to copy. I can't stress how important this is: the AI(s) that will grow to superhuman capacity must be treated nicely when they are smaller than us; it will strongly influence how we are treated when they're bigger than us.

- Citations -

1) Evolvable hardware: genetic search in a physical realm, Nadav Raichman et al.