There are many different types of neurons and synapses within biological neural systems, along with
other biological components such as glial cells; different areas of the brain have different neuron types,
synapse types, connectivity patterns, and supporting systems [REF].
Artificial neural networks have taken inspiration from different areas of the brain for different types of neural networks.
For example, the structure of convolutional neural networks is inspired by the organization and layout of the visual cortex
[LeCun2015], whereas hierarchical temporal memory (HTM) takes its inspiration from the organization of the neocortex [Hawkins2016, WSP:Kudithipudi] (Figure 7). When considering the model selection, it may be worthwhile to target a specific functionality or set of applications and take inspiration from a particular part of the brain that performs that functionality well in determining the characteristics of the model.
SNN consists of spiking neurons that do not generate outputs at each time step like other artificial neutral networks. Instead, a neuron produces a spike asynchronously when
its membrane potential (Vm) reaches a specific value. The information transfer in SNN hence takes place through precise spiking time or rates of spikes.
Spiking neural network implementation using CPU and CUDA engine
Offline training: weights are pre-defined by software training, just need one-time loading to the array; Conventional RRAM with gradual reset only is good enough
Online training: weights are updated during run-time; Special RRAM with both smooth set and reset is needed
Online, real-time learning in neuromorphic circuits have been implemented through variants of Spike Time Dependent Plasticity (STDP). Current implementations have used either floating-gate devices or memristors to implement such learning synapses together with non-volatile storage. However, these approaches require high voltages (3- 12V) for weight update and entail high energy for learning (4- 30pJ/write.
Hebbian learning is one of the oldest learning algorithms, and is based in large part on the dynamics of biological systems. A synapse between two neurons is strengthened when the neurons on either side of the synapse (input and output) have highly correlated outputs. In essence, when an input neuron fires, if it frequently leads to the firing of the output neuron, the synapse is strengthened. Following the analogy to an artificial system, the tap weight is increased with high correlation between two sequential neurons.
![[PukiWiki]](image/pukiwiki.png "[PukiWiki]")
ynamic Adaptive Neural Network Arrays: A Neuromorphic Architecture.pdf