![[PukiWiki]](image/pukiwiki.png "[PukiWiki]")
Although the brain is often compared to a computer, the electrical signals function very differently. Your brain contains tens of billions of neurons, each forming connections, called synapses, with thousands of other neurons. When stimulated by sensory information or other neurons, an electrical signal called an action potential spreads throughout the neuron. The action potential causes the release of chemicals called neurotransmitters that spread information to surrounding cells. The pattern of neurons firing action potentials is responsible for producing thoughts and actions.
Potassium in the Brain:
The action potential in neurons depends on electrolytes, mainly sodium and potassium. Low potassium levels cause your brain to slow down. Neurons with low potassium require more stimulation before firing an action potential and cannot fire action potentials rapidly. You may experience this as fatigue, confusion or the inability to start actions or finish trains of thought. The symptoms may be informally called brain fog and will not be corrected by taking stimulants, rest, good nutrition or removing stress.
Spiking neural networks (SNN) are a set of neurons that communicate through spikes and compute through the timing of the spike]. These spiking neurons have become popular since they mimic the spiking nature of biological neurons and can reproduce those neuron spiking patterns. There are cases where SNNs are more biologically plausible and more powerful than non-spiking ones [20] [21]. The Hodgkin-Huxley model [3] is one of the most detailed and best known models of spiking neurons. It clearly describes the subcellular level behaviors, the membrane current generation and propagation of neural spikes. And because of these high level of details, the Hodgkin-Huxley model is too complex to be used for a large scale simulation or hardware implementation. The Izhikevich model [4] is relatively recent model that is simple, has good performance both on computational efficiency and functional richness.
For large scale hardware implementations, simplified models such as integrate-andfire model are preferred. This is due to limited hardware resources for any design and same is the case for this thesis. Such simpler models can emulate the spiking nature of neurons, most of its behaviors and also keep the cost of computation at a comparatively low level [22]. In this thesis, the neuron model used is the leaky, integrate and fire model [Ref.5] which has a lesser level of complexity than Hodgkin-Huxley and Izhikevich, which we think would lead us to a more low power implementation. Details about Leaky, Integrate and Fire models are disccused in forthcoming sections.
Learning in spiking neural networks is devised based on synaptic plasticity change in biology. One of the most important rules for learning in such network is Spike Timing Dependent Plasticity (STDP). The main principle of STDP is that synaptic plasticity is updated according to the difference in spike timing between the pre and post synaptic neurons.
There are many variations of STDP, but all of them follow this basic principle. For example, in the upper figure, when the pre-synaptic neuron j fires within approximately 40ms before the post-synaptic neuron i does, the synaptic weight from j to i'' will increase. The closer the ring time of the two neurons are, the more the synaptic weight will increase. Vice versa, if the pre-synaptic neuron res within approximately 40ms after the post-synaptic neuron, the synaptic weight update will decrease.
Offline training: weights are pre-defined by software training, just need one-time loading to the array; Conventional RRAM with gradual reset only is good enough
Online training: weights are updated during run-time; Special RRAM with both smooth set and reset is needed. Online, real-time learning in neuromorphic circuits have been implemented through variants of Spike-Time Dependent Plasticity (STDP). Current implementations have used either floating-gate devices or memristors to implement such learning synapses together with non-volatile storage. However, these approaches require high voltages (3- 12V) for weight update and entail high energy for learning (4- 30pJ/write.
Definition: According to the characterizing mathematical relations, the memristor would hypothetically operate in the following way: The memristor's electrical resistance is not constant but depends on the history of current that had previously flowed through the device, i.e., its present resistance depends on how much electric charge has flowed in what direction through it in the past; the device remembers its history — the so-called non-volatility property.[2] When the electric power supply is turned off, the memristor remembers its most recent resistance until it is turned on again Wikipedia
...Theinput stimuli are a series of Poisson spike trains, generated artificially and sent via the AER protocol to the chip virtual synapses.
Long Term Potentiation and Memory Formation, Animation
They are responsible for the glutamate-mediated postsynaptic excitation of neural cells, and are important for neural communication, memory formation, learning, and regulation. Ref. https://en.wikipedia.org/wiki/Glutamate_receptor
http://kth.diva-portal.org/smash/search.jsf?dswid=-6128
Caffee is a framework developed by the Berkley Vision and Learning Center and community contributors [JSD+14]. As examples demonstrate, the software can be used to train a state of the art neural networks using the ImageNet data set. It utilizes only the available CPUs, or, if compatible, also GPUs. For the mobile nVidia Tegra K1 GPU, the support was recently added6 .
Torch is another computing framework, which supports deep learning algorithms [CBM02]. It can use GPUs to seed up computation. Recent ports brought support to run on mobile devices like iOS, Android and FPGAs.
Theano also supports deep learning algorithms with efficient calculation of multidimensional arrays [BLP+12]. The python library includes GPU support as well as symbolic differentiation. Support for mobile devices has not been a core feature of the software yet.
Deeplearning4j tries to solve some issues of the other frameworks, namely Java as a more robust and portable language and providing commercial support7 . This enables the framework to run on any platform with Java support. Additionally, strong support to scale over many parallel GPUs or CPUs is implemented.
Tensorflow is an open-source framework for numerical computations, which was recently released by Google [AAB+]. It facilitates large-scale machine learning on distributed systems. Without changing code, it can run on multiple CPUs or GPUs in a server or mobile device.
ynamic Adaptive Neural Network Arrays: A Neuromorphic Architecture.pdf