ASL Wiki
Specila ISssues†
Neuron models - There are three types of neurons:†
- 1. Binary signal neuron: The binary neuron model was jointly developed by McCulloch and Pitts in 1943. This model takes the weighted sum of the inputs and then compares the result with a threshold value: if the sum is larger than the threshold value, the neuron will give 1 as output, otherwise the output will be 0.
- 2. Continuous value neuron: The continuous value neuron model, as its name suggests, is different from the binary neuron model in the way that its output is a continuous value instead of a binary one. The activation
function of this neuron model is usually a sigmoid function like hyperbolic tangent or logistic. The output of a neuron can be interpreted as either the value itself, or as the probability of producing 1 as output. Most of the state-of-
the-art Machine Learning algorithms employ this type of neuron model.
- 3. Spiking neuron: The third type of neuron model is the spiking neuron model. This type of model takes spiking events as input and also outputs spiking events. Information is stored in the timing of spike
events instead of being interpreted as spiking frequency like in the binary and continuous value neuron models. Spiking neuron model is considered the most biologically plausible (معقول) among the three types, as transmitting spikes is how real neurons communicate with each other. In such a model, a neuron computes the weighted sum of all the spiking input currents integrated over time; when the membrane potential rises above a certain threshold, the neuron res a spike. A spiking neuron model usually can be described using an electronic circuit or a set of ordinary di erential equations, such as: leaky integrate and re model, Izhikevich model, Hodgkin - Huxley model, etc.
ANN Classification†
Neuromorphic†
SNN†
Spiking neural networks (SNN) are a set of neurons that communicate through spikes and compute through the timing of the spike]. These spiking neurons have become
popular since they mimic the spiking nature of biological neurons and can reproduce those neuron spiking patterns. There are cases where SNNs are more biologically
plausible and more powerful than non-spiking ones [20] [21]. The Hodgkin-Huxley model [3] is one of the most detailed and best known models of
spiking neurons. It clearly describes the subcellular level behaviors, the membrane current generation and propagation of neural spikes. And because of these high level
of details, the Hodgkin-Huxley model is too complex to be used for a large scale simulation or hardware implementation. The Izhikevich model [4] is relatively recent
model that is simple, has good performance both on computational efficiency and functional richness.
For large scale hardware implementations, simplified models such as integrate-andfire model are preferred. This is due to limited hardware resources for any design
and same is the case for this thesis. Such simpler models can emulate the spiking nature of neurons, most of its behaviors and also keep the cost of computation at a
comparatively low level [22]. In this thesis, the neuron model used is the leaky, integrate and fire model [Ref.5] which has a lesser level of complexity than Hodgkin-Huxley and Izhikevich, which we think
would lead us to a more low power implementation. Details about Leaky, Integrate and Fire models are disccused in forthcoming sections.
- SNN consists of spiking neurons that do not generate outputs at each time step like other artificial neutral networks. Instead, a neuron produces a spike asynchronously when
its membrane potential (Vm) reaches a specific value. The information transfer in SNN hence takes place through precise spiking time or rates of spikes.
- Spiking neural network implementation using CPU and CUDA engine
- Ref. 5 Z. Wang, L. Guo, and M. Adjouadi, A biological plausible Generalized Leaky
of the IEEE Engineering in Medicine and Biology Society, 2014, pp. 6810–6813.
CNN†
RNN†
Learning in spiking neural networks is devised based on synaptic plasticity change in biology. One of the most important rules for learning in such network is Spike Timing Dependent Plasticity (STDP).
The main principle of STDP is that synaptic plasticity is updated according to the difference in spike timing between the pre and post synaptic neurons.
There are many variations of STDP, but all of them follow this basic principle. For example, in the upper figure, when the pre-synaptic neuron j fires within approximately 40ms before the post-synaptic neuron i does, the synaptic weight from j to i'' will increase. The closer the ring time of the two neurons are, the more the synaptic weight will increase. Vice versa, if the pre-synaptic neuron res within approximately 40ms after the post-synaptic neuron, the synaptic weight update will decrease.
Offline training: weights are pre-defined by software training, just need one-time loading to the array; Conventional RRAM with gradual reset only is good enough
Online training: weights are updated during run-time; Special RRAM with both smooth set and reset is needed. Online, real-time learning in neuromorphic circuits have been implemented through variants of Spike-Time Dependent Plasticity (STDP). Current implementations have used either floating-gate devices or memristors to implement such learning synapses together with non-volatile storage. However, these approaches require high voltages (3- 12V) for weight update and entail high energy for learning (4- 30pJ/write.
- 1.G.-Q. Bi and M.-M. Poo, \Synaptic modi cations in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type," The Journal of Neuroscience, 1998. 10
Simulation†
- Spike Simulator -> fast event-driven simulator, called Spike, for simulating large networks of simple spiking neurons.
Applications†
- 2014 Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems Chicca, Elisabetta, Stefanini, Fabio, Bartolozzi, Chiara, Indiveri, Giacomo, Vol. 102, No. 9, September 2014 Proceedings of the IEEE
Memristor, others†
Definition: According to the characterizing mathematical relations, the memristor would hypothetically operate in the following way: The memristor's electrical resistance is not constant but depends on the history of current that had previously flowed through the device, i.e., its present resistance depends on how much electric charge has flowed in what direction through it in the past; the device remembers its history — the so-called non-volatility property.[2] When the electric power supply is turned off, the memristor remembers its most recent resistance until it is turned on again Wikipedia
Conductive-bridge random access memory (CBRAM)†
References/Projects†
Swiss - EPFL Lauzanne†
- [[PhD Thesis: Supernumerary Robotic Arm for Three-Handed Surgical Application: Behavioral Study and Design of Human-Machine Interface.]]
Neuromorphic Computing at Tennessee†
Europe†
Japan†
- UEC: The University of Electro-Communications, Brain Science Inspired Life Support Research Center
Multicast Routing†
- [41]Joshi, S.; Deiss, S.; Arnold, M.; Jongkil Park; Yu, T.; Cauwenberghs, G.; , "Scalable event routing in hierarchical neural array architecture with global synaptic connectivity," Cellular Nanoscale Networks and Their Applications (CNNA), 2010 12th International Workshop on , vol., no., pp.1-6, 3-5 Feb. 2010
- [44] Matos, D.; Concatto, C.; Carro, L.; Kastensmidt, F.; Kreutz, M.; Susin, A.; , "Highly efficient reconfigurable routers in Networks-on-Chip," Very Large Scale Integration (VLSI-SoC), 2009 17th IFIP International Conference on , vol., no., pp.165-170, 12-14 Oct. 2009.
How to Generate Artificial Spike train†
...Theinput stimuli are a series of Poisson spike trains, generated artificially and sent via the AER protocol to the chip virtual synapses.
Tools and†
Workshops†
Funds†
New Topics†
Tools†
![[PukiWiki]](image/pukiwiki.png "[PukiWiki]")
ynamic Adaptive Neural Network Arrays: A Neuromorphic Architecture.pdf