Design of Self-Adaptive Weighted Neuron model using Floating Gate

Technology

JAYANT KUMAR SINGH1, GARIMA KAPUR

Jaypee institute of information and technology, Noida, INDIA

Abstract: The paper presents an analog circuit solution for implementing models of synapses with short-term

short-term self-adaption with spiking input signal has been observed. The model can also demonstrate

homeostatic intrinsic plasticity, spike-based algorithms, and LMS algorithms. The model has a 4.5µV/℃

temperature coefficient, 0.675µW power consumption, and consumes a chip area of about 130×90 nm2. The

model is compact, low power, and stable. The proposed circuit has been applied to design a cell membrane (bio-

sensor CMOS-based circuit) depicting the effect of Sodium (NA) and Potassium (K) on synaptic action. With the

help of the Na and K feedback circuit, effects of polarization and depolarization on synapse output have been

demonstrated and thus depict spike-timing-dependent plasticity. The work can be extended to design a complete

neural architecture, an array of such complete neural cells, in turn, can design devices for assistive technology or

human-like machines.

List of Abbreviations

IOT Internet of things

SiNs Silicon neurons

NC Neuromorphic circuits

STDP Spike timing-dependent plasticity

LMS Least mean square

ADALINE Adaptive Linear Combiner

SP Structural-Plasticity

VLSI Very-large-scale integration

FG Floating Gate

information processing principles have been a standard in

building artificial intelligence (AI). AI-based algorithms allow

recognition tasks and can be implemented with the help of

very large-scale integration (VLSI) technology. The field of

integrating brain computation-based algorithms with

technology is referred to as “Neuromorphic engineering”

(NE). The term ‘Neuromorphic engineering’ was first

introduced by an American scientist Carver Mead [1]. NE

has two directive goals; one is to understand the

computational properties of biological neural systems

using models and the other is to exploit the known

properties of biological systems to design and implement

efficient devices for engineering applications (C S Thakur,

et al., 2018[2]). NE term is used for all neuro-inspired

techniques: hardware, algorithms, and recently this

discipline has emerged amongst top technologies

(information is given by the world economic forum [3]).

The market for neuromorphic systems is expected to

increase by roughly 1.8 billion dollars by the time 2025

solutions in the analog domain. Several Si-based analog

circuits for synapse modeling, learning methods, neural

spike generation, etc., have been developed [7,8,9]. With

the use of such compact models, digital communication

(large-scale neural networks) of spiking information has

also been developed which closely emulates the nature of

Semiconductor (FGMOS) devices are very well known for

memory application and other low-power analog computation

circuits. Recently they find applications in analog circuits that

are used for neuron modeling [12, 13], competitive learning

systems [14], implementation in learning algorithms [15], and

implementation of online unsupervised deep learning systems

[16]. Carver Mead noticed that the Si MOSFET operating

below threshold in current mode has similar characteristics as

“sigmoidal current-voltage”, which is similar to characteristics

of neural electrical(ion) channels [17]. Single MOSFET at

recent decade (electric signals, V/I, and ionic signal Na/K)

[18, 19]. NC emulating features like Homeostatic intrinsic

plasticity, Spiking-timing-dependent plasticity (STDP), Least-

mean square, perceptron, backpropagation, spiking-driven

synaptic plasticity and structural plasticity [20, 21, 22, 23]

have also been developed. Moreover, neural computation for

artificial intelligence requires complex integrated circuitry, for

which the designs need to be scaled down. Scaling of such

SiNs has several advantages like higher frequency response,

lower parasitic capacitances, and lower power consumption

1. Thus, we propose a floating gate-based PMOS (SiN)

simulation model at 45nm CMOS technology which shows

learning, storing, and self-adaptation with higher accuracy and

stability. The model can also emulate neuron features like

Spike-timing-dependent-plasticity (STDP) generation and

self-adaptation (Structural plasticity). The Model shows stable

learning, consumes low power, and smaller chip area. The

simulated results have also been compared with previous work

[20]. Our proposed SiN can be implemented in other adaptive

neuromorphic circuits which in turn can be used to build a

conditions in self-adaption. And in the last section, an

application has been simulated. The proposed SiN model can

be used to design a complete neural structure whose array can

develop a neural network. Such neural networks can be used

in two ways; either as an FPAA with an array of complete cell

boards implanted to innovate human-like machines. Or such

boards can be used to understand or emulate some part of the

defined and simplified method of modeling human-like

thought, emotion, behavioral, cognitive, and conjectural

processes.

while in the digital domain SiNs provide only a qualitative

approximation of real-time performance.

The digital domain models are not suitable for quantitative

investigations. In this decade several analog domain SiN

models have been developed for example Fitzhugh-

Nagumo (FHN) [25], and the modified Izhikevich neuron

model [26]. Memristor-based neuron models have also

been proposed [27], The commercial application of

neuromorphic chips includes low-power sensors, self-

learning robots, and power-efficient supercomputers.

change in weight multiplied by the input, which in turn results

in an impulse response. It has been observed (as shown in fig

3) that the charge in FG will be adapted automatically to

equilibrium position. Figure 5 represents the output voltage of

the neuron model (shown in fig 2) in response to the input

step. The response is similar to STDP obtained from a

mathematical equation (shown in fig 4). The response is also

comparable with the result of the paper [29].

explanation of LMS a simple neural network and adaptive

linear combiner (ADALINE) have been considered and

narrated by the equations as follows:

 (6)

 (7)

is the vector of weight, ds are the required output and e

denotes the corresponding error. LMS works on weight

updating algorithm as given as:

This feature is used to reduce error, and to emulate neural

computations and can also be designed in NCs.With the

change in high potential at the tunneling junction, the

amplification (dc level/equilibrium level) has been shifted, as

decreases so the dc level for 2.11V is 1.2V and for 2.19V it is

shifted to 0.95V. (The range of tunneling from 2-4V has been

reduced to 2.11-2.19V). With the feedback mechanism in our

proposed model if there is a difference in the desired output it

(in form of wiring) of an adult brain also undergoes certain

changes for example formation of new connections and

replacing them with old ones and this phenomenon is

known as structural plasticity (SP) [30,31]. The presence

and absence of connections are treated as 1-bit memory

signals and they are easy to store in CMOS latches with

less power and lesser chip size [32]. These algorithms are

recently applied to real-world problems. Based on SP

several algorithms have been developed which have

recently been applied for real- time applications [33]. The

with respect to input stimuli, weights need to be estimated.

As illustrated earlier these weights correspond to the

charge stored in the floating gate (equation 2). The

proposed SiN model is based on floating gate technology

where the charge at the floating gate(weights) is

programmed using external voltages using two quantum

mechanical techniques; tunneling and injection. The

experimental results of on-chip, precise, non-volatile

programming have been demonstrated in the next section.

The charge also self-adapts in correspondence to spiking

whose charge works like weights in our proposed SiN model

is once programmed, remains trapped on it for years, as FG is

covered from oxide from all sides and electronically ‘floating’.

The FGPMOS has been fabricated using on semiconductor C5

CMOS process using MOSIS fabrication services in the USA

in association with the University of Maryland. The

experimental results of tunneling and injection respectively are

demonstrated in Figures 8(a) and (b), respectively. The charge

at the floating gate can be programmed with seven bits of

resolution as illustrated in Tables I and II. The tunneling,

equilibrium is created when simultaneously tunneling and

injection of charge are performed after 0.2µs, as shown in

Figure 10. The final stable learned charge at the floating gate

is the difference between two currents, as shown in equation 9.

This change in charge after equilibrium is the stable weight of

our SiN neural model and according to it will produce output

when an input spike is provided. In the next section,

programming techniques have been explained in detail.

Tunneling Itun derived from experimental data.

Tunneling arises from the phenomenon of the electron

wave function which has a finite extent. It is a quantum

mechanics process that is used to tunnel out electrons from

the floating gate. For a thin barrier it is sufficient for an

electron to penetrate the barrier. In Fowler Nordheim

tunneling mechanism a high potential is applied at the

tunneling junction to create a strong electric field between

the tunneling junction and Floating gate which in turn

Where  and  are the extracted parameter and values

estimated are . With the increase in

tunneling junction voltage, the negative charge at the

floating gate reduces due to which the positive threshold

voltage of FGPMOS increases. The same has been

illustrated in Figure 9(a), which depicts the change in the

value of drain current at one time (200ps) with increasing

tunneling voltage from 2V to 4V.

The Physics of hot electron injection is to provide electrons

with enough kinetic energy and direction in the channel to

drain near the depletion region to surpass the Si-SiO2 barrier.

FGPMOS must have two requirements for injection of an

electron in FG. First, an electron must gain more than 3.1ev of

kinetic energy to cross the Si-SiO2 barrier. Second, the electric

field in the oxide is desired in a unique direction to receive

electrons once they cross the Si-SiO2barrier. The injection

current can be simulated using empirical equation 11, whose

parameter values have been extracted from fabricated results

gate. With the increase in the magnitude of the difference of

voltages between drain and source, the negative charge at the

floating gate increases and the threshold voltage of FGPMOS

decreases. The same has been illustrated in Figure 9(b), which

depicts the change in the value of drain current at one time

(1000ns) with increasing  voltage from 1v to -4v.

Table. II Injection programming precision

the cell membrane as shown in figure 12 can be used to

improve the insulation of the myelin sheath which is

responsible to transmit electrical impulses in the brain and

spinal cord and have the ability to emulate the STDP

feature of the human brain.

because the potassium channel opens a bit too long and this

state is stated as hyperpolarization and then the cell returns

back to resting potential at -70mV.

Action potential shown in figure 13 is obtained by applying a

step pulse from (-65mV to -10mV) at the input terminal of the

bio-sensor circuit shown in figure 11. It can be noted that due

to tunability of circuit and bias conditions a significant

voltage shown across the y axis of figure 13 is the voltage

spike at the cell membrane (󰇜 minus the voltage of the

circuit at rest. The concept of depolarization (rising phase)

In this paper a Si-based adaptive neuron model is proposed

and simulated using 45nm CMOS technology. It shows

features like storing, programming, and adaptability of charge

at FG. It can enhance the on-chip learning ability of Si

when the temperature is varied from 0 to 80-degree Celsius,

consumes 0.675µW power, and occupies a chip area of about

subthreshold conduction, is used to emulate a neuron and its

features such as LMS, STDP, and structural plasticity. The

architecture can be created. An array of these cells

depicting features like learning, storing, and self-adaption

can create a Neural network. Such analog based Neural

networks can be used in two ways; 1. it can get implanted

in machines to depict human-like behavior in bots. 2. It can

also be implanted inside a deficient brain to replicate the

correct cognitive behavior (can help in developing devices

for assistive technology).

[2] C.S. Thakur, J.L. Molin, etal. “Large-Scale

[3] Top 10 Emerging Technologies of 2015,

[8] S. Agarwal, D. Garland, J. Niroula, R.B. Jacobs-

[9] U. Ragavendran, and M. Ramachandran, “Low power

[11] F.D Broccard, S. Joshi, J. Wang, G. Cauwenberghs,

threshold operations”, IEEE Transactions on Electron

Devices, June 1992, vol. 39, pp. 1444–1455.

[13] V. Kornijcuk, H. Lim, Y.J. Seok, G. Kim, S.K. Kim, I.

Kim, B.J. Choi, D.S. Jeong, “Leaky integrate-and-fire

neuron circuit based on floating-gate integrator”,

Frontiers in Neuroscience, May 2016 vol. 10.

[14] D. Hsu, M. Figueroa, C. Diorio, “Competitive learning

with floating gate circuits”, IEEE Transactions on Neural

Networks, May 2002, vol. 13 (3) pp. 732–744.

[15] P. Hasler, and J. Dugger, “An analog floating-gate node

[17] C.S. Poon, K. Zhau, “Neuromorphic silicon neurons and

large-scale neural networks: challenges and

opportunities”, Front. Neuroscience, Sept. 2011, vol. 5.

[18] S. Nease, and C. Elisabetta, “Floating-Gate-Based

Intrinsic Plasticity with Low-Voltage Rate Control”,

international Symposium on Circuits and Systems

(ISCAS), IEEE, May 2016, pp. 4799-5341.

[19] I.E.Ebong, P. Mazumder, “CMOS and Memristor-Based

Neural Network Design for Position Detection”, Proc.

IEEE, June 2012, vol. 100, pp. 2050 – 2060.

Efficient Neuron, Synapse and STDP Integrated

systems, June 2012, vol. 6 (3), pp. 246-256.

[22] G. kapur, H. Mohan, V. Saxsena, C.M. Markan, “Design

of a bio-inspired CMOS circuit realizing features of

binocular rivalry between left and right eye”, Conference

IEEE, June 2013.

[23] B. Widrow, and M.A. Lehr, “30 years of adaptive neural

networks: Perceptron, Madaline, and backpropagation”,

Proc. IEEE, 1990, vol. 78 (9), pp. 1415–1442.

Analog Arrays”, IEEE, April 2008, pp. 772-775.

Model for Circuit Implementation of Spiking Neural

Networks”, IEEE Conference 2010, pp. 192-195.

[27] A. Thomas, “Memristor-based neural networks”, Journal

of applied physics, Feb 2013, vol. 46 (9).

Creative Commons Attribution License 4.0 (Attribution

4.0 International, CC BY 4.0)

This article is published under the terms of the Creative