7 views

Original Title: Artificial Neural Network

Uploaded by manish9890

Artificial Neural Network

ANN

© All Rights Reserved

- The Mind Asleep and Awake Video Assignment SonjaBrown02202013
- CP-Guillain Barre Syndrome
- PSYC 110 Midterm Exam
- Sentiment Analysis Algorithms and Applications - A Survey
- Early Brain Development
- 102249743-Neurons
- Science Quiz Bee Reviewer
- Exam Questions
- Exp Physiol 1989 Sakmann 1107 18
- ANALYSIS OF DATA MINING AND SOFT COMPUTING TECHNIQUES IN PROSPECTING DIABETES DISORDER IN HUMAN BEINGS: A REVIEW
- r5410503 Data Warehousing and Data Mining
- Grade 10 - 3rd Quarter BIOLOGY (Key Concepts: Nervous System)
- GENETIC ALGORITHM
- Neurology Speech
- Smith, LS - Neuromorphic Systems-Past, Present and Future
- Science-2014-Tomassy-319-24(3)
- Neuron
- Nicholas A. Kotov et al- Nanomaterials for Neural Interfaces
- Ann
- Assignment1 (2)

You are on page 1of 46

- Introduction -

Biological inspiration

Animals are able to react adaptively to changes in their

external and internal environment, and they use their nervous

system to perform these behaviours.

should be able to produce similar responses and behaviours in

artificial systems.

neurons, so copying their behaviour and functionality should

be the solution.

Biological inspiration

Dendrites

Axon

Biological inspiration

axon

dendrites

synapses

Biological neuron : A processing element

Dendrites: Input

Cell body: Processor

Synaptic: Link

Axon: Output

Biological neuron : A processing element

synapses

A processing element

combined.

Biological neuron : A processing element

an electrical pulse that travels from the body, down the axon,

to the next neuron(s)

Biological neuron : A processing element

the next neuron.

Biological neuron : A processing element

next is effected by neurotransmitters.

A processing element

first neuron and which bind to the

Second.

Biological neuron : A processing element

reaches the next neuron depends on factors such as the

amount of neurotransmitter available.

Elements of ANNs

Processing Elements (PE)

Processing Element (PE) : PE are artificial neurons

similar to biological neurons.

Each PE receives input, process them and deliver the

output.

Input can be raw input data or output of PE

The output can be the final result.

Or it can be an input to other neurons.

Artificial neurons

Neurons work by processing information. They receive and

provide information in form of spikes.

x1

w1

x2

Inputs

xn-1

xn

z wi xi ; y H ( z )

w2

x3

i 1

..

w3

wn-1

wn

The McCullogh-Pitts model

Output

y

Artificial neurons

The Mc Cullogh-Pitts model:

spikes are interpreted as spike rates;

synaptic strength are translated as synaptic weights;

excitation means positive product between the

incoming spike rate and the corresponding synaptic

weight;

incoming spike rate and the corresponding synaptic

weight;

Inputs

Output

neurons that are linked together according to a specific

network architecture. The objective of the neural network

is to transform the inputs into meaningful outputs.

Tasks to be solved by artificial neural networks:

controlling the movements of a robot based on selfperception and other information (e.g., visual

information);

deciding the category of potential food items (e.g.,

edible or non-edible) in an artificial world;

recognizing a visual object (e.g., a familiar face);

predicting where a moving object goes, when a robot

wants to catch it.

Inputs

Output

y11 2

1

2

2

y1 f ( y , w1 )

y

3

1

1

1

y 2 f ( x2 , w2 ) y 1 y 2 y 2 f ( y 1 , w 2 ) y 2 y 2 y f ( y 2 , w3 )

Out

1

3

1 2

2

1

1

2

y3 y 2 f ( y1 , w 2 )

y3 f ( x3 , w3 )

y3

y1 3

3

4

y14 f ( x4 , w14 )

y11 f ( x1 , w11 )

Standard Computers

one CPU

fast processing units

Neural Networks

highly parallel processing

slow processing units

reliable units

unreliable units

static infrastructure

dynamic infrastructure

control

classification

in general as

prediction

FUNCTION

APPROXIMATION

approximation

tasks.

build a neural network that approximates the g(x) values

for any input x.

There are two basic reasons why we are interested in

Technical viewpoint: Some problems such as

states of a system require massively parallel and

adaptive processing.

Biological viewpoint: ANNs can be used to

replicate and simulate components of the human

(or animal) brain, thereby giving us insight into

natural information processing.

Now, let us have a look at the model of an artificial neuron.

Input

xm

........

....

Processing

Output

x2

x1

= X1+X2 + .+Xm =y

Not all inputs are equal

Input

xm

weights

........

....

wm

...

..

x2

w2

Processing

Output

x1

w1

= X1w1+X2w2 + .+Xmwm

=y

The signal is not passed down to the

next neuron verbatim

Input

xm

........

....

weights

Processing

Transfer Function

(Activation Function)

Output

wm

...

..

x2

w2

f(vk)

x1

w1

affected by the weights, and the transfer

functions

Output

An ANN can:

1. compute any computable function, by the appropriate

selection of the network topology and weights values.

2. learn from experience!

Specifically, by trialanderror

Learning by trialanderror

Continuous process of:

Trial:

Processing an input to produce an output (In terms of

ANN: Compute the output function of a given input)

Evaluate:

Evaluating this output by comparing the actual output with

the expected output.

Adjust:

Adjust the weights.

Learning Paradigms

Supervised learning

Unsupervised learning

Reinforcement learning

Supervised learning

This is what we have seen so far!

A network is fed with a set of training samples (inputs

to learn the general relationship between the inputs

and the outputs.

This relationship is represented by the values of the

weights of the trained network.

Unsupervised learning

No desired output is associated with the training

data!

Faster than supervised learning

Used to find out structures within data:

Clustering

Compression

Reinforcement learning

Like supervised learning, but:

Weights adjusting is not directly related to the error

value.

The error value is used to randomly, shuffle weights!

Relatively slow learning due to randomness.

What is back propagation?

A neuron: receives input from many other neurons;

changes its internal state (activation) based on the

current input;

sends one output signal to many other neurons, possibly

including its input neurons (ANN is recurrent network).

Back-propagation is a type of supervised learning, used

response and the actual data.

Applications Areas

Function approximation/ Regression

including time series prediction and modelling.

Trained to predict whose values are integer numbers

Classification : Pattern reorganization

including patterns and sequences recognition, novelty

(radar systems, face identification, handwritten text recognition)

Data processing

including filtering, clustering blinds source separation and

compression.

(data mining, e-mail Spam filtering)

Applications Areas

Clustering : data set is so complicated that there is no way

features of dataset to classify and put then in a cluster.

Ex: used in Adaptive resonance theory

Association : ANN trained to remember number of unique

patterns, if the original pattern is distorted the network

associate with closet pattern,

Ex. Useful in noisy data or incomplete data

Advantages / Disadvantages

Advantages

Adapt to unknown situations

Powerful, it can model complex functions.

Ease of use, learns by example, and very little user

Disadvantages

Forgets

Not exact

Large complexity of the network structure

ANN application areas:

Tax form processing to identify tax fraud

Enhancing auditing by finding irregularities

Bankruptcy prediction

Customer credit scoring

Loan approvals

Credit card approval and fraud detection

Financial prediction

Energy forecasting

Computer access security (intrusion detection and

classification of attacks)

Fraud detection in mobile telecommunication networks

Conclusion

Artificial Neural Networks are an imitation of the biological

The computing would have a lot to gain from neural networks.

Their ability to learn by example makes them very flexible and

powerful furthermore there is need to device an algorithm in

order to perform a specific task.

Conclusion

Neural networks also contributes to area of research such a

parts of living organizations and to investigate the internal

mechanisms of the brain.

Many factors affect the performance of ANNs, such as the

transfer functions, size of training sample, network topology,

weights adjusting algorithm,

CLASSIFICATION AND

REGRESSION TREES (CART)

Also known as decision trees

Decision Trees:

following a path through a series of simple rules or questions, the

answers to which determine the next direction through the path.

Decision tree is a supervised learning algorithm which must be

provided with a training set that contains objects with class labels.

Looks like a cluster analysis dendro gram or partitioning diagram but

these are from unsupervised methods that take no account of preassigned class labels.

CLASSIFICATION AND

REGRESSION TREES (CART)

CART aims to use a set of predictor variables to estimate the

means of one or more response variables. A binary tree is

constructed by repeatedly splitting the data set into subsets. Each

individual split is based on a single predictor variable and is

chosen to minimise the variability of the response variables in

each of the resulting subsets.

The tree begins with the full data set and ends with a series of

terminal nodes. Within each terminal node, the means of the

response variables are taken as predictors for future observations.

Closer to ANOVA than regression in that data are divided into a

discrete number of subsets based on categorical predictors and

predictions are determined by subset means.

CLASSIFICATION AND

REGRESSION TREES (CART)

Must define two criteria:

1. A measure of impurity or inhomogeneity.

2. Rule for selecting optimum tree.

Produce a very large tree and then prune it into successively

smaller trees. Skill of each tree is determined by crossvalidation. Divide the full data into subsets, drop one subset,

grow the tree on the remaining data and test it on the omitted

subset.

CLASSIFICATION AND

REGRESSION TREES (CART)

A SUMMARY:

Explain variation of single response variable by one or more

explanatory or predictor variables.

Response variable can be quantitative (regression trees) or categorical

(classification trees).

Predictor variables can be categorical and/or quantitative.

Trees constructed by repeated splitting of data, defined by a simple rule

based on single predictor variable.

CLASSIFICATION AND

REGRESSION TREES (CART)

At each split, data partitioned into two mutually exclusive

groups, each of which is as homogeneous as possible. Splitting

procedure is then applied to each group separately.

Aim is to partition the response into homogeneous groups but to

keep the tree as small and as simple as possible.

Usually create an overlarge tree first, pruned back to desired size

by cross-validation.

(categorical response) or mean value (quantitative response) of

the response variable, group size, and the predictor variables that

define it.

- The Mind Asleep and Awake Video Assignment SonjaBrown02202013Uploaded bySonja Brown
- CP-Guillain Barre SyndromeUploaded bykathy
- PSYC 110 Midterm ExamUploaded bymelanie
- Sentiment Analysis Algorithms and Applications - A SurveyUploaded byTếu Tếu
- Early Brain DevelopmentUploaded byRitzuchan001
- 102249743-NeuronsUploaded byArchie Cuyacot
- Science Quiz Bee ReviewerUploaded byCatherine Joy Dela Cruz
- Exam QuestionsUploaded byathmercl
- Exp Physiol 1989 Sakmann 1107 18Uploaded by2begenius
- ANALYSIS OF DATA MINING AND SOFT COMPUTING TECHNIQUES IN PROSPECTING DIABETES DISORDER IN HUMAN BEINGS: A REVIEWUploaded byDr Manik Sharma
- r5410503 Data Warehousing and Data MiningUploaded byGireesh Kumar
- Grade 10 - 3rd Quarter BIOLOGY (Key Concepts: Nervous System)Uploaded byIonacer Viper
- GENETIC ALGORITHMUploaded bybalajibagavathi4
- Neurology SpeechUploaded byValerie Zimmermann
- Smith, LS - Neuromorphic Systems-Past, Present and FutureUploaded byAngel Martorell
- Science-2014-Tomassy-319-24(3)Uploaded byMakotoNatsui
- NeuronUploaded byiris danielle barola
- Nicholas A. Kotov et al- Nanomaterials for Neural InterfacesUploaded byNoScript
- AnnUploaded bySeema Gaude
- Assignment1 (2)Uploaded bySuriti Chawla
- machine-learning-lecture-3-1205567881737248-5.pdfUploaded byGayathri
- #Biology Nervous SystemUploaded bytareq_blay
- HectorUploaded byAndrés Chávez
- lics-chapter10 (1)Uploaded byAkhil Mehta
- TISSUES 2012(1)Uploaded byLexi Co
- 593-2375-1-PBUploaded bynhiar dwi
- ItchUploaded byemma rahma
- Aditya Vats-Data Mining-Final Project.pdfUploaded byAditya Vats
- Ch 2 Body Coordination TcUploaded byMissa MeiSan
- reportUploaded byapi-369423954

- FM Radio Listener Ship in TrichurUploaded byNicy Prethew
- About AstrumUploaded bymanish9890
- Msg FileUploaded bymanish9890
- msgfile.pdfUploaded bymanish9890
- Business Strategy-IT Industry-Tata Consultancy ServicesUploaded bysdhamangaonkar7264
- ResumeUploaded byAbhinav Juneja
- pqtFL2wGL2Uploaded bymanish9890
- Consumer Behavior All in One Batch 2015Uploaded bymanish9890
- MGI Indias Urban Awakening Full ReportUploaded bynitesshverma
- CommerceUploaded bymanish9890
- Authorized Persons - CommodityUploaded bymanish9890
- Kirana Case Study_final222Uploaded byAseem Jain
- AmwayUploaded bymanish9890
- Corporate Restructurin BSUploaded bymanish9890
- Corporate Restructuring 2014Uploaded bymanish9890
- Future of Competition 130904Uploaded bymanish9890
- Strategic Alliance JV M&a Dec14Uploaded bymanish9890
- Marketing Strategies HaldiramUploaded byApoorva Saxena
- Society Strat Plan 2008-2018Uploaded bymanish9890
- Super Final Report on RADIOUploaded bymanish9890
- ConflictUploaded bymanish9890
- Busniess Strategy PptUploaded bymanish9890
- Business Startegy -IUploaded bymanish9890
- 03a Consumer Perceptions & Expectations-SUploaded bymanish9890
- 03 Consumer Behaviour-S(1)Uploaded bymanish9890
- 02 Managing People-SUploaded bymanish9890
- 04 Strategies-Positioning & Differentiation-SUploaded bymanish9890
- Petrolium ProductsUploaded bymanish9890

- 01the Nerve Impulsepc (1)Uploaded byHaruka Suzuki
- Anatomy CH 7 Test Review 2010Uploaded byjbradee
- BiofeedbackUploaded byspeedy7767
- The Human ConnectomeUploaded bySanford Pepper
- Consciousness: Confessions of a Romantic Reductionist - Christof KochUploaded byDrWhoF
- The Learning BrainUploaded byMaria Sandu
- An Antioxidant is a Molecule Capable of Inhibiting the Oxidation of Other MoleculesUploaded byRia Talukdaar
- Circuit HubsUploaded byTristan2001
- ABPG1103 - INTRODUCTION TO PSYCHOLOGY. 2.pptUploaded byAnonymous HK3LyOqR2g
- 2011 - Mohaghegh etal - Artificial Intelligence & Data Mining Applications in the E&P Industry.pdfUploaded byRiska Laksmita Sari
- 9 00SCF11 Read Kr2Uploaded byClaire Hsu
- KPCP(1)Uploaded bystay312
- NeuronsUploaded byMohamed
- NeuronUploaded byPrashant Chahar
- Modul Gemilang1Uploaded byRoslinda Mohamed Othman
- Maturana, Humberto - Biology of CognitionUploaded byJeanDef
- whatismodeling-versIIIUploaded byBace Hickam
- neural networkUploaded byfarhan danish
- Scheme of Instruction IIScUploaded byRS1678
- Neurobiology - Option - IBUploaded byJarek Oleszczynski
- Ayurvedic Medicinal Plants for Alzheimer’s DiseaseUploaded byjcamacho95
- An Approach Based on Wavelet Decomposition and Neural Network for ECG Noise ReductionUploaded byHoang Manh
- Nervous Tissue (2)Uploaded byDavid Calalo
- 117Uploaded byriofd
- CH-2-BrainUploaded bysenthil_77
- Smith, LS - Neuromorphic Systems-Past, Present and FutureUploaded byAngel Martorell
- NeuroscienceUploaded bySalam Matalka
- 2. IJMPERD - OPTIMIZATION OF CUTTING FORCE AND TOOL TEMPERATURE USING.pdfUploaded byAnonymous Y0wU6InI
- Young blood reverses age-related impairments in cognitive function and synaptic plasticity in mice SummaryUploaded bythu
- Neurons.. 11...Uploaded byAndrea Pineda