Вы находитесь на странице: 1из 26

Semester Core Project

Final Report

Semester Core Project Final Report Submitted by: Vaibhav Singh Submitted to: Mrs. Nishtha phutela

Submitted by:

Vaibhav Singh

Semester Core Project Final Report Submitted by: Vaibhav Singh Submitted to: Mrs. Nishtha phutela

Submitted to:

Mrs. Nishtha phutela

Semester Core Project Final Report Submitted by: Vaibhav Singh Submitted to: Mrs. Nishtha phutela

TABLE OF CONTENTS

AboutDeeplearning

3

LinearRegression

5

R2

6

Code:

7

Challenge

8

Code

8

UseofLinearregression

10

NeuralNetwork

10

Process

10

Code

11

UseofNeuralNetwork

22

TensorFlowforclassification

23

10 ​ Process 10 Code 11 UseofNeuralNetwork 22 TensorFlowforclassification 23

About Deep learning

Traditionally, programing has been about defining every single step for a program to reach an outcome. The machine learning flips that approach. With machine learning, we define the outcome and program learn to steps get there. So if we build an app that can recognize street signs so instead of writing the code to recognize hundreds of different feature of street signs like this shape of letters and the colors. We will just say here are some examples of a street signs, learn the shape you need to recognize it. Sometimes, we have not any idea what steps could possibly be. Machine learning is already everywhere on the internet.Every major services uses it in some way. Youtube uses it to decide which other videos we might like as we watch and its uses will only grow over time. There are lot of machine learning models are there and one of them are neural networks. When we use a neural network that’s not just one or two but many layers deep to make a prediction, we called that deep learning. It’s a subset of machine learning that has outperformed every other type of model almost every time on a huge range of tasks.

a subset of machine learning that has outperformed every other type of model almost every time

Linear Regression

Simple linear regression is a statistical method that allows us to summarize and study relationships between two continuous (quantitative) variables:

One variable, denoted x, is regarded as the predictor, explanatory, or independent variable.

The other variable, denoted y, is regarded as the response, outcome, or dependent variable.

With simple linear regression we want to model our data as follows:

y = B0 + B1 * x (similar to y = mx + c)

This is a line where y is the output variable we want to predict, x is the input variable we know and B0 and B1 are coefficients that we need to estimate that move the line around.

x is the input variable we know and B0 and B1 are coefficients that we need

R2

The R2 term

fits the observed data. The coefficient of determination is usually given by,

is the coefficient of determination and it usuallyreflects how well the model

and it ​ usually ​ reflects how well the model Where Y ​ i ​ is

Where Yi is an observed response, is the mean of the observed responses,is a prediction of the response made by the linear model, and (Yi -) is the residual, i.e., the difference between the response and the prediction. Also, SSE is called the sum of the squared error, or SSR the sum of the squared residuals, and SST is called the total sum of squares.

called the sum of the squared error, or SSR the sum of the squared residuals, and

Code:

importpandas aspd

fromsklearn importlinear_model

importmatplotlib.pyplot asplt

# Read data

dataframe =pd.read_fwf('brain_body.txt')

print(dataframe)

x_values =dataframe[['Brain']]

y_values =dataframe[['Body']]

# train model on data

body_regression =linear_model.LinearRegression()

body_regression.fit(x_values,y_values)

# visualize results

plt.scatter(x_values,y_values)

plt.plot(x_values,body_regression.predict(x_values))

plt.show()

plot ​ ( ​ x_values ​ , ​ body_regression ​ . ​ predict ​ ( ​

Challenge

challenge is to use scikit-learn to create a line of best fit for the included 'challenge_dataset'. Then, make a prediction for an existing data point and see how close it matches up to the actual value. Print out the error you get. You can use scikit-learn's documentation for more help.

Code

importpandas aspd

importnumpy asnp

fromsklearn importlinear_model

importmatplotlib.pyplot asplt

frompylab importfigure

# Read dataset

dataframes =pd.read_csv('challenge_dataset.txt',sep=',',header=None)

dataframes.columns =["X_data","Y_data"]

x_values =dataframes[['X_data']]

y_values =dataframes[['Y_data']]

# Train data with linear regression model

model =linear_model.LinearRegression()

model.fit(x_values,y_values)

​ = ​ linear_model ​ . ​ LinearRegression ​ () model ​ . ​ fit ​

# function for calculating residuals

defcalculate_residual(actual,predicted):

res =(np.array(actual)-np.array(predicted)**2)

returnres

# Evaluating fit model

residuals =calculate_residual(y_values,model.predict(x_values))

print("Mean squared error: ",np.mean(residuals))

print("Coefficient of determination: ",model.score(x_values,y_values))

# Plotting on matplotlib

plt.figure(1)

plt.scatter(x_values,y_values,color="blue")

plt.savefig("Scatter.png")

plt.figure(2)

plt.plot(x_values,model.predict(x_values),color="black")

plt.figure(3)

plt.plot(residuals)

plt.savefig("residual.png")

plt.show()

plt ​ . ​ plot ​ ( ​ residuals) plt ​ . ​ savefig ​ (

Use of Linear regression

1. In environmental science it is used to try to establish how much one quantity, say atmospheric greenhouse gasses, influences another, say global surface temperature.

2. In quantitative finance linear regression is core to everything, as they use something called a linear factor model. Long story short it estimates how much the price of one asset will move when another thing (oil prices) move.

Neural Network

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurons) working in unison to solve specific problems. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurones. This is true of ANNs as well.

Process

We will give each input a weight, which can be a positive or negative number. An input with a large positive weight or a large negative weight, will have a strong effect on the neuron’s output.

An input with a large positive weight or a large negative weight, will have a strong

1. Take the inputs from a training set example, adjust them by the weights, and pass them through a special formula to calculate the neuron’s output.

2. Calculate the error, which is the difference between the neuron’s output and the desired output in the training set example.

3. Depending on the direction of the error, adjust the weights slightly.

4. Repeat this process 10, 000 times.

Code

fromnumpy importexp,array,random,dot

importnumpy asnp

classNeuralNetwork():

"""docstring for ."""

def

init

(self):

# seed the random number generation, so it generates the same number

# every time the program runs

random.seed(1)

# we model a single neuron, with 3 input connections and 1 output connection.

# we assign random weights to 3 x 1 matrix, with a value in the range

# of -1 to 1 a mean of 0

# our neuron have 3 input connections and 1 output connections

with a value in the range ​ # of -1 to 1 a mean of 0

self.synaptic_weights =2*np.random.random((3,1))-1

# Activation Function

# The sigmoid function, which describes an s shaped curve

# we pass the weighted sum of the inputs through this function

# to normalise them between 0 and 1

def

sigmoid(self,x):

return1/(1+exp(-x))

# gradient of the sigmoid curve

def

sigmoid_deravative(self,x):

returnx *(1-x)

deftrain(self,training_set_inputs,training_set_outputs,number_of_training_iterations):

foriteration inrange(number_of_training_iterations):

# pass the training set through our neural net

output =self.predict(training_set_inputs)

# calculate the error

error =(training_set_outputs -output)

( ​ training_set_inputs) ​ # calculate the error error ​ = ​ ​ ( ​ training_set_outputs

# multiply the error by the input ad again by the gradient of the

# sigmoid curve

adjustments =dot(training_set_inputs.T,error *

self.sigmoid_deravative(output))

# adjust the weights

self.synaptic_weights +=adjustments

defpredict(self,inputs):

# pass inputs through our neural network (our singe neuron)

if

returnself.

name

=='

main

sigmoid(dot(inputs,

':

self.synaptic_weights))

# initialise a single neuron neural network

neural_network =NeuralNetwork()

print('Random starting synaptic weights: ')

print(neural_network.synaptic_weights)

# the training set. We have 4 examples, each consisting of 3 input values

# and 1 output value.

synaptic_weights) ​ # the training set. We have 4 examples, each consisting of 3 input values

training_set_inputs =array([[0,0,1],[1,1,1],[1,0,1],[0,1,1]])

training_set_outputs =array([[0,1,1,0]]).T

# train the neural network using the training sets

# Do it 10,000 times amd making small adjustments each time

neural_network.train(training_set_inputs,training_set_outputs,10000)

print('New synaptic weights after training: ')

print(neural_network.synaptic_weights)

# Test the neural network

print('predicting

')

print('considering new situation [1, 0, 0] -> ?: ')

print(neural_network.predict(array([1,0,0])))

print ​ ( ​ neural_network ​ . ​ predict ​ ( ​ array ​ ([ ​

Challenge is to create a 3 layer feedforward neural network using only numpy as dependency.

Challenge.py

fromnumpy importexp,array,random,dot

importnumpy asnp

importmatplotlib.pyplot asplt

# %matplotlib inline

classNeuralNetwork():

"""docstring for NeuralNetwork."""

def

init

(self):

# seed the random number generation, so it generates the same number

# every time

random.seed(1)

# setting the number of nodes in layer 2 and layer 3

# more the number of nodes in hidden layer more the confident the

# neural network in predicting

l2 =5

l3 =6

of nodes in hidden layer more the confident the ​ # neural network in predicting l2

# assign random weights to the matrix

# (no. of nodes in previous layer) x (no. of nodes in following layer)

self.synaptic_weights1 =2*np.random.random((3,l2))-1

self.synaptic_weights2 =2*np.random.random((l2,l3))-1

self.synaptic_weights3 =2*np.random.random((l3,1))-1

def

sigmoid(self,x):

return1/(1+exp(-x))

def

sigmoid_deravative(self,x):

returnx *(1-x)

# train neural network, adjusting synaptics weight each time

deftrain(self,training_set_inputs,training_set_outputs,number_of_training_iterations):

foriteration inrange(number_of_training_iterations):

# passing training set through our neual network

# a2 means activation fed to the second layer

a2 =self.

sigmoid(dot(training_set_inputs,

self.synaptic_weights1))

print("============================================================")

print("Dot product output of 2nd Layer")

​ print ​ ( ​ "Dot product output of 2nd Layer")

print(dot(training_set_inputs,self.synaptic_weights1))

print("Sigmoid function output")

print(a2)

print("============================================================")

a3 =self.

sigmoid(dot(a2,

self.synaptic_weights2))

print("============================================================")

print("Dot product output of 3rd Layer")

print(dot(a2,self.synaptic_weights2))

print("Sigmoid function output")

print(a3)

print("============================================================")

output =self.

sigmoid(dot(a3,

self.synaptic_weights3))

print("============================================================")

print("Dot product output layer")

print(dot(a3,self.synaptic_weights3))

print("Sigmoid function output")

print(output)

print("============================================================")

# calculatng error

delta4 =(training_set_outputs -output)

​ # calculatng error delta4 ​ = ​ ​ ( ​ training_set_outputs ​ - ​ output)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

print("Error:")

print(delta4)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

# finding errors in each layer

delta3 =dot(self.synaptic_weights3,delta4.T)*(self.sigmoid_deravative(a3).T)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

print("Error in Layer 3:")

print(delta3)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

delta2 =dot(self.synaptic_weights2,delta3)*(self.sigmoid_deravative(a2).T)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

print("Error in LAyer 2:")

print(delta2)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

# adjustments(gradient) in each layer

adjustment3 =dot(a3.T,delta4)

adjustment2 =dot(a2.T,delta3.T)

adjustment1 =dot(training_set_inputs.T,delta2.T)

delta3 ​ . ​ T) adjustment1 ​ = ​ dot ​ ( ​ training_set_inputs ​ .

# adjusting the weight accordingly

self.synaptic_weights1 +=adjustment1

self.synaptic_weights2 +=adjustment2

self.synaptic_weights3 +=adjustment3

defforward_pass(self,inputs):

# passing our inputs into neural networks

a2 =self.

sigmoid(dot(inputs,

self.synaptic_weights1))

a3 =self.

sigmoid(dot(a2,

self.synaptic_weights2))

 

output =self.

sigmoid(dot(a3,

self.synaptic_weights3))

returnoutput

 

if

name

=='

main

':

# initializing neural network

neural_network =NeuralNetwork()

# printing the initial weight assigned to the neural network

print("Random starting synaptics weights (layer 1): ")

print(neural_network.synaptic_weights1)

"Random starting synaptics weights (layer 1): ") ​ print ​ ( ​ neural_network ​ . ​

print("Random starting synaptics weights (layer 2): ")

print(neural_network.synaptic_weights2)

print("Random starting synaptics weights (layer 3): ")

print(neural_network.synaptic_weights3)

# loading the training set

training_set_inputs =array([[0,0,1],[1,1,1],[1,0,1],[0,1,1]])

plt.imshow(training_set_inputs)

plt.show()

# Interpolation calculates what the color or value of a pixel "should" be,

# according to different mathematical schemes. One common place that this

# happens is when you resize an image. The number of pixels change, but you

# want the same information. Since pixels are discrete, there's missing

# space. Interpolation is how you fill that space.

plt.imshow(training_set_inputs,interpolation='nearest')

plt.show()

training_set_outputs =array([[0,1,1,0]]).T

# training the neural network using the training sets

# Doing 10,000 times and making small adjustments every time

​ # training the neural network using the training sets ​ # Doing 10,000 times and

neural_network.train(training_set_inputs,training_set_outputs,10000)

print("\nnew synaptic weights (layer 1) after training: ")

print(neural_network.synaptic_weights1)

print("\nnew synaptic weights (layer 2) after training: ")

print(neural_network.synaptic_weights2)

print("\nnew synaptic weights (layer 3) after training: ")

print(neural_network.synaptic_weights3)

# Testing the neural network

print("Predicting

")

print("considering new situation [1, 0, 0] -> ?: ")

print(neural_network.forward_pass(array([1,0,0])))

​ ( ​ neural_network ​ . ​ forward_pass ​ ( ​ array ​ ([ ​ 1

Use of Neural Network

CharacterRecognition- The idea of character recognition has become very important as handheld devices like the Palm Pilot are becoming increasingly popular. Neural networks can be used to recognize handwritten characters.

ImageCompression- Neural networks can receive and process vast amounts of information at once, making them useful in image compression. With the Internet explosion and more sites using more images on their sites, using neural networks for image compression is worth a look.

StockMarketPrediction- The day-to-day business of the stock market is extremely complicated. Many factors weigh in whether a given stock will go up or down on any given day. Since neural networks can examine a lot of information quickly and sort it all out, they can be used to predict stock prices.

TravelingSaleman'sProblem- Interestingly enough, neural networks can solve the traveling salesman problem, but only to a certain degree of approximation.

Medicine,ElectronicNose,Security,andLoanApplications- These are some applications that are in their proof-of-concept stage, with the acception of a neural network that will decide whether or not to grant a loan, something that has already been used more successfully than many humans.

MiscellaneousApplications- These are some very interesting (albeit at times a little absurd) applications of neural networks.

​ - These are some very interesting (albeit at times a little absurd) applications of neural

Tensor Flow for classification

Classification is one of the most important parts of machine learning, as most of people’s communication is done via text. We write blog articles, email, tweet, leave notes and comments. All this information is there but is really hard to use compared to a form or data collected from some sensor.

There been classic NLP techniques dealing with this, by mostly using words as symbols and running linear models. This techniques worked but were very brittle. Recent adoption of embeddings and deep learning opened up a new ways of handling text.

"""Supervised problem."""

importpandas aspd # work with data as tables

importnumpy asnp # use number matrics

importmatplotlib.pyplot asplt

importtensorflow astf

# step 1: Load data

dataframe =pd.read_csv('data.csv')# dataframe objects

# remove the features we not care about

dataframe =dataframe.drop(['index','price','sq_price'],axis=1)

# we only use first 10 rows of dataset

dataframe =dataframe[0:10]

], ​ axis ​ = ​ 1) # we only use first 10 rows of dataset

print(dataframe)

# Step 2: Adding labels

# 1 is good buy and 0 is bad buy

dataframe.loc[:,('y1')]=[1,1,1,0,0,1,0,1,1,1]

# y2 is a negation of y1, opposite

dataframe.loc[:,('y2')]=dataframe['y1']==0

# turns True/False values to 1s and 0s

dataframe.loc[:,('y2')]=dataframe['y2'].astype(int)

# Step 3: Prepare data for tensorflow (tensors)

# tensors are a generic version of matrix and vectors

# vector - is a list of numbers (1D tensor)

# matrix is a list of list of numbers (2D tensor)

# list of list of list of numbers (3D tensors)

# converting features in input tensor

inputX =dataframe.loc[:,['area','bathrooms']].as_matrix()

# convert labels into tensors

inputY =dataframe.loc[:,['y1','y2']].as_matrix()

# print(inputX)

# print(inputY)

[:, ​ ​ [ ​ 'y1' ​ , ​ ​ 'y2' ​ ]]. ​ as_matrix ​

# Step 4: Write out our hyperparameters

learning_rate =0.000001

training

epochs

=2000

display_steps =50

n_samples =inputY.size

#

Step 5: create our compuatatin graph/Neural network

#

for feature input tensor, none means any number of examples

#

placeholders are gateways for data in our computational graph

x

=tf.placeholder(tf.float32,[None,2])

#

create weights

#

2x2 float matrix, that We'll keep updating through our training process

#

variable in tf hold and update parameters

#

in memory bufers contating tensors

w

=tf.Variable(tf.zeros([2,2]))

#

add biases (exampe c in y = mx + c, c is a bias)

b

=tf.Variable(tf.zeros([2]))

c in y = mx + c, c is a bias) b ​ = ​ tf

# multiply our weights by our inputs,

# weights are how the data flows in our compuattion graph

# multiply inputs with weights and biases

y_values =tf.add(tf.matmul(x,w),b)

# apply softmax to values we just created i.e "y_values"

# softmax is our activation function

y =tf.nn.softmax(y_values)

# feeding in a matrix labels

y_ =tf.placeholder(tf.float32,[None,2])

# perform training

# create our cost function, mean squared error

# reduced sum computes the sum of elements across dimensions of a tensor

cost =tf.reduce_sum(tf.pow(y_-y,2))/(2*n_samples)

# Gradient descent

optimizer =tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)

# initialize variables and tensorflow sessions

init =tf.global_variables_initializer()

sess =tf.Session()

sess.run(init)

# training loop

​ () sess ​ = ​ tf ​ . ​ Session ​ () sess ​ .

fori inrange(training epochs):

sess.run(optimizer,feed_dict={x:inputX,y_:inputY})# Take a gradient descent step using our inputs and labels

# That's all! The rest of the cell just outputs debug messages.

# Display logs per epoch step

if(i)%display_steps ==0:

cc =sess.run(cost,feed_dict={x:inputX,y_:inputY})

print("Training step:",'%04d'%(i),"cost=","{:.9f}".format(cc))#, \"W=", sess.run(W), "b=", sess.run(b)

print("Optimization Finished!")

training_cost =sess.run(cost,feed_dict={x:inputX,y_:inputY})

print("Training cost=",training_cost,"W=",sess.run(w),"b=",sess.run(b),'\n')

sess.run(y,feed_dict={x:inputX })

# So It's guessing they're all good houses. That makes it get 7/10 correct

sess.run(tf.nn.softmax([1.,2.]))

it get 7/10 correct sess ​ . ​ run ​ ( ​ tf ​ . ​