Вы находитесь на странице: 1из 26

Semester Core Project

Final Report Submitted by:

Vaibhav Singh Submitted to:

Mrs. Nishtha phutela AboutDeeplearning 3 LinearRegression 5 R2 6 Code: 7 Challenge 8 Code 8 UseofLinearregression 10 NeuralNetwork 10 ​Process 10 Code 11 UseofNeuralNetwork 22 TensorFlowforclassification 23 Traditionally, programing has been about defining every single step for a program to reach an outcome. The machine learning flips that approach. With machine learning, we define the outcome and program learn to steps get there. So if we build an app that can recognize street signs so instead of writing the code to recognize hundreds of different feature of street signs like this shape of letters and the colors. We will just say here are some examples of a street signs, learn the shape you need to recognize it. Sometimes, we have not any idea what steps could possibly be. Machine learning is already everywhere on the internet.Every major services uses it in some way. Youtube uses it to decide which other videos we might like as we watch and its uses will only grow over time. There are lot of machine learning models are there and one of them are neural networks. When we use a neural network that’s not just one or two but many layers deep to make a prediction, we called that deep learning. It’s a subset of machine learning that has outperformed every other type of model almost every time on a huge range of tasks. Linear Regression

Simple linear regression is a statistical method that allows us to summarize and study relationships between two continuous (quantitative) variables:

One variable, denoted x, is regarded as the predictor, explanatory, or independent variable.

The other variable, denoted y, is regarded as the response, outcome, or dependent variable.

With simple linear regression we want to model our data as follows:

y = B0 + B1 * x (similar to y = mx + c)

This is a line where y is the output variable we want to predict, x is the input variable we know and B0 and B1 are coefficients that we need to estimate that move the line around. R2

The R2 term

fits the observed data. The coefficient of determination is usually given by,

is the coefficient of determination and it usuallyreflects how well the model Where Yi is an observed response, is the mean of the observed responses,is a prediction of the response made by the linear model, and (Yi -) is the residual, i.e., the difference between the response and the prediction. Also, SSE is called the sum of the squared error, or SSR the sum of the squared residuals, and SST is called the total sum of squares. Code:

importpandas aspd

fromsklearn importlinear_model

importmatplotlib.pyplot asplt

print(dataframe)

x_values =dataframe[['Brain']]

y_values =dataframe[['Body']]

# train model on data

body_regression =linear_model.LinearRegression()

body_regression.fit(x_values,y_values)

# visualize results

plt.scatter(x_values,y_values)

plt.plot(x_values,body_regression.predict(x_values))

plt.show() Challenge

challenge is to use scikit-learn to create a line of best fit for the included 'challenge_dataset'. Then, make a prediction for an existing data point and see how close it matches up to the actual value. Print out the error you get. You can use scikit-learn's documentation for more help.

Code

importpandas aspd

importnumpy asnp

fromsklearn importlinear_model

importmatplotlib.pyplot asplt

frompylab importfigure

dataframes.columns =["X_data","Y_data"]

x_values =dataframes[['X_data']]

y_values =dataframes[['Y_data']]

# Train data with linear regression model

model =linear_model.LinearRegression()

model.fit(x_values,y_values) # function for calculating residuals

defcalculate_residual(actual,predicted):

res =(np.array(actual)-np.array(predicted)**2)

returnres

# Evaluating fit model

residuals =calculate_residual(y_values,model.predict(x_values))

print("Mean squared error: ",np.mean(residuals))

print("Coefficient of determination: ",model.score(x_values,y_values))

# Plotting on matplotlib

plt.figure(1)

plt.scatter(x_values,y_values,color="blue")

plt.savefig("Scatter.png")

plt.figure(2)

plt.plot(x_values,model.predict(x_values),color="black")

plt.figure(3)

plt.plot(residuals)

plt.savefig("residual.png")

plt.show() Use of Linear regression

1. In environmental science it is used to try to establish how much one quantity, say atmospheric greenhouse gasses, influences another, say global surface temperature.

2. In quantitative finance linear regression is core to everything, as they use something called a linear factor model. Long story short it estimates how much the price of one asset will move when another thing (oil prices) move.

Neural Network

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurons) working in unison to solve specific problems. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurones. This is true of ANNs as well.

Process

We will give each input a weight, which can be a positive or negative number. An input with a large positive weight or a large negative weight, will have a strong effect on the neuron’s output. 1. Take the inputs from a training set example, adjust them by the weights, and pass them through a special formula to calculate the neuron’s output.

2. Calculate the error, which is the difference between the neuron’s output and the desired output in the training set example.

3. Depending on the direction of the error, adjust the weights slightly.

4. Repeat this process 10, 000 times.

Code

fromnumpy importexp,array,random,dot

importnumpy asnp

classNeuralNetwork():

"""docstring for ."""

def

init

(self):

# seed the random number generation, so it generates the same number

# every time the program runs

random.seed(1)

# we model a single neuron, with 3 input connections and 1 output connection.

# we assign random weights to 3 x 1 matrix, with a value in the range

# of -1 to 1 a mean of 0

# our neuron have 3 input connections and 1 output connections self.synaptic_weights =2*np.random.random((3,1))-1

# Activation Function

# The sigmoid function, which describes an s shaped curve

# we pass the weighted sum of the inputs through this function

# to normalise them between 0 and 1

def

sigmoid(self,x):

return1/(1+exp(-x))

# gradient of the sigmoid curve

def

sigmoid_deravative(self,x):

returnx *(1-x)

deftrain(self,training_set_inputs,training_set_outputs,number_of_training_iterations):

foriteration inrange(number_of_training_iterations):

# pass the training set through our neural net

output =self.predict(training_set_inputs)

# calculate the error

error =(training_set_outputs -output) # multiply the error by the input ad again by the gradient of the

# sigmoid curve

self.sigmoid_deravative(output))

defpredict(self,inputs):

# pass inputs through our neural network (our singe neuron)

if

returnself.

name

=='

main

sigmoid(dot(inputs,

':

self.synaptic_weights))

# initialise a single neuron neural network

neural_network =NeuralNetwork()

print('Random starting synaptic weights: ')

print(neural_network.synaptic_weights)

# the training set. We have 4 examples, each consisting of 3 input values

# and 1 output value. training_set_inputs =array([[0,0,1],[1,1,1],[1,0,1],[0,1,1]])

training_set_outputs =array([[0,1,1,0]]).T

# train the neural network using the training sets

# Do it 10,000 times amd making small adjustments each time

neural_network.train(training_set_inputs,training_set_outputs,10000)

print('New synaptic weights after training: ')

print(neural_network.synaptic_weights)

# Test the neural network

print('predicting

')

print('considering new situation [1, 0, 0] -> ?: ')

print(neural_network.predict(array([1,0,0]))) Challenge is to create a 3 layer feedforward neural network using only numpy as dependency.

Challenge.py

fromnumpy importexp,array,random,dot

importnumpy asnp

importmatplotlib.pyplot asplt

# %matplotlib inline

classNeuralNetwork():

"""docstring for NeuralNetwork."""

def

init

(self):

# seed the random number generation, so it generates the same number

# every time

random.seed(1)

# setting the number of nodes in layer 2 and layer 3

# more the number of nodes in hidden layer more the confident the

# neural network in predicting

l2 =5

l3 =6 # assign random weights to the matrix

# (no. of nodes in previous layer) x (no. of nodes in following layer)

self.synaptic_weights1 =2*np.random.random((3,l2))-1

self.synaptic_weights2 =2*np.random.random((l2,l3))-1

self.synaptic_weights3 =2*np.random.random((l3,1))-1

def

sigmoid(self,x):

return1/(1+exp(-x))

def

sigmoid_deravative(self,x):

returnx *(1-x)

# train neural network, adjusting synaptics weight each time

deftrain(self,training_set_inputs,training_set_outputs,number_of_training_iterations):

foriteration inrange(number_of_training_iterations):

# passing training set through our neual network

# a2 means activation fed to the second layer

a2 =self.

sigmoid(dot(training_set_inputs,

self.synaptic_weights1))

print("============================================================")

print("Dot product output of 2nd Layer") print(dot(training_set_inputs,self.synaptic_weights1))

print("Sigmoid function output")

print(a2)

print("============================================================")

a3 =self.

sigmoid(dot(a2,

self.synaptic_weights2))

print("============================================================")

print("Dot product output of 3rd Layer")

print(dot(a2,self.synaptic_weights2))

print("Sigmoid function output")

print(a3)

print("============================================================")

output =self.

sigmoid(dot(a3,

self.synaptic_weights3))

print("============================================================")

print("Dot product output layer")

print(dot(a3,self.synaptic_weights3))

print("Sigmoid function output")

print(output)

print("============================================================")

# calculatng error

delta4 =(training_set_outputs -output) print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

print("Error:")

print(delta4)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

# finding errors in each layer

delta3 =dot(self.synaptic_weights3,delta4.T)*(self.sigmoid_deravative(a3).T)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

print("Error in Layer 3:")

print(delta3)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

delta2 =dot(self.synaptic_weights2,delta3)*(self.sigmoid_deravative(a2).T)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

print("Error in LAyer 2:")

print(delta2)

print("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++") defforward_pass(self,inputs):

# passing our inputs into neural networks

 a2 ​=​ ​self​.​ sigmoid​(​dot​(​inputs​,​ ​self​.​synaptic_weights1​)) a3 ​=​ ​self​.​ sigmoid​(​dot​(​a2​,​ ​self​.​synaptic_weights2​))
 output ​=​ ​self​.​ sigmoid​(​dot​(​a3​,​ ​self​.​synaptic_weights3​)) ​return​ output if​ name ​==​ ​' main ':

# initializing neural network

neural_network =NeuralNetwork()

# printing the initial weight assigned to the neural network

print("Random starting synaptics weights (layer 1): ")

print(neural_network.synaptic_weights1) print("Random starting synaptics weights (layer 2): ")

print(neural_network.synaptic_weights2)

print("Random starting synaptics weights (layer 3): ")

print(neural_network.synaptic_weights3)

training_set_inputs =array([[0,0,1],[1,1,1],[1,0,1],[0,1,1]])

plt.imshow(training_set_inputs)

plt.show()

# Interpolation calculates what the color or value of a pixel "should" be,

# according to different mathematical schemes. One common place that this

# happens is when you resize an image. The number of pixels change, but you

# want the same information. Since pixels are discrete, there's missing

# space. Interpolation is how you fill that space.

plt.imshow(training_set_inputs,interpolation='nearest')

plt.show()

training_set_outputs =array([[0,1,1,0]]).T

# training the neural network using the training sets

# Doing 10,000 times and making small adjustments every time neural_network.train(training_set_inputs,training_set_outputs,10000)

print("\nnew synaptic weights (layer 1) after training: ")

print(neural_network.synaptic_weights1)

print("\nnew synaptic weights (layer 2) after training: ")

print(neural_network.synaptic_weights2)

print("\nnew synaptic weights (layer 3) after training: ")

print(neural_network.synaptic_weights3)

# Testing the neural network

print("Predicting

")

print("considering new situation [1, 0, 0] -> ?: ")

print(neural_network.forward_pass(array([1,0,0]))) Use of Neural Network

CharacterRecognition- The idea of character recognition has become very important as handheld devices like the Palm Pilot are becoming increasingly popular. Neural networks can be used to recognize handwritten characters.

ImageCompression- Neural networks can receive and process vast amounts of information at once, making them useful in image compression. With the Internet explosion and more sites using more images on their sites, using neural networks for image compression is worth a look.

StockMarketPrediction- The day-to-day business of the stock market is extremely complicated. Many factors weigh in whether a given stock will go up or down on any given day. Since neural networks can examine a lot of information quickly and sort it all out, they can be used to predict stock prices.

TravelingSaleman'sProblem- Interestingly enough, neural networks can solve the traveling salesman problem, but only to a certain degree of approximation.

Medicine,ElectronicNose,Security,andLoanApplications- These are some applications that are in their proof-of-concept stage, with the acception of a neural network that will decide whether or not to grant a loan, something that has already been used more successfully than many humans.

MiscellaneousApplications- These are some very interesting (albeit at times a little absurd) applications of neural networks. Tensor Flow for classification

Classification is one of the most important parts of machine learning, as most of people’s communication is done via text. We write blog articles, email, tweet, leave notes and comments. All this information is there but is really hard to use compared to a form or data collected from some sensor.

There been classic NLP techniques dealing with this, by mostly using words as symbols and running linear models. This techniques worked but were very brittle. Recent adoption of embeddings and deep learning opened up a new ways of handling text.

"""Supervised problem."""

importpandas aspd # work with data as tables

importnumpy asnp # use number matrics

importmatplotlib.pyplot asplt

importtensorflow astf

# remove the features we not care about

dataframe =dataframe.drop(['index','price','sq_price'],axis=1)

# we only use first 10 rows of dataset

dataframe =dataframe[0:10] print(dataframe)

dataframe.loc[:,('y1')]=[1,1,1,0,0,1,0,1,1,1]

# y2 is a negation of y1, opposite

dataframe.loc[:,('y2')]=dataframe['y1']==0

# turns True/False values to 1s and 0s

dataframe.loc[:,('y2')]=dataframe['y2'].astype(int)

# Step 3: Prepare data for tensorflow (tensors)

# tensors are a generic version of matrix and vectors

# vector - is a list of numbers (1D tensor)

# matrix is a list of list of numbers (2D tensor)

# list of list of list of numbers (3D tensors)

# converting features in input tensor

inputX =dataframe.loc[:,['area','bathrooms']].as_matrix()

# convert labels into tensors

inputY =dataframe.loc[:,['y1','y2']].as_matrix()

# print(inputX)

# print(inputY) # Step 4: Write out our hyperparameters

learning_rate =0.000001

training

epochs

=2000

display_steps =50

n_samples =inputY.size

 # Step 5: create our compuatatin graph/Neural network # for feature input tensor, none means any number of examples # placeholders are gateways for data in our computational graph x ​=​ tf​.​placeholder​(​tf​.​float32​,​ ​[​None​,​ ​2​]) # create weights # 2x2 float matrix, that We'll keep updating through our training process # variable in tf hold and update parameters # in memory bufers contating tensors w ​=​ tf​.​Variable​(​tf​.​zeros​([​2​,​ ​2​])) # add biases (exampe c in y = mx + c, c is a bias) b ​=​ tf​.​Variable​(​tf​.​zeros​([​2​])) # multiply our weights by our inputs,

# weights are how the data flows in our compuattion graph

# multiply inputs with weights and biases

# apply softmax to values we just created i.e "y_values"

# softmax is our activation function

y =tf.nn.softmax(y_values)

# feeding in a matrix labels

y_ =tf.placeholder(tf.float32,[None,2])

# perform training

# create our cost function, mean squared error

# reduced sum computes the sum of elements across dimensions of a tensor

cost =tf.reduce_sum(tf.pow(y_-y,2))/(2*n_samples)

# initialize variables and tensorflow sessions

init =tf.global_variables_initializer()

sess =tf.Session()

sess.run(init)

# training loop fori inrange(training epochs):

sess.run(optimizer,feed_dict={x:inputX,y_:inputY})# Take a gradient descent step using our inputs and labels

# That's all! The rest of the cell just outputs debug messages.

# Display logs per epoch step

if(i)%display_steps ==0:

cc =sess.run(cost,feed_dict={x:inputX,y_:inputY})

print("Training step:",'%04d'%(i),"cost=","{:.9f}".format(cc))#, \"W=", sess.run(W), "b=", sess.run(b)

print("Optimization Finished!")

training_cost =sess.run(cost,feed_dict={x:inputX,y_:inputY})

print("Training cost=",training_cost,"W=",sess.run(w),"b=",sess.run(b),'\n')

sess.run(y,feed_dict={x:inputX })

# So It's guessing they're all good houses. That makes it get 7/10 correct

sess.run(tf.nn.softmax([1.,2.])) 