Logistic Regression with Stochastic Gradient Descent

5
Dataset
X1 X2 Y
2.7810836 2.550537 0 4
1.46548937 2.36212508 0
3.39656169 4.40029353 0 3
1.38807019 1.85022032 0
3.06407232 3.00530597 0 0
7.62753121 2.75926224 1 2
1
5.33244125 2.08862678 1
6.92259672 1.77106367 1 1
8.67541865 -0.2420687 1
7.67375647 3.50856301 1
0
Learning Rate 1 2 3 4 5 6 7 8 9 10
0.3 -1

Iteration Bias X1 X2 Y B0 B1
1 1 2.7810836 2.550537 0 0 0
1.1 1 1.46548937 2.36212508 0 -0.0375 -0.1042906
1.2 1 3.39656169 4.40029353 0 -0.0660511 -0.146132
1.3 1 1.38807019 1.85022032 0 -0.0771603 -0.183865
1.4 1 3.06407232 3.00530597 0 -0.098688 -0.213747
1.5 1 7.62753121 2.75926224 1 -0.1067283 -0.238383
1.6 1 5.33244125 2.08862678 1 -0.0899579 -0.110466
1.7 1 6.92259672 1.77106367 1 -0.0484417 0.11091644
1.8 1 8.67541865 -0.2420687 1 -0.021043 0.30058666
1.9 1 7.67375647 3.50856301 1 -0.0197308 0.311971
2 1 2.7810836 2.550537 0 -0.0162259 0.33886677
2.1 1 1.46548937 2.36212508 0 -0.0606627 0.21528421
2.2 1 3.39656169 4.40029353 0 -0.0939223 0.16654263
2.3 1 1.38807019 1.85022032 0 -0.1152085 0.0942427
2.4 1 3.06407232 3.00530597 0 -0.1380512 0.06253544
2.5 1 7.62753121 2.75926224 1 -0.1507378 0.02366275
2.6 1 5.33244125 2.08862678 1 -0.1098186 0.33577537
2.7 1 6.92259672 1.77106367 1 -0.0935723 0.42240779
2.8 1 8.67541865 -0.2420687 1 -0.0912943 0.4381772
2.9 1 7.67375647 3.50856301 1 -0.0911489 0.43943896
3 1 2.7810836 2.550537 0 -0.0886254 0.45880357
3.1 1 1.46548937 2.36212508 0 -0.1321875 0.33765377
3.2 1 3.39656169 4.40029353 0 -0.1569223 0.30140509
3.3 1 1.38807019 1.85022032 0 -0.170105 0.25662951
3.4 1 3.06407232 3.00530597 0 -0.1906622 0.22809459
3.5 1 7.62753121 2.75926224 1 -0.2037116 0.18811044
3.6 1 5.33244125 2.08862678 1 -0.1605017 0.51769499
3.7 1 6.92259672 1.77106367 1 -0.1535572 0.55472641
3.8 1 8.67541865 -0.2420687 1 -0.1527432 0.56036133
3.9 1 7.67375647 3.50856301 1 -0.1527241 0.56052679
 4 1 2.7810836 2.550537 0 -0.1512393 0.57192103
4.1 1 1.46548937 2.36212508 0 -0.1931718 0.45530327
4.2 1 3.39656169 4.40029353 0 -0.212037 0.42765641
4.3 1 1.38807019 1.85022032 0 -0.2211896 0.39656916
4.4 1 3.06407232 3.00530597 0 -0.2397437 0.37081471
4.5 1 7.62753121 2.75926224 1 -0.2530068 0.33017581
4.6 1 5.33244125 2.08862678 1 -0.2213731 0.57146232
4.7 1 6.92259672 1.77106367 1 -0.2132843 0.61459563
4.8 1 8.67541865 -0.2420687 1 -0.2125819 0.61945767
4.9 1 7.67375647 3.50856301 1 -0.2125747 0.6195203
5 1 2.7810836 2.550537 0 -0.2106325 0.63442462
5.1 1 1.46548937 2.36212508 0 -0.2482419 0.52982979
5.2 1 3.39656169 4.40029353 0 -0.2617266 0.51006802
5.3 1 1.38807019 1.85022032 0 -0.2672246 0.49139363
5.4 1 3.06407232 3.00530597 0 -0.283293 0.46908969
5.5 1 7.62753121 2.75926224 1 -0.2953913 0.43201956
5.6 1 5.33244125 2.08862678 1 -0.273842 0.5963871
5.7 1 6.92259672 1.77106367 1 -0.2631391 0.6534601
5.8 1 8.67541865 -0.2420687 1 -0.2624264 0.6583932
5.9 1 7.67375647 3.50856301 1 -0.2624226 0.65842642
6 1 2.7810836 2.550537 0 -0.2596765 0.67949925
6.1 1 1.46548937 2.36212508 0 -0.2922813 0.58882259
6.2 1 3.39656169 4.40029353 0 -0.3023027 0.57413641
6.3 1 1.38807019 1.85022032 0 -0.3057923 0.56228363
6.4 1 3.06407232 3.00530597 0 -0.3197221 0.54294809
6.5 1 7.62753121 2.75926224 1 -0.3305152 0.50987742
6.6 1 5.33244125 2.08862678 1 -0.3154055 0.62512708
6.7 1 6.92259672 1.77106367 1 -0.3030353 0.69109014
6.8 1 8.67541865 -0.2420687 1 -0.3023736 0.69567084
6.9 1 7.67375647 3.50856301 1 -0.3023715 0.69568883
7 1 2.7810836 2.550537 0 -0.2990558 0.72113296
7.1 1 1.46548937 2.36212508 0 -0.3276049 0.64173558
7.2 1 3.39656169 4.40029353 0 -0.3354733 0.63020451
7.3 1 1.38807019 1.85022032 0 -0.3379007 0.62195967
7.4 1 3.06407232 3.00530597 0 -0.3501994 0.60488825
7.5 1 7.62753121 2.75926224 1 -0.3599754 0.57493381
7.6 1 5.33244125 2.08862678 1 -0.3492103 0.65704506
7.7 1 6.92259672 1.77106367 1 -0.3361333 0.7267773
7.8 1 8.67541865 -0.2420687 1 -0.3355464 0.73084032
7.9 1 7.67375647 3.50856301 1 -0.3355452 0.73085036
8 1 2.7810836 2.550537 0 -0.3318835 0.75894968
8.1 1 1.46548937 2.36212508 0 -0.3571991 0.68854486
8.2 1 3.39656169 4.40029353 0 -0.363599 0.6791658
8.3 1 1.38807019 1.85022032 0 -0.365384 0.6731031
8.4 1 3.06407232 3.00530597 0 -0.3763871 0.65783003
8.5 1 7.62753121 2.75926224 1 -0.3853272 0.63043703
8.6 1 5.33244125 2.08862678 1 -0.3774576 0.69046204
8.7 1 6.92259672 1.77106367 1 -0.3643785 0.76020554
8.8 1 8.67541865 -0.2420687 1 -0.3638693 0.76373108
 8.9 1 7.67375647 3.50856301 1 -0.3638686 0.76373689
9 1 2.7810836 2.550537 0 -0.3600312 0.79318422
9.1 1 1.46548937 2.36212508 0 -0.3827368 0.73003806
9.2 1 3.39656169 4.40029353 0 -0.3880735 0.72221721
9.3 1 1.38807019 1.85022032 0 -0.3894365 0.71758761
9.4 1 3.06407232 3.00530597 0 -0.3993797 0.70378568
9.5 1 7.62753121 2.75926224 1 -0.4076091 0.67857032
9.6 1 5.33244125 2.08862678 1 -0.4016925 0.72369969
9.7 1 6.92259672 1.77106367 1 -0.389016 0.79129623
9.8 1 8.67541865 -0.2420687 1 -0.3885777 0.79433019
9.9 1 7.67375647 3.50856301 1 -0.3885773 0.79433368
10 1 2.7810836 2.550537 0 -0.3846794 0.82424538
10.1 1 1.46548937 2.36212508 0 -0.4052421 0.76705863
10.2 1 3.39656169 4.40029353 0 -0.4097766 0.7604135
10.3 1 1.38807019 1.85022032 0 -0.4108468 0.75677825
10.4 1 3.06407232 3.00530597 0 -0.4199046 0.74420546
10.5 1 7.62753121 2.75926224 1 -0.4275175 0.72087908
10.6 1 5.33244125 2.08862678 1 -0.4229472 0.75573869
10.7 1 6.92259672 1.77106367 1 -0.4108663 0.82015957
10.8 1 8.67541865 -0.2420687 1 -0.4104896 0.82276745
10.9 1 7.67375647 3.50856301 1 -0.4104893 0.82276962

Coefficients
B0 B1 B2
-0.4066055 0.85257332 -1.1047463

Prediction
X1 X2 Prediction Crisp Y Error Accuracy
2.7810836 2.550537 0.29875699 0 0 0
1.46548937 2.36212508 0.14595106 0 0 0
3.39656169 4.40029353 0.08533327 0 0 0
1.38807019 1.85022032 0.21973731 0 0 0
3.06407232 3.00530597 0.247059 0 0 0
7.62753121 2.75926224 0.95470213 1 1 0
5.33244125 2.08862678 0.86203419 1 1 0
6.92259672 1.77106367 0.97177291 1 1 0
8.67541865 -0.2420687 0.99929545 1 1 0
7.67375647 3.50856301 0.90548932 1 1 0 100
 Accuracy
120.00% 0.5
0.45
100.00% 0.4
0.35
0 80.00%
0.3
1
60.00% 0.25
0.2
40.00% 0.15
0.1
20.00%
8 9 10 0.05

0.00% 0

B2 Prediction B0(t+1) B1(t+1) B2(t+1) Sharp PredictionSquared Err

0 0.5 -0.0375 -0.1042906 -0.0956451 1 0.25
-0.0956451 0.39741142 -0.0660511 -0.146132 -0.1630864 0 0.15793584
-0.1630864 0.2175459 -0.0771603 -0.183865 -0.2119701 0 0.04732622
-0.2119701 0.32638764 -0.098688 -0.213747 -0.2518011 0 0.10652889
-0.2518011 0.18088489 -0.1067283 -0.238383 -0.2759646 0 0.03271934
-0.2759646 0.06377702 -0.0899579 -0.110466 -0.2296906 0 0.87651346
-0.2296906 0.23889458 -0.0484417 0.11091644 -0.1429788 0 0.57928146
-0.1429788 0.6144753 -0.021043 0.30058666 -0.094454 1 0.1486293
-0.094454 0.93147281 -0.0197308 0.311971 -0.0947716 1 0.00469598
-0.0947716 0.88511098 -0.0162259 0.33886677 -0.0824745 1 0.01319949
-0.0824745 0.67168959 -0.0606627 0.21528421 -0.1958123 1 0.4511669
-0.1958123 0.4482611 -0.0939223 0.16654263 -0.2743756 0 0.20093802
-0.2743756 0.32397074 -0.1152085 0.0942427 -0.3680412 0 0.10495704
-0.3680412 0.33953878 -0.1380512 0.06253544 -0.4103052 0 0.11528658
-0.4103052 0.23513659 -0.1507378 0.02366275 -0.4484323 0 0.05528921
-0.4484323 0.23012743 -0.1098186 0.33577537 -0.3355255 0 0.59270378
-0.3355255 0.72708765 -0.0935723 0.42240779 -0.301593 1 0.07448115
-0.301593 0.90858228 -0.0912943 0.4381772 -0.2975586 1 0.0083572
-0.2975586 0.97773246 -0.0911489 0.43943896 -0.2975938 1 0.00049584
-0.2975938 0.90351211 -0.0886254 0.45880357 -0.28874 1 0.00930991
-0.28874 0.61085406 -0.1321875 0.33765377 -0.3998467 1 0.37314268
-0.3998467 0.35850761 -0.1569223 0.30140509 -0.4582736 0 0.1285277
-0.4582736 0.24054031 -0.170105 0.25662951 -0.516281 0 0.05785964
-0.516281 0.31666978 -0.1906622 0.22809459 -0.5543164 0 0.10027975
-0.5543164 0.23909377 -0.2037116 0.18811044 -0.5935337 0 0.05716583
-0.5935337 0.39972563 -0.1605017 0.51769499 -0.4743064 0 0.36032932
-0.4743064 0.83333188 -0.1535572 0.55472641 -0.4598018 1 0.02777826
-0.4598018 0.94645758 -0.1527432 0.56036133 -0.4583602 1 0.00286679
-0.4583602 0.99199444 -0.1527241 0.56052679 -0.4583648 1 6.4089E-05
-0.4583648 0.9269273 -0.1512393 0.57192103 -0.4531552 1 0.00533962
-0.4531552 0.57041192 -0.1931718 0.45530327 -0.5601055 1 0.32536975
-0.5601055 0.29964919 -0.212037 0.42765641 -0.6046677 0 0.08978964
-0.6046677 0.19463167 -0.2211896 0.39656916 -0.6449416 0 0.03788149
-0.6449416 0.29650267 -0.2397437 0.37081471 -0.6792709 0 0.08791383
-0.6792709 0.24141132 -0.2530068 0.33017581 -0.7191304 0 0.05827943
-0.7191304 0.56982821 -0.2213731 0.57146232 -0.6318449 1 0.18504777
-0.6318449 0.81850152 -0.2132843 0.61459563 -0.6149503 1 0.0329417
-0.6149503 0.95036724 -0.2125819 0.61945767 -0.6137064 1 0.00246341
-0.6137064 0.99508253 -0.2125747 0.6195203 -0.6137081 1 2.4181E-05
-0.6137081 0.9159261 -0.2106325 0.63442462 -0.6068936 1 0.00706842
-0.6068936 0.50146267 -0.2482419 0.52982979 -0.7028178 1 0.25146481
-0.7028178 0.24380595 -0.2617266 0.51006802 -0.7346704 0 0.05944134
-0.7346704 0.14653819 -0.2672246 0.49139363 -0.7588634 0 0.02147344
-0.7588634 0.27107007 -0.283293 0.46908969 -0.7885933 0 0.07347898
-0.7885933 0.22865307 -0.2953913 0.43201956 -0.8249524 0 0.05228223
-0.8249524 0.67339735 -0.273842 0.5963871 -0.7654924 1 0.10666929
-0.7654924 0.7870994 -0.2631391 0.6534601 -0.7431379 1 0.04532666
-0.7431379 0.94999611 -0.2624264 0.6583932 -0.7418758 1 0.00250039
-0.7418758 0.99642067 -0.2624226 0.65842642 -0.7418768 1 1.2812E-05
-0.7418768 0.89909961 -0.2596765 0.67949925 -0.7322419 1 0.01018089
-0.7322419 0.44089206 -0.2922813 0.58882259 -0.8154017 0 0.19438581
-0.8154017 0.20498099 -0.3023027 0.57413641 -0.8390733 0 0.0420172
-0.8390733 0.11462129 -0.3057923 0.56228363 -0.8544288 0 0.01313804
-0.8544288 0.24858287 -0.3197221 0.54294809 -0.880202 0 0.06179344
-0.880202 0.21393519 -0.3305152 0.50987742 -0.9126384 0 0.04576827
-0.9126384 0.73892374 -0.3154055 0.62512708 -0.8709468 1 0.06816082
-0.8709468 0.76834065 -0.3030353 0.69109014 -0.8451102 1 0.05366606
-0.8451102 0.9518625 -0.3023736 0.69567084 -0.8439382 1 0.00231722
-0.8439382 0.99736742 -0.3023715 0.69568883 -0.8439387 1 6.9305E-06
-0.8439387 0.8884656 -0.2990558 0.72113296 -0.8323053 1 0.01243992
-0.8323053 0.39739044 -0.3276049 0.64173558 -0.9051208 0 0.15791916
-0.9051208 0.17870343 -0.3354733 0.63020451 -0.9237069 0 0.03193491
-0.9237069 0.09453089 -0.3379007 0.62195967 -0.9343882 0 0.00893609
-0.9343882 0.23087073 -0.3501994 0.60488825 -0.9571435 0 0.05330129
-0.9571435 0.20208928 -0.3599754 0.57493381 -0.9865234 0 0.04084008
-0.9865234 0.78638537 -0.3492103 0.65704506 -0.9568197 1 0.04563121
-0.9568197 0.760606 -0.3361333 0.7267773 -0.9295067 1 0.05730949
-0.9295067 0.9547323 -0.3355464 0.73084032 -0.9284672 1 0.00204916
-0.9284672 0.99803354 -0.3355452 0.73085036 -0.9284675 1 3.867E-06
-0.9284675 0.88238747 -0.3318835 0.75894968 -0.9156201 1 0.01383271
-0.9156201 0.36435679 -0.3571991 0.68854486 -0.9801885 0 0.13275587
-0.9801885 0.15929651 -0.363599 0.6791658 -0.995306 0 0.02537538
-0.995306 0.08043814 -0.365384 0.6731031 -1.0031603 0 0.00647029
-1.0031603 0.21633777 -0.3763871 0.65783003 -1.0235184 0 0.04680203
-1.0235184 0.19205157 -0.3853272 0.63043703 -1.050386 0 0.03688381
-1.050386 0.82128245 -0.3774576 0.69046204 -1.028672 1 0.03193996
-1.028672 0.76058306 -0.3643785 0.76020554 -1.0013546 1 0.05732047
-1.0013546 0.95790239 -0.3638693 0.76373108 -1.0004527 1 0.00177221
-1.0004527 0.9985048 -0.3638686 0.76373689 -1.0004528 1 2.2356E-06
-1.0004528 0.87939469 -0.3600312 0.79318422 -0.986989 1 0.01454564
-0.986989 0.3381674 -0.3827368 0.73003806 -1.0449005 0 0.11435719
-1.0449005 0.14417237 -0.3880735 0.72221721 -1.0575064 0 0.02078567
-1.0575064 0.06989153 -0.3894365 0.71758761 -1.0635041 0 0.00488483
-1.0635041 0.20406297 -0.3993797 0.70378568 -1.0819013 0 0.04164169
-1.0819013 0.18326611 -0.4076091 0.67857032 -1.1066331 0 0.03358647
-1.1066331 0.84744701 -0.4016925 0.72369969 -1.0903075 1 0.02327241
-1.0903075 0.76497444 -0.389016 0.79129623 -1.0638311 1 0.05523702
-1.0638311 0.96101062 -0.3885777 0.79433019 -1.0630549 1 0.00152017
-1.0630549 0.998842 -0.3885773 0.79433368 -1.063055 1 1.341E-06
-1.063055 0.87837707 -0.3846794 0.82424538 -1.0493789 1 0.01479214
-1.0493789 0.31672495 -0.4052421 0.76705863 -1.101825 0 0.10031469
-1.101825 0.13195596 -0.4097766 0.7604135 -1.1125358 0 0.01741237
-1.1125358 0.06166044 -0.4108468 0.75677825 -1.1172453 0 0.00380201
-1.1172453 0.19348297 -0.4199046 0.74420546 -1.1340042 0 0.03743566
-1.1340042 0.17542815 -0.4275175 0.72087908 -1.1568832 0 0.03077504
-1.1568832 0.86748085 -0.4229472 0.75573869 -1.1442727 1 0.01756133
-1.1442727 0.77153982 -0.4108663 0.82015957 -1.1190401 1 0.05219405
-1.1190401 0.96390632 -0.4104896 0.82276745 -1.1183729 1 0.00130275
-1.1183729 0.9990872 -0.4104893 0.82276962 -1.118373 1 8.332E-07
-1.118373 0.87861317 -0.4066055 0.85257332 -1.1047463 1 0.01473476
 RMSE
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0

RMSE Error Accuracy

1
0
0
0
0
1
1
0
0
0.47083224 0 70.00%
1
0
0
0
0
1
0
0
0
0.40161992 0 80.00%
1
0
0
0
0
1
0
0
0
0.33366955 0 80.00%
 1
0
0
0
0
0
0
0
0
0.28753776 0 90.00%
1
0
0
0
0
0
0
0
0
0.24956579 0 90.00%
0
0
0
0
0
0
0
0
0
0.22219219 0 100.00%
0
0
0
0
0
0
0
0
0
0.2029182 0 100.00%
0
0
0
0
0
0
0
0
0
0.18811377 0 100.00%
0
0
0
0
0
0
0
0
0
0.17609058 0 100.00%
0
0
0
0
0
0
0
0
0
0.16599202 0 100.00%