]]>

I am into this weird problem where i loaded a pre trained keras model which was trained for 300 iteration on cifar10 dataset and had a test accuracy of 67%. Now i loaded the model and trained it for few iterations more but this time got test accuracy of 267%.

How is that possible? What would be possibly wrong?

]]>May I know how to modify my own Python programming so that I will get the **same picture **as refer to the attached file - Adaline Stochastic gradient descent

(I am using the Anaconda Python 3.7)

Tron Orino Yeong

tcynotebook@yahoo.com

0916643858

```
from matplotlib.colors import ListedColormap
import matplotlib.pyplot as plt
import numpy as np
from numpy.random import seed
import pandas as pd
# Stochastic Gradient Descent
class SGD(object):
def __init__(self, rate = 0.01, niter = 10,
shuffle=True, random_state=None):
self.rate = rate
self.niter = niter
self.weight_initialized = False
# If True, Shuffles training data every epoch
self.shuffle = shuffle
# Set random state for shuffling and initializing the weights.
if random_state:
seed(random_state)
def fit(self, X, y):
"""Fit training data
X : Training vectors, X.shape :
y : Target values, y.shape :
"""
# weights
self.initialize_weights(X.shape)
# Cost function
self.cost = []
for i in range(self.niter):
if self.shuffle:
X, y = self.shuffle_set(X, y)
cost = []
for xi, target in zip(X, y):
cost.append(self.update_weights(xi, target))
avg_cost = sum(cost)/len(y)
self.cost.append(avg_cost)
return self
def partial_fit(self, X, y):
"""Fit training data without reinitializing the weights"""
if not self.weight_initialized:
self.initialize_weights(X.shape)
if y.ravel().shape > 1:
for xi, target in zip(X, y):
self.update_weights(xi, target)
else:
self.up
return self
def shuffle_set(self, X, y):
"""Shuffle training data"""
r = np.random.permutation(len(y))
return X, y
def initialize_weights(self, m):
"""Initialize weights to zeros"""
self.weight = np.zeros(1 + m)
self.weight_initialized = True
def update_weights(self, xi, target):
"""Apply SGD learning rule to update the weights"""
output = self.net_input(xi)
error = (target - output)
self.weight += self.rate * xi.dot(error)
self.weight += self.rate * error
cost = 0.5 * error**2
return cost
def net_input(self, X):
"""Calculate net input"""
return np.dot(X, self.weight) + self.weight
def activation(self, X):
"""Compute linear activation"""
return self.net_input(X)
def predict(self, X):
"""Return class label after unit step"""
return np.where(self.activation(X) >= 0.0, 1, -1)
def plot_decision_regions(X, y, classifier, resolution=0.02):
# setup marker generator and color map
markers = ('s', 'x', 'o', '^', 'v')
colors = ('red', 'blue', 'lightgreen', 'gray', 'cyan')
cmap = ListedColormap(colors)
# plot the decision surface
x1_min, x1_max = X.min() - 1, X.max() + 1
x2_min, x2_max = X.min() - 1, X.max() + 1
xx1, xx2 = np.meshgrid(np.arange(x1_min, x1_max, resolution),
np.arange(x2_min, x2_max, resolution))
Z = classifier.predict(np.array().T)
Z = Z.reshape(xx1.shape)
plt.contourf(xx1, xx2, Z, alpha=0.4, cmap=cmap)
plt.xlim(xx1.min(), xx1.max())
plt.ylim(xx2.min(), xx2.max())
# plot class samples
for idx, cl in enumerate(np.unique(y)):
plt.scatter(x=X, y=X,
alpha=0.8, c=cmap(idx),
marker=markers, label=cl)
df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None)
y = df.iloc.values
y = np.where(y == 'Iris-setosa', -1, 1)
X = df.iloc[0:100, ].values
# standardize
X_std = np.copy(X)
X_std = (X - X.mean()) / X.std()
X_std = (X - X.mean()) / X.std()
sgd1 = SGD(niter=100, rate=0.01, random_state=1)
sgd2 = SGD(niter=50, rate=0.01, random_state=1)
sgd3 = SGD(niter=10, rate=0.01, random_state=1)
sgd1.fit(X_std, y)
sgd2.fit(X_std, y)
sgd3.fit(X_std, y)
plt.plot(range(1, len(sgd1.cost) + 1), sgd1.cost,
marker='o', linestyle='oo', label='batch=1')
plt.plot(range(1, len(sgd2.cost_) + 1), np.array(sgd2.cost_) / len(y_train),
marker='o', linestyle='--', label='batch=2')
plt.plot(range(1, len(sgd3.cost_) + 1), np.array(sgd3.cost_) / len(y_train),
marker='o', linestyle='xx', label='batch=3')
plt.xlabel('Epochs')
plt.ylabel('Average Cost')
plt.show()
```

I am trying to build a binary classification model using the model we made in course 2 last assignment using tensorflow. The problem I am facing is that it always shows an accuracy of 50% on my training and test set and gives only one class output on unseen images. I have tried loads of combos of batch size and learning rate.

Can anyone provide some insights here?

Thank You!

]]>

I started coding a library in C to run experiments on neural network size, activation function, batch size etc. I am currently facing two problems.

**1.** My NN trains itself on the mnist data set and only learns the training data after about 200 epochs which is as i understand a lot. I also only get an error rate of 20% minimum and anything lower takes extremely long. This does not make sense as tutorials i have seen with tensorflow the error rate drops really low (<10%) just after 5 epochs. My NN uses stochastic gradient decent, i.e. the weight values are updated every sample. I thought this could maybe be a problem so i tried changing my code to implement mini batch gradient decent which lead me to my second problem.

**2. **I don't understand how to update the weights with mini batch gradient decent. When i feed a training sample into the network i can calculate the cost of that specific sample and the overall error that must be propagated back. I save this error into a variable that accumulates over my minibatch loop. Once done i have a variable that contains the accumulated error of all the samples in my mini batch. Now i must update my weights and this is where i get stuck. To update the weights the error must be multiplied with the hidden unit values which are dependent on the input sample you feed the network. Which values for the hidden units do i use? I hope my question makes sense

]]>