Killing an ant with a sword #1

I am going to play/practice with keras and neural network in this series of posts. We will make a model emulate some commonly used operations in real world.

Here i try to make the network lean a simple addition operation. With a very limited data the model was able to produce a decent result.

Addition is a linear operation which is a cake walk for the model to learn.
we will have positive as well as negative numbers for our training.

import pandas as pd
import numpy as np
import os
import matplotlib.pyplot as plt
import numpy as np
from keras.models import Sequential
from keras.layers import Dense,Input
from keras.optimizers import Adam, RMSprop
from keras import metrics,regularizers
#Initialize random training data
v1 = np.random.randint(low=-100, high=100, size=40)
v2 = np.random.randint(low=-100, high=100, size=40)

Our model will be capable of adding two numbers at a time. So we generate train_x (input) and train_y (output) using generateData function.

And we construct our network as below.

We have
1 input layer
3 hidden layers
1 output layer.

We use Adam’s optimiser for tuning our params in the network and MSE(mean square error) as our loss metric.

def generateData(x,y):
    inp = []
    for a in x:
        for b in y:
            inp.append([a,b,a+b])
    inp_arr =np.array(inp)
    train_x = inp_arr[:,0:2]
    train_y = inp_arr[:,2:3]
    return train_x,train_y
def fitting(X,y):
    # define the keras model
    model = Sequential()
    model.add(Dense(256, input_dim=2, activation='elu'))
    model.add(Dense(128, activation='sigmoid'))
    model.add(Dense(50, activation='sigmoid'))
    model.add(Dense(1))
# compile the keras model
    opt = optimizers.adam(lr=0.001)
    model.compile(loss='mean_squared_error', optimizer=opt)
    # fit the keras model on the dataset (CPU)
    model.fit(X, y, epochs=600, batch_size=10)
    return model
def evaluate(mod,train_x,train_y):
    pred_squares = mod.predict(train_x)
    for i in range(10):
        print('%s => %.2f (expected %.2f)' % (str(train_x[i][0])+"+"+str(train_x[i][1]), pred_squares[i], train_y[i]))

We will fit and evaluate

train_x,train_y = generateData(v1,v2)
mod = fitting(train_x,train_y)
evaluate(mod,train_x,train_y)

Given that we have trained with less number of data but with a high number of epochs. The model is bound to be overfit. But it doesn’t matter as it is simple addition and the model works well for the unseen data as well.

The output is not so bad may be ceil or a floor here and there would give a better result ;-)

See you in another interesting experiment ✋