Using Gaussian functions to estimate 0.5 + 0.4sin(2 * pi * x)
This project was created for CSE 5526: Introduction to Neural Networks in Spring 2018.
The goal of this project was to create a Radial Basis Function network that used K-Means clustering to find k centers and create Gaussian functions centered at those. From there variances for each Gaussian are calculated based on the number of sampled points in that cluster.
In the network, we train weights and biases for each of the functions. Since the actual Gaussian function doesn't have the weight inside it, differentiating is simple, and we treat the weight update as if it were linear regression.
I implemented this project in Processing, which is built upon Java. Processing includes a graphical user interface (GUI) for me to draw the UI elements on, and allow the user to see the network learning (my favorite part!).
The RED line represents the RBF networks prediction, the BLUE dots represent the 75 randomly sampled data points with a noise within [-0.1, 0.1] (these were seeded), and the YELLOW line represents the true function we're trying to estimate. The faint lines each represent one of the Gaussian functions used in that simulation.
Training was done over 100 epochs with varying learning rates and number of Gaussian functions. Above, only four configurations were used, but we see a pattern emerging. Overfitting start happening when we use a higher number of Gaussians. I found that between 3 and 6 Gaussians all work well for this function, using a learning rate of 0.02. In the future, I'll calculate errors using area approximation in order to decide the "optimal" RBF network.
Code available here