Time series modeling/forecasting plays an important role that contributes to the success in business. Time series modeling/ forecasting is the use of a model to predict future values based on previously observed values. Therefore, the time series modeling applications will vary from market analysis, stock prediction, sale planning to non-linear dynamic system modeling in robotics and manufacturing applications.
There existing many different time series modeling methods, and machine learning equipped with desired function approximation characteristic would be the great candidate to model any nonlinear time series data.
In this article, we will explore how a neural network can learn a generalized model of a function, and we choose a simple sine wave with a small dataset that contains historical sine wave value at a certain time.
For time series modeling application, the small dataset will cause the generalization problem. This problem is caused by overfitting issue that occurred during neural network training. The error on the training set is driven to a very small value, but when new data is presented to the network the error is large. The network has memorized the training examples, but it has not learned to generalize to new situations.
To demonstrate that Ying Xie has used both Multilayer Perceptron Layer (MLP) neural network, Convolution Neural Network and Deep Learning LSTM Recurrent Neural Network to model a simple time series sine wave. However, the results are not very promising due to overfitting issue.
To improve the neural network generalization and avoid overfitting issue, MathWorks has recommended using the automated regularization technique.
This is a complicated technique as it involves in probability theory to train a neural network, but let see how ANNHUB can handle it with ease and can export the trained time series model directly into an Arduino device.
Note: The final trained Bayesian Neural Network time series model that can be deployed in the Arduino device can be downloaded here.
2. Data preparation
In this article, we are going to design a neural network time series model to approximate the sine wave function described as follows:
y=sin(2*pi*t), where t is a time in the range -1 to 1.
For each value of time t, we can generate the according to output value for y, that leads to a training data set in a 2D array.
The dataset is small with only 400 data samples that present the historical data of the sine wave.
3. Design Bayesian Neural Network in ANNHUB
In this article, we are going to design a Bayesian Neural Network (BNN). A BNN refers to extending standard MLP neural networks with posterior inference to achieve auto regularization that will improve the neural network generalization and overfitting for small dataset application. With this special technique, the validation set is not required during the training process, so we can save more data for training.
Designing BBN in ANNHUB is simple, the following steps are required.
Step 1: Load time series data into ANNHUB
When the time series data in the correct CSV format, it can be easily imported into the ANNHUB.
Step 2: Design BNN time series model in ANNHUB
To configure the Bayesian Neural Network, we only need to choose Bayesian Regularization as the training engine (1), and ANNHUB will automatically treat this network as BNN.
ANNHUB will auto-configure this neural network structure, we only need to choose a number of hidden nodes (if we want) in (5).
Step 3: Train BNN time series model in ANNHUB
ANNHUB will suggest the stopping criteria and training parameters for us, we only need to change changing epochs and number of Max Fails if the training algorithm cannot find optimal performance index.
Being configured as a BNN, auto regularization technique is automatically applied in the training algorithm, and the validation set for early stopping technique is not required. ANNHUB will run through iterations and determine the best solution for us. Easy as a piece of cake!
Step 4: Evaluate the trained BNN time series model in ANNHUB
Although the advanced auto regularization technique has been applied in the training process, we still need to evaluate the trained neural network to see if it is able to correctly model a random time series data.
ANNHUB provides a built-in evaluation tool, which is regression curves to allow us to evaluate the trained model in both training data set an test data set that gives us the confidence to use the trained model in actual production.
Step 5: Test the trained BNN time series model in ANNHUB with the unseen test dataset.
We still need to verify if the trained BNN time series model can actually model the sine time-series waveform. ANNHUB provides test interface that allows us to load test data to verify our trained model before we decide to export it for production/deployment.
As can be seen in the test interface bellow, the trained BNN time series model can correctly approximate the sine wave with the very desired accuracy, much better than other methods mentioned by Ying Xie.
Step 6: Export the trained BNN time series model for deployment.
After being evaluated, tested and verified, the trained BNN time series is ready to be exported into supported programming languages for deployment.
In this article, we export the trained model into Arduino IDE directly. The exported file format is in Arduino format.
4. Deploy the trained neural network into the Arduino device
The trained BNN time series model exported from ANNHUB can be loaded directly into the Arduino IDE environment as an Arduino program. No external libraries or dependencies are required. We can modify this Arduino program to meet our requirements before we can compile and upload this time series modeling program into an Arduino device.
After being loaded by the time series modeling program, the Arduino device now can predict/forecast the sine waveform correctly. We can confirm it by using Serial plotter to observe predicted waveform value generated by the trained BNN time series model (red) and actual time series sine waveform (blue).
In this article, we have demonstrated how machine learning can be applied to model a time series of data based on previously observed values. We also can see that how Bayesian Neural Network would be the most suitable neural network to avoid the over-fitting issue in small data set problem and achieve very promising prediction results. Most importantly, we can easily design this Bayesian Neural Network in ANNHUB and deploy it into any environments, including small Arduino devices.
Let try it yourself as ANNHUB is FREE TO USE. You can get the latest ANNHB version in our download page (https://www.anscenter.com/Customer/Products/Download/annhub)