Finally making fast progress on my embedded neural net training and classification library.

I ran into a road block and could not see what was causing the code to not train.  It turns out that an AI disregarded my instructions and dropped the input size onto the ground. It just made up a number for the first layers input, instead of using the input size in the create neural network function call.  This has held me up since Christmas. It appeared to be getting pulled in and used, but it was not.  

Now, you are going to ask me, there are tons of libraries already written to do this in python that use pytorch, why not use those.  Because pytorch doesn't run on small embedded hardware.  This library is aimed squarely at arduino, pico, and raspberry pi. 

The intention is to run the models on large desktop or server machines to train and test them, then load the finished model onto a thermostat, a kids toy,  a clock, a media player, or other embedded devices to give limited intelligence to those machines. Imagine a kids toy that could drive around a room without crashing into stuff. 

This is the neural_net.h file.

This is an example of training and testing the MNIST data set.

This is what the output from that program looks like:

These are the activation functions I have in the library:

These are the gradient descents I have implemented:

Once I have all this tested I will be putting it up on github. 

And yes, I will make agents that can use these neural nets in my Dynamic AI Agent Workflow program.