Initial commit with activation function and tests

.gitignore

@@ -3,7 +3,3 @@ build/
 
 # Google Tests
 tests/lib/
-
-# Jet Brains
-.idea/
-cmake-build-debug/

CMakeLists.txt

@@ -1,5 +1,5 @@
 cmake_minimum_required(VERSION 3.9)
-project(MyProject)
+project(NeuralNet)
 
 set(CMAKE_CXX_STANDARD 17)
 
@@ -35,4 +35,4 @@ if(DOXYGEN_FOUND)
             VERBATIM)
 else(DOXYGEN_FOUND)
     message("Doxygen needs to be installed to generate the documentation.")
-endif(DOXYGEN_FOUND)
+endif(DOXYGEN_FOUND)

README.md

@@ -1,80 +1,59 @@
-# C++ Project Template
-When setting out on a new project in C++ there are a few configuration steps
-which need to be completed prior to actually getting down to writing code.
-This repository is going to be a C++ project template that already has the
-following components:
+# Neural Net
 
-- Directory Structure
-- Make Build (CMake)
-- Unit Test Framework (Google Test)
-- API Documentation (Doxygen)
+## Overview
 
-Feel free to fork this repository and tailor it to suit you.
+`neural_net` is a personal learning project focused on implementing a basic neural network from scratch using C++. The goal of this project is to gain a deep understanding of neural network architecture, training algorithms, and fundamental machine learning concepts.
 
-## Procedure
-1. Download Bash script to create new C++ projects 
-    ```bash
-    curl -O https://raw.githubusercontent.com/TimothyHelton/cpp_project_template/master/new_cpp_project.sh
-    chmod u+x new_cpp_project.sh
-    ```
-1. Create new C++ project
-    ```bash
-    ./new_cpp_project.sh NewProjectName
-    ```
-1. In the project top level **CMakeLists.txt**:
-    1. Line 2: Change the variable **MyProject** to the name of your project.
-        ```cmake
-        project(NewProject)
-        ```
-        - This variable will be used in a couple of different places.
-            - MyProject_run: will be the main executable name
-            - MyProject_lib: will be the project library name
-    1. Line 4: Set the version of C++ to use.  For example, let's set up the
-    NewProject to use C++ 11.
-        ```cmake
-        set(CMAKE_CXX_STANDARD 11)
-        ```
-1. Update project name and description in the `Doxyfile` located in the `docs`
-directory.
-    1. Update line `PROJECT_NAME`
-        1. This name will appear on each documentation page.
-    1. Update line `PROJECT_NUMBER`
-        1. This is the version number of your project.
-    1. Update line `PROJECT_BRIEF`
-        1. Any text entered here will also appear on each documentation page.
-        Try not to make this one too long.
-1. Reload the top CMake file.
+## Features
 
-## CLION IDE Specific Instructions
-I started using an IDE from [JET Brains](https://www.jetbrains.com/) tailored
-for Python called [PyCharm](https://www.jetbrains.com/pycharm/) and thought
-it helped me write better code.
-I'd been wanting to learn C++ and decided to give JET Brains C/C++ IDE called
-[CLion](https://www.jetbrains.com/clion/) a try.
-The code completion, interactive suggestions, debugger, introspection tools,
-and built-in test execution are very handy.
-There are a couple extra details to set when using this IDE.
+- Custom neural network implementation in C++
+- Support for basic feed-forward neural network architecture
+- Configurable number of layers and neurons
+- Implementation of key activation functions
+- Simple data loading and preprocessing utilities
+- Basic performance metrics and evaluation
 
-1. The IDE allows you to mark directories with their desired purpose.
-To mark a directory right click on the directory name in the `Project` window
-and select `Mark Directory as` from the drop-down menu.
-    1. Mark the `src` directory as `Project Sources and Headers`
-    1. Mark the `tests/lib/googletest` directory as  `Library Files`
-1. Setup the `Run/Debug Configuration` by selecting `Edit Configurations...`
-from the pull-down menu from the run button (green triangle) in the upper right
-corner.
-    1. Update Doxygen Build to execute the unit test suite.
-        1. Select Doxygen from the Application menu on the left.
-        1. Choose the **executable** for Doxygen to be `Unit_Tests_run`.
-    1. Create a `Google Test` configuration
-        1. In the upper left corner select the plus symbol.
-        1. Chose `Google Test` from the drop-down menu.
-        1. Set **Name** to `Unit Tests`.
-        1. Set **Target** to `Unit_Tests_run`.
+## Prerequisites
 
-## Wrap Up
-That should be all it takes to start writing code.
-If you find any issues or bugs with this repository please file an issue on
-[GitHub](https://github.com/TimothyHelton/cpp_project_template/issues).
+- C++17 or later
+- CMake (version 3.10 or higher)
+- A modern C++ compiler (GCC, Clang, or MSVC)
 
-Hope you find this template useful and enjoy learning C++!
+## Installation
+
+1. Clone the repository:
+   ```bash
+   git clone https://github.com/aselimov/neural_net.git
+   cd neural_net
+   ```
+
+2. Create a build directory and compile:
+   ```bash
+   mkdir build
+   cd build
+   cmake ..
+   make
+   ```
+
+## Learning Objectives
+
+- Understand neural network architecture
+- Implement core machine learning algorithms
+- Practice advanced C++ programming techniques
+- Explore computational efficiency in ML implementations
+
+## Roadmap
+
+- [x] Activation functions
+- [ ] Basic neural network structure
+- [ ] Backpropagation algorithm
+- [ ] Regularization techniques
+- [ ] Performance optimizations
+
+## License
+
+This project is [MIT](LICENSE) licensed.
+
+## Contact
+
+Alex Selimov  - [alex@alexselimov.com](mailto:alex@alexselimov.com)

src/CMakeLists.txt

@@ -1,6 +1,8 @@
 project(${CMAKE_PROJECT_NAME}_lib)
 
 set(HEADER_FILES
+    activation_function.hpp
+    neural_net.hpp
 )
 set(SOURCE_FILES
 )
@@ -14,4 +16,4 @@ if (EXISTS ${SOURCE_FILES})
 else()
     # The library only contains header files.
     add_library(${CMAKE_PROJECT_NAME}_lib INTERFACE)
-endif()
+endif()

src/activation_function.hpp

@@ -0,0 +1,19 @@
+#ifndef ACTIVATION_FUNCTION_H
+#define ACTIVATION_FUNCTION_H
+
+#include <algorithm>
+#include <cmath>
+/**
+ * Functor to set the activation function as a Sigmoid function
+ */
+struct Sigmoid {
+  double operator()(double z) { return 1 / (1 + exp(-z)); };
+};
+
+/**
+ * Functor to set the activation function as Rectified Linear Unit
+ */
+struct ReLU {
+  double operator()(double z) { return std::max(z, 0.0); };
+};
+#endif

src/neural_net.hpp

@@ -0,0 +1,5 @@
+#ifndef NEURAL_NET_H
+#define NEURAL_NET_H
+
+template <class ActivationFunction> class NeuralNet : ActivationFunction {};
+#endif

tests/unit_tests/CMakeLists.txt

@@ -1,8 +1,8 @@
 include_directories(${gtest_SOURCE_DIR}/include ${gtest_SOURCE_DIR})
 
 add_executable(Unit_Tests_run
-    test_example.cpp
+    test_activation_functions.cpp
 )
 
 target_link_libraries(Unit_Tests_run gtest gtest_main)
-target_link_libraries(Unit_Tests_run ${CMAKE_PROJECT_NAME}_lib)
+target_link_libraries(Unit_Tests_run ${CMAKE_PROJECT_NAME}_lib)

tests/unit_tests/test_activation_functions.cpp

@@ -0,0 +1,41 @@
+#include <gtest/gtest.h>
+#include "../../src/activation_function.hpp"
+#include <cmath>
+
+TEST(ActivationFunctionTest, SigmoidTest) {
+    Sigmoid sigmoid;
+    
+    // Test sigmoid at x = 0 (should be 0.5)
+    EXPECT_NEAR(sigmoid(0.0), 0.5, 1e-6);
+    
+    // Test sigmoid at large positive value (should approach 1)
+    EXPECT_NEAR(sigmoid(10.0), 1.0, 1e-4);
+    
+    // Test sigmoid at large negative value (should approach 0)
+    EXPECT_NEAR(sigmoid(-10.0), 0.0, 1e-4);
+    
+    // Test sigmoid at x = 1
+    EXPECT_NEAR(sigmoid(1.0), 1.0 / (1.0 + exp(-1.0)), 1e-6);
+    
+    // Test sigmoid at x = -1
+    EXPECT_NEAR(sigmoid(-1.0), 1.0 / (1.0 + exp(1.0)), 1e-6);
+}
+
+TEST(ActivationFunctionTest, ReLUTest) {
+    ReLU relu;
+    
+    // Test ReLU at x = 0 (should be 0)
+    EXPECT_DOUBLE_EQ(relu(0.0), 0.0);
+    
+    // Test ReLU at positive value (should be same value)
+    EXPECT_DOUBLE_EQ(relu(5.0), 5.0);
+    
+    // Test ReLU at negative value (should be 0)
+    EXPECT_DOUBLE_EQ(relu(-5.0), 0.0);
+    
+    // Test ReLU at very small positive value
+    EXPECT_DOUBLE_EQ(relu(0.0001), 0.0001);
+    
+    // Test ReLU at very small negative value
+    EXPECT_DOUBLE_EQ(relu(-0.0001), 0.0);
+}