Building a Holographic Digit Classifier with NumPy and MNIST
In this post, we’ll explore how to build a simple, yet powerful, holographic memory system to classify handwritten digits from the popular MNIST dataset. Using NumPy
, we’ll create a system that represents digits as holographic encodings—dense vector representations—and compares new images using cosine similarity.
This approach draws inspiration from vector symbolic architectures (VSAs) and holographic reduced representations (HRRs) used in associative memory and cognitive modeling.
📦 Step 1: Organizing the Dataset
Assume you have MNIST digit data where each digit (0
to 9
) is stored in its own folder. Each image has been preprocessed and saved as a .npy
file (a NumPy array), possibly as a vector encoding or 2D array.
Folder structure:
🧪 Step 2: Create the Holographic Memory File
We’ll create a single file (digit_holograms.npz
) that stores a prototype vector for each digit. This is done by summing all the vectors in each folder and normalizing the result.
This gives us a single .npz
file with 10 normalized vectors—one per digit.
🔍 Step 3: Classify a New Image
To classify a new handwritten digit, we compare it to each digit vector in the hologram model using cosine similarity.
🧪 Step 4: Run It
🧠 Why This Works
This method leverages distributed representation—each digit is represented not by a single example, but by a summed, normalized memory vector that generalizes over 100s of examples.
Advantages:
-
Fast comparisons (just a dot product!)
-
Compact model file
-
Easy to interpret and extend
📈 Next Steps
-
Integrate with a real MNIST loader like
tensorflow.keras.datasets.mnist
. -
Replace
.npy
vectors with encodings from an autoencoder or other neural net. -
Apply to non-digit domains like gestures, audio, or other symbol-based input.
🔚 Conclusion
By combining simple tools like NumPy with ideas from holographic memory, we can build elegant and effective classifiers for high-dimensional symbolic data. This approach shows how cognitive science and machine learning can come together in practical, scalable ways.
Here is the results of using the MNST testing set:
I find your work very interesting. I have tried to reach out to you, if you are still doing this type of work, I would like to talk to you about a project I am involved with that has some very high level similar topics. A group of engineers, including myself and others, are creating a very advanced digital representation of the C Elegans. This our approach is using a feedback based system that takes outcome type into account and the power of memory writing, recalling, and tuning. The tuning is a very different approach as the system resembles human behavior much more than the C Elegans, with a focus on tuning towards the actions taken, versus trying actions to achieve a goal and assigning reward against the goal, etc…
ReplyDeleteRegardless of whether or not you have the time, or desire, to work with us, I really respect your work, it is fantastic. My opinion is you would have a massive following on this site if you defined what you mean by AGI. I can understand why you have used that term, but it typically is interpreted as an AI system that can perform tasks of many kinds at least as good as humans. Your system, to me, could have the potential with the right components as you have an incredible run time cycle that may represent a consciousness and free will, albeit, perhaps the most primitive.
I would use this to describe yours in regards to AGI…
“Our biologically inspired AI has the appropriate prerequisites to achieve a general AI and even a Super AI if the secondary actions of the system have the capability to perform the actions, or through the execution of external processes where the outcomes are measurable by our AI, that could result, in any combination of usage, directly, indirectly or through emergence capabilities, in the achievement of the abilities defined as a GAI or SAI.”
Lastly, what your system really has now is a biologically inspired autonomous real time mode of operation. Which is incredible. And I can see by your venture into the fruit fly that you know exactly where to go next.
Incredible work! I commend you!
Please reach out if you’d like to work together.
Trevor E. Chandler
Trevor:
DeleteThank you for your acknowledgment of my research. Greatly appreciated!
Yes, I am a co-founder of the OpenWorm project but left in 2014 to work on higher level animals. I am always interested in working with C elegans in-slico. I am developing a robot currently that has a metal detector and I am using the C elegan's nervous system emulation to operate autonomously. Please contact me at tbusbice@connectomicagi.com to give me more details. I look forward to hearing more!