Mohammed Hamdy

mmhamdy

👁 hugging-science

👁 Image
ydnysh's profile picture 👁 Image
fractalego's profile picture 👁 Image
MN0099's profile picture

https://surfingmanifolds.substack.com/

AI & ML interests

AI4Sci | NLP | Reinforcement Learning

Recent Activity

upvoted a collection about 20 hours ago

📝 Research & Long-Form Blog Posts

upvoted an article about 20 hours ago

Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler

reacted to theirpost with 🚀 1 day ago

What if you could train a model on just 10 images instead of 60,000 and still get close to the same performance? Traditional machine learning requires thousands, even millions, of data points to achieve high accuracy. But what if we could "distill" the entire dataset into just a few synthetic samples? This is what Dataset Distillation offers. Unlike traditional knowledge distillation, we keep the model fixed and distill the knowledge contained in a massive training set into a tiny set of synthetic distilled images. The goal is to train a model on this ultra-small set and achieve performance that almost matches what the same model would get when trained on the massive original dataset. For example, training on only 10 distilled MNIST images (this is equivalent to a single image per class) yields 94% accuracy, compared to 99% when training on the full 60,000 images. Interestingly, these distilled images look significantly different (as you can see in the image below) from natural images because they are optimized for model training rather than for matching the correct data distribution. But that's not all. Most importantly, this same method opens the door to a potent form of data poisoning. Because distilled images are specifically optimized for rapid learning, an attacker can create a tiny set of adversarial distilled images to cause a well-trained model to forget or misclassify a specific category. What I find fascinating about dataset distillation is this: it mimics human-like learning by letting a model grasp a concept from a single example, but it does so using alien synthetic images that mean absolutely nothing to a human eye! What about you? What are your thoughts on it?

View all activity

Organizations

mmhamdy 's models 17

Text-to-Speech • Updated Aug 28, 2023

Automatic Speech Recognition • Updated Aug 26, 2023 • 1

Audio Classification • Updated Aug 25, 2023

Reinforcement Learning • Updated May 26, 2023

Reinforcement Learning • Updated May 22, 2023

Reinforcement Learning • Updated May 19, 2023

Reinforcement Learning • Updated May 18, 2023

Reinforcement Learning • Updated May 17, 2023

Reinforcement Learning • Updated Mar 16, 2023

Reinforcement Learning • Updated Mar 8, 2023 • 22

Reinforcement Learning • Updated Mar 8, 2023 • 2

Reinforcement Learning • Updated Mar 7, 2023

Reinforcement Learning • Updated Mar 3, 2023 • 3

Reinforcement Learning • Updated Mar 2, 2023

Reinforcement Learning • Updated Feb 26, 2023

URL: https://huggingface.co/mmhamdy/models

⇱ mmhamdy (Mohammed Hamdy)

Mohammed Hamdy

AI & ML interests

Recent Activity

Organizations

mmhamdy 's models 17