Robust Deep Learning via Derivative Manipulation and IMAE
in Blogs
For source codes, the usage is conditioned on academic use only and kindness to cite our work: Derivative Manipulation and IMAE.
As a young researcher, your interest and kind citation (star) will definitely mean a lot for me and my collaborators.
For any specific discussion or potential future collaboration, please feel free to contact me.
Selected work partially impacted by our work
How do you think of requesting kind citations?

SIGUA: Forgetting May Make Learning with Noisy Labels More Robust
Reason1: Reducing a learning rate on ‘bad’ examples, is intrinsically equivalent to, reducing the weights (derivative magnitudes) of ‘bad’ data points. “SIGUA works in each minibatch: it implements SGD on good data as usual, and if there are any bad data, it implements stochastic gradientascent (SGA) on bad data with a reduced learning rate.”
In [DM] and [IMAE], we have studied on how to model examplelevel weighting from the perspective of gradient/derivative. Concretely, we have claimed that those ‘bad’ examples are assigned with smaller derivative magnitude at the final layer. Mathematically, a point’s final gradient for backpropagation = its derivative * learning rate. You do not modify the derivative, instead you adjust the learning rate. But fundamentally, the principle is the same. Therefore, our work [DM] and [IMAE] should be discussed.Reason 2: Although [DM] and [IMAE] are unpublished in conferences or journals, they have been released in arXiv for more than 1 year by now. Therefore, it is improper to ignore them. Furthermore, [DM] and [IMAE] are included my PhD thesis and passed the examination. PhD Thesis: Example Weighting for Deep Representation Learning
I am looking forward to your ideas. If I am wrong, please feel free to tell me. Otherwise, I will appreciate it significantly if you agree to discuss our work in your paper. Many thanks.