Twitter | Pretraživanje | |
Pierre-Yves Oudeyer
Amazing: every week I see a paper comparing algorithms using mean performance over *3* seeds ! Yes, ****3**** !!! Please please community, your great ideas will be served better using standard scientific methods!
Reply Retweet Označi sa "sviđa mi se" More
Colombian To World 30. sij
Odgovor korisniku/ci @pyoudeyer
Agree ... Any suggestions / guidelines of how we could make it more scientifically rigorous and produce more trustworthy results ?
Reply Retweet Označi sa "sviđa mi se"
Florian Laurent 30. sij
Odgovor korisniku/ci @colombia2World @pyoudeyer
"How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments"
Reply Retweet Označi sa "sviđa mi se"
Antoine Cully 31. sij
Odgovor korisniku/ci @pyoudeyer
Totally agree! On the bright side, it is already 3 times more than what I used to see in this community a few years ago, and before that we often had the seed as part of the hyper parameters....
Reply Retweet Označi sa "sviđa mi se"
Will Dabney 31. sij
Odgovor korisniku/ci @pyoudeyer
First, a quick mea culpa, I’m definitely guilty of this too. Solutions? As problematic as p-hacking is, it would already be an improvement if we were all reporting a reasonable statistical significance vs just eye-balling the learning curves (or worse).
Reply Retweet Označi sa "sviđa mi se"
Sandu Ursu 1. velj
Odgovor korisniku/ci @pyoudeyer
Interestingly, this same practice of 18th century observational astronomers: averaging the 3 best measurements of a particular phenomenon motivated Laplace to develop the Central Limit Theorem (1810) & lay the foundation for what is now called "Bayesian statistics".
Reply Retweet Označi sa "sviđa mi se"
Artem Molchanov 31. sij
Odgovor korisniku/ci @pyoudeyer
Well, give everyone access to clusters for free and the problem will disappear. It is never the issue of laziness - simply lack of money and time constraints.
Reply Retweet Označi sa "sviđa mi se"