|
Colombian To World
@colombia2World
|
30. sij |
|
Agree ... Any suggestions / guidelines of how we could make it more scientifically rigorous and produce more trustworthy results ?
|
||
|
|
||
|
Florian Laurent
@MasterScrat
|
30. sij |
|
"How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments" arxiv.org/abs/1806.08295
|
||
|
|
||
|
Antoine Cully
@CULLYAntoine
|
31. sij |
|
Totally agree! On the bright side, it is already 3 times more than what I used to see in this community a few years ago, and before that we often had the seed as part of the hyper parameters....
|
||
|
|
||
|
Will Dabney
@wwdabney
|
31. sij |
|
First, a quick mea culpa, I’m definitely guilty of this too. Solutions? As problematic as p-hacking is, it would already be an improvement if we were all reporting a reasonable statistical significance vs just eye-balling the learning curves (or worse).
|
||
|
|
||
|
Sandu Ursu
@SanduUrsu
|
1. velj |
|
Interestingly, this same practice of 18th century observational astronomers: averaging the 3 best measurements of a particular phenomenon motivated Laplace to develop the Central Limit Theorem (1810) & lay the foundation for what is now called "Bayesian statistics".
|
||
|
|
||
|
Artem Molchanov
@amolchanov86
|
31. sij |
|
Well, give everyone access to clusters for free and the problem will disappear. It is never the issue of laziness - simply lack of money and time constraints.
|
||
|
|
||