Twitter | Pretraživanje | |
Steven Hansen
Fair point, but it's worth noting that there is a trade-off between # seeds, # baselines, and # and complexity of environment(s). Personally, I'd prefer a method /w 3 seeds eval'd on 57 tasks with 5 baselines to one with 10 seeds eval'd on 1 task with 2 baselines.
Reply Retweet Označi sa "sviđa mi se" More
Roman Werpachowski 30. sij
Odgovor korisniku/ci @Zergylord
Tasks are not a straightforward replacement for seeds because seends can be randomised, and hand-made tasks are not. Unless you're procedurally generating the tasks, in which case you effectively have more seeds.
Reply Retweet Označi sa "sviđa mi se"
Steven Hansen 30. sij
Odgovor korisniku/ci @RWerpachowski
I agree, not a replacement -- but they do compete for computing resources. And generally I've found that it's easier to overfit to a single task than it is to be misled by too few seeds. Both are problematic, but without infinite compute hard decisions must be made.
Reply Retweet Označi sa "sviđa mi se"