Follow-up post with some consequences twitter.com/AndreasZeller/…

Science is fundamentally built on trust, between society and academia, between PhD advisor and PhD student, and between reviewer and authors. Hence, it is important to hold everyone accountable. This is how we can start with ourselves: mboehme.github.io/manifesto.html #openscience

Great work! External validation is highly important in research. I also think we should praise the Angora authors for making their tool open-source and enable such scrutiny. Many don't publish their code and this significantly hinders overall development.

I got similar findings after investigating source code of a well known research tech. Feel bad about SE research when I found that many papers do those ''tricks'': telling a nice story in paper, doing something different in code to make results better.

I have a picture for this LoL. pic.twitter.com/tmvpFeWEdv

This is really great and important work, hats off to you and your student for doing this!

Did the gradient descent prove to be effective against checksums or other constraints that are a function over input rather than using values directly from input as it states in their email?

Very interesting! I believe this happens in a lot of cases. I have also dropped lots of information from publications because I perceived it to be "technical and non-interesting" for scientific audience. I also agree that replication packages must be obligatory to publish.

Thanks for writing this

Magic bytes/LAVA-M is quite biased so it would be more interesting to see how GD performs on more realistic constraints, vs random search, binary search, and SMT solver