|
@nidhi_s91 | |||||
|
1/8 Our paper “Movement science needs different pose tracking algorithms” is on arxiv. #tweeprint
arxiv.org/abs/1907.10226
Thread👇
|
||||||
|
||||||
|
Nidhi Seethapathi
@nidhi_s91
|
25. srp |
|
2/8 In this paper, we give ideas for how pose estimation algorithms should change to best serve movement science -- by quantifying different variables, better ground truth, tracking in time, and more...
|
||
|
|
||
|
Nidhi Seethapathi
@nidhi_s91
|
25. srp |
|
3/8 Many fields of science and engineering rely on movement data for research. Insights from movement data impact neuroscience, bioengineering, sports science, psychology, physiology, biophysics, robotics and even more fields pic.twitter.com/MlhK9pUJZE
|
||
|
|
||
|
Nidhi Seethapathi
@nidhi_s91
|
25. srp |
|
4/8 Exciting progress in pose estimation in-the-wild promises to take movement science outside the lab: study real-world non-contrived movements, increase number of subjects, realize low-cost science and medicine & do science on existing videos i.e. not run experiments ourselves.
|
||
|
|
||
|
Nidhi Seethapathi
@nidhi_s91
|
25. srp |
|
5/8 But, existing algorithms don’t serve the needs of movement science yet. Main reason for this is: they largely ignore underlying dynamics and treat each video frame as independent from its neighbors. In reality, each frame imposes a strong prior for poses in nearby frames.
|
||
|
|
||
|
Nidhi Seethapathi
@nidhi_s91
|
25. srp |
|
6/8 Secondly, because they ignore the physical range of motion constraints imposed by our body. These oversights (among others) result is weird errors when tracking videos of interest to movement science: pic.twitter.com/2B5HGJzcM5
|
||
|
|
||
|
Nidhi Seethapathi
@nidhi_s91
|
25. srp |
|
7/8 A summary of our main suggestions on how to change pose tracking to best serve movement science is provided in the table below: pic.twitter.com/DXoMOvZlTL
|
||
|
|
||
|
Nidhi Seethapathi
@nidhi_s91
|
25. srp |
|
|
||
|
|
||
|
Babak Taati
@Babak_Taati
|
25. srp |
|
I wouldn't say it's an "oversight". There has been lots of effort towards some of the things that you've suggested, it just had it hasn't worked as well as hoped for, so far (e.g. incorporating temporal dependencies, lifting into 3D pose, physics/dynamics modeling, etc.)
|
||
|
|
||
|
Nidhi Seethapathi
@nidhi_s91
|
25. srp |
|
Hi! Yes, we summarize these efforts in the paper especially those in temporal dependencies. See figure 2, where we review how many papers in leading computer vision journals in the last two years incorporate temporal tracking. We don't use the word "oversight" in the paper :)
|
||
|
|
||
|
KordingLab 👨💻🧠∇🔬📈,🏋️♂️⛷️🏂🛹🕺⛰️☕🦖
@KordingLab
|
26. srp |
|
As us movement scientists are somewhat outsiders let me tag some pose folks: @drfeifei @facebookai @ChrSzegedy @Oxford_VGG @NataliaNeverova @trevor_darrell_ @alexttoshev @KostasPenn @umariqb @EldarIsTyping @subail @Michael_J_Black Pose matters for movement science!
|
||
|
|
||