This week I got the opportunity to meet the robot Pepper and Alex Yuan Gao of the lab of social robotics in Uppsala [https://usr-lab.github.io/].
There are essential differences between my clay embodiment and the propper pepper. For one the texture is reflective and some part are lit up. This is the main reason why I decided on using fiducials istead of tracking the whole object using 3D scans – tracked surfaces cannot be reflective.
Other key difference is that the whole head moves. This of course can be . As it is by default it moves the head to watch the phone. I am pretty sure that this should not be activated in my study. For one, that behaiviour not achivable with the clay version. Alto this is a result from peppers facial tacking and it had a tendency to dip when the phone was in the way of the face. “If a human is detected, Pepper tries to keep eye contact with him/her”. Pepper did follow the phone, but I suspect that it was trying to find eye contact.
When Pepper dipped with its head – the tracking is lost. I belive that a better tracker would solve the issue – or make it better at least.
Step 2 is to try a new tracker it hope that the tracking is more stable. I also will try to fit a human shaped head in some aesthetically acceptable way.
The full augmented pepper face was interesting. First thought is that this is pointless as the facial element all are present in the example above. There are of course use cases in reseatch where a full face augmentation is interesting (i.e when some new features concering the full face is tested). But for it to work it has to be a more smooth model (easy to fix in blender), and the tracker has to be much more stable. The stuttering and shaking is much more noticable when the full face is overlayed.
With this said, the light is well approximated. This is stimulated using the method previously written about (blurred background photos as skyboxes).
Clay pepper 