Interactive demos of the paper
Time Consistent Surface Mapping for Deformable Object Shape Control
Shape control deforms an object into a desired shape with robots. Its reference is hard to define for 3D objects without visual texture or distinctive features. The paper defines it with a surface map between the perceived mesh of the object and the target mesh, computed with functional maps: every point of the object is compared with its image on the target, using all of the object's geometry.
A map computed anew at every frame is not a usable reference: each frame brings a new mesh, with another number of points in another order, and the map can jump between solutions of similar cost. The method keeps the eigenbasis, the tracked points, and the functional map consistent from one frame to the next, copes with targets that are not isometric to the current shape, and runs above 5 Hz. A shape controller then learns the response of the object to each gripper from the tracked points.
These pages compute the method in the browser: on meshes perceived by a simulated depth camera, on objects deformed with ARAP as in the simulations of the paper.
Interactive pages
in the order of the paper
Functional maps
Sec. 2. Laplace–Beltrami eigenfunctions of two perceived meshes, the functional map \C(I) at every step of ZoomOut, the point map it gives, and the image of a point selector.
Time consistency
Secs. 3.1–3.3. A deforming object perceived anew at every frame: ZoomOut from scratch against the time-consistent map, with the tracking map \CM(k), the sign corrections, and the warm start.
Non-isometries
Sec. 3.4. A beam mapped to a shorter or longer, bent beam: the eigenvalues, the choice of J, the slanted functional map, and a comparison with ZoomOut against the true correspondence.
Shape control
Sec. 4. The closed loop on four foam pieces simulated with ARAP: the shape error, the buffers and the estimated interaction matrices of the grippers, the actions, and the error distributions of Fig. 15.
Video
the method, the simulations, and the experiments of the paper
The video presents the shape control framework (Fig. 1): a 3D sensor provides the mesh of the object at every iteration, the time-consistent surface map compares it with the target mesh, and the control law defines a 6 DoF action for each robot.
It shows the simulations with ARAP of Fig. 8 and the five experiments of Sec. 4.3 with two ABB IRB120 robots and an Intel RealSense D415 camera: a Mexican hat, a T-shaped pool noodle, a pillow, a rectangular foam sheet, and a foam cut-out, with the surface maps of the method next to those of ZoomOut computed at every frame. Open on YouTube
Framework
Fig. 1 of the paper
Results in the paper
click a figure to enlarge it
Code
the MATLAB code of the paper, a Python implementation, and the source of these demos
The repository of the paper contains the MATLAB code that reproduces the surface maps of the five experiments (timeconsistentsurfacemapping.m) from the recorded meshes, and compares them with ZoomOut in shape error, time, and mesh size. A Python implementation of the method, with a 3D viewer of the experiments, is in python/.
% MATLAB R2022b (Statistics, Computer Vision, Lidar, Optimization toolboxes) timeconsistentsurfacemapping % the five experiments of Sec. 4.3 python python/demo.py --experiment 5 # Python (see python/README.md)
The demos are static files (HTML, CSS, and JavaScript, with KaTeX and three.js included) in the folder docs of the same repository, next to the web version of the paper (docs/paper). The method, the simulator, and the camera (docs/js/sim) also run in Node.js, where tests/test_sim.mjs checks them.
python run.py # http://127.0.0.1:8765/ python tools/build_web.py # after editing the sources in docs (needs Node.js) node tests/test_sim.mjs # checks of the simulation modules