I am presently a fifth-year Ph.D. student in the Department of Computer Science at Shanghai Jiao Tong University. As an active member of the
esteemed SJTU MVIG lab, I work closely under the expert guidance
of Prof. Cewu
Lu.
Before that, I acquired my Bachelor's degree (Computer Science) in 2019
from Huazhong University of Science and
Technology. My research pursuits predominantly revolve around Computer
Vision, 3D Vision, and Embodied
AI.
Rearrangement operations form the crux of interactions between humans and their environment. The
ability to generate natural, fluid sequences of this operation is of essential value in AR/VR and
CG. Bridging a gap in the field, our study introduces Favor: a novel dataset for Full-body
AR-driven Virtual Object Rearrangement that uniquely employs motion capture systems and AR
eyeglasses. We also present a pipeline Favorite for
producing digital human rearrangement motion sequences guided by instructions. Experimental results,
both qualitative and quantitative, suggest that this dataset and pipeline deliver high-quality
motion sequences.
We proposed a new method Chord which exploits the
categorical shape prior for reconstructing the
shape of intra-class objects. In addition, we constructed a new dataset, COMIC, of category-level
hand-object interaction. Comic encompasses a diverse
collection of object instances, materials, hand
interactions, and viewing directions, as illustrated.
Learning how humans manipulate objects requires machines to acquire knowledge from two perspectives:
one for understanding object affordances and the other for learning human interactions based on
affordances. In this work, we propose a multi-modal and rich-annotated knowledge repository,
OakInk,
for the visual and cognitive understanding of hand-object interactions. Check our website for more
details!
We propose a lightweight online data enrichment method that boosts articulated hand-object pose
estimation
from the data perspective.
During training, ArtiBoost alternatively performs data exploration and synthesis.
Even with a simple baseline, our method can boost it to outperform the previous SOTA on several
hand-object benchmarks.
In this paper, we extend MANO with more Diverse Accessories and Rich Textures, namely DART.
DART is comprised of 325 exquisite hand-crafted texture maps which vary in appearance and cover
different kinds of blemishes, make-ups, and accessories.
We also generate large-scale (800K), diverse, and high-fidelity hand images, paired with
perfect-aligned 3D labels, called DARTset.
Color-NeuS focuses on mesh reconstruction with color. We remove view-dependent color while using a
relighting network to maintain volume rendering performance. Mesh is extracted from the SDF network,
and vertex color is derived from the global color network. We conceived a in hand object scanning
task and gathered several videos for it to evaluate Color-NeuS.
We highlight contact in the hand-object interaction modeling task by proposing an
explicit representation named Contact Potential Field (CPF). In CPF, we treat each contacting
hand-object
vertex pair as a spring-mass system, Hence the whole system forms a potential field with minimal
elastic
energy
at the grasp position.
Misc
Conference reviewer for CVPR 2024, ICCV
2023, ECCV
2022, CVPR 2022, and 3DV 2022.