Research
🏠Home
Resume
More
Future City --- Towards Real-Time Architecture Design by GenAI and AR

XR / Architecture Design

Github [Github]   Youtube [Youtube]

Future City is a project that aims to create a real-time architecture design by using GenAI and AR. It aims to facilitate the design process by providing a more intuitive and efficient way to create and visualize architecture designs for architects. GenAI is designed to generate the architecture materials,facades, layouts and models, while AR is designed to visualize the architecture design in the real world to increase situational awareness and engagement. By integrating these two technologies, we can minimize the cost of context switch and augment authoring.

Real-time Human X-Ray Vision via Mixed Reality and Mobile Robot

XR / Robotics

Youtube [Youtube]

Real-time Human X-Ray Vision aims to support flexible and efficient human-robot co-exploration in any indoor physical environment. It would capsulate and translate the real world into perceivable digital representations for human to interact with, and further enhance human's engagement and situational awareness, while unlocking numerous opportunities for human-robot and human-space interaction, a step towards more expressive spatial computing era.

Fundamental WebXR Model Interaction Methods

XR / WebXR

Github [Github]   Youtube [Youtube]   Bilibili [Bilibili]

This project demonstrates and showcases 5 fundamental interaction methods with object in WebXR. The 5 methods are hand ray control, free manipulation, handlers assisted control, hand sync control and direct manipulation. To fasten and help first-time WebXR developes, I provide a sample here to showcase the use of WebXR interaction. There is also multi modal design, like including the spatial audio in.

Transcrisense

Accessability

Github [Github]   Youtube [Youtube]   Bilibili [Bilibili]

TranscriSense, designed for individuals with hearing impairments. Our(me, Yicheng Cai and lcy) goal was to enhance the Asian Games viewing experience for those with hearing challenges. This multi-model system, which combines visual effect, speech recognition, haptic feedback, and data visualization, can fully bring a more immersive experience for the hearing-impaired people.

Cut Fruits 3D

Virtual Reality

Github [Github]   Youtube [Youtube]   Bilibili [Bilibili]

This 3D rendition of the fruit-cutting game, crafted with Unity3D, represents a collaborative effort between myself, Qirui Hu, and Xiuyuan Wang. With a focus on delivering an immersive gaming experience, the project supports controller interaction, haptic feedback, and sound effects, aiming to provide players with a deeply engaging gameplay environment that also contributes positively to their physical and mental well-being.

Handbot---Apple WWDC 2023 Swift Student Challenge

Human AI Interaction

Github [Github]   Youtube [Youtube]   Bilibili [Bilibili]

This playground is structured into three components: text, image, and video—primary modes of information dissemination in contemporary society. While personally, I don't object to AI-generated content, my central belief is that individuals should maintain their critical thinking and deliberation before deciding to incorporate AIGC into their work. Otherwise, there's a risk of surrendering control to AI, particularly in media contexts.


Data Visualization

Github [Github]   Youtube [Youtube]   Bilibili [Bilibili]

Three-pointers are a strategic offensive technique in basketball. In the 2014-2015 season, athlete Stephen Curry emerged onto the scene, and the Warriors, under his leadership, experienced a meteoric rise, ushering in a league-wide era of the three-point revolution. Coincidentally, both Ryan and @tyf, as well as myself, are avid NBA basketball fans.

Eyecompass

Wearable Empherical Device

Github [Github]   Youtube [Youtube]   Bilibili [Bilibili]

Eyecompass is a revolutionary smart glasses solution tailor-made for food delivery couriers. Developed by the Group 6 'cancanneed' Team (@zzk, Jiajun Pan and @ly, and myself). Equipped with advanced features such as navigation assistance, real-time information reminders, and haptic feedback through vibrations, Eyecompass stands out as a game-changer in the delivery industry. The hallmark of this intelligent eyewear lies in its ability to liberate the hands of delivery personnel, offering unparalleled convenience and efficiency.