ArchTA
Research Assistant, Team Lead, Generative AI Engineer & Researcher
Jun 2025 – Jun 2026
A research project on domain-specialized multimodal AI for architectural critique, formative feedback, and design learning. It explores how evaluative reasoning in architecture can be partially structured and studied through multimodal models and interface-based user evaluation.Multimodal AI, HCI, design education, React, AWS
How can multimodal AI be adapted for architecture-specific critique rather than generic visual description?
How can tacit evaluative knowledge in design be made more interpretable without reducing it to rigid rules?
What differences do users perceive between alternative forms of AI-generated critique in learning and review contexts?
How can interface design support the evaluation of AI feedback quality in realistic use settings?
Reframed architectural AI from generation-oriented assistance toward critique-oriented evaluation
Developed a case-based multimodal pipeline for architecture-specific training and analysis
Explored interpretable structuring of tacit evaluative dimensions in AI feedback
Designed a web interface for side-by-side comparison of critique outputs and collection of user judgments
Combined model experimentation with user-centered evaluation and qualitative analysis
This project investigates how domain-specialized multimodal AI can support architectural critique in a more grounded and interpretable way. It approaches critique as an evaluative task that links project representations with design reasoning and examines how AI systems may assist formative feedback in architecture.Using curated public architectural references, I developed a multimodal workflow for case organization, model adaptation, and critique-oriented evaluation. A central research interest was how architectural judgment, including spatial, contextual, and technical concerns, might be partially externalized in forms that remain useful for both analysis and interaction design.We also developed a web-based comparison interface through which users could review alternative AI-generated critiques and provide ratings and short rationales. This made it possible to study not only model behavior, but also how different forms of feedback were perceived in terms of clarity, grounding, usefulness, and professional quality. Overall, the project sits at the intersection of HCI, multimodal AI, and design education, with a focus on evaluative reasoning in architectural learning and review.
Curios
Research Assistant, Full-stack developer
Sep 2024 – May 2025
HKU’s metaverse concept-related immersive learning platform.React, Firebase
Implemented instrumentation hooks to capture user interaction data and stored it securely in the cloud.
Gained hands-on R3F (React Three Fiber) development experience, combined with Firebase and database-backed web development.
Curios offers 5 selectable classroom scenes where students can customize their appearance and control a 3D avatar to greet others in a virtual environment. They can also make calls and send private or group messages.Students collaborate on group assignment whiteboards, editing content and adding 3D decorations. Professors, visitors, and students can like and comment on assignments.The project includes flexible controls for smooth facilitation—for example, quick spatial teleportation and convenient instructor tools to modify group setups: swapping group assignment panels, adding members, and adjusting management permissions.A rich set of hooks captures in‑scene interaction data (such as editing durations and dwell time). This data is stored in a cloud database and powers features like in‑platform leaderboards, as well as research use.To ensure a seamless user experience, the codebase is substantial: at least 100 scripts with 500+ lines each, spanning frontend and backend deployment, user authentication and authorization, researcher-facing control pages, end‑to‑end system logic from R3F (React Three Fiber) scene rendering to UI interactions, and cloud functions that interface with the database and use caching to accelerate loading.I was the sole maintainer and developer on the project. This was my first transition from Unity to web development, and I resolved 150+ issues across the stack. Additional contributions include upgrading rendering across all five scenes to a more efficient system, optimizing the existing UI, and implementing group broadcasting and communication features. In a previously sparsely documented codebase, I added hundreds of comments, compiled an index of code entry points for major features, and wrote a concise handover guide to help the next developer continue the work.
Digital Reality Painter
Nvidia Joint Lab STEM Intern, Technical Lead and concept designer
Oct 2024 – Oct 2024
VR painting tool integrating AIGC methods for 3D, 2D, and projection-based art creation.Unity (Oculus), ComfyUI
What is the impact of AI-augmented interaction on interaction fluency and perceived experience in creative design?
How does collaborating with AI reshape reasoning patterns, attention allocation, and exploration behaviors?
How do participants negotiate agency and trust with AI suggestions and scaffolding over time?
IEEE TVCG special issue (third author)
Hong Kong International Licensing Show, 2025
PolyVerse - Culture and Art Tech Exhibition 2025
Rapidly developed functional demos and integrated them into the project framework.
Gained development experience on next-gen VR headsets, including hand-gesture interaction and in-headset UI control.
Drew insights from prior VR painting literature to identify research gaps.
Integrated a ComfyUI-Unity generative pipeline into the VR project.
The project integrates AI across three layers of VR painting: 3D spatial strokes generate 3D models, 2D surface strokes generate 2D decals, and lighting strokes drive an AI-powered projector that casts dynamic lighting. All generative features are driven by ComfyUI workflows.The system includes a suite of editing tools, such as grouping selected objects for unified scaling and translation, duplication and deletion, and a magnet-based deformation tool (similar to a “liquify” effect in digital painting).The project supports both hand-tracking gestures and controllers.Development took just one week. A senior from PolyU validated the core logic for 3D strokes and magnet deformation, while I completed the remaining development, including system refinement and all AI workflow implementation.
Beauty of Canton Porcelain
Nvidia Joint Lab STEM Intern, Generative AI Engineer & Unity Developer
Jul 2024 – Jul 2024
The project utilizes Lora to learn intangible cultural heritage patterns and integrates AI into collaborative art creation.Unity, ComfyUI, GPT-API
How can content generated by AI maintain the traditional characteristics and authenticity of Intangible Cultural Heritage (meet the expectations of users)?
How to balance the automation of GenAI with the creative engagement of novices?
Journal of Human Computer Interaction (T&F) (second author)
Featured on television channels such as Channels News Asia
PolyVerse - Culture and Art Tech Exhibition 2025
Optimized training data and trained LoRA models.
Took over and refactored research project codebases.
Designed and validated a ComfyUI-Unity generation pipeline.
The project began with a senior from PolyU gathering image samples of Guangcai porcelain and validating the basic interaction interface.Based on these samples, I re-cropped and optimized the dataset and trained a LoRA model suitable for creative production.I then refactored the interaction system from the ground up to improve performance and deliver smoother mouse-and-keyboard control, and I developed GPT–ComfyUI workflow integrations. I also added editing features for inserting decals with scaling and rotation.
GeneyMAP
Part-time Research Assistant, Sole developer
Feb 2024 – Jul 2024
Tool for designing GenAI-empowered user journey mapping.Unity, GPT-API
How can GenAI be systematically integrated into journey mapping without disrupting established designer workflows, given that current tools are manual and struggle with large, complex data?
The authors co-designed GeneyMAP with eight GenAI functions and a four-step human-in-the-loop process that automates analysis, mapping, synthesis, and ideation. Studies showed improved efficiency and output quality, but raised concerns about transparency, trust, and control.
CHI 2025 Late-Breaking Work (co-first author)
Built human-AI co-design prototypes using the GPT API.
Implemented rich Unity UI interactions based on Figma design specs.
Followed the full HCI workflow from interviews to development to user studies.
As one of the earliest projects at the PolyU Design School to integrate the GPT-API, I first explored the capability boundaries of GPT-4 for creating User Journey Maps. I proposed a table-based editing workflow and an AI submission/retrieval mechanism, and validated a system where stable JSON structures enable both granular and global table adjustments via AI.Over the next two iterations, working from a colleague’s Figma designs at PolyU, I implemented complex, precise UI interactions in Unity (though a web stack would have been more suitable). Features included AI-suggested or custom table dimensions, collapsible/expandable sections, adding and removing rows and columns with reflow, rapid AI corrections for individual cells, and AI-generated reference imagery for inspiration.
GPT-World
Master’s thesis project, Supervisor: Prof. Henry Duh
Jan 2024 – May 2024
An AI-driven DnD game featuring text-based role-playing, with JSON-based object generation, multi-agent AI integration, quantified game mechanics, and automated data retrieval via LLM.Unity (DIY Framework, FEEL), GPT-API
Does formalizing game assets as quantitative variables significantly reduce logical inconsistencies and uncontrollable behaviors in LLM-based interactive narrative games?
Can a readable–writable JSON asset system reduce inconsistencies in narrative state information ?
Does a modular multi-agent architecture improve performance and playability across different functional modules of the system?
Can the combined approach support long-term, scalable AI-native game operation?
Using asset quantification, JSON read/write archives, and a modular multi-agent setup—versus pure text LLMs—reduces logic errors, state drift, and runaway behavior, and improves maintainability and gameplay depth.
Current evidence is from engineering runs and iterations, with positive qualitative feedback. To prove statistical gains, we still need controlled user studies and offline metrics.
Recognized differences between academic and engineering problem framing and methods.
Gained experience in academic writing.
Explored the limits of GPT-API creativity and stability, enabling flexible AI-integrated development.
Independently iterated through multiple design-and-build cycles for AI games.
This is the project I’m most proud of and passionate about. Players start by describing the world they want to explore and defining their own character. The system then automatically generates class templates needed for the game (for example, Place and Character). Each class comes with well-defined attributes; a Character might have dimensions like Magic, Strength, Wisdom, Values, and Secrets. The system also creates Events and instantiates entities under each class, such as concrete locations and NPCs.Compared to AI Dungeon (aidungeon.com), the gameplay is more structured and engaging. The system proactively introduces challenges, and player choices have quantifiable, systemic effects on entities. For instance, successfully cultivating land increases a location’s Prosperity, rather than just advancing a narrative beat. It’s more than a storytelling simulator—it’s a dynamic, stateful game world.The original goal was to build a fully AI-generated and AI-driven homage to Taiko Risshiden V. Over five iterations, I developed a stable JSON-based content pipeline. The system evolved from directly generating entities to first producing flexible class templates, enabling a modular, extensible architecture. A multi-agent setup now handles different subsystems, progressively increasing flexibility. As the project evolved from GPT-3.5 to GPT-4o, the game became notably more adaptable and fun.
The rhythm of life
Master’s Assignment
Oct 2023 - Nov 2023
An art installation aims to express vibrant vitality through the rhythm of sound and visuals.Visitors can use various methods to rotate or shake the water bottles in their hands, and our device will provide auditory and visual feedback based on their interactions.Ableton & Max, Microcontroller
Cɑs1n0
Bachelor’s thesis project
Oct 2022 – Apr 2023
This project focuses on the four elements of "Interaction," "Metaverse," "Fun," and "AI" and is dedicated to helping players improve their adaptability and observation skills in virtual environments while learning the basics of classical cryptography.Unity (DIY Framework, Inworld AI)
In Metaverse/VR settings where criminals may hide information with scene-based cues and ciphers, how can police adapt to effectively detect and decode encrypted clues and receive targeted training?
Developed a Unity-based VR training/puzzle game system that integrates six classic cryptography challenges, a scene “view highlight” aid, a graffiti-style note-taking Paint System, radial VR keyboards, achievement and autosave systems, AI chat assistance, and AI-generated music, to enhance officers’ observation and decoding skills in virtual environments.
GridSystem
Tech Intern - Unity, Supervisor: Zhang Liming (Tech Director)
Jul 2022 – Sep 2022
This Demo project is my work as an intern in Unity, aiming to verify the feasibility and effect of grid system construction.
By rewriting the plug-in -- Grid Placement System, more complex construction functions are realized, such as withdrawing instructions, archiving, area operations, etc.Unity (Rewrite Plugin)
Mono
Leader & Developer & Artist
Jan 2022 - Jan 2022
In this AR game, white squares in bright light are separated from opposite black shadows. Players need to build blocks in a way that matches the given puzzle projection. Players can switch between black and white environments to observe the existence of blocks or shadows. The game runs through a fairy tale full of philosophical sayings.Unity (Vuforia, DOTween)
Best AR Game, Global Game Jam 2022, Hong Kong
Li QingZhao
Leader & Developer
Oct 2019 - Feb 2020
A VR visual novel game that tells the life of Li Qingzhao, a famous ancient female lyricist. The game includes small games such as answering questions and escaping.Unity (DOTween, Fungus)
Commendation Award, PolyU Virtual Reality Contest