This project is sponsored by the Open Source Foundation in Taiwan. We aimed to build localized content and AI models for general use by gamifying AI Reinforcement Learning (RLHF) through the Taiwanese ritual of Zhijiao (Moon Blocks).
- My positionProduct Designer
- In Charge ofDesigned the Information Architecture (IA) and UI/UX using Figma, integrating cultural rituals into the RLHF data collection workflow
Problem Statement
Global LLMs lack Taiwanese cultural nuance, and traditional methods for gathering human feedback (data labeling) are too tedious to sustain user engagement. While models like DeepSeek from China are powerful, they often lack the cultural nuance, political context, and Traditional Chinese vocabulary required for Taiwan's industries.
We needed to fine-tune these models for local usage. To fine-tune effectively, we needed human feedback (RLHF: Reinforcement Learning from Human Feedback), but asking volunteers to rate hundreds of text responses is boring and tedious.
Objective
To democratize local model training by building a gamified "Arena" that transforms complex data evaluation into an engaging, culturally resonant social activity.
Discovery & Research
To identify the drivers that would motivate users to voluntarily train our AI, we conducted a quantitative interest survey with approximately 600 participants. The results were decisive: users showed negligible interest in generic technical tasks, instead overwhelmingly favoring hyper-local topics such as food recommendations, neighborhood issues, and relationship advice, specifically appealing to 'Love Gods.'
This data revealed a critical strategic insight: users were not seeking a 'better ChatGPT,' but rather a 'Digital Oracle.' The standard, sterile chat interface was fundamentally ill-suited for the intimate, culturally specific conversations users actually cared about, necessitating a pivot toward a more ritualistic design.
The Design Strategy
Our goal was to create an "Arena" website interface where users blind-test two models (Model A vs. Model B) and vote on the better response. We designed a streamlined flow that allows rapid comparison of complex text outputs without cognitive overload.
The Zhijiao System – Digital Folklore
Instead of the generic Western binary of "Thumbs Up/Down," we introduced a vernacular design metaphor based on Taiwanese folk religion: Zhijiao (Moon Blocks).
Sheng Bei (聖茶 - Divine Answer) is used for "Good/Accepted" — it implies the model gave a blessing or a correct, harmonious answer. Xiao Bei (笑茶 - Laughing Answer) is used for "Bad/Hallucination" — it implies the model is "joking" or speaking nonsense.
This design choice reduced the friction of the task. It turned the sterile act of "data labeling" into a familiar, ritualistic action. It respects the user's cultural context while still enabling us to label the meaning of the comments from the users. This idea was approved by the early test users in our testing study, especially from the elderly users.
Information Architecture
The Trinity of Interaction: to sustain high-quality human feedback without losing users' interest, the architecture is divided into three pillars — Chat Mode for blind evaluation using ritual metaphors, Social Mode for community ranking of cultural trends, and Mission Mode for targeted stress-testing.
Impact & Outcome
The Arena interface replaces the sterile "A/B Testing" usually found in engineering with a culturally resonant ritual. We mapped the complex task of RLHF to Zhijiao (Moon Blocks). The Sheng Bei meaning Divine Approval equals "Good Response." We lowered the cognitive load for elderly and local users, turning the tedious work of data labeling into a familiar, intuitive act of "seeking truth."
The Social Mode acts as a community amplifier. By integrating Crowdsourced Rankings and seamless Social Media Sharing, we transform private model interactions into public discourse. This allows the community to collectively define what constitutes a "good" answer for Taiwanese society, turning individual feedback into a consensus on local values.
What I Learned
This project challenged me as a designer to take the standard engineering approach with AI training into consideration. While the technical goal was simply to gather human feedback, the design challenge was sustaining human engagement. I learned that gamification is most effective when it taps into intrinsic cultural behaviors rather than generic point systems. By mapping 'Sheng Bei' (Divine Approval) to 'Model Approval,' we prompted Taiwanese users' interest in engaging with the arena and turned community feedback into a collective social activity. It demonstrated how cultural nuance can serve as critical infrastructure for high-tech systems.