Brush and Beyond-Reserch Report

"Brush and Beyond" is a virtual reality game that immerses players in an artists' working environments, capturing their struggle with diseases, and the joy of creating fine art. The game aims to enhance curiosity about artists' stories and offers a dynamic way to learn art history beyond traditional methods.

Heuristic Evaluation

Evaluation Objectives

Heuristic Evaluation:

Conduct a thorough game review using Jakob Nielsen's 10 heuristic evaluation principles to ensure comprehensive coverage.

Identify Design Issues:

Document critical pain points and inefficiencies within the game's design to pinpoint areas for improvement.

Enhance User Experience:

Improve overall player satisfaction by addressing common user issues, refining gameplay, and streamlining interactions.

Team:

Xueer Xia (Lead Designer/Developer, Tester, UX Researcher)

Dr. Ching-hua Chuan (Research Author, XR Researcher, Tester)

Hao Zhong (UX Researcher, Content Designer, Tester)

Tools & Equipment:

Key Products Features

Painting

View with Cataracts

Adventure for the Lost Color

Target Audiences

The target audience includes independent gamers, students, and educators with an interest in art history who visit museums and actively participate in interactive experiences.

Evaluation Criteria

Cosmetic problem only: fix if time is available

Minor usability problem: fixing this should be given low priority

Major usability problem: important to fix, given high priority

Usability catastrophe: fix this before product can be released

First Round Test

26 Participants

Casual

Beginner

First experience

XR developer

Features Users Love

  • I really enjoy painting in a VR world — it feels so fresh and exciting, and it’s like magic to draw on a virtual canvas.

  • I enjoy Monet’s voice — even though I know it’s AI-generated, it truly sounds like an old artist speaking.

  • “It feels so crazy — I’m actually walking through the painting and jump into Monet’s garden. It’s so much fun!”

  • The cataract filter made me feel a little uncomfortable, but it really helped me understand how patients struggle with their vision.

  • I love the layout of the garden — it feels so immersive! Look at that bridge!

  • I love how the designer broke the artwork into steps — I don’t even need to know how to draw! It just unlocks the next part automatically.

Problems and Development

Voices - Nonlinear Storyline

  • “I am not sure about the initial interaction with the gallery and the blank canvas.”

    “It’s creepy to see there is only one painting in the gallery, others is all blank.”

  • Focus on iconic paintings at Giverny Garden

    Representing Monet’s career and the effects of cataracts

Voices - Teleportation, Painting Tubes Clues bugs

  • “Consider visual cues as well as audio cues for figuring out what color is next to make this more playful or inspire us to find a calming challenge.”

  • Dog footprints as visual clues for users to find color tubes

    Verbal hints for better storytelling and guide users to the next clues

  • “The teleportation takes me to everywhere expect the place I want be… I don’t like to go forward and back.”

  • Uses anchors to limit users’ teleportation area and guide them to the main interactable objects

After the heuristic evaluation, we learned that this VR project really works for users — they enjoy the main features and find it immersive and fun.

But one big question remains: How much do they actually take away from the experience?

Do they see it as something that could help teach kids or make a real impact?

That’s why we decided to move forward with more research!

Next Step - Usability Testing

Usability
Testing Plan

Problem Severity

4: Unusable: The user is not able to or will not want to use a particular part of the product because of the way that the product has been designed and implemented.

3: Severe: The user will probably use or attempt to use the product here, but will be severely limited in his or her ability to do so.

2: Moderate: The user will be able to use the product in most cases, but will have to undertake some moderate effort in getting around the problem.

1: Irritant: The problem occurs only intermittently, can be circumvented easily, or is dependent on a standard that is outside the product’s boundaries. Could also be a cosmetic problem

Task1

Let’s start by taking a walk around the gallery and completing a short tutorial on how to use the controller and navigate in the VR environment.

  • Users can view three artworks displayed in the gallery.

  • They can use the left and right icons to read the instructions for the experience.

Task3

In the garden scene, users can find the easel and canvas, and they can pick up the brush.

  • Users press the trigger button to grab the painting brush.

  • By pressing the trigger button, users can teleport around the scene.

Task5

Users hear visual and audio cues that guide them to unlock the next color, and once they discover it, they need to return to the front of the canvas.

  • Users hear and see clues about the next color.

  • Users can find the new color.

  • Users can grab the new color and hear a clue for the next step.

  • Users return to the front of the canvas.

Task2

Next, after reviewing instructions, users can walk into the painting of the Japanese garden and transition into the garden scene.

  • By pressing the trigger button, users can teleport to the front of the painting.

  • The user could smoothly transfer from the galley scene to the garden scene.

Task4

Users begin by dipping the brush into the colors on the palette and then painting on the canvas.

  • Users can hold the painting brush with green color.

  • Users can create brush strokes on the canvas.

Task6

Users return to the canvas, dip the brush into the new color, and paint to create a new stroke, unlocking the next step of the artwork.

  • Users can change the color of their brush.

  • Users can create brush strokes on the canvas with the new color.

  • The next step of the painting can be unlocked.

How should we evaluate the result?

We will compare completion rates, time on task, and satisfaction scores to evaluate the overall user experience and effectiveness of the project.

Post-Event Servery

  1. What did you enjoy most about this experience?

  2. What new insights or knowledge did you gain from participating?

  3. After completing the experience, how do you feel toward individuals with cataracts?

  4. How did the first-person perspective influence your understanding or engagement?

  5. Is there anything you would improve about the way the VR elements interact?

  6. Did any part of the experience make you feel emotionally connected or inspired?

  7. Do you think this experience could be useful for teaching or learning about art history or visual impairment?

  8. How likely are you to participate in one of our experiences again in the future?

Where the Testing Happens?

The team is continuing to refine the testing phase of this project, with plans to present the project and its testing framework at IEEE VR 2026.