Motion-based video games have a variety of positive effects on older adults, but their design remains challenging. Traditional game design often turns towards a user-centered design process, and game design experts are readily available. This paper discusses the challenges and opportunities of this approach in motion-based game design for older adults. We identify core challenges and show how users and experts can be involved in the design process to help game developers create safe, accessible and enjoyable motion-based games for older adults.