BackgroundMany research domains still heavily rely on paper-based data collection procedures, despite numerous associated drawbacks. The QuestionSys framework is intended to empower researchers as well as clinicians without programming skills to develop their own smart mobile apps in order to collect data for their specific scenarios. ObjectiveIn order to validate the feasibility of this model-driven, end-user programming approach, we conducted a study with 80 participants. MethodsAcross 2 sessions (7 days between Session 1 and Session 2), participants had to model 10 data collection instruments (5 at each session) with the developed configurator component of the framework. In this context, performance measures like the time and operations needed as well as the resulting errors were evaluated. Participants were separated into two groups (ie, novices vs experts) based on prior knowledge in process modeling, which is one fundamental pillar of the QuestionSys framework. ResultsStatistical analysis (t tests) revealed that novices showed significant learning effects for errors (P=.04), operations (P