A metrics-based model for programming skill

Abstract

Programming is a core skill in computing. Much of computing education and work in industry and research has programming skill is an important factor. Human subjects have varying levels of programming experience which affects their output. The inability to quantify programming skill threatens the validity of research results. Previous research has attempted to model programming skill and experience by finding a correlation between programmer characteristics and number of correct answers in a set of programming tasks. However, the model was not validated. This study is therefore designed to model the skill of programmers and to validate the model using experimental proof approach. Undergraduate students were presented with programming tasks and a questionnaire to elicit programming experience and demographic data. Using Stepwise regression, a linear model was extracted which showed two variables: lengthSchoolJava and schoolHours6Months contributed most to the number of correct answers. Furthermore, factor analysis extracted two factors: Java knowledge and Practical Java experience, that explained the number of correct answers. For the future direction, we want to analyse programming error logs to improve our quality metric and confirm the extracted model by collecting more qualitative data.