An experiment on subjective evolvability evaluation of object-oriented software: Explaining factors and interrater agreement (bibtex)
by
Abstract:
Recent trends in software development have emphasized the importance of refactoring in preserving software evolvability. We performed two experiments on software evolvability evaluation, i.e. evaluating the existence of certain code problems called code smells and the refactoring decision. We studied the agreement of the evaluators. Interrater agreement was high for simple code smells and low for the refactoring decision. Furthermore, we analyzed evaluators' demographics and source code metrics as factors explaining the evaluations. The code metrics explained over 70% of the variation regarding the simple code smell evaluations, but only about 30% of the refactoring decision. Surprisingly, the demographics were not useful predictors neither for evaluating code smells nor the refactoring decision. The low agreement for the refactoring decisions may indicate difficulty in building tool support simulating real-life subjective refactoring decisions. However, code metrics tools should be effective in highlighting straightforward problems, e.g. simple code smells.
Reference:
An experiment on subjective evolvability evaluation of object-oriented software: Explaining factors and interrater agreement (), In 2005 International Symposium on Empirical Software Engineering, ISESE 2005, 2005.
Bibtex Entry:
@inproceedings{Maentylae2005,
abstract = { Recent trends in software development have emphasized the importance of refactoring in preserving software evolvability. We performed two experiments on software evolvability evaluation, i.e. evaluating the existence of certain code problems called code smells and the refactoring decision. We studied the agreement of the evaluators. Interrater agreement was high for simple code smells and low for the refactoring decision. Furthermore, we analyzed evaluators' demographics and source code metrics as factors explaining the evaluations. The code metrics explained over 70{\%} of the variation regarding the simple code smell evaluations, but only about 30{\%} of the refactoring decision. Surprisingly, the demographics were not useful predictors neither for evaluating code smells nor the refactoring decision. The low agreement for the refactoring decisions may indicate difficulty in building tool support simulating real-life subjective refactoring decisions. However, code metrics tools should be effective in highlighting straightforward problems, e.g. simple code smells.},
author = {M{\"{a}}ntyl{\"{a}}, Mika V.},
booktitle = {2005 International Symposium on Empirical Software Engineering, ISESE 2005},
doi = {10.1109/ISESE.2005.1541837},
isbn = {0780395085},
keywords = {cocome_lit-review,object-oriented programming;software metrics;code smell evaluation;evaluator demographics analysis;interrater agreement;object-oriented software;refactoring decision;software development;software evolvability evaluation;source code metrics;Demography;Humans;Object oriented modeling;Performance evaluation;Programming profession;Software performance;Software quality;Software systems},
mendeley-tags = {cocome_lit-review},
pages = {287--296},
title = {{An experiment on subjective evolvability evaluation of object-oriented software: Explaining factors and interrater agreement}},
year = {2005}
}
Powered by bibtexbrowser