“Finding Open Options” — a MSc thesis by Karin van den Berg, touches on a model for OSS evaluation.
The summary is as follows:
The Open Source software market is getting more and more attention. Large IT corporations such as IBM and Novell are investing in Open Source software. Open Source software development is very different from traditional proprietary software. In order to understand Open Source software better, this thesis offers a model for
Open Source software evaluation, which can be used as a tool to find the right software package to meet the user�s needs.
This research project was performed at Tilburg University in the Department of Information Systems and Management. The goal was to get a better understanding of Open Source software and to make the Open Source software process more understandable for those who evaluate this type of software.
An introduction to Open Source software is followed by the Open Source software evaluation model, using the criteria found in Open Source literature.
Community � the driving force behind an Open Source project
Release Activity � showing the progress made by the developers
Longevity � how long the product has been around
License � is one of the general Open Source licenses used
Support � from the community as well as paid support options
Documentation � user manuals and tutorials, developer documentation
Security � responding to vulnerabilities
Functionality � testing against functional requirements
Integration � standards, modularity and collaboration with other products
Goal and Origin � why was the project started and what is the current goal
These criteria form the key terms of the model. The evaluation process is described using these criteria. The practical part of the model consists of two steps. In the first step selection on the candidate list is performed, using four of the above criteria: Functionality, Community, Release Activity and Longevity. These criteria were selected
because they can be evaluated quickly for each candidate in order to eliminate non-viable candidates and select the best ones. This step results in a �short list� of candidates that can be evaluated in depth in the second step, taking a closer look at the software and the project using all ten criteria.
In order to test this model on real Open Source software, a case study was performed on Course Management Systems. In this case study the model is applied on a candidate list of 36 systems, and evaluation is performed on the top two systems found in the selection step. This evaluation led to a clear conclusion. The best system in this
evaluation is the Course Management System called Moodle. The results of the case study are consistent with real life performance of the Course Management Systems.