How We Grade Final Theses (Grading Framework)
Update 2017-10-30: This grading scheme has been superseded by a new version. The version below may still be of interest for the grading rationale based on the various Prüfrungsordnungen.
With increasing student volume, we have to review more and more final theses from students of various degree programs. Thus, we were looking for a model that would ensure fair and even evaluation of student theses. Unfortunately, we didn’t find any, so we had to develop the model ourselves. We describe this model in this article. We first look at the requirements from the rules and regulations of the various degree programs and find that there aren’t many. We then look at existing evaluation models and find that there aren’t many either and that those few we find don’t match our needs. Thus, in a third section, we describe our model as derived from the prior reviews and our general understanding of what a final thesis should provide. It is our hope that this will make more explicit to students what we expect of a final thesis and how we will evaluate it.
Review of Prüfungsordnungen
You can find bits and pieces of the expectations for final thesis in the various Prüfungsordnungen (degree program regulations) of the degree programs we service.
The general rules and regulations of the TechFak (Faculty of Engineering) stipulate for the Bachelor thesis:
Die Bachelorarbeit soll nachweisen, dass die Studierenden im Stande sind, innerhalb einer vorgegebenen Frist eine Fragestellung selbstständig mit wissenschaftlichen Methoden zu bearbeiten und die Ergebnisse sachgerecht darzustellen.
And for the Master thesis:
Die Masterarbeit ist eine Prüfungsarbeit, die die wissenschaftliche Ausbildung abschließt. Sie soll zeigen, dass die Studierende oder der Studierende in der Lage ist, innerhalb einer vorgegebenen Frist ein Problem aus ihrem oder seinem Fach selbständig und nach wissenschaftlichen Methoden zu bearbeiten.
The general rules are extended by the specific rules of a particular degree program. In the Computer Science rules and regulations, for example, it says for the Bachelor thesis:
Die schriftliche Bachelorarbeit soll ein wissenschaftliches Thema aus dem Bereich der Informatik behandeln.
And for the Master thesis:
Die Masterarbeit dient dazu, die selbständige Bearbeitung von wissenschaftlichen Aufgabenstellungen der Informatik nachzuweisen.
So they just repeat the general rules while restricting the thesis domain to computer science.
The same holds true for Embedded Systems (“Informations- und Kommunikationssysteme”)
Die Bachelorarbeit dient dazu, die selbständige Bearbeitung von Aufgabenstellungen der Informations- und Kommunikationstechnik zu erlernen.
Die Masterarbeit dient dazu, die selbständige Bearbeitung von wissenschaftlichen Aufgabenstellungen der Informations- und Kommunikationstechnik nachzuweisen.
as well as Computational Engineering:
Die Bachelorarbeit dient dazu, die selbständige Bearbeitung von Aufgabenstellungen des Computational Engineerings zu erlernen.
Die Masterarbeit dient dazu, die selbständige Bearbeitung von wissenschaftlichen Aufgabenstellungen des Computational Engineerings nachzuweisen.
We also serve the Information Systems (WINF and IIS) degree programs. These programs are provided by the Faculty of Economics. For the WINF Bachelor thesis:
Die Bachelorarbeit soll nachweisen, dass die Studierenden im Stande sind, innerhalb einer vorgegebenen Frist ein Problem selbstständig nach wissenschaftlichen Methoden zu bearbeiten und die Ergebnisse sachgerecht darzustellen.
And for the IIS Master thesis:
Die Masterarbeit ist eine Prüfungsarbeit, die die wissenschaftliche Ausbildung abschließt. Sie soll zeigen, dass die Studierende oder der Studierende in der Lage ist, innerhalb einer vorgegebenen Frist ein Problem aus ihrem oder seinem Fach selbständig und nach wissenschaftlichen Methoden zu bearbeiten. Sie muss einen internationalen Bezug aufweisen und kann entweder anwendungs- oder forschungsorientiert ausgerichtet werden. Sie soll im Rahmen eines Auslandsaufenthaltes in Kooperation mit einer ausländischen Hochschule oder mit einem international tätigen Unternehmen angefertigt werden. […] Die Masterarbeit muss mit einer Erklärung der Studierenden oder des Studierenden versehen sein, dass die Arbeit selbst verfasst und keine anderen als die darin angegebenen Quellen und Hilfsmittel benutzt wurden.
Beyond this, the various rules and regulations talk about time frames, language, and process, but not about quality criteria. Thus, there is no officially approved framework that ensures fair and repeatable evaluation of final theses.
We felt we needed to create a superset of requirements from what we just reviewed, given that so few requirements are spelled out. Thus, for Bachelor and the Master theses, here are minimal and necessary (but not sufficient) criteria. The student is supposed to
- perform work on a scientific task,
- demonstrate ability to perform independent work,
- use and demonstrate ability to apply scientific methods,
- finish the task within the given time-frame,
- present the results adequately, and
- declare all employed sources and tools.
In addition, depending on the thesis domain, there are additional criteria. For one, all theses need to be within the degree program’s domain. For IIS, additional things need to be checked off. The thesis needs to
- touch on international (information systems) issues and
- have been developed as part of some international cooperation.
Review of Existing Frameworks
Prof. Schneider established a framework for grading theses at the Chair of Programming Systems; his successor Prof. Philippsen has been using it as well. It is based on the following criteria:
- Schwierigkeitsgrad (difficulty)
- Originalität (originality)
- Wissenschaftliche Arbeitstechnik (method competence)
- Stil (style)
- Form (form)
For each criterion, a point value is provided; the values get added up using a formula. A couple of constraints serve as safe-guards, for example, one is to check that criterion 3 has at least 4 pointsand that criteria 3-5 add up to at least 8 points. Where exactly the point system comes from I don’t know.
Our Grading Framework
These criteria make a lot of sense to me, but after reviewing some more discussions, I’m adjusting them to this more orthogonal and comprehensive set of criteria:
Pass/Non-Pass Contributions
- Completeness of declaration of materials built on, tools used, and help received (Vollständigkeit in der Benennung aller Hilfe und Hilfsmittel)
- How well the thesis focusses on the problem domain of the degree program studied (Bearbeitung angemessener Probleme (CS, IuK, IIS, …))
If a student doesn’t properly acknowledge or reference appropriately what he or she built on, I might be looking at a case of plagiarism. If there was not enough of a focus on the domain of the degree program, the student failed the degree requirements, see above. For (1), the student should have learned in Nailing your Thesis how to properly cite, and for (2) the agreement I write up with the student before work begins should have ensured that students focus properly.
Graded Contributions
- Independence of work performed by student (Selbständigkeit der Arbeit): A student should know when to come asking for help and when to solve problems on their own. To the extent this happens, the student demonstrates maturity as to what he or she was supposed to have learned.
- Clarity of the presentation of results and methods (Klarheit der Darstellung der Ergebnisse und Methoden): The thesis itself is a written presentation of the work performed with main parts being the thesis question, process followed and methods applied as well as results achieved. The clarity of that presentation is an important skill in itself and also demonstrates the student’s clarity of thinking and understanding of the subject matter.
- Extent of achieved scientific progress (Ausmaß des erreichten wissenschaftlichen Fortschritts). This category has the following measurable components:
- Clarity of thought of question and resulting answers (Klarheit der formulierten Aufgabe und der resultierenden Antworten)
- Difficulty of challenge and creativity of solution (Schwierigkeit des Vorhabens und Kreativität in der Lösung)
- Quality of validation resp. plausibilisation of results (Validierung resp. Nachvollziehbarkeit der gegebenen Antworten)
- Quality and competence of relevant method application (Qualität der Anwendung und Kompetenz in den relevanten Methoden)
- Quality of conceptualization and consistence of work presented (Qualität der Konzeption und Durchgängigkeit der Arbeit)
- Scientific and societal value of work presented (wissenschaftliche und gesellschaftliche Bedeutung der Arbeit)
That’s it. All these components, taken together, form the framework from which we grade final theses. We review each thesis on all of these components, determine their grade, and combine them into one final grade. The actual grading is not based on a point system but uses the departmental average grade and compares the student with how much better or worse he or she has been doing when compared with “the average student”. This way, we remain fair to everyone and adjust to changing standards.