grading-criteria.md

Grading Criteria of the KTH Devops Course

This page presents the grading criteria to assess the contribution in each task category: "presentation", "essay", "demo", "executable tutorial", "contribution to open-source", "course automation", "feedback".

Note 1: we are all aware that this assessment is subjective by nature. In case of disagreement, the informed judgment of the Professor is the final decision.

Note 2: in the grading criteria presented below, the mandatory parts do not count as "yes" for grading.

Presentations

The concept: The students prepare a 7 minute presentation on a topic that is relevant to DevOps. All students in a presentation must be present and speak in a balanced manner.

	Yes	No
timing: The presentation's length is between 6:30-7:30 minutes (hard limit)	Yes	No
well-structured: Structure is announced and graphically visible	Yes	No
motivation: The presentation contains a good, motivating introduction telling why this presentation is important	Yes	No
technical: The presentation contains one part that is deeply technical	Yes	No
code: The presentation contains valuable and readable code snippets	Yes	No
originality: The presentation contains one part that is original	Yes	No
google: Less than 100 results on Google on this topic.	Yes	No
reflection: The presentation contains a reflective part	Yes	No
sota: There is one good slide positioning the presentation in the state of the art	Yes	No
take-home: The last slide contains a good and concise take-home message	Yes	No
loudly: The speakers talk loudly and clearly	Yes	No
speech: The speakers engage with the audience	Yes	No
interactivity: usage of an interaction platform (eg menti)	Yes	No
humour: The speakers are fun, have humour	Yes	No
readable-slides: The slides don't have too much text	Yes	No
illustration: The slides contain nice illustrations	Yes	No

To pass, you need at least 10 "yes".

Essays

The concept: The students write an essay on a topic relevant to the course. Example of essays, see 2019 essays, 2020 essays

	Yes	No
format: The essay is in PDF	Mandatory	-
length: The essay length is between 2400-2600 words, incl. references (hard limit)	Mandatory	-
relevant: The essay addresses a topic that is relevant for DevOps	Mandatory	-
title: The essay has a good and focused title	Yes	No
well-structured: The essay is well structured	Yes	No
problem: The intro clearly states a relevant problem	Yes	No
sota: There is one part positioning the essay in the state of the art	Yes	No
introduction: The intro outlines the solution to the problem	Yes	No
conclusion: The conclusion contains an emphasized key take-away of the essay	Yes	No
self-contained: The essay is self-contained, one can understand it without reading something else (expected knowledge of the reader: a master student in computer science)	Yes	No
innovative: The essay contains innovative ideas or material	Yes	No
figures: The essay contains relevant and informative figures	Yes	No
listings: The essay contains relevant and informative listings	Yes	No
sound: The essay is sound, factual, and accurate	Yes	No
references: The essay contains references, appropriate in number and quality (10 good refs is a minimum, incl. at least 3 academic references)	Yes	No
elegant: The essay presentation is elegant and visually appealing (eg LaTeX, InDesign)	Yes	No
reflection: The essay contains a reflective part	Yes	No

To pass, you need at least 9 "yes".

Demos

The concept: Students prepare a demonstration involving DevOps technology, to be performed during the lecture. For example, a demo typically involves multiple virtual machines, likely deployed in the cloud. A demonstration is scripted, prepared and lasts 6:30-7:30 minutes.

	Yes	No
timing: The demo lasts between 6:30-7:30 minutes (hard limit)	Mandatory	-
motivation: The demonstration is clearly motivated (why it matters for Devops?)	Yes	No
narrative: The demo contains a good narrative	Yes	No
difficulty: The demonstration is difficult to do	Yes	No
speech: The demo is accompanied by a clearly and structured speech	Yes	no
originality: The demonstration is original (there are few demos/tutos on this topic on the Internet)	Yes	No
aesthetics: The demo is visually appealing	Yes	No
easter-egg: The demo contains an easter egg related to the demo topic	Yes	No
repo: There is a good code repo, documented with a README, to run the demo.	Yes	No
take-home: The demo includes a clear and visible take-home message	Yes	No

To pass, you must have at least 7 "yes".

Open-source contributions

The concept: you contribute to one open-source project related to DevOps. Yout get at least one merged pull-request.

Criteria for the selection of the open-source project: 1) The project is related to DevOps 2) The project has more than 10 Stars 3) The project has more than 100 Commits 4) The project has an active community on GitHub.

	Yes	No
bug: The contribution fixes bugs	Yes	No
documentation: The contribution improves documentation	Yes	No
feat: The contribution adds new features	Yes	No
difficulty: The contribution is a difficult piece of engineering	Yes	No
conversation: There is an interesting engineering conversation with the maintainers	Yes	No
merge: The contribution is merged in the main branch.	Yes	No

To pass, you must have 2 yes.

Executable Tutorials

The concept: you create an executable tutorial about a specific technology related to Devops. You deliver your tutorial on a platform supporting execution, such as Katacoda, mybinder.org, collab or equivalent.

	Yes	No
executable: The tutorial can be automatically executed from beginning to the end, in the browser or in CI (see below)	Mandatory	-
ilo: The tutorial states the intended learning outcomes.	Mandatory	-
motivation: The tutorial is clearly motivated (why it matters for Devops?)	Yes	No
browser-based: The tutorial can be successful executed in the browser (katacoda is recommended)	Yes	No
ci-based: The tutorial can successful be executed as a CI job	Yes	No
background: The tutorial gives enough background	Yes	No
illustrated: The tutorial is illustrated with an informative figure (eg a flowchart)	Yes	No
pedagogical: The tutorial is easy to follow	Yes	No
original: The tutorial is original, no or few similar tutorials exist on the web	Yes	No
easter-eggs: The tutorial contains an easter egg	Yes	No
language: The language is appropriate (structure, grammar, spelling)	Yes	No

To pass, you must have the mandatory parts and at least 5 "yes".

Course automation

The concept: you create an automation task for the course to improve the teaching experience and effectiveness (eg a Github action task). Simple tasks can be bundled in a package to get a pass.

Make sure that you meet the following requirements in your final submission:

1. The automation task (ex., Github Action) should be implemented in a stand-alone repository, not as a part of the Deveops-Course repository. As a bonus, you can do this by publishing your Github Action in the Github marketplace. The final submission must include a link to the repository that implements the task.
2. You should demonstrate that your automation technique actually works and automatically produces some results. For example, you can fork the Devops-Course repo and activate the implemented Github Action on it and show how it works by making changes in your forked repo. In case of this example, a link to your forked repository should be included in the final submission.

Suggestions for course automation tasks: KTH#916

	Yes	No
timeliness: the automation is done before the first task deadline (in order to be useful for the course)	Mandatory	-
repo: the code for the task is available in a public repo	Mandatory	-
outcome: the automation task produces a PR status or an issue / PR comment	Yes	No
reuse: the automation task is reusable (next year for this course)	Yes	No
platform: the task runs on a standard platform (eg Github action)	Yes	No
doc: the repo is documented with a good readme	Yes	No
figure: the README is illustrated with at least an informative figure (eg a flowchart)	Yes	No
availability: The action is available on a market place (Github Marketplace, Azure Marketplace, etc.)	Yes	No

To pass, you must have the mandatory parts and at least 4 "yes".

Feedback

The concept: you provide constructive and timely feeback about one task from categories "essay" and "executable tutorial". The feedback is provided in a written manner, as a well-structured comment on the PR of the task. The students who receive feedback act upon the feedback they receive.

	Yes	No
substance: the feedback is substantiated (at least 500 words)	Mandatory	-
high-level: the feedback starts with a list of high-level strengths and high-level weaknesses about the work	Yes	No
timeliness: the feedback is provided 2 business days (48h) after the "go" from the authors	Yes	No
constructive: all feedback points are constructive and clearly actionable	Yes	No
structure: the feedback is well-structured (eg. along the outline of the work under feedback)	Yes	No
pointers: the feedback contains valuable pointers to additional material	Yes	No

To pass, you must have at least 4 "yes" and the TA assessment.