Docket
Marks handwritten exam scripts and re-marks every affected one when the scheme changes.
A vision model transcribes what the student wrote. Plain code checks it against the answer key and applies the official marking scheme. A teacher decides anything two readers disagree on, through a Strands Graph interrupt. AWS KMS signs each grade, and anyone can verify it in a browser with no account. When the scheme changes, one file landing in S3 re-marks every script, re-issues the grades that moved and revokes the old ones on a signed list.
Problem
When an exam board corrects its marking scheme, nobody can say which students' grades are now wrong without reading every script again by hand. A grade sitting in a database is also something anyone with access can edit.
Approach
- The model reads, code decides. A vision model transcribes the student's answer, and plain code checks it against an answer key it has verified itself, then applies the scheme.
- Two readers from different model families read every script. Any criterion they disagree on stops the Strands Graph for a teacher, and the paused Graph waits in S3 until the teacher answers.
- A SigningGuard hook blocks signing on a missing reading, a stale scheme or an undecided criterion. AWS KMS signs each grade, and the verify page checks it in the browser with WebCrypto.
- A new scheme file in S3 starts EventBridge and a Step Functions Map that re-marks every script on Lambda, re-issues the grades that moved and re-signs the revocation list.
Outcome
Built for the AWS Builder Center hackathon Strands SDK: Workshop to Ship. On the 21 task-13 scripts of CHECK-MAT, real handwritten exam solutions with official grades, Docket matched the official grade on 16, where the dataset authors' best published result is 12 (p = 0.29, not significant at this size). On tasks 15, 16 and 18 it matched 17 of 26, below the published 19. My first design let two models judge and signed when they agreed, and only 2 of its 6 unattended grades were right. That is why correctness moved into code. The two-reader version of design 2 is not measured yet.
Highlights
Routing lives in Strands Graph edge conditions, so a model never picks the route.
A teacher interrupt survives the process through the S3 session manager.
Grades are KMS-signed credentials that a static page verifies with no login.
Appeals rerun the Graph on cached readings with the readers swapped, and every outcome is signed.
Stack
- Python
- Strands Agents
- AWS KMS
- AWS Step Functions
- AWS Lambda
- Amazon EventBridge
- Amazon S3
