This started as a question rather than a product: could the DSPy framework, running against fully local models, handle a job that usually gets thrown at cloud APIs? Essay grading was the test case - slow and subjective when done by hand, and exactly the kind of task where consistent, structured LLM output matters more than raw model size.
The result is a self-learning project that grades essays across multiple criteria with detailed feedback, plus a DSPy guide on Medium documenting what I learned about running DSPy on local models.
Live dashboard of Interface
Two decisions defined the build
- Local models via Ollama, not an API: Student essays are exactly the kind of data that should not be shipped to third parties, and grading at volume through a metered API adds cost per essay. Local inference removes both concerns - accepting weaker raw model quality as the trade.
- DSPy, not raw prompts: An evaluator that grades "creatively" is useless. DSPy's structured signatures pin the model to defined inputs and outputs for each criterion, keeping feedback consistent from one essay to the next - the property a grader needs most.
What it evaluates
- Multiple Criteria: Content quality, structure, grammar, and argumentation, each assessed separately.
- Structured Feedback: Comprehensive comments per aspect of the essay, not just a number.
- Sentiment Analysis: Emotion and bias detection alongside the quality criteria.
Workflow
- Text Input: Essay submission through the interface
- Preprocessing: Text cleaning and structure analysis
- LLM Analysis: Multi-criteria evaluation using local models
- Feedback Generation: Comprehensive report with improvement suggestions
- Score Assignment: Quantitative grade across evaluation criteria
Multi-Criteria Evaluation
Grade and Evaluation Downloadable
Conclusion
The experiment held up: structured evaluation on local models produces objective, instant, repeatable feedback - feasible for educators and students without sending a single essay off-machine.
For more details and to explore the implementation, visit the GitHub repository.