Analyzing Code Provenance in LLMs for Programming Education
Summary
- Researchers investigate code provenance in LLMs to address challenges in programming education caused by instant code generation.
- The study extends the Min-K% probability approach with synthetic non-member code generation for Python membership inference and provenance tracking.
- Evaluations cover free models like GPT-2, DistilGPT-2, and DialoGPT, alongside commercial models including GPT-5, Gemini 2.5, Mistral, DeepSeek, and LLaMA 3.3.
- Results indicate that free models enable more sensitive code memorization detection than commercial models, balancing detection accuracy and privacy protection.
Why it matters
Tracing code provenance helps educational institutions verify student effort and learning authenticity amidst widespread AI code generation.