Skip to content
Agentic Engineering Primer

Learning Impacts

AI boosts immediate task performance but can undermine the deep learning that builds lasting skill.

The Performance-Learning Paradox

Section titled “The Performance-Learning Paradox”
  • OECD Digital Education Outlook (2026): AI-assisted students scored 48% higher on practice tasks but 17% lower on unassisted exams.
  • March 2025 meta-analysis (13 studies): Positive effect size g=0.86 for AI in education, but effectiveness depends heavily on instructional model.
  • Wang & Fan (2025): ChatGPT meta-analysis found significant performance improvement (g=0.867) but only moderate effects on higher-order thinking.

Cognitive Effects

Section titled “Cognitive Effects”
  • MIT Media Lab “Your Brain on ChatGPT” (2025, preprint): EEG study of 54 participants. ChatGPT users showed lowest brain engagement, reduced frontal-parietal connectivity (working memory, executive function), and struggled with recall and originality afterward.
  • Gerlich (2025): Negative correlation between frequent AI use and critical thinking. Younger users showed highest dependence.
  • Walther (2026, Stanford): Introduced the concept of “cognitive surrender” — offloading reasoning itself, not just memory.

Coding-Specific Findings

Section titled “Coding-Specific Findings”
  • Anthropic RCT (Jan 2026): 52 developers learning a Python library. AI group scored 17% lower on comprehension (50% vs 67%). Largest gap appeared in debugging. Developers who used AI for conceptual questions retained more.
  • Shen & Tamkin (2026): Heavy AI reliance reduced new skill acquisition. Delegating code generation led to failed unassisted quizzes.
  • Prather et al. (2024, ACM): AI exacerbates metacognitive difficulties for novice programmers.
  • METR (early 2025): Experienced open-source developers were 19% slower with AI tools.

The Competence Illusion

Section titled “The Competence Illusion”
  • IJRSI (2025): Documented an “illusion of competence” — the fluency of AI output gets mistaken for personal mastery.
  • Zhang & Xu (2025): AI boosts perceived self-efficacy while increasing dependence and reducing actual autonomy.
  • Ray et al. (2024): AI accelerates decay of existing skills and hinders new skill acquisition, often without user awareness.

When AI Helps Learning

Section titled “When AI Helps Learning”
  • Tutor CoPilot RCT (Wang et al., 2024/2025): AI-human hybrid tutoring produced a 4 percentage point increase in math mastery, +9 p.p. for students of lower-rated tutors. Worked by promoting probing questions rather than direct answers.
  • Ponti (2025, Nature): A custom pedagogical AI tutor outperformed traditional active learning.
  • Park et al. (2024): AI that asks questions instead of answering them improved decision-making outcomes.
  • PNAS (2025): AI without guardrails actively harmed learning. With guardrails, outcomes improved.
  • The pattern: AI helps when it scaffolds thinking; hurts when it replaces thinking.

How to Apply This

Section titled “How to Apply This”
  • Attempt problems before turning to AI. Struggle is where encoding happens.
  • Use AI to explain concepts and ask follow-up questions, not to generate solutions you paste in.
  • Test yourself without AI regularly. If you can’t reproduce what you built, you didn’t learn it.
  • For practical patterns, see Learning with AI.

Sources

Section titled “Sources”