What:
A research paper published at CHI '25 (Conference on Human Factors in Computing Systems) that investigates how generative AI affects critical thinking among knowledge workers. The authors surveyed 319 knowledge workers who regularly use GenAI tools to understand when and how they enact critical thinking when using these tools, and when they perceive increased or decreased effort for critical thinking due to GenAI.
Who:
The research was conducted by Hao-Ping (Hank) Lee from Carnegie Mellon University and a team from Microsoft Research Cambridge including Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson.
In a Nutshell:
Knowledge workers perceive GenAI as reducing the effort required for critical thinking tasks, especially when they have higher confidence in AI capabilities.
Higher confidence in GenAI is associated with less critical thinking, while higher self-confidence in one's own abilities is associated with more critical thinking.
GenAI shifts the nature of critical thinking toward:
Information verification (rather than information gathering)
Response integration (rather than problem-solving)
Task stewardship (rather than task execution)
Three key motivators for critical thinking with GenAI:
Desire to improve work quality
Avoid potential negative outcomes
Develop professional skills
Three main inhibitors of critical thinking with GenAI:
Awareness barriers (e.g., overreliance on AI)
Motivation barriers (e.g., time pressure)
Ability barriers (e.g., difficulty improving AI responses)
Key Stats & Facts:
79% perceived less effort for comprehension tasks with GenAI.
72% perceived less effort for analysis with GenAI.
76% perceived less effort for synthesis with GenAI.
Top 3...:
Top 3 GenAI tools used by participants: ChatGPT (96.87%), Microsoft Copilot website (23.20%), and Gemini website (21.63%).
Top 3 occupations represented: Computer and Mathematical (18.50%), Arts/Design/Entertainment/Sports/Media (13.79%), and Office/Administrative Support (11.91%).
Golden nuggets:
The most crucial finding is the inverse relationship between confidence in AI and critical thinking effort: as users trust GenAI more, they engage in less critical evaluation of its outputs. This creates a potential "irony of automation" where the tool that is meant to enhance productivity may ultimately diminish users' critical thinking capabilities through reduced practice, especially for routine tasks.
What's In It For Marketers?
What Marketers Can Do to Maintain Critical Thinking While Using GenAI?
Implement a "verify-first" habit: Before accepting any GenAI output for marketing content, establish a routine of verifying key claims, statistics, and brand messaging. Create a checklist of verification sources (industry reports, brand guidelines, legal requirements) that you review for each AI-generated piece.
Practice regular AI-free ideation sessions: Set aside dedicated time (perhaps weekly) where you generate marketing concepts completely without AI assistance first, then compare with AI-assisted approaches afterward. This maintains your creative muscles while helping you recognize when AI is truly adding value.
Adopt the "three alternatives" approach: When using GenAI for marketing strategy or content creation, always request multiple versions, then critically evaluate each against your objectives. The act of comparison forces critical assessment rather than passive acceptance.
Document your reasoning: When making marketing decisions based on GenAI suggestions, write down your specific reasoning for accepting or rejecting each recommendation. This metacognitive practice strengthens analytical thinking and prevents over-reliance. If possible, do it the 'old school' way: with a pencil and a white sheet of paper ;).
Use AI as a sounding board, not a decision-maker: Approach GenAI as a collaborative thought partner by presenting your initial marketing ideas first, then asking for feedback or enhancements, rather than starting with an empty prompt expecting the AI to do all the thinking.
Schedule regular skill maintenance: Identify core marketing skills (copywriting, data analysis, audience research) that GenAI might automate, and deliberately practice these manually at regular intervals to maintain proficiency, even as you leverage AI for efficiency. Don't fully delegate to Gen AI. You are and remain in charge! Always.
Create feedback loops: After implementing AI-assisted marketing campaigns, systematically analyze performance against your own predictions. This critical evaluation of outcomes strengthens your judgment and helps identify AI's strengths and weaknesses in your specific marketing context.
"...generate marketing concepts completely without AI assistance first, then compare with AI-assisted approaches afterward" <--- Yes, this is a great suggestion. That way you cross-check your thought processes and see where you can improve, as well as where you may be better than AI. I also think that the recommendation to schedule regular skill maintenance is so important! So often we are running like hamsters in a wheel trying to get everything done, and we miss keeping our skills in the process. Like doctors, dentists, lawyers, educators and others need to do continuing education, those who do not have those professional requirements must be proactive in doing so, in order to ensure that they are not overtaken by those who have built this in to their personal development program.