Tom Barber challenges the AI hype cycle, arguing that users care about outcomes, not architecture. Learn why slapping an 'AI-powered' label on everything is the wrong approach, and discover how to thoughtfully integrate LLMs into products without falling into common pitfalls like dependency on unstable APIs or unnecessary chatbot interfaces.
Show Notes
Episode Overview
Tom Barber returns with a critical examination of AI integration in modern software development, challenging teams to focus on user outcomes rather than jumping on the AI hype train.
Key Topics Covered
The AI Marketing Problem
- Why 'AI-powered' labels are often meaningless marketing
- The difference between machine learning (which has existed for decades) and modern LLMs
- Examples of invisible AI: spam filtering, fraud detection, map rerouting
- Users grade products on consistency, not on the impressiveness of the underlying model
Engineering Considerations for LLM Integration
- Choosing the right model for your specific use case (Opus, Sonnet, GPT-4, etc.)
- Tradeoffs between cost, speed, and inference quality
- Building evaluation systems and fallback paths
- Managing latency budgets and graceful degradation
- Handling API outages from providers like Anthropic and OpenAI
- The risks of depending on frontier models that can be deprecated
Trust and Transparency
- AI as a potential trust liability
- Managing user expectations around hallucinations
- The importance of data provenance and quality (garbage in, garbage out)
- When and how to disclose AI usage to users
- The ethical obligation to be transparent when AI makes consequential decisions
Product Strategy
- Why you can't charge an 'AI tax' on top of existing pricing
- Pricing based on outcomes, not on the technology stack
- How to use LLMs to deliver genuine efficiency gains
- Reducing user overhead and friction through thoughtful AI integration
Beyond Chatbots
- Why chatbots may be the most inefficient way to interact with LLMs
- The challenge: How to integrate LLMs without forcing users to type everything
- Asking 'What's now instant that wasn't?' instead of 'How do we add AI?'
- Innovation opportunities for those who can solve the chatbot problem
Key Takeaways
- Users care about reliable outcomes, not whether you're using AI
- Engineer for model availability issues and API outages from day one
- Select and tune models specifically for your use case rather than defaulting to frontier models
- Be transparent about AI usage, especially for consequential decisions
- Focus on delivering value through AI rather than adding an 'AI-powered' label for marketing
- The future belongs to products that leverage LLMs without relying on chatbot interfaces
Resources Mentioned
- Various LLM providers: Anthropic (Claude/Opus/Sonnet), OpenAI (ChatGPT-4)
- Example of model deprecation: Fable model being pulled
Connect
Engineering Evolved is hosted by Tom Barber. If you found this episode valuable, please leave a rating and review to help other leaders discover the show.
Chapters
- 0:00 - Introduction: Users Don't Care If It's AI
- 1:01 - Machine Learning Has Always Been Here
- 2:19 - The AI Marketing Problem: Selling Architecture vs Outcomes
- 5:16 - Engineering Realities: Models, Consistency, and Reliability
- 10:11 - The Cost of the AI Label: Trust and Pricing
- 14:39 - When Users Do Care: Transparency and Consequential Decisions
- 17:15 - Beyond Chatbots: The Future of LLM Integration