26 Key Takeaways from Building 150+ Agents in 9 months
YouTube Video Title
0:00:00.000
1. Intro
(0:00:23.000)
▼
Built over 150 AI agents in 9 months
Sharing 26 key takeaways
Aim to prevent others from repeating mistakes
0:00:00.000
1.1. Introduction to AI Agents
(0:00:23.000)
▼
Overview of building AI agents
Purpose of sharing takeaways
Importance of learning from mistakes
0:00:23.000
2. Takeaway 1
(0:01:16.000)
▼
AI agents differ from employees and automations
Agents require specific training
Think of agents in terms of Sops
0:00:23.000
2.1. Understanding AI Agents
(0:00:38.000)
▼
Agents not employees or automations
Automations have hardcoded steps
Agents have less autonomy than employees
0:01:01.000
2.2. Training and Thinking About Agents
(0:00:38.000)
▼
Agents need training on specific instructions
Cannot learn by trial and error
Consider agents in terms of Sops
0:01:39.000
3. Takeaway 2
(0:00:51.000)
▼
Start with well-documented processes
Sops simplify agent training
Use existing onboarding materials
0:01:39.000
3.1. Importance of Sops
(0:00:51.000)
▼
Begin with well-documented processes
Sops contain necessary training data
Utilize existing onboarding materials
0:02:30.000
4. Takeaway 3
(0:01:12.000)
▼
Business owners won't build their own agents
Agent platforms will increase demand for developers
Determining which agents to build is crucial
0:02:30.000
4.1. Business Owners and Agent Development
(0:00:36.000)
▼
Business owners won't build agents
Agent platforms will increase demand
Developers needed to determine agent needs
0:03:06.000
4.2. The Role of Agent Developers
(0:00:36.000)
▼
Developers crucial for agent creation
Determining which agents to build is key
Platforms will not replace developers
0:03:42.000
5. Takeaway 4
(0:01:12.000)
▼
Business owners often unsure of needed agents
Consulting is vital for identifying valuable agents
Start with customer journey mapping
0:03:42.000
5.1. Identifying Valuable Agents
(0:00:36.000)
▼
Owners unsure of needed agents
Consulting identifies valuable agents
Start with customer journey mapping
0:04:18.000
5.2. Mapping Customer Journeys
(0:00:36.000)
▼
Map customer journeys on Figma
Identify automation opportunities
Focus on valuable parts of the process
0:04:54.000
6. Takeaway 5
(0:01:22.000)
▼
Start with as few agents as possible
Begin with one small agent
Add more agents after fine-tuning
0:04:54.000
6.1. Starting with Few Agents
(0:00:41.000)
▼
Avoid building too many agents
Start with as few as possible
Begin with one small agent
0:05:35.000
6.2. Fine-Tuning and Expansion
(0:00:41.000)
▼
Fine-tune the initial agent
Deploy and test with client
Add more agents as needed
0:06:16.000
7. Takeaway 6
(0:01:12.000)
▼
Data-driven decisions are important
Combining data with actions yields results
Scrape both internal and external sources
0:06:16.000
7.1. Data-Driven Decisions
(0:00:36.000)
▼
Importance of data-driven decisions
Garbage in, garbage out principle
Data quality affects agent outputs
0:06:52.000
7.2. Combining Data and Actions
(0:00:36.000)
▼
Combine data with relevant actions
Achieve higher results with both
Scrape internal and external sources
0:07:28.000
8. Takeaway 7
(0:01:18.000)
▼
Prompt engineering is an art
Provide examples and consider order
Iterate and test prompts constantly
0:07:28.000
8.1. The Art of Prompt Engineering
(0:00:39.000)
▼
Prompt engineering is a real job
Importance grows with model evolution
Write prompts like essays or blog posts
0:08:07.000
8.2. Tips for Effective Prompts
(0:00:39.000)
▼
Provide examples in prompts
Order of sentences matters
Iterate and test prompts constantly
0:08:46.000
9. Takeaway 8
(0:01:18.000)
▼
Integrations are as important as functionality
Agents must work in existing systems
Convenience for users is crucial
0:08:46.000
9.1. Importance of Integrations
(0:00:39.000)
▼
Integrations as important as functionality
Focus on agent capabilities often
Integrations affect user convenience
0:09:25.000
9.2. Integrating Agents into Systems
(0:00:39.000)
▼
Integrate agents into existing systems
Example: customer support agent in Zendesk
Convenience drives value for users
0:10:04.000
10. Takeaway 9
(0:01:07.000)
▼
Agent reliability has been solved
Use Pydantic for data validation
Developers are responsible for reliability
0:10:04.000
10.1. Solving Agent Reliability
(0:00:33.500)
▼
Agent reliability issue resolved
Startups focus on reliability
Developer responsibility for reliability
0:10:37.500
10.2. Using Pydantic for Validation
(0:00:33.500)
▼
Jason Liu's Pydantic solution
Validate agent inputs and outputs
Prevents major consequences
0:11:11.000
11. Takeaway 10
(0:00:54.000)
▼
Tools are the most important component
Agents generate value through actions
Focus on building and structuring tools
0:11:11.000
11.1. Importance of Tools
(0:00:54.000)
▼
Tools crucial for agent building
70% of work goes into building actions
Tools provide value through actions
0:12:05.000
12. Takeaway 11
(0:00:54.000)
▼
Limit tools to four to six per agent
Complexity affects tool limits
Splitting agents prevents hallucinations
0:12:05.000
12.1. Limiting Tools per Agent
(0:00:54.000)
▼
Four to six tools per agent
Depends on tool complexity
Prevents agent hallucinations
0:12:59.000
13. Takeaway 12
(0:00:43.000)
▼
Model costs are not a major concern
Focus on ROI rather than costs
Significant ROI from AI agents
0:12:59.000
13.1. Model Costs and ROI
(0:00:43.000)
▼
Model costs not a major concern
Focus on ROI from AI agents
Example: significant cost reduction
0:13:42.000
14. Takeaway 13
(0:00:46.000)
▼
Clients don't care about the model used
Use Azure OpenAI for data privacy
Developer experience with OpenAI API
0:13:42.000
14.1. Client Focus on Value
(0:00:46.000)
▼
Clients focus on value, not models
Use Azure OpenAI for privacy
Developer experience with OpenAI API
0:14:28.000
15. Takeaway 14
(0:00:52.000)
▼
Don't automate until value is established
Manual process validation is crucial
Automate after confirming value
0:14:28.000
15.1. Establishing Value Before Automation
(0:00:52.000)
▼
Automate after establishing value
Manual process validation needed
Development costs are a concern
0:15:20.000
16. Takeaway 15
(0:01:03.000)
▼
Focus on ROI, not just use cases
Use ROI formula to calculate value
Example calculation of ROI
0:15:20.000
16.1. Calculating ROI
(0:01:03.000)
▼
Focus on ROI over use cases
Use ROI formula for calculations
Example: ROI calculation for a process
0:16:23.000
17. Takeaway 16
(0:00:50.000)
▼
Agent development is iterative
Test different architectures
Compare and refine agent solutions
0:16:23.000
17.1. Iterative Agent Development
(0:00:50.000)
▼
Agent development is iterative
Test various architectures
Compare and refine solutions
0:17:13.000
18. Takeaway 17
(0:01:03.000)
▼
Use divide and conquer approach
Deliver solutions incrementally
Automate by departments first
0:17:13.000
18.1. Divide and Conquer Strategy
(0:01:03.000)
▼
Use divide and conquer approach
Deliver solutions incrementally
Automate by departments first
0:18:16.000
19. Takeaway 18
(0:01:00.000)
▼
Evals are important for big companies
Evals improve solutions over time
SMBs may not need evals initially
0:18:16.000
19.1. Importance of Evals
(0:01:00.000)
▼
Evals crucial for big companies
Improve solutions over time
SMBs may not need evals initially
0:19:16.000
20. Takeaway 19
(0:01:11.000)
▼
Two types of agents: agents and workflows
Workflows have predetermined steps
Combine workflows with agentic steps
0:19:16.000
20.1. Types of Agents
(0:01:11.000)
▼
Agents and agentic workflows
Workflows have set steps
Combine workflows with agentic steps
0:20:27.000
21. Takeaway 20
(0:00:53.000)
▼
Agents need to be adaptable on feedback
Add tools for analyzing results
Ensure agents can read and validate data
0:20:27.000
21.1. Adaptability and Feedback
(0:00:53.000)
▼
Agents need adaptability on feedback
Add tools for result analysis
Ensure data reading and validation
0:21:20.000
22. Takeaway 21
(0:00:59.000)
▼
Don't build around limitations
Models will continue to improve
Avoid building obvious general use cases
0:21:20.000
22.1. Building Beyond Limitations
(0:00:59.000)
▼
Avoid building around limitations
Models will continue to improve
Don't focus on obvious general use cases
0:22:19.000
23. Takeaway 22
(0:00:57.000)
▼
Deploying agents is harder than building
Integration into client processes is key
Building a platform for deployment
0:22:19.000
23.1. Challenges of Deploying Agents
(0:00:57.000)
▼
Deployment harder than building
Integration into client processes
Building a platform for deployment
0:23:16.000
24. Takeaway 23
(0:00:55.000)
▼
Waterfall projects don't work for agents
Use subscription-based models
Transition to agile service agreements
0:23:16.000
24.1. Agile Approach to Agent Projects
(0:00:55.000)
▼
Waterfall projects unsuitable
Use subscription-based models
Transition to agile service agreements
0:24:11.000
25. Takeaway 24
(0:00:48.000)
▼
Include humans in the loop for critical agents
Review and approve agent actions
Remove human loop after fine-tuning
0:24:11.000
25.1. Human in the Loop for Critical Agents
(0:00:48.000)
▼
Include humans for critical agents
Review and approve agent actions
Remove human loop after fine-tuning
0:24:59.000
26. Takeaway 25
(0:00:53.000)
▼
2025 is the year of vertical AI agents
Vertical agents serve specific industries
Start with horizontal agents before vertical
0:24:59.000
26.1. Vertical AI Agents in 2025
(0:00:53.000)
▼
2025: year of vertical AI agents
Serve specific industries
Start with horizontal agents first
0:25:52.000
27. Takeaway 26
(0:00:56.000)
▼
Agents help businesses scale
Automation leads to higher revenues
Employees focus on higher-level tasks
0:25:52.000
27.1. Agents and Business Scaling
(0:00:56.000)
▼
Agents help businesses scale
Lead to higher revenues
Employees focus on higher-level tasks