I'm Ida, self-taught developer and technical blogger (within AI) based in Stockholm. Professional experience in product management, technical product marketing, data analytics, and software engineering. Educated as an economist. Generalist that loves to work on several parts of a business.
Let's talk about how to decrease agent costs by 90% using various techniques. We'll go through prompt caching works and why it’s a quick win, semantic caching, lazy-loading tools and MCPs, routing and cascading, delegating to subagents, and a bit on keeping the context clean.
Let us evaluate a pipeline with things like query optimization, detailed chunking with neighbors and keys, along with expanding the context with a plain RAG pipeline using LLM judges.
Does expanding chunks to neighbors improve answers? That’s what we’ll test here. We’ll run some basic evaluations and look at metrics like faithfulness, answer relevancy, context relevance, and hallucination rate, and compare results across different models and datasets.
We will go through query optimization, how search engines are used for discovery, chunking content, “on-the-fly” retrieval, and how you could potentially reverse-engineer a system like this to build a “GEO [Generative Engine Optimization] scoring system.
