GenAI RAG Platform
30+ min document hunts → sub-10s answers; hybrid RAG + semantic cache cut LLM spend ~60%.
- Role
- Senior AWS Solution Architect
- Context
- Enterprise / SaaS
- Duration
- 5 months
- Team
- 3 engineers + 1 architect (me)
Tech stack
- AWS Bedrock
- Claude
- OpenSearch Serverless
- Lambda
- S3
- ElastiCache
- Step Functions
- Python
- React
- Terraform
Business problem and production context.
Business problem
Enterprise client had 2M+ internal documents (PDFs, Confluence, Slack threads) spread across systems with no unified search. Engineers spent 30+ minutes finding answers to questions already documented. Traditional keyword search missed semantic context and returned irrelevant results.
Architecture decision
Built an end-to-end RAG pipeline on AWS Bedrock with Claude as the LLM. Implemented document ingestion via Lambda + S3 triggers, chunking with overlap strategies, and vector embeddings stored in Amazon OpenSearch Serverless. Added semantic caching with ElastiCache to reduce repeated LLM calls by 60%. Built a React chat interface with streaming responses and source attribution.
The useful proof is the decision surface, not only the result number.
These are the parts of the work that show production judgment: ownership, constraints, rollback, cost, and observability.
Owned scope
- RAG architecture on AWS Bedrock with ingestion, chunking, vector storage, semantic cache, and streaming UI boundaries
- Retrieval quality and source-attribution path so engineers could trust the answer, not only the generated text
- Cost-per-query controls using cache hits, context sizing, and repeated-query reduction
Evidence artifacts
What would fail first?
Every project has constraints. The useful work is naming them early enough that rollback, cost, and ownership are designed before an incident.
Data volume
2M+ documents across mixed internal sources required incremental ingestion and traceable source chunks.
Trust
Enterprise users needed source attribution before they would rely on AI answers in daily work.
Cost
Repeated questions would have made model usage expensive without semantic caching and context control.
Trade-offs accepted on purpose.
Production risks and how they were controlled.
Small steps, visible changes, fewer surprises.
- 01
Cataloged document sources. 2M+ files across S3 buckets, Confluence spaces, and Slack exports. Built Lambda-based connectors for each source with incremental sync.
- 02
Implemented chunking pipeline using Step Functions: PDF extraction → text splitting (512 tokens, 50-token overlap) → embedding generation via Amazon Titan → vector storage in OpenSearch Serverless.
- 03
Built semantic search layer combining vector similarity search with keyword filtering: a hybrid approach that outperformed pure vector search by 23% on relevance benchmarks.
- 04
Added ElastiCache-based semantic cache layer, hashing query embeddings and returning cached responses for semantically similar questions (cosine similarity > 0.95).
- 05
Deployed React chat interface with streaming responses via WebSocket API Gateway, including source attribution showing exactly which document chunks informed each answer.
Before and after.
Rollback path
Ingestion, retrieval, and answer generation were separable, so a bad retrieval change could be rolled back without disabling the full chat surface.
Cost considerations
Semantic caching and context discipline were the main cost levers; the visible result was about 60% lower repeated LLM spend in the sanitized case story.
Observability notes
Useful signals were retrieval latency, cache hit rate, no-source answers, failed ingestion jobs, and user feedback on source relevance.
Related services
Have a similar production problem?
Send the current architecture, bill concern, deployment pain, or GenAI reliability issue. I will help you find the first real bottleneck.
Book a production review