Skip to content
Back to case studies
SanitizedGenAI / AI2024

GenAI RAG Platform

30+ min document hunts → sub-10s answers; hybrid RAG + semantic cache cut LLM spend ~60%.

Role
Senior AWS Solution Architect
Context
Enterprise / SaaS
Duration
5 months
Team
3 engineers + 1 architect (me)

Tech stack

  • AWS Bedrock
  • Claude
  • OpenSearch Serverless
  • Lambda
  • S3
  • ElastiCache
  • Step Functions
  • Python
  • React
  • Terraform
Executive summary

Business problem and production context.

Business problem

Enterprise client had 2M+ internal documents (PDFs, Confluence, Slack threads) spread across systems with no unified search. Engineers spent 30+ minutes finding answers to questions already documented. Traditional keyword search missed semantic context and returned irrelevant results.

Architecture decision

Built an end-to-end RAG pipeline on AWS Bedrock with Claude as the LLM. Implemented document ingestion via Lambda + S3 triggers, chunking with overlap strategies, and vector embeddings stored in Amazon OpenSearch Serverless. Added semantic caching with ElastiCache to reduce repeated LLM calls by 60%. Built a React chat interface with streaming responses and source attribution.

awsAWS Account / RAG prodRequestResponseQUERY INGRESSINGESTION + RETRIEVAL WORKERSVECTOR + CACHE + MODEL123456789CloudFrontedge cache + WAFAPI Gatewayauth + rate limitLambdaquery handlerS3raw doc storeLambdachunker 512/50BedrockTitan embed v2Lambdaanswer composerOpenSearchhybrid vector + kwElastiCachesemantic cache 60%BedrockClaude w/ citationsCROSS-CUTTING: IAM scoped per role · KMS doc encryption · X-Ray traces · Eval set + golden Q
What Rahul owned

The useful proof is the decision surface, not only the result number.

These are the parts of the work that show production judgment: ownership, constraints, rollback, cost, and observability.

Owned scope

  • RAG architecture on AWS Bedrock with ingestion, chunking, vector storage, semantic cache, and streaming UI boundaries
  • Retrieval quality and source-attribution path so engineers could trust the answer, not only the generated text
  • Cost-per-query controls using cache hits, context sizing, and repeated-query reduction

Evidence artifacts

Hybrid RAG designOpenSearch Serverless vector storeSource attributionSemantic cacheBefore/after retrieval metrics
Constraints

What would fail first?

Every project has constraints. The useful work is naming them early enough that rollback, cost, and ownership are designed before an incident.

Data volume

2M+ documents across mixed internal sources required incremental ingestion and traceable source chunks.

Trust

Enterprise users needed source attribution before they would rely on AI answers in daily work.

Cost

Repeated questions would have made model usage expensive without semantic caching and context control.

Decision table

Trade-offs accepted on purpose.

Option
Decision
Reason
OpenSearch Serverless vs Pinecone
Chose OpenSearch for AWS-native integration, no data leaving the VPC, and hybrid keyword+vector search
OpenSearch Serverless vs Pinecone: Chose OpenSearch for AWS-native integration, no data leaving the VPC, and hybrid keyword+vector search.
Claude vs GPT-4
Claude selected for longer context window, better instruction following, and AWS Bedrock native availability
Claude vs GPT-4: Claude selected for longer context window, better instruction following, and AWS Bedrock native availability.
Semantic caching vs no caching
Added ElastiCache layer
Semantic caching vs no caching: Added ElastiCache layer. 60% cache hit rate on repeated queries, cutting LLM costs by around 60%.
Chunking strategy
512-token chunks with 50-token overlap, balanced between retrieval precision and context preservation
Chunking strategy: 512-token chunks with 50-token overlap, balanced between retrieval precision and context preservation.
Risk table

Production risks and how they were controlled.

Risk
Mitigation
Owner
Wrong answer looked confident
Returned source chunks, tracked retrieval quality, and treated unsupported answers as product risk.
AI platform + product owner
Embedding or chunk changes broke relevance
Kept chunking strategy explicit and compared relevance before rollout.
Platform engineer
LLM cost scaled with repeated questions
Added semantic cache and context-size control before exposing broader usage.
Platform + finance
Implementation approach

Small steps, visible changes, fewer surprises.

  1. 01

    Cataloged document sources. 2M+ files across S3 buckets, Confluence spaces, and Slack exports. Built Lambda-based connectors for each source with incremental sync.

  2. 02

    Implemented chunking pipeline using Step Functions: PDF extraction → text splitting (512 tokens, 50-token overlap) → embedding generation via Amazon Titan → vector storage in OpenSearch Serverless.

  3. 03

    Built semantic search layer combining vector similarity search with keyword filtering: a hybrid approach that outperformed pure vector search by 23% on relevance benchmarks.

  4. 04

    Added ElastiCache-based semantic cache layer, hashing query embeddings and returning cached responses for semantically similar questions (cosine similarity > 0.95).

  5. 05

    Deployed React chat interface with streaming responses via WebSocket API Gateway, including source attribution showing exactly which document chunks informed each answer.

Result

Before and after.

Time to Find Answer
30+ minutes
<10 seconds
LLM Cost vs no caching
baseline
~60% lower
Search Relevance (MRR@10)
0.31 (keyword)
0.78 (hybrid RAG)
Engineer Productivity
Baseline
+35% (measured via DORA)

Rollback path

Ingestion, retrieval, and answer generation were separable, so a bad retrieval change could be rolled back without disabling the full chat surface.

Cost considerations

Semantic caching and context discipline were the main cost levers; the visible result was about 60% lower repeated LLM spend in the sanitized case story.

Observability notes

Useful signals were retrieval latency, cache hit rate, no-source answers, failed ingestion jobs, and user feedback on source relevance.

Related services

Forward Deployed AI EngineeringAWS Production Architecture ReviewGenAI / RAG Production Readiness

Have a similar production problem?

Send the current architecture, bill concern, deployment pain, or GenAI reliability issue. I will help you find the first real bottleneck.