Skip to content
Back to case studies
InternalGenAI / AI2025

LLM Infrastructure Training

Valid syntax, dangerous IaC → RLHF + curated patterns; quality 42% → 89%, security violations <5% of gens.

Role
LLM Infrastructure Trainer
Context
AI / Enterprise
Duration
8 months
Team
Specialized trainer (me) + Amazon AI team

Tech stack

  • AWS CDK
  • CloudFormation
  • TypeScript
  • Python
  • AWS Bedrock
  • Lambda
  • VPC
  • IAM
  • Terraform
  • RLHF
Executive summary

Business problem and production context.

Business problem

Amazon needed LLMs that could generate production-quality AWS CDK and CloudFormation code. Existing models produced syntactically correct but architecturally flawed infrastructure code, missing security best practices, cost optimization patterns, and Well-Architected Framework alignment.

Architecture decision

Created high-quality training datasets of AWS infrastructure patterns covering VPC design, IAM policies, Lambda configurations, and multi-account setups. Designed reusable TypeScript CDK constructs as reference implementations. Validated AI-generated infrastructure code against security, cost efficiency, and Well-Architected principles through systematic RLHF feedback loops.

awsAWS Account / Turing-AmazonRequestResponsePROMPT + DATASETTRAINING + EVALMODEL + ARTIFACTS12345678S3prompt datasetStep Functionstraining orchestrationSageMakerRLHF jobsLambdareward modelBatchrollouts on GPUEventBridgeeval triggersBedrockbase + tuned modelS3checkpointsAthenaeval metricsCROSS-CUTTING: Curated CDK ref pool · Static IaC linter · Security violation <5% · Per-job spend cap
What Rahul owned

The useful proof is the decision surface, not only the result number.

These are the parts of the work that show production judgment: ownership, constraints, rollback, cost, and observability.

Owned scope

  • Infrastructure review and training feedback for LLM-related IaC patterns
  • AWS infrastructure judgment applied to model-training and evaluation workflows
  • Quality feedback on infrastructure tasks where correctness, security, and maintainability mattered

Evidence artifacts

Internal proof levelAWS IaC reviewLLM infrastructure trainingProduction-risk feedback
Constraints

What would fail first?

Every project has constraints. The useful work is naming them early enough that rollback, cost, and ownership are designed before an incident.

Internal proof

Details are intentionally limited because the work is internal/NDA-sensitive.

IaC correctness

Training tasks needed secure and maintainable infrastructure patterns, not only syntactically valid snippets.

Evaluation quality

Feedback had to distinguish acceptable cloud patterns from risky production shortcuts.

Decision table

Trade-offs accepted on purpose.

Option
Decision
Reason
CDK TypeScript vs Python
Chose TypeScript for stronger type safety and better IDE support in training data
CDK TypeScript vs Python: Chose TypeScript for stronger type safety and better IDE support in training data. Models learn better from typed code.
Broad coverage vs deep specialization
Focused on the 20 most common AWS patterns rather than covering all 200+ services
Broad coverage vs deep specialization: Focused on the 20 most common AWS patterns rather than covering all 200+ services. Higher quality on critical paths.
Manual RLHF vs automated evaluation
Combined both, with automated checks for syntax/security and manual review for architectural quality and rationale
Manual RLHF vs automated evaluation: Combined both, with automated checks for syntax/security and manual review for architectural quality and rationale.
Risk table

Production risks and how they were controlled.

Risk
Mitigation
Owner
Leaking confidential detail
Keep public proof at role/context level and discuss deeper examples only when allowed.
Rahul
Training examples normalized bad cloud practice
Reviewed IaC for security, maintainability, and production fit.
Infrastructure reviewer
AI output judged only by syntax
Evaluated architecture trade-offs and operational risk, not only code shape.
LLM infra trainer
Implementation approach

Small steps, visible changes, fewer surprises.

  1. 01

    Audited 500+ existing CDK/CloudFormation templates to identify common anti-patterns in AI-generated infrastructure code: security gaps, missing encryption, overly permissive IAM roles.

  2. 02

    Created structured training datasets: correct CDK constructs paired with explanations of why each design decision was made (security, cost, reliability rationale).

  3. 03

    Designed reusable TypeScript CDK constructs following AWS Well-Architected Framework. These became reference implementations for model training.

  4. 04

    Built RLHF feedback pipeline: reviewed model outputs, ranked by infrastructure quality, flagged security violations, and provided corrective signals.

  5. 05

    Validated trained model outputs against production deployment scenarios, measuring IAM policy correctness, resource tagging compliance, and cost estimation accuracy.

Result

Before and after.

CDK Code Quality Score
42% (baseline)
89% (post-RLHF)
Security Violations in Output
34% of generations
<5% of generations
Well-Architected Compliance
Partial
Full alignment
Training Data Coverage
0 patterns
500+ validated patterns

Rollback path

Public details are constrained; the transferable lesson is to treat generated IaC as production-risky until reviewed.

Cost considerations

The public value is expertise signal rather than client savings disclosure.

Observability notes

The work reinforces why AI-generated infrastructure must be evaluated against operations, security, and maintenance.

Related services

Forward Deployed AI EngineeringAWS Production Architecture ReviewGenAI / RAG Production Readiness

Have a similar production problem?

Send the current architecture, bill concern, deployment pain, or GenAI reliability issue. I will help you find the first real bottleneck.