Current State Analysis: AI Course Generation & Lesson Enhancement

Date: 2025-12-01 Author: Claude Code Analysis Purpose: Implementation planning for AI-powered course generation and lesson enhancement features

Executive Summary

This document provides a comprehensive analysis of the Momentum codebase’s current state and a detailed implementation plan for two critical AI features:

AI Course Generation: Automated generation of complete courses (7/14/21 day courses) with lessons, content, and action items
Lesson Enhancement: AI-powered enhancement of existing lessons with improved content, action items, and resources

These features will showcase the application’s core AI capabilities and provide significant value to admin users.

Current State Findings
Architecture Overview
Implementation Plan
Cost Analysis
Dependencies & Risks
Implementation Timeline
Critical Files
Testing Strategy

Current State Findings

What EXISTS ✅

1. Infrastructure (Complete AWS Setup)

VPC: Fully configured with public/private subnets, NAT gateways, route tables
- Location: infrastructure/terraform/vpc.tf
Aurora PostgreSQL: Serverless v2 cluster with auto-scaling
- Location: infrastructure/terraform/rds.tf
Lambda: Execution roles and basic configuration
- Location: infrastructure/terraform/lambda.tf
API Gateway: REST API with CORS support
- Location: infrastructure/terraform/api-gateway.tf
S3 Buckets:
- momentum-dev-content: User uploads and course content
- momentum-dev-website: Static hosting
- Location: infrastructure/terraform/s3.tf

2. Database Schema (Complete)

Location: backend/shared/db/schema.sql

Tables:

courses: Title, description, category, duration, price, status, metadata
lessons: Course FK, day, title, content, action_items, resources, order_index
categories: Name, slug, description
enrollments: User-course relationships with payment tracking
progress: User-lesson completion tracking
users: Synced from Cognito, role-based access

Indexes: Optimized for common queries (category, status, user, course)

3. Backend Repositories (Full CRUD)

Location: backend/shared/repositories/

CourseRepository.ts:
- createCourse(), findById(), findAll(), updateCourse(), deleteCourse()
- Query builder with filters, pagination, sorting
LessonRepository.ts:
- createLesson(), findById(), findByCourseId(), updateLesson(), deleteLesson()
- Lesson ordering and validation
CategoryRepository.ts: Category management
EnrollmentRepository.ts: User enrollment tracking
ProgressRepository.ts: Lesson completion tracking

4. Lambda Functions (Existing)

Location: backend/functions/

courses/handler.ts: Course CRUD operations
lessons/handler.ts: Lesson CRUD operations
enrollments/handler.ts: Enrollment management
progress/handler.ts: Progress tracking
analytics/handler.ts: Usage analytics

5. Frontend Admin Panel (Complete UI)

Location: frontend/app/admin/

Course management pages (list, create, edit)
Lesson management pages
Category management
Analytics dashboard
API client functions in frontend/lib/api/

6. API Integration

RESTful API with API Gateway + Lambda
Authentication via Cognito JWT tokens
CORS configured for frontend domain
API key protection

What DOES NOT EXIST ❌

1. AWS Bedrock Integration

No Bedrock SDK imports
No IAM permissions for Bedrock
No Bedrock client implementation

2. Step Functions

No state machine definitions
No orchestration workflows
No IAM roles for Step Functions

3. SQS Queues

No async job processing infrastructure
No dead letter queues
No SQS Lambda triggers

4. AI Lambda Functions

No AI-specific handlers
No prompt engineering code
No content generation logic

5. Video Generation Integration

No third-party API clients (Synthesia/D-ID/HeyGen)
No MediaConvert setup
No video transcoding workflows

6. Course Generation Endpoints

No /courses/generate API endpoint
No generation status polling
No Step Functions execution tracking

Architecture Overview

High-Level Architecture

┌─────────────────┐
│  Admin Dashboard│
│   (Next.js)     │
└────────┬────────┘
         │
         │ POST /courses/generate
         │ {courseId: "..."}
         ▼
┌─────────────────────────────────────────────────────────────┐
│                    API Gateway (REST API)                    │
│              API Key + Cognito Authorization                 │
└────────┬────────────────────────────────────────────────────┘
         │
         │ Invoke
         ▼
┌─────────────────────────────────────────────────────────────┐
│         Lambda: AI Orchestrator (orchestrator.ts)            │
│  - Validate course exists                                    │
│  - Check course is in DRAFT status                           │
│  - Start Step Functions execution                            │
└────────┬────────────────────────────────────────────────────┘
         │
         │ StartExecution
         ▼
┌─────────────────────────────────────────────────────────────┐
│            Step Functions: Course Generation                 │
│                                                               │
│  1. Validate Input                                           │
│     └─► Lambda: outline.ts (action: "validate")              │
│                                                               │
│  2. Generate Outline                                         │
│     └─► Lambda: outline.ts (action: "generateOutline")       │
│         └─► Bedrock: Claude 3 Sonnet                         │
│             - Returns 7/14/21 lesson outlines                │
│                                                               │
│  3. Parallel Lesson Generation (Map State)                   │
│     ├─► Lambda: lesson.ts (Lesson 1) ──┐                     │
│     ├─► Lambda: lesson.ts (Lesson 2)   │                     │
│     ├─► Lambda: lesson.ts (Lesson 3)   │ Parallel            │
│     ├─► Lambda: lesson.ts (Lesson 4)   │ MaxConcurrency: 5   │
│     └─► Lambda: lesson.ts (Lesson N) ──┘                     │
│         │                                                     │
│         └─► Bedrock: Claude 3 Sonnet                         │
│             - Generate lesson content (800-1200 words)       │
│             - Generate action items (3-5 items)              │
│             - Generate key takeaways                         │
│             - Suggest resources                              │
│             └─► S3: Store content & metadata                 │
│                 └─► Aurora: Insert lesson record             │
│                                                               │
│  4. Update Course Status                                     │
│     └─► Lambda: courses/handler.ts                           │
│         └─► Aurora: UPDATE status = 'READY_FOR_REVIEW'       │
│                                                               │
└─────────────────────────────────────────────────────────────┘

Data Flow

Course Generation Flow

Admin Action: Admin clicks “Generate Course with AI” button
API Request: Frontend → API Gateway /courses/generate
Orchestration: Lambda validates course and starts Step Functions
Step Functions Workflow:
- State 1: Validate course is DRAFT status
- State 2: Generate outline (titles, topics, learning outcomes for N days)
- State 3: Parallel map over lessons (up to 5 concurrent)
  - Each iteration: Generate full lesson content via Bedrock
  - Store content in S3 (courses/{courseId}/lessons/day-{N}/content.md)
  - Insert lesson record in Aurora
- State 4: Update course status to READY_FOR_REVIEW
Frontend Polling: Poll execution status every 10 seconds
Completion: Display success message and refresh lessons list

Lesson Enhancement Flow

Admin Action: Admin clicks “Enhance Lesson” on existing lesson
API Request: Frontend → API Gateway /lessons/enhance
Lambda Processing:
- Fetch existing lesson content
- Generate enhancement prompt with goals
- Invoke Bedrock for improvements
- Update lesson record with enhanced content
Response: Return enhanced lesson to frontend

Implementation Plan

Phase 1: Core AI Course Generation (Weeks 1-2)

Week 1: Infrastructure & Backend

Day 1-2: Terraform Infrastructure

File: infrastructure/terraform/ai-services.tf (NEW)

Components to create:

IAM role policy for Lambda → Bedrock access
IAM role policy for Lambda → Step Functions access
SQS queue for video generation (Phase 3)
SQS dead letter queue
Step Functions state machine resource
IAM role for Step Functions execution
CloudWatch log groups

File: infrastructure/terraform/lambda.tf (UPDATE)

Add three new Lambda functions:

ai-generate-outline: 120s timeout, 1024 MB memory
ai-generate-lesson: 180s timeout, 1536 MB memory
ai-orchestrator: 30s timeout, 512 MB memory

Environment variables:

BEDROCK_REGION: us-east-1
BEDROCK_MODEL_ID: anthropic.claude-3-sonnet-20240229-v1:0
STEP_FUNCTION_ARN: State machine ARN
S3_CONTENT_BUCKET: Content bucket name

File: infrastructure/terraform/step-functions/course-generation.json (NEW)

State machine definition with 4 states:

ValidateInput
GenerateOutline
ParallelLessonGeneration (Map state)
UpdateCourseStatus

Retry configuration:

Max attempts: 3
Backoff rate: 2
Interval: 5 seconds

Day 3-5: Backend Lambda Functions

Directory Structure:

backend/functions/ai-generation/
├── src/
│   ├── outline.ts          # Course outline generation
│   ├── lesson.ts           # Individual lesson generation
│   ├── orchestrator.ts     # Step Functions trigger
│   └── lib/
│       ├── bedrock-client.ts    # Bedrock SDK wrapper
│       ├── prompts.ts           # AI prompt templates
│       ├── s3-storage.ts        # S3 storage helpers
│       └── types.ts             # TypeScript types
├── package.json
├── tsconfig.json
└── __tests__/
    ├── bedrock-client.test.ts
    ├── prompts.test.ts
    └── outline.test.ts

Key Implementation Details:

bedrock-client.ts:
- invokeBedrockModel(): Call Bedrock with retry logic
- parseBedrockJSON(): Extract JSON from markdown code blocks
- Error handling for rate limits and timeouts
prompts.ts:
- generateOutlinePrompt(): Course outline generation prompt
- generateLessonPrompt(): Lesson content generation prompt
- generateLessonEnhancementPrompt(): Enhancement prompt
- Structured prompts that request JSON responses
outline.ts:
- Validate course exists and is DRAFT status
- Generate outline with N lessons (7/14/21)
- Return lesson titles, topics, learning outcomes
lesson.ts:
- Generate full lesson content (800-1200 words markdown)
- Generate 3-5 specific action items
- Generate key takeaways
- Suggest 2-3 learning resources
- Store content in S3
- Insert lesson record in Aurora
orchestrator.ts:
- API Gateway handler
- Validate request body
- Start Step Functions execution
- Return execution ARN for status polling

Day 6-7: API Gateway Integration

File: infrastructure/terraform/api-gateway.tf (UPDATE)

Add new resource:

/courses/generate (POST)
API key required
CORS enabled
Lambda proxy integration to ai-orchestrator

Add new endpoint:

/courses/generation-status (GET)
Query parameter: executionArn
Returns Step Functions execution status

Week 2: Frontend Integration & Testing

Day 8-9: Frontend API Client

File: frontend/lib/api/ai-generation.ts (NEW)

Functions:

generateCourseContent(courseId): Start generation
checkGenerationStatus(executionArn): Poll status
pollGenerationStatus(): Auto-polling with callback

File: frontend/components/admin/GenerateCourseButton.tsx (NEW)

Features:

“Generate Course with AI” button with sparkle icon
Loading state with progress bar
Status updates during generation
Success/error messages
Auto-refresh on completion

File: frontend/app/admin/courses/[id]/edit/page.tsx (UPDATE)

Integration:

Add GenerateCourseButton component
Show only for DRAFT courses
Display below course details form
Hide after generation completes

Unit Tests:

Bedrock client JSON parsing
Prompt template generation
S3 storage helpers

Integration Tests:

Lambda function handlers with mocked Bedrock
Step Functions state machine with LocalStack

E2E Tests:

Admin signs in
Creates draft course
Clicks generate button
Waits for completion
Verifies lessons created

Phase 2: Lesson Enhancement (Weeks 3-4)

Week 3: Enhancement Backend

Day 11-12: Enhancement Lambda

File: backend/functions/ai-generation/src/enhance-lesson.ts (NEW)

Features:

Fetch existing lesson content
Accept enhancement goals array
Generate enhancement prompt
Invoke Bedrock for improvements
Update lesson with enhanced content
Track enhancement history in metadata

Default enhancement goals:

Improve clarity and explanations
Add more practical examples
Enhance action items specificity
Suggest better learning resources

File: infrastructure/terraform/lambda.tf (UPDATE)

Add Lambda function:

ai-enhance-lesson: 180s timeout, 1536 MB memory

File: infrastructure/terraform/api-gateway.tf (UPDATE)

Add endpoint:

/lessons/enhance (POST)
Body: {lessonId, enhancementGoals}

Day 13-14: Enhancement Frontend

File: frontend/components/admin/EnhanceLessonButton.tsx (NEW)

Features:

“Enhance with AI” button
Optional: Enhancement goals checkboxes
Loading state
Side-by-side diff view (before/after)
Confirm/reject changes

File: frontend/app/admin/lessons/[id]/edit/page.tsx (UPDATE)

Integration:

Add EnhanceLessonButton component
Show enhancement history
Display “Enhanced by AI” badge

Week 4: Polish & Documentation

Day 15-16: Testing & Documentation

Tests:

Enhancement Lambda unit tests
Frontend component tests
E2E enhancement flow

Documentation:

Admin user guide for AI features
AI prompt engineering guide
Troubleshooting guide

Phase 3: Video Generation (Optional - Weeks 5-6)

NOT INCLUDED IN MVP - Future Enhancement

This phase would add:

Integration with Synthesia/D-ID/HeyGen APIs
SQS-based async video processing
MediaConvert for video transcoding
CloudFront for video delivery

Estimated additional cost: $10-15 per video (API fees)

Cost Analysis

Per-Course Cost Breakdown

7-Day Course:

Bedrock (Outline): ~500 input + 2,500 output tokens = $0.015
Bedrock (7 Lessons): 7 × $0.015 = $0.105
AWS Services: Step Functions ($0.001) + Lambda ($0.005) + S3 ($0.001) = $0.007
Total: ~$0.13 per 7-day course

14-Day Course:

Bedrock (Outline): ~$0.015
Bedrock (14 Lessons): 14 × $0.025 = $0.35
AWS Services: ~$0.007
Total: ~$0.37 per 14-day course

21-Day Course:

Bedrock (Outline): ~$0.015
Bedrock (21 Lessons): 21 × $0.035 = $0.735
AWS Services: ~$0.007
Total: ~$0.76 per 21-day course

Bedrock Pricing (Claude 3 Sonnet)

Input tokens: $3.00 per million tokens
Output tokens: $15.00 per million tokens

Estimated Monthly Costs (100 courses/month)

Assuming distribution: 40% 7-day, 40% 14-day, 20% 21-day

40 × 7-day courses: $5.20
40 × 14-day courses: $14.80
20 × 21-day courses: $15.20
Total: ~$35/month for AI text generation

Cost Optimization Strategies

Use Claude 3 Haiku for simple tasks ($0.80/$4.00 per million tokens)
Cache common prompts (future Bedrock feature)
Batch lesson generation where possible
Monitor and optimize token usage

Dependencies & Risks

Critical Dependencies

1. AWS Bedrock Access ⚠️

Status: Required before implementation

Action needed:

Go to AWS Console → Bedrock → Model access
Request access to Claude 3 Sonnet and Claude 3 Haiku
Wait for approval (usually 1-2 business days)

2. IAM Permissions

Status: Will be created via Terraform

Required permissions:

bedrock:InvokeModel
bedrock:InvokeModelWithResponseStream
states:StartExecution
states:DescribeExecution
sqs:SendMessage, sqs:ReceiveMessage, sqs:DeleteMessage
s3:PutObject (content bucket)

3. VPC Configuration

Status: Exists

Lambda functions need:

VPC access for Aurora database
NAT Gateway for external API calls (Bedrock)
Security group rules

4. Terraform State

Status: Using local state

Consideration: Migrate to S3 backend for team collaboration

Technical Risks

Risk 1: Bedrock Rate Limits

Severity: Medium Likelihood: High (if generating many courses simultaneously)

Limit: 5 requests per second per account
Mitigation:
- Step Functions Map state MaxConcurrency: 5
- Exponential backoff retry logic
- Queue-based rate limiting

Risk 2: Lambda Timeouts

Severity: Medium Likelihood: Medium

Issue: Bedrock may take 30-60s per lesson
Mitigation:
- Increased Lambda timeout to 180s (3 minutes)
- Step Functions handles orchestration (15 min timeout)
- Async processing via SQS if needed

Risk 3: JSON Parsing Errors

Severity: Low Likelihood: Medium

Issue: Bedrock may return malformed JSON or wrapped in markdown
Mitigation:
- Robust parsing logic (extract from code blocks)
- Retry with improved prompt
- Fallback to text parsing
- Validation before database insert

Risk 4: Token Limits

Severity: Low Likelihood: Low

Issue: Claude 3 Sonnet has 200K input + 4K output token limits
Mitigation:
- Lessons designed for 1500-2000 tokens output
- Well within limits
- Chunking strategy if needed

Risk 5: Cost Overruns

Severity: Low Likelihood: Low

Issue: Unexpected high usage
Mitigation:
- CloudWatch billing alarms at $50, $100, $200
- Admin rate limiting (max N courses per day)
- Token usage monitoring and alerts

Risk 6: Content Quality

Severity: Medium Likelihood: Medium

Issue: AI-generated content may need editing
Mitigation:
- Course status: READY_FOR_REVIEW (not auto-publish)
- Admin review required before PUBLISHED
- Enhancement feature for improvements
- A/B testing of prompts

External Dependencies

Bedrock Service Availability

SLA: 99.9% uptime
Fallback: Retry logic, admin manual creation

Database Connectivity

Dependency: Aurora cluster in private subnet
Requirement: Lambda in VPC with proper security groups

Operational Risks

Monitoring & Debugging

Challenge: Distributed workflow across Lambda + Step Functions
Solution:
- AWS X-Ray tracing enabled
- CloudWatch Logs for each Lambda
- Step Functions execution history
- Custom CloudWatch dashboard

Implementation Timeline

Week 1-2: Core AI Course Generation

Week 1: Infrastructure & Backend

Day	Task	Deliverable	Owner
1	Terraform: IAM roles & policies	`ai-services.tf`	Backend
1	Terraform: SQS queues	SQS resources	Backend
2	Terraform: Step Functions	State machine JSON	Backend
2	Terraform: Lambda functions	Lambda resources	Backend
3	Backend: Bedrock client	`bedrock-client.ts`	Backend
3	Backend: Prompt templates	`prompts.ts`	Backend
4	Backend: Outline Lambda	`outline.ts`	Backend
4	Backend: Lesson Lambda	`lesson.ts`	Backend
5	Backend: Orchestrator Lambda	`orchestrator.ts`	Backend
5	Backend: S3 storage helpers	`s3-storage.ts`	Backend

Week 2: Frontend & Testing

Day	Task	Deliverable	Owner
6	API Gateway: New endpoints	Terraform update	Backend
6	Frontend: API client	`ai-generation.ts`	Frontend
7	Frontend: Generate button	`GenerateCourseButton.tsx`	Frontend
7	Frontend: Integration	Course edit page update	Frontend
8	Unit tests	Jest test suites	Both
9	Integration tests	Lambda test suites	Backend
9	E2E tests	Playwright tests	Frontend
10	Terraform apply	Deploy to dev	DevOps
10	Manual testing	Test report	QA

Week 3-4: Lesson Enhancement

Week 3: Enhancement Backend

Day	Task	Deliverable	Owner
11	Enhancement Lambda	`enhance-lesson.ts`	Backend
11	Enhancement prompts	Prompt templates	Backend
12	API Gateway endpoint	`/lessons/enhance`	Backend
12	Unit tests	Test suites	Backend
13	Deploy to dev	Terraform apply	DevOps

Week 4: Enhancement Frontend & Polish

Day	Task	Deliverable	Owner
13	Frontend: Enhance button	`EnhanceLessonButton.tsx`	Frontend
14	Frontend: Diff view	Before/after comparison	Frontend
14	Integration testing	Full flow testing	QA
15	Documentation	Admin user guide	Docs
15	Documentation	AI prompt guide	Docs
16	Documentation	Troubleshooting guide	Docs
16	Final testing	Production readiness	QA

Optional: Week 5-6: Video Generation

NOT IN MVP - Future Enhancement

This would add:

Video API integration (Synthesia/D-ID)
SQS worker for async processing
MediaConvert for transcoding
Video management UI

Critical Files

Must Create (NEW Files)

Infrastructure (Terraform)

infrastructure/terraform/ai-services.tf
- Priority: P0 (Blocker)
- Lines: ~350
- Components: IAM roles, SQS, Step Functions, CloudWatch
- Dependencies: None
infrastructure/terraform/step-functions/course-generation.json
- Priority: P0 (Blocker)
- Lines: ~150
- Components: State machine definition
- Dependencies: Lambda ARNs
infrastructure/terraform/monitoring.tf
- Priority: P1 (Important)
- Lines: ~100
- Components: CloudWatch alarms, SNS topics
- Dependencies: Lambda, Step Functions

Backend Lambda Functions

backend/functions/ai-generation/src/lib/bedrock-client.ts
- Priority: P0 (Blocker)
- Lines: ~150
- Components: Bedrock SDK wrapper, retry logic
- Dependencies: @aws-sdk/client-bedrock-runtime
- MOST CRITICAL: All AI features depend on this
backend/functions/ai-generation/src/lib/prompts.ts
- Priority: P0 (Blocker)
- Lines: ~300
- Components: Prompt templates for outline, lessons, enhancement
- Dependencies: None
- MOST CRITICAL: Determines content quality
backend/functions/ai-generation/src/outline.ts
- Priority: P0 (Blocker)
- Lines: ~200
- Components: Course outline generation
- Dependencies: bedrock-client, prompts, CourseRepository
backend/functions/ai-generation/src/lesson.ts
- Priority: P0 (Blocker)
- Lines: ~250
- Components: Individual lesson generation
- Dependencies: bedrock-client, prompts, LessonRepository, s3-storage
backend/functions/ai-generation/src/orchestrator.ts
- Priority: P0 (Blocker)
- Lines: ~150
- Components: Step Functions trigger, API handler
- Dependencies: @aws-sdk/client-sfn, CourseRepository
backend/functions/ai-generation/src/lib/s3-storage.ts
- Priority: P1 (Important)
- Lines: ~100
- Components: S3 content storage helpers
- Dependencies: @aws-sdk/client-s3
backend/functions/ai-generation/src/enhance-lesson.ts (Phase 2)
- Priority: P2 (Nice to have)
- Lines: ~200
- Components: Lesson enhancement handler
- Dependencies: bedrock-client, prompts, LessonRepository

Frontend

frontend/lib/api/ai-generation.ts
- Priority: P0 (Blocker)
- Lines: ~150
- Components: API client for generation endpoints
- Dependencies: auth/cognito-client
frontend/components/admin/GenerateCourseButton.tsx
- Priority: P0 (Blocker)
- Lines: ~200
- Components: UI for triggering generation
- Dependencies: lucide-react, ai-generation API
frontend/components/admin/EnhanceLessonButton.tsx (Phase 2)
- Priority: P2 (Nice to have)
- Lines: ~180
- Components: UI for lesson enhancement
- Dependencies: lucide-react, ai-generation API

Must Update (EXISTING Files)

infrastructure/terraform/lambda.tf
- Priority: P0 (Blocker)
- Changes: Add 3 Lambda function resources
- Lines added: ~200
infrastructure/terraform/api-gateway.tf
- Priority: P0 (Blocker)
- Changes: Add /courses/generate and /lessons/enhance endpoints
- Lines added: ~100
frontend/app/admin/courses/[id]/edit/page.tsx
- Priority: P1 (Important)
- Changes: Add GenerateCourseButton component
- Lines added: ~20
frontend/app/admin/lessons/[id]/edit/page.tsx (Phase 2)
- Priority: P2 (Nice to have)
- Changes: Add EnhanceLessonButton component
- Lines added: ~20
backend/shared/repositories/LessonRepository.ts
- Priority: P1 (Important)
- Changes: Add methods for AI metadata handling
- Lines added: ~50

Testing Files

backend/functions/ai-generation/__tests__/bedrock-client.test.ts
- Priority: P1 (Important)
- Lines: ~150
- Components: Unit tests for Bedrock client
tests/e2e/admin-ai-generation.spec.ts
- Priority: P1 (Important)
- Lines: ~100
- Components: E2E tests for generation flow

Testing Strategy

Unit Tests

Backend Tests

Location: backend/functions/ai-generation/__tests__/

Test Files:

bedrock-client.test.ts
- invokeBedrockModel() with mocked SDK
- parseBedrockJSON() with various formats
- Error handling for rate limits
- Retry logic validation
prompts.test.ts
- generateOutlinePrompt() output format
- generateLessonPrompt() with different contexts
- generateLessonEnhancementPrompt() structure
- Prompt length validation
s3-storage.test.ts
- storeLessonContent() with mocked S3
- storeLessonMetadata() with mocked S3
- Error handling for S3 failures
outline.test.ts
- validateCourse() with valid/invalid courses
- generateOutline() with mocked Bedrock
- JSON parsing validation
- Lesson count verification
lesson.test.ts
- handler() with mocked dependencies
- S3 storage integration
- Database insertion validation
- Error handling

Running Tests:

cd backend/functions/ai-generation
npm test
npm run test:coverage  # Aim for 80%+ coverage

Frontend Tests

Location: frontend/__tests__/components/admin/

Test Files:

GenerateCourseButton.test.tsx
- Button render in idle state
- Loading state display
- Success state display
- Error state display
- Progress bar updates
- API call mocking

Running Tests:

cd frontend
npm test

Integration Tests

Lambda Integration Tests

Location: backend/functions/ai-generation/__tests__/integration/

Test Scenarios:

Full course generation flow:
- Create draft course in test database
- Call outline Lambda with real Bedrock (dev environment)
- Verify outline structure
- Call lesson Lambda for each day
- Verify S3 storage
- Verify database records
Step Functions local testing:
- Use AWS SAM or LocalStack
- Mock Bedrock responses
- Verify state transitions
- Test parallel lesson generation
- Test error handling and retries

Running Tests:

# Using LocalStack
localstack start
cd infrastructure/terraform
terraform init -backend-config=localstack
terraform apply

cd backend/functions/ai-generation
npm run test:integration

E2E Tests (Playwright)

Location: tests/e2e/admin-ai-generation.spec.ts

Test Scenarios:

Complete course generation flow:
- Admin signs in
- Navigates to course creation
- Creates draft course (7-day)
- Clicks “Generate Course with AI”
- Waits for generation (max 2 minutes)
- Verifies success message
- Navigates to lessons page
- Verifies 7 lessons exist
- Opens lesson detail
- Verifies content, action items, resources
Lesson enhancement flow:
- Admin opens existing lesson
- Clicks “Enhance with AI”
- Waits for enhancement
- Reviews before/after diff
- Confirms changes
- Verifies lesson updated
Error handling:
- Test with invalid course ID
- Test with non-DRAFT course
- Test network failure scenario
- Verify error messages display

Running E2E Tests:

cd tests/e2e
npx playwright test admin-ai-generation.spec.ts
npx playwright test --ui  # Debug mode

Manual Testing Checklist

Pre-deployment:

Bedrock access approved in AWS account
Terraform plan shows all expected resources
Environment variables set in Lambda
API Gateway endpoints deployed
Frontend environment variables set

Post-deployment:

Performance Testing

Load Testing:

# Generate 10 courses concurrently
for i in {1..10}; do
  curl -X POST https://api.momentum.cloudnnj.com/courses/generate \
    -H "x-api-key: $API_KEY" \
    -H "Authorization: Bearer $TOKEN" \
    -d "{\"courseId\": \"course-$i\"}" &
done

Metrics to Monitor:

Step Functions execution time (target: <5 minutes for 14-day course)
Lambda cold start time (target: <2 seconds)
Bedrock response time (target: <30 seconds per lesson)
Token usage per course (for cost tracking)
Error rate (target: <1%)

Monitoring & Observability

CloudWatch Dashboards

Create Custom Dashboard: Momentum-AI-Generation

Widgets:

Execution Metrics:
- Step Functions executions started
- Step Functions executions succeeded
- Step Functions executions failed
- Average execution duration
Lambda Metrics:
- AI Lambda invocations
- AI Lambda errors
- AI Lambda duration (p50, p95, p99)
- AI Lambda throttles
Bedrock Metrics:
- Total tokens consumed (input + output)
- Average response time
- Error rate
Cost Metrics:
- Daily Bedrock costs
- Daily Lambda costs
- Daily Step Functions costs

CloudWatch Alarms

Critical Alarms:

AI Lambda Errors > 5 in 5 minutes
- Action: Send SNS to admin email
- Threshold: 5 errors
- Period: 5 minutes
Step Functions Failed Executions > 1
- Action: Send SNS to admin email
- Threshold: 1 failure
- Period: 5 minutes
Bedrock Token Usage > 1M per day
- Action: Send SNS to admin email
- Threshold: 1,000,000 tokens
- Period: 24 hours
Daily AI Costs > $50
- Action: Send SNS to admin email + disable generation
- Threshold: $50
- Period: 24 hours

Logging Strategy

Log Levels:

ERROR: Failures, exceptions, unrecoverable errors
WARN: Retry attempts, degraded performance, rate limits
INFO: Execution starts, completions, major state changes
DEBUG: Detailed execution flow (dev only)

Structured Logging Format:

{
  "level": "INFO",
  "timestamp": "2025-12-01T10:00:00Z",
  "requestId": "abc-123",
  "courseId": "course-456",
  "executionArn": "arn:aws:states:...",
  "message": "Course generation started",
  "metadata": {
    "durationDays": 14,
    "category": "business"
  }
}

Log Retention:

Production: 30 days
Development: 7 days

Security Considerations

IAM Least Privilege

Lambda Execution Role:

Only Bedrock models used by the application
Only S3 bucket and paths for course content
Only Step Functions state machines for generation
Only Aurora clusters in the VPC

Secrets Management

Sensitive Data:

Bedrock model IDs: Environment variables (not sensitive)
S3 bucket names: Environment variables
Database credentials: AWS Secrets Manager
API keys: AWS Secrets Manager

Input Validation

Course Generation Request:

Validate courseId is UUID format
Verify course exists in database
Verify user is admin role (Cognito JWT)
Verify course is DRAFT status

Lesson Enhancement Request:

Validate lessonId is UUID format
Verify lesson exists in database
Verify user is admin role
Sanitize enhancementGoals array

Output Sanitization

AI-Generated Content:

Strip any malicious HTML/JavaScript
Validate markdown format
Check for PII or sensitive data leaks
Profanity filter (optional)

Rate Limiting

API Gateway:

100 requests per minute per API key
10 course generations per hour per admin user

Bedrock:

Respect 5 requests/second service limit
Implement exponential backoff

Rollout Strategy

Phase 1: Development Environment

Timeline: Week 1-2

Steps:

Deploy infrastructure to dev account
Manual testing by development team
Generate 5 test courses
Review content quality
Fix bugs and iterate

Success Criteria:

5 courses generated successfully
Content quality reviewed and approved
No errors in CloudWatch logs
Costs within expected range

Phase 2: Beta Testing

Timeline: Week 3

Steps:

Deploy to production (feature flag OFF)
Enable for 2-3 admin users
Generate 10-15 real courses
Collect feedback
Iterate on prompts and UX

Success Criteria:

10 courses generated
Admin feedback positive (NPS > 8)
<5% content requires major edits
No production errors

Phase 3: General Availability

Timeline: Week 4

Steps:

Enable feature for all admin users
Add documentation to admin panel
Monitor usage and costs
Optimize based on usage patterns

Success Criteria:

All admins can access feature
Documentation complete
Monitoring dashboards live
Cost alarms configured

Future Enhancements

Short-term (3-6 months)

Batch Course Generation:
- Generate multiple courses at once
- CSV import for bulk generation
- Category-based templates
Content Templates:
- Industry-specific templates
- Difficulty level templates
- Learning style templates
A/B Testing:
- Test different prompts
- Measure content quality metrics
- Optimize for user engagement
Enhanced Analytics:
- Track AI-generated vs manual courses
- Compare completion rates
- Measure content quality over time

Long-term (6-12 months)

Video Generation:
- Integrate Synthesia/D-ID
- Automatic video narration
- Video thumbnail generation
Multi-language Support:
- Translate courses automatically
- Localize content for regions
- Cultural adaptation
Personalization:
- Adapt content to user level
- Personalized action items
- Adaptive learning paths
Interactive Elements:
- Generate quizzes automatically
- Create interactive exercises
- Generate discussion prompts

Appendix

A. Bedrock Model Comparison

Model	Input Cost	Output Cost	Context	Speed	Use Case
Claude 3 Opus	$15/M	$75/M	200K	Slow	Complex reasoning
Claude 3 Sonnet	$3/M	$15/M	200K	Medium	Course generation
Claude 3 Haiku	$0.80/M	$4/M	200K	Fast	Simple tasks

Recommendation: Use Sonnet for course/lesson generation, Haiku for enhancements.

B. Alternative Architectures Considered

Option 1: Direct Lambda (Without Step Functions)

Pros: Simpler, fewer components Cons: No orchestration, no parallel processing, harder error handling Decision: Rejected - Step Functions provides better reliability

Option 2: ECS Batch Jobs

Pros: No Lambda timeout limits, can use larger compute Cons: More complex, slower cold starts, higher costs for small jobs Decision: Rejected - Lambda sufficient for text generation

Option 3: SQS + Lambda Workers

Pros: Simple async processing, good for high volume Cons: No built-in orchestration, manual state tracking Decision: Partially adopted - Will use for video generation (Phase 3)

C. Prompt Engineering Best Practices

Be Specific: Define exact output format (JSON structure)
Provide Examples: Include example responses in prompts
Set Constraints: Specify word counts, tone, style
Use System Prompts: Separate role definition from task
Iterate: A/B test prompts and measure quality
Handle Errors: Request JSON only, no markdown wrappers

D. Useful Commands

Check Bedrock Model Access:

aws bedrock list-foundation-models \
  --region us-east-1 \
  --query 'modelSummaries[?contains(modelId, `claude`)].modelId'

Start Step Functions Execution:

aws stepfunctions start-execution \
  --state-machine-arn arn:aws:states:us-east-1:123456789012:stateMachine:momentum-course-generation-dev \
  --input '{"courseId":"uuid-here"}'

Check Execution Status:

aws stepfunctions describe-execution \
  --execution-arn arn:aws:states:us-east-1:123456789012:execution:momentum-course-generation-dev:...

Query CloudWatch Logs:

aws logs tail /aws/lambda/momentum-ai-outline-dev --follow

Check Bedrock Usage:

aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name InvocationCount \
  --start-time 2025-12-01T00:00:00Z \
  --end-time 2025-12-01T23:59:59Z \
  --period 3600 \
  --statistics Sum

Conclusion

This implementation plan provides a comprehensive roadmap for building AI-powered course generation and lesson enhancement features in the Momentum platform. The plan is:

Achievable: Builds on existing infrastructure and patterns
Scalable: Uses serverless architecture with auto-scaling
Cost-effective: ~$0.37 per 14-day course
Maintainable: Clean separation of concerns, well-tested
Production-ready: Includes monitoring, error handling, security

Next Steps:

Review and approve this plan
Request AWS Bedrock model access
Create feature branches for each phase
Begin Phase 1 infrastructure setup

Questions or Concerns? Please reach out to the development team for clarification on any aspect of this implementation plan.

Document Version: 1.0 Last Updated: 2025-12-01 Status: Ready for Implementation

Current State Analysis: AI Course Generation & Lesson Enhancement

Executive Summary

Table of Contents

Current State Findings

What EXISTS ✅

1. Infrastructure (Complete AWS Setup)

2. Database Schema (Complete)

3. Backend Repositories (Full CRUD)

4. Lambda Functions (Existing)

5. Frontend Admin Panel (Complete UI)

6. API Integration

What DOES NOT EXIST ❌

1. AWS Bedrock Integration

2. Step Functions

3. SQS Queues

4. AI Lambda Functions

5. Video Generation Integration

6. Course Generation Endpoints

Architecture Overview

High-Level Architecture

Data Flow

Course Generation Flow

Lesson Enhancement Flow

Implementation Plan

Phase 1: Core AI Course Generation (Weeks 1-2)

Week 1: Infrastructure & Backend

Day 1-2: Terraform Infrastructure

Day 3-5: Backend Lambda Functions

Day 6-7: API Gateway Integration

Week 2: Frontend Integration & Testing

Day 8-9: Frontend API Client

Day 10: Testing & Refinement

Phase 2: Lesson Enhancement (Weeks 3-4)

Week 3: Enhancement Backend

Day 11-12: Enhancement Lambda

Day 13-14: Enhancement Frontend

Week 4: Polish & Documentation

Day 15-16: Testing & Documentation

Phase 3: Video Generation (Optional - Weeks 5-6)

Cost Analysis

Per-Course Cost Breakdown

7-Day Course:

14-Day Course:

21-Day Course:

Bedrock Pricing (Claude 3 Sonnet)

Estimated Monthly Costs (100 courses/month)

Cost Optimization Strategies

Dependencies & Risks

Critical Dependencies

1. AWS Bedrock Access ⚠️

2. IAM Permissions

3. VPC Configuration

4. Terraform State

Technical Risks

Risk 1: Bedrock Rate Limits

Risk 2: Lambda Timeouts

Risk 3: JSON Parsing Errors

Risk 4: Token Limits

Risk 5: Cost Overruns

Risk 6: Content Quality

External Dependencies

Bedrock Service Availability

Database Connectivity

Operational Risks

Monitoring & Debugging

Implementation Timeline

Week 1-2: Core AI Course Generation

Week 3-4: Lesson Enhancement

Optional: Week 5-6: Video Generation

Critical Files

Must Create (NEW Files)

Infrastructure (Terraform)

Backend Lambda Functions

Frontend

Must Update (EXISTING Files)

Testing Files

Testing Strategy

Unit Tests

Backend Tests

Frontend Tests