📖 Guide 04 — EPICBackend Deep Dive (Node.js Lambda)

What you’ll learn: Every folder, every file, every important function in the EPICBackend module.
Prerequisite: Read Guide 01 (JavaScript Basics) first.

Section 1 — EPICBackend Overview

The EPICBackend is a Node.js application that runs as AWS Lambda functions. It handles all REST API calls from the frontend and stores/retrieves data from DynamoDB and MySQL.

Location: /home/theja/EPIC/EPIC/src/EPICBackend/src/epiclambda/

Folder Structure:

epiclambda/
├── api/           ← REST API handlers (one file per resource)
├── operations/    ← Database query logic
├── clients/       ← Database connection wrappers
├── common/        ← Constants and utility functions
├── notification/  ← Email sending logic
├── sqs/           ← SQS message handling
├── stream/        ← DynamoDB stream processing
├── Script/        ← One-time scripts and data migration tools
└── templates/     ← SIM ticket markdown templates

Section 2 — API Handlers (The api/ Folder)

Each file in api/ handles CRUD operations for one resource. Every method follows the same pattern.

Pattern for ALL API handlers:

static async methodName(event) {
    // 1. Parse and validate input
    try {
        body = JSON.parse(event.body);
        if (!required fields present) throw TypeError();
        // extract fields
    } catch (err) {
        return Util.handleErr(400, "Bad request", err);
    }
    
    // 2. Do database work
    try {
        const result = await operations.doSomething(data);
        return Util.handleResponse(200, JSON.stringify(result));
    } catch (err) {
        return Util.handleErr(503, "Server error", err);
    }
}

Fleet.js — Fleet Management

A Fleet is a group of servers running a service in one region (e.g., “FORTRESSService-NA-X0”).

Methods:

Key concept — Version Tracking:

// Fleet uses versioning. Every update creates a NEW version.
// DynamoDB stores ALL versions.
// VersionId 1 = original, VersionId 2 = first update, etc.
// "LatestVersionId" field always points to the current version.

async createFleet(event) {
    // ...
    let newVersionId = 1;  // first version
    const item = {
        FleetId: fleetId,
        VersionId: newVersionId,
        LatestVersionId: newVersionId,
        // ... rest of fleet data
    };
    await dynamodb.put(item);
}

Service.js — Service Management

A Service is an Amazon microservice that owns fleets (e.g., “FORTRESSService”).

Methods:

Service Types:

// From ServiceConstants.js
SERVICE_TYPE_REGISTERED = 'Registered';  // fully onboarded to EPIC
SERVICE_TYPE_ONBOARDING = 'Onboarding';  // currently onboarding

Event.js — Event Management

An Event is a peak event like “PrimeDay2024” or “BlackFriday2024”.

Methods:

EventPlan.js — Milestone Tracking ⭐ (Very Important)

An EventPlan tracks milestone progress for one fleet × one event combination.

EventPlanId = “{EventId}#{FleetId}” (e.g., “PD2024#RIPE-NA”)

Methods:

Milestone Status Values:

• NotStarted — milestone hasn’t begun
• InProgress — work is underway
• Completed — done!
• PendingOnEPIC — EPIC system needs to act
• PendingOnServiceOwner — owner needs to act

Milestone Flow:

GatherProjections → HardwareOrder → HardwareFulfillment → CommunicateTPM → ThrottlingUpdate

HOTW.js — Hardware Order Management ⭐ (Core)

The HOTW (Head of the Week) API manages hardware ordering data.

Methods:

Key data structures:

// What createOrUpdateDashboardDetailsAndAsgDetails receives:
{
    "ServiceId": "RIPE",
    "FleetId": "RIPE-NA",
    "EventId": "PD2024",
    "Purpose": "Upscaling",
    "CapacityOverrideSum": "50",
    "ASGDetails": [
        {
            "AutoScalingGroupName": "RIPE/NA/PROD/1a",
            "AsgTag": "1a",
            "HostType": "c5.2xlarge",
            "EAPOnboardingStatus": true,
            "EAPEnableStatus": true,
            "AsgCapacityLowerBound": 5
        }
    ]
}

// What createHotwExecutionDetails receives:
{
    "FleetId": "RIPE-NA",
    "EventId": "PD2024",
    "RunId": 2,
    "Status": "Success",  // or "Fail" or "PartialSuccess"
    "Reasoning": "EAP enabled successfully",
    "HotwExecutionDetails": {
        "PeakTPM": 50000,
        "BAUTPM": 30000,
        "CTPeakFactor": 1.5,
        "BufferFactor": 1.1,
        "RequiredHosts": 200,
        "HostsPresentInApollo": 150,
        "CapacityOverrideSum": 50,
        "PendingEPICFMCHostOrder": 0,
        "HostThroughPutTPM": 250
    },
    "CapacityOverrideDetails": [ ... ],
    "FulfillmentDetails": [ ... ],
    "ASGDetails": [ ... ]
}

Projection.js — Traffic Projections

Projections are estimates of how much traffic each fleet will get during a peak event.

Methods:

Throttling.js — Gizmo/SDC Throttling

Throttling limits traffic to prevent overload. EPIC manages throttling via Gizmo (also called SDC).

Methods:

Ticket.js — SIM Ticket Management

SIM tickets are Amazon’s internal task tracking system (like JIRA). EPIC auto-creates tickets for hardware orders.

Methods:

Other API Files

Section 3 — Operations (The operations/ Folder)

Operations files contain the actual database logic. They are called by API handlers.

HOTWOperations.js

class HOTWOperations {
    constructor() {
        this.db = new AuroraDB();  // MySQL/Aurora connection
    }
    
    // SQL: INSERT INTO hotwdashboard (...)
    async createOrUpdateHotwDashBoardTable(tableName, entries) { ... }
    
    // SQL: INSERT INTO asg_details (...)
    async createOrUpdateAsgDetailsTable(tableName, entries) { ... }
    
    // SQL: INSERT INTO hotw_run (EventIndexId, RunStatus)
    async updateRunTable(tableName, entries, eventIndexId) { ... }
    
    // SQL: INSERT INTO hotw_execution (PeakTPM, BAUTPM, ...)
    async insertHotwExecutionDetails(tableName, fleetIndexId, runId, ...) { ... }
    
    // SQL: INSERT INTO capacity_override_details (...)
    async insertCapacityOverrideDetails(tableName, fleetIndexId, runId, details) { ... }
    
    // SQL: INSERT INTO fulfillment_details (...)
    async insertFulfillmentDetails(tableName, fleetIndexId, runId, details) { ... }
    
    // SQL: INSERT INTO asg_details_run (...)
    async insertAsgDetailsWithRunId(tableName, fleetIndexId, runId, asgDetails) { ... }
    
    // SQL: SELECT * FROM hotw_dashboard WHERE EventId = ?
    async getHotwDashboardDetails(eventId, purpose) { ... }
    
    // SQL: SELECT * FROM hotw_execution WHERE fleetIndexId = ? AND runId = ?
    async getHOTWExecutionHistoryForFleetWithEvent(eventIndexId, fleetIndexId) { ... }
}

FleetOperations.js

class FleetOperations {
    // DynamoDB: get fleet by FleetId
    async getLatestFleet(fleetId) { ... }
    
    // DynamoDB: create fleet item
    async createFleet(fleetId, data) { ... }
    
    // DynamoDB: get numerical index for a fleet (used in MySQL)
    async getFleetIndex(fleetId) {
        // fleets have a string ID in DynamoDB but a numeric index in MySQL
        // this returns the numeric index
    }
    
    // Check if fleet exists for event
    async checkIfFleetIdExistsForEvent(fleetId, eventId) { ... }
}

EventPlanOperations.js

class EventPlanOperations {
    // DynamoDB: get latest EventPlan
    async getLatestEventPlan(eventPlanId) { ... }
    
    // DynamoDB: create or update EventPlan (versioned)
    async createOrUpdateEventPlanInDDB(eventPlanId, fleetOps, body, version, audit) { ... }
    
    // DynamoDB: check if EventPlan exists
    async checkIfEventPlanIdExists(eventPlanId) { ... }
    
    // DynamoDB: get specific version of EventPlan
    async getEventPlanByVersionId(eventPlanId, versionId) { ... }
    
    // DynamoDB: get event plans for multiple fleets
    async getBatchEventPlanForService(fleetIds, eventId) { ... }
}

Section 4 — Clients (The clients/ Folder)

DynamoDbDocumentClient.js — NoSQL Database

class DynamoDbDocumentClient {
    constructor() {
        this.client = new AWS.DynamoDB.DocumentClient();
    }
    
    // Create or replace an item
    async put(tableName, item) { ... }
    
    // Get one item by key
    async get(tableName, key) { ... }
    
    // Query items (uses index)
    async query(tableName, params) { ... }
    
    // Scan all items (slow, use carefully)
    async scan(tableName, params) { ... }
    
    // Delete an item
    async delete(tableName, key) { ... }
    
    // Update specific fields
    async update(tableName, key, expression) { ... }
    
    // Get multiple items at once
    async batchGet(tableName, keys) { ... }
}

AuroraMysqlClient.js — SQL Database

class AuroraMysqlClient {
    // Execute a SELECT query
    async queryPromise(sql, values) { ... }
    
    // Execute an INSERT query (single row)
    async insertSingleQueryPromise(sql, values) { ... }
    
    // Execute an INSERT query (multiple rows)
    async insertQueryPromise(sql, valuesArray) { ... }
    
    // Execute an UPDATE query
    async updateQueryPromise(sql, values) { ... }
    
    // Transaction management
    async startTransaction() { ... }
    async commitTransaction() { ... }
    async rollbackTransaction() { ... }
}

When is each used?

• DynamoDB — for main data (Fleet, Service, Event, EventPlan, Projections, Schema)
• MySQL/Aurora — for HOTW run data (execution details, ASG details, fulfillment details)

SIMClient.js — SIM Ticket Integration

class SIMClient {
    // Create a new SIM ticket
    async createTicket(title, description, assignee) { ... }
    
    // Add a comment to a ticket
    async addComment(ticketId, comment) { ... }
    
    // Update ticket status
    async updateStatus(ticketId, status) { ... }
}

SQSClient.js — Queue Messaging

class SQSClient {
    // Send a message to a queue
    async sendMessage(queueUrl, messageBody, attributes) { ... }
    
    // Receive messages from a queue
    async receiveMessage(queueUrl) { ... }
    
    // Delete message after processing
    async deleteMessage(queueUrl, receiptHandle) { ... }
}

Section 5 — Common Files (The common/ Folder)

Util.js — Helper Functions

const Util = {
    // Validate that all required keys exist in body
    validateKeys(body, requiredKeys) {
        return requiredKeys.every(key => key in body && body[key] !== null);
    },
    
    // Format HTTP response
    handleResponse(statusCode, body) {
        return {
            statusCode,
            headers: { 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*' },
            body
        };
    },
    
    // Format HTTP error response
    handleErr(statusCode, message, error) {
        console.error(message, error);
        return {
            statusCode,
            body: JSON.stringify({ message, error: error?.message })
        };
    },
    
    // Get current timestamp as string
    getCurrentTimestampString() {
        return new Date().toISOString();
    }
};

Constants Files (Very Important!)

These define all the string keys used throughout the backend:

Why these matter: Instead of hardcoding strings everywhere, EPIC uses constants:

// BAD (typo-prone):
const fleetId = body["FleetId"];  // what if someone writes "FleetID"?

// GOOD (EPIC's way):
const fleetId = body[FleetConstants.FLEET_ID];  // FleetConstants.FLEET_ID = "FleetId"

Section 6 — Notifications (The notification/ Folder)

Notification.js — SNS Publishing

class Notification {
    // Publish service update to SNS
    static async publishServiceUpdate(serviceId, updateType) {
        await sns.publish({
            TopicArn: process.env.SNS_TOPIC_ARN,
            Message: JSON.stringify({ serviceId, updateType }),
        });
    }
    
    // Publish service creation notification
    static async publishServiceCreation(service) { ... }
}

EmailNotification.js — SES Email Sending

Sends HTML emails for various EPIC events:

class EmailNotification {
    // Send email using template
    static async sendEmail(template, recipients, variables) { ... }
    
    // Send hardware order summary
    static async sendHardwareOrderSummary(fleetId, eventId, orderDetails) { ... }
    
    // Send peak readiness report
    static async sendPeakReadinessReport(eventId, report) { ... }
}

operations/EmailNotificationOperations.js

Handles the complex logic of building email content before sending.

operations/PeakReadinessReportOperation.js

Generates the peak readiness report email — shows green/yellow/red status for every fleet.

Section 7 — SQS Receivers (The sqs/ Folder)

Some Lambda functions are triggered by SQS messages instead of API Gateway:

FleetReceiver.js

Processes messages from the EventFleetCreationQueue. When a new event is created, it automatically creates event plans for all fleets of all registered services.

// When triggered by SQS message:
async function handler(event) {
    for (const record of event.Records) {
        const { eventId, fleetId } = JSON.parse(record.body);
        await eventPlanOperations.createOrUpdateEventPlanInDDB(
            `${eventId}#${fleetId}`,
            // ... creates initial milestone list
        );
    }
}

EventTicketReceiver.js

Processes messages from EventTicketCreationQueue. Creates SIM tickets for new events.

Sender.js

Generic SQS message sender utility.

Section 8 — Scripts (The Script/ Folder)

One-time scripts or maintenance utilities:

Section 9 — Full Request Flow Example

User clicks “Get HOTW Dashboard” for PrimeDay2024:

1. Browser → GET /hotw/dashboard/PD2024/Upscaling
                                  ↓
2. API Gateway routes to HOTWLambdaStack.getHotwDashboardDetailsLambda
                                  ↓
3. Lambda runs HOTW.getHotwDashboardDetails(event)
                                  ↓
4. Parse path params: eventId="PD2024", purpose="Upscaling"
                                  ↓
5. Call serviceOperations.getAllServices("Registered")
   └── DynamoDB: scan ServiceTable where ServiceType=Registered
                                  ↓
6. Call eventOperations.getLatestEvent("PD2024")
   └── DynamoDB: get EventTable item where EventId=PD2024
                                  ↓
7. Build list of all fleet IDs from all services
                                  ↓
8. Call eventPlanOperations.getBatchEventPlanForService(fleetIds, "PD2024")
   └── DynamoDB: batch get EventPlanTable for all fleet+event combinations
                                  ↓
9. Call hotwOperations.getHotwDashboardDetails("PD2024", "Upscaling")
   └── MySQL: SELECT * FROM hotwdashboard WHERE EventId=...
                                  ↓
10. Merge HOTW dashboard data with EventPlan milestone status
    (adds StatusReasoning and ActionPendingOn to each row)
                                  ↓
11. Return sorted list organized by service
                                  ↓
12. Frontend receives data and renders the table

Section 10 — Error Codes Used in EPIC Backend

Section 11 — Key SQL Tables (MySQL)

These are the MySQL/Aurora tables that HOTW uses: