WIP: Reverse Engineering OpenAI Node SDK

This documentation reverse engineers the internal architecture of the openai-node SDK. It's intended for developers who want to understand how the SDK works under the hood.

Overview

• What: Official OpenAI Node.js/TypeScript SDK
• Generated by: Stainless from an OpenAPI spec
• Runtime dependencies: Zero
• Platforms: Node.js, Deno, Bun, browsers (with opt-in), edge runtimes

Architecture at a Glance

┌─────────────────────────────────────────────────────────────┐
│                     User Code                               │
│   const client = new OpenAI({ apiKey: '...' });             │
│   await client.chat.completions.create({ ... });            │
└──────────────────────┬──────────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────────┐
│  Resources  (src/resources/)                                │
│  Each API group is a class extending APIResource.           │
│  e.g., Chat, Completions, Models, Files                    │
│  Resources delegate HTTP calls to the client.               │
└──────────────────────┬──────────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────────┐
│  OpenAI Client  (src/client.ts)                             │
│  Central HTTP orchestrator. Builds URLs, headers, body.     │
│  Handles retries, timeouts, auth, logging.                  │
└──────────────────────┬──────────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────────┐
│  Core Layer  (src/core/)                                    │
│  APIPromise  - Lazy promise with .withResponse()            │
│  Stream      - SSE async iterable                           │
│  Pagination  - Auto-paginating page classes                 │
│  Errors      - Status-specific error hierarchy              │
└──────────────────────┬──────────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────────┐
│  Internal Layer  (src/internal/)                            │
│  Types, headers, response parsing, platform detection,      │
│  line decoders, upload encoding, utility functions          │
└─────────────────────────────────────────────────────────────┘

Source Code Layout

Entry Point

File: src/index.ts

The entry point is a thin barrel export that re-exports only the public API surface. Everything a user needs is accessible from import OpenAI from 'openai'.

Exports

// Default export - the main client
export { OpenAI as default } from './client';
 
// Upload utilities
export { type Uploadable, toFile } from './core/uploads';
 
// Promise subclass
export { APIPromise } from './core/api-promise';
 
// Named client export + options type
export { OpenAI, type ClientOptions } from './client';
 
// Pagination promise (for typed auto-pagination)
export { PagePromise } from './core/pagination';
 
// Full error hierarchy (13 classes)
export {
  OpenAIError, APIError, APIConnectionError,
  APIConnectionTimeoutError, APIUserAbortError,
  NotFoundError, ConflictError, RateLimitError,
  BadRequestError, AuthenticationError,
  InternalServerError, PermissionDeniedError,
  UnprocessableEntityError, InvalidWebhookSignatureError,
} from './core/error';
 
// Azure variant
export { AzureOpenAI } from './azure';

Usage Patterns

// Default import
import OpenAI from 'openai';
 
// Named imports
import { OpenAI, APIError, toFile } from 'openai';
 
// Type imports
import type { ClientOptions } from 'openai';

TypeScript Namespace

The OpenAI client class also declares a TypeScript namespace src/client.ts:1115-1372 that re-exports all resource classes and types. This enables the OpenAI.Chat.Completions.ChatCompletion pattern for type access:

// Access types via namespace
const params: OpenAI.ChatCompletionCreateParams = {
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'Hello' }],
};
 
// Access pagination types
type Page = OpenAI.CursorPage<OpenAI.Model>;

The namespace exports include:

• All resource classes (Completions, Chat, Files, etc.)
• All request/response types for each resource
• Pagination types (Page, CursorPage, ConversationCursorPage)
• Shared types (Metadata, Reasoning, FunctionDefinition, etc.)

Client

File: src/client.ts (1,373 lines)

The OpenAI class is the central hub of the SDK. It handles configuration, HTTP requests, retries, authentication, and hosts all API resource instances.

ClientOptions

interface ClientOptions {
  apiKey?: string | ApiKeySetter;    // Static key or async function for rotation
  organization?: string | null;      // OpenAI organization ID
  project?: string | null;           // OpenAI project ID
  webhookSecret?: string | null;     // For webhook signature verification
  baseURL?: string | null;           // Override API base URL
  timeout?: number;                  // Request timeout in ms (default: 600,000 = 10 min)
  fetch?: Fetch;                     // Custom fetch implementation
  fetchOptions?: MergedRequestInit;  // Extra options passed to every fetch call
  maxRetries?: number;               // Max retry attempts (default: 2)
  defaultHeaders?: HeadersLike;      // Headers included in every request
  defaultQuery?: Record<string, string | undefined>;  // Query params on every request
  dangerouslyAllowBrowser?: boolean; // Allow browser usage (unsafe, exposes key)
  logLevel?: LogLevel;               // 'debug' | 'info' | 'warn' | 'error' | 'off'
  logger?: Logger;                   // Custom logger (default: console)
}

Configuration Resolution

Each option follows the same resolution chain:

Constructor argument  →  Environment variable  →  Default value

Constructor Behavior

The constructor (client.ts:383-433) performs these steps:

1. Reads environment variables as defaults for unset options
2. Throws OpenAIError if no apiKey is provided
3. Blocks browser usage unless dangerouslyAllowBrowser: true
4. Sets all instance properties
5. Stores original options for withOptions() cloning

API Key Rotation

The apiKey option accepts an async function for dynamic credentials:

const client = new OpenAI({
  apiKey: async () => {
    // Fetch a fresh token from your auth service
    return await getTokenFromVault();
  },
});

How it works (client.ts:497-520):

• _callApiKey() is called in prepareOptions() before each request
• If apiKey is a function, it's invoked and must return a non-empty string
• The returned token replaces this.apiKey for that request
• Errors from the function are wrapped in OpenAIError with the original as cause

HTTP Methods

All five HTTP methods follow the same pattern (client.ts:564-594):

get<Rsp>(path: string, opts?: RequestOptions): APIPromise<Rsp>
post<Rsp>(path: string, opts?: RequestOptions): APIPromise<Rsp>
patch<Rsp>(path: string, opts?: RequestOptions): APIPromise<Rsp>
put<Rsp>(path: string, opts?: RequestOptions): APIPromise<Rsp>
delete<Rsp>(path: string, opts?: RequestOptions): APIPromise<Rsp>

They all delegate to methodRequest(), which creates FinalRequestOptions and calls request():

private methodRequest<Rsp>(method, path, opts): APIPromise<Rsp> {
  return this.request(
    Promise.resolve(opts).then(opts => ({ method, path, ...opts }))
  );
}

Note: opts can be a Promise<RequestOptions>, enabling lazy option resolution.

request() and makeRequest()

request() (client.ts:596-601) creates an APIPromise wrapping the makeRequest() call. The actual HTTP work is deferred until the promise is awaited.

makeRequest() (client.ts:603-762) is where the real work happens:

1. Await the options (may be a promise)
2. Call prepareOptions() (resolves API key function)
3. Call buildRequest() (build URL, headers, body)
4. Call prepareRequest() (extension hook for subclasses)
5. Generate a log ID for request correlation
6. Check if signal is already aborted
7. Execute fetchWithTimeout()
8. Handle connection errors (retry or throw)
9. Handle HTTP error responses (retry or throw)
10. Return APIResponseProps on success

Building Requests

URL Building (client.ts:522-544)

buildURL(path, query, defaultBaseURL): string

• Resolves baseURL (custom override or default)
• Handles absolute URLs (pass-through)
• Merges default query params with request-specific params
• Stringifies query via stringifyQuery()

Header Building (client.ts:928-965)

Headers are built by merging multiple sources in priority order:

1. Idempotency headers (if method is not GET)
2. Standard headers:
   - Accept: application/json
   - User-Agent: OpenAI/JS {version}
   - X-Stainless-Retry-Count: {n}
   - X-Stainless-Timeout: {seconds}
   - Platform headers (OS, arch, runtime)
   - OpenAI-Organization
   - OpenAI-Project
3. Auth headers (Authorization: Bearer {apiKey})
4. Client default headers
5. Body content-type headers
6. Per-request headers

Later sources override earlier ones. Setting a header to null explicitly removes it.

Body Building (client.ts:973-1016)

The body builder handles multiple content types:

• Raw types (ArrayBuffer, Uint8Array, Blob, ReadableStream): passed through as-is
• FormData: passed through (multipart/form-data)
• URLSearchParams: passed through (application/x-www-form-urlencoded)
• AsyncIterable: converted to ReadableStream
• Objects: JSON-serialized via FallbackEncoder (application/json)

Retry Strategy

Which Requests Retry (client.ts:824-845)

The shouldRetry() method checks:

1. x-should-retry header: if the server explicitly says true/false, obey
2. Status 408: Request Timeout
3. Status 409: Conflict (lock timeout)
4. Status 429: Rate Limited
5. Status >= 500: Server Error

Connection errors and timeouts are always retried.

Backoff Calculation (client.ts:886-899)

Exponential backoff with jitter:

sleepSeconds = min(0.5 * 2^retryNumber, 8.0)
jitter = 1 - random() * 0.25
timeout = sleepSeconds * jitter * 1000

This produces:

• Retry 0: ~0.5s (375ms - 500ms)
• Retry 1: ~1.0s (750ms - 1000ms)
• Retry 2: ~2.0s (1500ms - 2000ms)
• Max: 8.0s

The server can override this with Retry-After or retry-after-ms headers (client.ts:847-884).

Pagination Support

getAPIList<Item, PageClass>(path, Page, opts): PagePromise<PageClass, Item>
requestAPIList<Item, PageClass>(Page, options): PagePromise<PageClass, Item>

These methods return a PagePromise that resolves to a page object. See Pagination for details.

Extension Hooks

Three protected methods allow subclasses (like AzureOpenAI) to customize behavior:

// Called before each request - default: resolves API key function
protected async prepareOptions(options: FinalRequestOptions): Promise<void>
 
// Called after request is built - default: no-op
protected async prepareRequest(request: RequestInit, { url, options }): Promise<void>
 
// Returns auth headers - default: Bearer token
protected async authHeaders(opts: FinalRequestOptions): Promise<NullableHeaders>

Resource Instances

The client instantiates all API resources as properties (client.ts:1041-1089):

completions: API.Completions = new API.Completions(this);
chat: API.Chat = new API.Chat(this);
embeddings: API.Embeddings = new API.Embeddings(this);
files: API.Files = new API.Files(this);
images: API.Images = new API.Images(this);
audio: API.Audio = new API.Audio(this);
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
uploads: API.Uploads = new API.Uploads(this);
responses: API.Responses = new API.Responses(this);
realtime: API.Realtime = new API.Realtime(this);
conversations: API.Conversations = new API.Conversations(this);
evals: API.Evals = new API.Evals(this);
// ... and more

Each resource receives this (the client) and uses it for all HTTP calls.

Static Properties

The OpenAI class also exposes static references (client.ts:1018-1036):

OpenAI.DEFAULT_TIMEOUT  // 600000 (10 minutes)
OpenAI.OpenAIError      // Error classes accessible from the class
OpenAI.APIError
OpenAI.RateLimitError
// ... all error classes
OpenAI.toFile           // Upload utility

withOptions()

Creates a new client instance with merged options (client.ts:438-455):

const newClient = client.withOptions({ timeout: 30000 });

Useful for creating a modified client without affecting the original.