Pigment as a Source Connector

Configure Pigment EPM as a source to extract list and table data for downstream targets. Covers auth scopes, object types, rate limits, and troubleshooting.

Overview

The Pigment connector is dual-role: the same API key credential works as both a target (push data into Pigment via the CSV Push API) and a source (extract data out of Pigment via the Export API). Source extraction enables Pigment-to-Pigment syncs between applications, Pigment-to-Adaptive data migration, and GL history backfill where Pigment is the system of record.

VersionForge registers Pigment as a source alongside the existing target, mirroring the dual-role pattern used by Adaptive Planning. Source and target share one credential; the auth schema is unchanged from the target-only setup.

Prerequisites

A Pigment workspace with at least one application populated with list and/or table data
An API key with Export and Modeling Metadata scopes (see below)
The applicationId of the Pigment application you intend to extract from

Credential Fields

| Field | Required | Description | |-------|----------|-------------| | apiKey | Yes | Pigment API key with both Export and Modeling Metadata scopes. | | applicationId | Yes (for source) | Stored in connection metadata. Identifies which Pigment application's lists and tables to enumerate. |

If you already configured a Pigment connection for the target (push) side, the same apiKey works for the source side as long as the key was issued with both scopes. Add the applicationId to the connection's metadata to enable source discovery.

Setup Steps

Generate an API key with dual scopes
In Pigment, navigate to Workspace Settings > API Keys > Create API Key. Give it a descriptive name like "VersionForge Source + Target".
Assign both of the following scopes to the key:
- Export — required to read data out of lists and tables
- Modeling Metadata — required to enumerate lists and tables and to fetch column schemas
A key with only the Export scope will extract data but cannot discover which objects are available to extract. A key with only the Modeling Metadata scope can list objects but will return a 403 when VersionForge attempts to pull their contents.
Copy the key immediately. Pigment only displays it once.
Find your applicationId
The Pigment application ID appears in the URL when you open an application in the Pigment UI (for example, https://pigment.app/app/<applicationId>/...). Alternatively, ask a workspace administrator.
The source connector is scoped to one application per connection. If you need to extract from multiple applications, create one connection per application.
Create or update the connection
In VersionForge, go to Connections. If you already have a Pigment connection for pushing data, edit it and add the applicationId field. Otherwise, click New Connection, select Pigment EPM, and provide the API key and applicationId.
VersionForge validates the credential on save by listing the application's lists and tables. A 403 at this step indicates a narrow-scope key.

Supported Object Types

| Object Type | What it is | How it maps to RawRecords | |-------------|-----------|--------------------------| | list | A dimension member catalog — members of a Pigment list with their attributes. | One record per member. The first column marked isKey (or the first column, if none) serves as sourceRecordId. | | table | Fact-style data: dimensions + measures per row. Primary source for GL history extraction. | One record per row. Same key-column convention. |

Pigment blocks (views built over tables) are not supported as a direct source. Blocks apply filters and calculations on top of table data; VersionForge exposes the underlying tables instead, since the view-level transformations can be reproduced in the transform step of the pipeline.

How Extraction Works

During the prepare() phase, VersionForge fetches the full catalog of lists and tables for the configured applicationId and caches it in the run context. This avoids duplicate catalog lookups when the same run extracts multiple objects.

In extract(), VersionForge issues a POST to the Pigment Export API for the selected list or table:

POST /api/v1/export/list/{listId}
POST /api/v1/export/table/{tableId}

The response is a CSV payload parsed into header-keyed row objects. Each row becomes a RawRecord with a deterministic SHA-256 contentHash derived from its field values, enabling row-level diffing across subsequent runs.

Rate Limits

Pigment enforces 500 requests per 5-minute window across all API endpoints, shared between push and pull. VersionForge's export client:

Uses exponential backoff with full jitter on HTTP 429 (base 1s, ceiling 30s)
Retries up to 3 times before surfacing a PigmentExportError
Applies a 60-second timeout per export call (bulk payloads)
Increments ctx.metrics.apiCallsMade for every outbound call

If your sync schedule would push you near the limit, stagger extracts across multiple runs or reduce cadence. The target-side CSV Push also consumes the same quota.

Known Limitations

No historical snapshots. The Export API returns the current materialization of a list or table. There is no public way to request a point-in-time snapshot.
No calculated-metric grain control. Tables return the stored materialization; metrics derived by block-level views are not accessible through table exports.
No native CDC. Every extraction is a full snapshot. VersionForge handles change detection on the staging side via contentHash comparison — this is efficient for moderate data volumes but is not a substitute for true change data capture on very large tables.
Synchronous exports only. The CSV comes back in the HTTP response body. Very large exports may hit the 60-second timeout; split large tables by filtering or by exporting contiguous time ranges.

Troubleshooting

403 on schema discovery. The API key is missing the Modeling Metadata scope. Regenerate the key with both scopes and update the connection.

403 on export but schema discovery works. The API key has Modeling Metadata but not Export. Same fix: regenerate with both scopes.

404 on a list or table. The object no longer exists in Pigment, or the applicationId on the connection does not contain it. Re-run schema discovery to refresh the catalog.

Empty rows but 200 OK. The list or table has no data in Pigment. Confirm in the Pigment UI before investigating VersionForge.

429 after a large sync. You've exhausted the 500-req / 5-min window. The client will back off automatically, but if the error persists, wait five minutes and re-run. For ongoing heavy use, schedule syncs with wider spacing.

Validation says "applicationId not configured". Add applicationId to the connection's metadata (not the credential body). Source-side discovery is gated on this field.