Type Alias ExperimentParams<Input, ExpectedOutput, Metadata>

Array of data items to process.

Can be either custom ExperimentItem[] or DatasetItem[] from Langfuse. Each item should contain input data and optionally expected output.

Optional ISO 8601 timestamp (RFC 3339, Section 5.6) in UTC (e.g., "2026-01-21T14:35:42Z"). If provided, returns state of dataset at this timestamp. If not provided, returns the latest version. Recorded on each dataset item's observation as the experiment item version.

Optional description explaining the experiment's purpose.

Provide context about what you're testing, methodology, or goals. This helps with experiment tracking and result interpretation.

Optional array of evaluator functions to assess each item's output.

Each evaluator receives input, output, and expected output (if available) and returns evaluation results. Multiple evaluators enable comprehensive assessment.

Optional ID of the Langfuse experiment to record this run into.

Langfuse groups experiment items by this ID, so separate run calls that pass the same ID (for example from parallel workers, or when resuming an interrupted run) land in one experiment. Use it for local data, which otherwise gets a new random ID on every call.

When set, it is used as-is for every item of the run, including items from Langfuse datasets, and as the datasetRunId of run-level scores. Re-runs on a dataset with the same runName then join this ID rather than the derived one. When omitted, runs on Langfuse datasets derive a stable ID from the project, dataset and runName, and runs on local data get a random ID. Must not be empty or whitespace-only.

Pass the same runName on every worker as well; otherwise each one records its own timestamped run name.

Generate the ID once with createTraceId from @langfuse/tracing and share it with every worker, or derive it on each worker from a seed they all know. Seeded IDs match the Python SDK's create_trace_id(seed=...), so Python and JS workers can share one experiment:

import { createTraceId } from "@langfuse/tracing";

// Random: create once, then pass to every worker
const experimentId = await createTraceId();

// Deterministic: the same seed yields the same ID on every worker
const experimentId = await createTraceId("nightly-eval-2026-10-20");

await langfuse.experiment.run({
  name: "Nightly eval",
  runName: "nightly-eval-2026-10-20",
  experimentId,
  data: shardForThisWorker,
  task,
});
Copy

Re-running an item records a new attempt: the item view shows the latest attempt, but run-level aggregates such as cost and score averages include every attempt. Run-level evaluators only see the items of their own run call.

Maximum number of concurrent task executions (default: Infinity).

Controls parallelism to manage resource usage and API rate limits. Set lower values for expensive operations or rate-limited services.

Optional metadata to attach to the experiment run.

Store additional context like model versions, hyperparameters, or any other relevant information for analysis and comparison.

Human-readable name for the experiment.

This name will appear in Langfuse UI and experiment results. Choose a descriptive name that identifies the experiment's purpose.

Optional array of run-level evaluators to assess the entire experiment.

These evaluators receive all item results and can perform aggregate analysis like calculating averages, detecting patterns, or statistical analysis.

Optional exact name for the experiment run.

If provided, this will be used as the exact dataset run name if the data contains Langfuse dataset items. If not provided, this will default to the experiment name appended with an ISO timestamp.

The task function to execute on each data item.

This function receives input data and produces output that will be evaluated. It should encapsulate the model or system being tested.