Docs

Client

@cuitty/scrape/client is the frontend half of the package. It talks to a Scrape server over HTTP, or it can run against an in-process mock server when mock: true is passed.

Creating a Client

Use baseUrl for a running server on port 4340:

import { createScrapeClient } from "@cuitty/scrape/client";

const client = createScrapeClient({
  baseUrl: "http://127.0.0.1:4340",
});

console.log(await client.health());

Use mock: true for offline tests and examples:

import { createScrapeClient } from "@cuitty/scrape/client";

const client = createScrapeClient({ mock: true });
const health = await client.health();
console.log(health.status);

Running a Scrape

client.run(request, options) creates a job, starts a run, polls until the run is terminal, and returns a ScrapeResult.

import { createScrapeClient } from "@cuitty/scrape/client";
import type { InteractionResponse } from "@cuitty/scrape/types";

const client = createScrapeClient({ mock: true });

const result = await client.run(
  {
    url: "https://example.com/pricing",
    intent: "Extract pricing plan names as JSON",
    mode: "auto",
    emit: { artifact: true, markdown: true },
  },
  {
    pollIntervalMs: 25,
    timeoutMs: 30_000,
    onInteraction: async (request): Promise<InteractionResponse> => {
      console.log(request.title);
      return { id: request.id, outcome: "aborted" };
    },
  },
);

console.log(result.status, result.path, result.answer);

Polling and Events

The current client polls GET /api/scrape/runs/:runId inside run(). streamEvents(runId) reads the server's current SSE-formatted event buffer and returns parsed ScrapeEvent[].

import { createScrapeClient } from "@cuitty/scrape/client";

const client = createScrapeClient({ mock: true });
const result = await client.run({ url: "https://example.com", intent: "Read the headline" });
const events = await client.streamEvents(result.runId);

for (const event of events) {
  console.log(event.type, event.runId);
}

Interaction Callback

When a run parks in needs-interaction, run() calls onInteraction. Return an InteractionResponse; the client posts it to /api/scrape/runs/:runId/interactions/:interactionId and continues polling.

If no callback is supplied, the client aborts the pending interaction. That keeps headless automation deterministic.

Retrieving Runs and Artifacts

getRun(runId) returns only completed runs. If the run is still queued, running, or waiting for interaction, it throws with the current status.

getArtifact(artifactId) fetches an artifact that was emitted by a completed run. There is not yet a list-runs or list-artifacts endpoint in the shipped API, so callers should persist IDs they need later.

import { createScrapeClient } from "@cuitty/scrape/client";

const client = createScrapeClient({ mock: true });
const result = await client.run({ url: "https://example.com", intent: "Extract the page summary" });

const complete = await client.getRun(result.runId);
if (complete.artifact) {
  const artifact = await client.getArtifact(complete.artifact.id);
  console.log(artifact.id, artifact.healing.enabled);
}

Error Handling

HTTP failures throw Error("scrape: <status> <body>"). Timeouts throw from run(). Terminal engine failures return a ScrapeResult with status: "failed" and an error object.