Skip to content

design: Calibrate get-dataset-schema output detail #882

Description

@jirispilka

Problem

get-dataset-schema (generateSchemaFromItems over 5 items) returns a thin schema; raw Apify dataset.schema is too bloated. No agreed token budget for what a schema response should look like.

Proposed solution

Open investigation:

  • derive-from-items vs. trim dataset.schema vs. hybrid
  • evals to measure agent success vs. schema verbosity
  • pick a budget, land the strategy

Probe finding — same problem in get-dataset

get-dataset also dumps the raw Apify dataset.schema field — the exact bloated structure this issue is calibrating away from. Whatever trim/inference strategy lands for get-dataset-schema should also apply when surfacing schema info via get-dataset (or get-dataset should stop returning the schema entirely and defer to get-dataset-schema).

Parent: #784

Metadata

Metadata

Assignees

Labels

t-aiIssues owned by the AI team.

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions