LLM & MCP Integration

AutoCarver can be driven by a Large Language Model to qualify a dataset’s columns (decide which are numerical, categorical, ordinal, datetime or nested hierarchies) and then carve them against a target — the two steps that usually require a data scientist’s judgment.

There are two ways to use this, depending on where you work:

  • As an MCP server, exposed to an MCP-aware assistant (VS Code Copilot, Claude Desktop, Cursor, …) that drives the workflow through tools.

  • In a notebook, with an LLM client you instantiate yourself.

Both paths share the same logic: the read-only inspection helpers (AutoCarver.mcp.inspection), the type-routing (specs_to_features_kwargs()) and the CarverSession. The MCP server is only a thin transport layer on top of that session.

Note

The scope is qualify + carve: the workflow stops at the carved-feature artifact (the carving summary, per-feature content and a saved carver). It does not train a downstream estimator. The saved .json embeds the carved Features, so loading it back with load() restores both the carver and its features.

Warning

Carving quality depends on the LLM. Feature qualification — deciding which columns are ordinal, which form hierarchies and which to ignore (ids, free text, leakage) — is only as good as the model you point it at, and different models (or even different runs) can make different calls. Treat the LLM’s output as a first draft, not a final answer: a human should review and confirm the feature definitions before any production use.

Note

Your data stays local. The MCP server runs entirely on your machine — it reads files from your filesystem and never sends your dataset to AutoCarver or any other external service. The only data that leaves your machine is whatever your own LLM client/provider transmits as part of the conversation (e.g. the column summaries the assistant chooses to share). That exchange is governed by your provider’s terms, not by AutoCarver.

Path 1 — As an MCP server (VS Code Copilot, Claude Desktop, …)

AutoCarver ships a local Model Context Protocol server, so an MCP-aware assistant can build carved features straight from a file path — loading the data, inspecting it, choosing types and carving, all through tool calls.

Installing the server

The server depends on fastmcp, which is not part of the core install. Pull it in with the mcp extra:

uv add "autocarver[mcp]"
pip install "autocarver[mcp]"

Running the server

The server speaks stdio (the default transport for MCP clients):

uv run python -m AutoCarver.mcp
python -m AutoCarver.mcp

Clients usually launch the command for you via a config file (below) rather than you running it by hand.

Configuring the client

Point your client at the command. For VS Code / GitHub Copilot, add a .vscode/mcp.json to your workspace:

{
  "servers": {
    "autocarver": {
      "command": "uv",
      "args": ["run", "python", "-m", "AutoCarver.mcp"]
    }
  }
}
{
  "servers": {
    "autocarver": {
      "command": "python",
      "args": ["-m", "AutoCarver.mcp"]
    }
  }
}

For Claude Desktop the shape is identical but the top-level key is mcpServers (in claude_desktop_config.json).

A typical conversation

Once connected, you can ask the assistant in natural language, e.g.:

“Load ``data/loans.parquet`` with target ``default``, profile the columns, qualify the feature types, carve them and save the result to ``loans_carver.json``.”

The assistant will chain the tools below: load_dataset → inspect (list_columns, feature_distribution, validate_nesting …) → draft (suggest_features / set_feature) → run_carver → save_carver, and report the carving summary back to you.

Available tools

Inspection (read-only). Return summaries only — never raw columns wholesale — so they stay safe within a model’s context window.

  • load_dataset(path, target=None) — load a .csv / .parquet as the working dataset.

  • list_columns() — dtype, cardinality, missingness and suggested kind for every column.

  • profile_column(column, top_n=20) — quantiles (numeric) or top modalities (qualitative).

  • feature_distribution(column, min_freq=None, top_n=50) — modality counts, target rate and rare-modality flags (Wilson confidence interval, matching how the carvers judge rarity).

  • validate_nesting(child, parents) — check a column rolls cleanly (many-to-one) into coarser parents before declaring a nested feature.

  • datetime_reference_candidates() — span and coverage of datetime columns to pick a reference.

Drafting & carving.

  • suggest_features() — fill the draft with dtype-based suggestions (skips the target).

  • set_feature(column, kind, values=None, reference=None, parents=None) — set/override one column’s kind (numerical / categorical / ordinal / datetime / nested / ignore).

  • drop_feature(column) — remove a column from the draft.

  • preview_features() — return the current draft as {column: spec}.

  • run_carver(task="auto", min_freq=0.05, max_n_mod=5) — build Features from the draft and carve them against the target; returns the kept/dropped features, carved content and summary.

  • save_carver(path) — save the fitted carver and its carved features to a .json file (run run_carver first); the file restores both via BinaryCarver.load.

  • evaluate_stability(path, target=None) — score a holdout or production extract against the carved features’ train reference: PSI, chi-square drift and rank inversions (see Stability monitoring). target defaults to the session’s; an extract without it gets the population metrics only.

Path 2 — In a notebook, with your own LLM

You already have a dataset loaded and an LLM client instantiated. AutoCarver gives you a provider-agnostic qualifier: you supply llm_fn, a callable that takes a prompt string and returns the model’s raw text. No provider SDK is imported by AutoCarver, so any backend works.

Qualifying columns into Features

from anthropic import Anthropic
from AutoCarver.features import qualify_with_llm

client = Anthropic()

def llm_fn(prompt: str) -> str:
    msg = client.messages.create(
        model="claude-opus-5-5",
        max_tokens=2000,
        messages=[{"role": "user", "content": prompt}],
    )
    return msg.content[0].text

# X is your training DataFrame, X_dev a held-out sample with the same columns
target = "default"  # your binary 0/1 target column, kept out of the prompt
features = qualify_with_llm(X.drop(columns=[target]), llm_fn)

qualify_with_llm() builds a prompt describing each column (dtype, cardinality, a value sample), asks the model to return a JSON {column: spec} mapping, and routes it into a Features. Any backend works — swap llm_fn for an OpenAI / local-model call.

Then carve exactly as you would by hand:

from AutoCarver import BinaryCarver

carver = BinaryCarver(features=features, min_freq=0.02, max_n_mod=5)
carver.fit(X, X[target], X_dev=X_dev, y_dev=X_dev[target])  # dev set: viability checks
x_discretized = carver.transform(X)
carver.save("my_carver.json")

Note

Without an LLM, Features.from_dataframe() gives the same dtype-based first pass deterministically. qualify_with_llm adds the semantic judgment a dtype cannot infer: ordering (ordinals), hierarchies (nested) and which columns to ignore (ids, free text, leakage).

Driving the session programmatically

For a guided, tool-by-tool workflow (the same one the MCP server exposes) without running a server, use CarverSession directly. This lets the model — or you — interrogate the data before committing to types:

from AutoCarver.mcp import CarverSession

session = CarverSession()
session.load_dataset("data.parquet", target="default")

session.list_columns()                         # dtype / cardinality / missingness / suggested kind
session.feature_distribution("job", min_freq=0.05)   # modality rates + rare-modality flags
session.validate_nesting("city", ["region", "country"])  # check a hierarchy is many-to-one

session.suggest_features()                      # dtype-based draft (skips the target)
session.set_feature("grade", "ordinal", values=["low", "medium", "high"])
result = session.run_carver(task="auto", min_freq=0.05, max_n_mod=5)
session.save_carver("my_carver.json")          # carver + carved features, reloadable
session.evaluate_stability("holdout.csv")      # PSI / drift / rank inversions vs. the train reference

run_carver returns the resolved task, the kept/dropped features, the carved content per feature and the carving summary. save_carver persists the fitted carver — features included — so it can later be restored with BinaryCarver.load("my_carver.json"). evaluate_stability scores a later sample against the reference statistics stored on each carved feature — see Stability monitoring.

Architecture

The MCP package is layered so the logic is importable and unit-tested without fastmcp:

The draft specs use the same schema as qualify_with_llm(), and both go through specs_to_features_kwargs() — a single source of truth for routing a column’s declared type into a Features.

API reference

AutoCarver.features.qualify_with_llm(X: DataFrame, llm_fn: Callable[[str], str], *, sample_size: int = 20, config: FeaturesConfig | None = None) → Features

Builds a Features by asking llm_fn to qualify every column of X.

Parameters:
  • X (pd.DataFrame) – DataFrame whose columns are qualified.

  • llm_fn (Callable[[str], str]) – Callable taking the prompt and returning the model’s raw text answer.

  • sample_size (int, optional) – Maximum number of sample values shown per column, by default 20.

  • config (FeaturesConfig, optional) – Collection-level config propagated to each feature, by default None.

AutoCarver.features.specs_to_features_kwargs(mapping: dict) → dict

Routes a {column: {"type": ..., ...}} mapping into Features kwargs.

Returns a dict with the categoricals, numericals, ordinals, datetimes and nested keys expected by Features(**kwargs). ignore columns are dropped. Shared by the LLM qualifier and the MCP session so the type-routing has a single source of truth.

Raises:

ValueError – If a column declares an unknown / incomplete type.

class AutoCarver.mcp.session.CarverSession

Holds the dataset, target and feature draft for one qualify-and-carve workflow.

datetime_reference_candidates() → list[dict]

Summarises datetime columns (span + coverage) to help pick a reference.

drop_feature(column: str) → dict

Removes a column from the draft (it will not become a feature).

evaluate_stability(path: str, target: str | None = None) → dict

Scores a new sample against the fitted carver’s train reference.

Loads a .csv/.parquet holdout or production extract and compares each carved feature’s modality distribution and target rate to the reference stored at carving time: PSI, a chi-square homogeneity test on counts, per-modality target drift, and the carver’s own viability tests (rank inversion, Wilson min_freq, distinct rates). target names the target column in the loaded file; it defaults to the session’s target. An unlabelled extract (one that simply doesn’t carry the session’s target column) falls back to the frequency-based metrics only — but an explicitly named target that is missing is an error, not a silent downgrade. Requires a prior run_carver().

feature_distribution(column: str, min_freq: float | None = None, top_n: int = 50) → dict

Modality distribution of a column, against the loaded target when one is set.

list_columns() → list[dict]

Per-column summary (dtype, cardinality, missingness, suggested kind).

load_dataset(path: str, target: str | None = None) → dict

Loads a .csv or .parquet file as the session dataset.

target, when given, names the column carved against; it is excluded from feature suggestions. Resets any previous draft.

preview_features() → dict

Returns the current draft as {column: spec}.

profile_column(column: str, top_n: int = 20) → dict

Detailed profile of a single column.

run_carver(task: str = 'auto', min_freq: float = 0.05, max_n_mod: int = 5) → dict

Builds Features from the draft and carves them against the target.

task is binary/continuous/multiclass/ordinal/one_vs_rest or auto (inferred from the target; never resolves to ordinal or one_vs_rest, which must be requested explicitly). multiclass is the joint carver — one carving per feature against the full K-class target; one_vs_rest fits a separate carving per class instead. Returns the carving summary, the carved content per feature, and any features the carver dropped as non-viable.

save_carver(path: str) → dict

Saves the fitted carver (carved features included) to a .json file.

The carver serialises its Features inside the same file, so the saved artifact restores both via BaseCarver.load(). Requires a prior run_carver().

set_feature(column: str, kind: str, values: list[str] | None = None, reference: str | None = None, parents: list[str] | None = None) → dict

Sets or overrides one column’s spec in the draft.

kind is one of numerical/categorical/ordinal/datetime/nested/ignore. ordinal needs values (ordered), datetime needs reference (column or date literal), nested needs parents.

suggest_features() → dict

Fills the draft with dtype-based suggestions (the deterministic first pass).

Mirrors Features.from_dataframe() (including datetime reference resolution) but skips the target column. Overwrites the current draft.

validate_nesting(child: str, parents: list[str]) → dict

Checks that child rolls cleanly into parents (many-to-one hierarchy).

AutoCarver.mcp.server.create_server() → FastMCP

Builds a FastMCP server bound to a fresh CarverSession.

AutoCarver.mcp.inspection.profile_dataframe(X: DataFrame) → list[dict]

One summary row per column: dtype, cardinality, missingness, suggested feature kind.

AutoCarver.mcp.inspection.profile_column(X: DataFrame, column: str, *, top_n: int = 20) → dict

Detailed profile of a single column: cardinality, missingness, and a kind-specific view.

Numeric / datetime columns report min/max/quantiles; qualitative columns report their most frequent modalities (capped at top_n).

AutoCarver.mcp.inspection.feature_distribution(X: DataFrame, column: str, y: Series | None = None, *, min_freq: float | None = None, alpha: float = 0.05, top_n: int = 50) → dict

Distribution of a qualitative column’s modalities, optionally against the target.

Reports per-modality count and frequency, the target rate when y is given, and — when min_freq is provided — flags modalities whose frequency is significantly below it (Wilson upper bound, matching how the carvers decide rarity).

AutoCarver.mcp.inspection.validate_nesting(X: DataFrame, child: str, parents: list[str]) → dict

Checks that child rolls cleanly into parents (a many-to-one hierarchy).

For each consecutive (finer, coarser) level pair, verifies that every finer modality maps to exactly one coarser modality. Returns whether the hierarchy is valid, the cardinality of each level, and any violations (a finer modality spread across several coarser ones).

AutoCarver.mcp.inspection.datetime_reference_candidates(X: DataFrame) → list[dict]

Summarises each datetime column (span + coverage) to help pick a reference.

A good fixed reference / anchor is typically the column with the widest, most-complete coverage; another datetime column can also be referenced row-wise.