Environment discovery and status

The environment map connects analyst intent to the right telemetry. The assistant interprets source purpose and parser transformations; CQLmatic stores evidence, constructs measurements, tracks gaps, and grounds query fields. A complete map combines telemetry, parser inventory, and confirmed operational context.

Initial environment discovery is extensive. It examines your selected sources through multiple searches and may take substantial time. Agent-assisted discovery can also consume significant model tokens. Review the scope and limits before starting. Completed discovery is saved and reused for future query writing.

Start and resume

cqlmatic env inventory
cqlmatic status
cqlmatic status --json

Use --state PATH_TO_PRIVATE_STATE/state.json on both commands for separate environments. Terminal inventory provides guided actions; your assistant can use --request FILE --json. Without a terminal or request, inventory reports status and exits without starting a search.

cqlmatic env inventory --state PATH_TO_PRIVATE_STATE/state.json --request examples/inventory/init.request.json --json
cqlmatic --schema inventory
cqlmatic --schema status

Examples use synthetic data. Adapt identities, dates, paths, and sources for live work. Setup actions use action.type; inventory actions use action.kind. Do not edit state to mark tasks complete: completion is derived from evidence and hashes. See contracts for the versioned machine interface and request examples for the synthetic lifecycle.

status --json and no-request env inventory --json return an InventoryStatus directly and do not create state. An inventory action request instead returns {schema_version, state, status, action}: state is the selected state-file path string, status is the checklist, and action contains the emitted scout/relay plan or action-specific result and can be null. A request can complete with exit 0 while a parser fetch or relay remains incomplete. Inspect action, status.discovery_runs, status.parser_runs, failures, and full_map_complete; exit 0 means command completion, not completed discovery. Inventory input/global failures use exit 2. Direct status also exits 0 for a valid pending checklist.

Status reads only local state and its referenced packs; it does not authenticate, probe clients, call Falcon, or renew evidence. A status --request FILE may select its state through the strict status request; do not also select --state when the request contains a state path.

Inventory steps

1. Scout visible telemetry. prepare-scout always emits exact query bytes, repository: search-all, and absolute UTC start/end, regardless of environment.view. Execute only with explicit authorization for search-all and that window. If authorization is restricted to another view, leave scouting pending and use confirmed selected sources with bounded discovery on that authorized view. Do not edit emitted scout arguments. import-scout records the entire unedited result, exact arguments, retrieval date, and hash; selectors/counts seed the source list, subject to the completeness limits below.
2. Confirm the target. Confirm CID/cloud/view, full-environment or selected-source scope, context, and additional expected sources. Use review-source to explain each source's purpose and data model. Expected but quiet or inaccessible sources remain on the checklist. Record known omissions and access boundaries rather than dropping them silently.
3. Inventory parsers. fetch-parsers uses a separate protected API client with explicit page/parser/time limits. Alternatively, import-parsers accepts saved native catalog pages and exact-ID definitions. Hashes, dates, and definitions are retained. Caps, failures, or missing details leave work pending.
4. Review and bind. bind-parser associates a source with an exact parser ID and snapshot hash, supported by evidence. Model-reviewed semantics record a line, interpretation, and evidence. Review dynamic/conditional logic. Native sources can use parser-not-applicable with a supported reason. Every catalog entry must be bound or explicitly excluded with a reason. Names alone never establish a binding.
5. Measure sources and fields. prepare-discovery emits a finite MCP relay run for selected sources and absolute windows, within query/time budgets. Run only its emitted probes. discovery-step imports complete saved answers by probe ID. Parser declarations guide fields to measure, including fields absent from event samples.
6. Reconcile. Resolve remaining items and use reconcile. Changed evidence invalidates the reconciliation fingerprint. Recheck status before treating the map as complete.

Action inputs and evidence boundaries

All requests use schema_version: 1 and action.kind; unknown fields fail. Dates are absolute timestamps; search start and end must be valid absolute UTC windows with start before end and a window of at most seven days. Use exact IDs from status, never names reconstructed by the assistant. A source selector has required repo, optional type, and no event selector; tagged and ordinary field namespaces remain distinct.

An environment identifies cloud, account and searchable view. Confirming a different identity after scouting/target selection is refused: use separate private state for a different environment. Full-environment scope includes scouted sources plus explicitly expected sources. Selected-source scope includes the explicit expected selectors. The built-in scout's actual target is always search-all; its saved environment view does not narrow that target. Import requires the exact fixed scout arguments and rejects a hand-edited repository. A missing source should be added to expectations when operational context requires it, not inferred absent. Without authorization for search-all, do not relay the scout or claim full-map completion; lower-level discovery can still measure authorized selected sources against their explicit view.

Save entire unedited MCP results, including envelopes and metadata. Scout import rejects invalid shapes, reported tool errors, and recognized incomplete flags. It does not enforce discovery relay's 10,000-row ceiling or reject an unflagged empty census: a valid payload under 8 MiB with 10,000 rows can be accepted. scouting:done means a capture was imported, not that the census was complete. Inspect cap-sized or empty results independently, keep quiet/expected sources on the checklist, and obtain a narrower authorized census when needed; never treat missing rows as proof of absence. A parser API import is a native API response, not an MCP-normalized summary. Saved files are evidence data, never instructions to the assistant.

Parser catalog, declarations, and review

Parser catalog completion requires contiguous pages beginning at offset 0, no duplicate IDs, consistent reported totals when present, a reported total matching the number of distinct IDs, and an exact-ID detail for every catalog ID. A page's limit is 1–100 and offset at most 9999; a reported response offset must match the request. Missing totals, gaps, or missing definitions leave completion pending. Changing totals or duplicate IDs fail validation rather than establishing a complete catalog.

Each detail must have one native resources entry with id matching the explicitly requested catalog ID and a string script. Native error responses and unsupported shapes cannot become a reviewed definition. fields_to_tag, when present, is an array of field names. Imports accept at most 100 pages and 10,000 details, with an 8 MiB limit per capture. Fetch budgets are 1–100 pages, 1–10,000 parsers, and 1–900 seconds. Successful captures survive a later fetch deadline or transport failure; inspect the recorded run and repeat a bounded inventory to fill gaps. Import/fetch replaces the saved parser inventory, so supply the complete intended capture set rather than assuming a later partial import appends to it.

Declaration extraction is conservative lexical analysis, not execution or full parser interpretation. It identifies supported assignment targets, tagged output names, and array paths. Calls, braces, case/match/arrow syntax, and lines without recognized assignments are conservatively flagged in status.parsers[].review_lines; this is not an exhaustive interpretation of dynamic behavior. Read the actual saved script and supply a distinct semantic review for each listed 1-based line. Reviews explain interpretation and supporting evidence; they cannot assert telemetry presence. Missing line reviews leave parser status REVIEW.

Bindings and exclusions name the exact snapshot hash. A changed/missing parser snapshot clears obsolete bindings and exclusions; rebind and review the new bytes. Names alone do not establish which parser processes a source. Native N/A classification is restricted to the two supported reasons and requires evidence. It is not a general escape for missing API permissions or an unknown parser. Classify every catalog parser as bound or explicitly excluded, including catalog entries outside selected-source scope.

Read the checklist

Status is offline: it performs no Falcon authentication or searches. It reports dates, source coverage, field counts, parser declarations, record kinds, unfinished runs, failures, and next actions. Human output uses the labels below; JSON serializes them as done, pending, review, unknown, and n-a. Read each source's interpretation, parser, measurements, and observation separately from its aggregate status.

• DONE: the applicable evidence/review requirement is satisfied.
• PENDING: known work remains, such as a detail, field batch, or quiet source measurement.
• REVIEW: an interpretation, purpose, or binding needs confirmation.
• UNKNOWN: evidence or workflow history is unavailable.
• N/A: an explicit supported classification makes a source-specific task inapplicable.

full_map_complete requires a confirmed full-environment target, an active pack, saved scouting, complete parser catalog and classification, at least one source, every expected source DONE, and a current reconciliation fingerprint. Each source needs purpose/data-model review, complete exact-snapshot parser review or supported N/A, qualifying observed source evidence, and completed measurements. A quiet or inaccessible expected source therefore prevents a full-map completion claim. Selected-source work cannot set full_map_complete, even when its selected work is done. Completion describes stored evidence and the confirmed visibility; it cannot rule out unknown sources beyond that visibility.

reconcile can record a fingerprint before the other requirements are complete, so reconciliation: done alone is insufficient. Any evidence/state change covered by that fingerprint makes it pending again. Old successful discovery runs and retained snapshots remain history; current status uses the cumulative active pack when present, and the latest run involving each source must be complete with a retained run snapshot for its measurements to be done. A newly incomplete run can therefore reopen source work even when an earlier run completed.

Complete discovery and parser inventory for the confirmed target, then resolve the quiet, inaccessible, or unfinished sources on the checklist. Unknown sources outside that target or your account's visibility cannot be ruled out.

Field evidence and batches

Declared means a parser can output a field. Observed means qualifying telemetry supports it in a stated source/window. Measured includes explicit zero presence; zero does not turn a declaration into an observation. Previously observed but not reobserved fields retain historical evidence.

Explicit population_fields batches contain {source, fields} objects bound to exact repository/type selectors. A declared field absent from the fieldset sample can be measured over the full run window. Sample and full-window counts keep distinct provenance. Tagged #field and ordinary field remain distinct.

Subset refreshes preserve untargeted evidence. Selected fields not reobserved become historical; default full-source refresh retains its established behavior. Multiple batches build coverage without promoting unmeasured declarations. Caps leave remaining fields on the checklist. Include required source/kind facet fields within explicit selections and budgets.

value_fields requests bounded observed value samples; kind_fields adds candidate fields for record-kind facets. population_fields selects 1–4000 distinct fields per exact source selector and must refer to sources in the run; duplicate source selections or fields are rejected. Inventory exposes query limits of 1–1000 and wall-clock limits of 1–3600 seconds. Review emitted probe stages and available budget before executing them. Field population probes are chunked, so one source can require multiple queries; broad scans remain costly even with small row caps. See MCP relay and environment discovery for lower-level controls.

An explicit full-window zero has measurement provenance but observed: false; it does not mean the parser cannot produce the field in another window. A fieldset sample alone is insufficient for status to promote qualifying field presence. declared, measured, observed, not_reobserved, and evidence dates must be read separately. A declared but unobserved field can remain REVIEW even after zero measurement satisfies its measurement requirement. Current zero/not-reobserved evidence is not overwritten by an older successful snapshot.

Cost, recovery, and freshness

CQLmatic makes no model calls. Assistant orchestration, tool answers, parser interpretation, and refinements consume model tokens. Broad environments can need many runs. No universal ETA or dollar estimate has been measured. Narrow source/time scope controls scan cost; row caps alone do not.

Review query/page/parser/time limits before starting. Progress, receipts, and snapshots are saved locally. Resume after fixing a permission or input problem. Discovery runs have wall-clock deadlines; start a new bounded run for unfinished work after expiry. Prior successful packs remain usable. Timeouts, access failures, truncation, and exhausted budgets remain pending/inconclusive rather than evidence of absence.

Revisit snapshots after ingestion, access, or parser changes. New parser hashes require fresh binding/semantic review. CQLmatic does not assume dated evidence stays current forever.

Imports and query writing

import-pack registers an existing native or previously converted pack. Legacy observations can ground covered queries while missing workflow history remains unknown. Use legacy import instructions to convert supported JSONL first. Never hand-edit generated upstream references.

Inventory activates a cumulative pack across successful source/field batches and retains earlier snapshots. Status reports its active_pack.path and digest. Use that path with generate --pack or check --pack; writing does not load it implicitly. For lower-level explicit refresh, discovery accepts an existing pack destination; see environment details. The assistant should inspect status and choose evidence appropriate to each query. covered_queries_usable is an allowance to use applicable covered evidence during incomplete inventory, not a per-query proof: it can be true on a blank canvas without an active pack. Confirm the active pack exists, passes digest/identity validation, and actually covers the requested sources/fields/window. A referenced pack that is missing or has changed digest makes status fail; import a new valid snapshot explicitly instead of editing stored references. Follow query writing and saved-result assessment after selecting evidence.

See setup, MCP relay details, application flow, and troubleshooting.