Interview Questions: Ingestion and Indexing Pipelines

Strong answer: a repeatable system that moves approved sources through parsing, cleaning, metadata validation, chunking, embedding, indexing, validation, publishing, and monitoring.

Follow-up: what is the difference between ingestion and online retrieval?

Strong answer: first corpus build, major rebuild, embedding model change, schema migration, or full quality reset.

Follow-up: what gates should block a batch release?

Strong answer: stale chunks, duplicate chunks, missed deletes, mixed versions, and partial publish states.

Follow-up: how do you test delete propagation?

Strong answer: index versions support traceability, reproducibility, rollback, release comparison, and debugging bad answers.

Follow-up: what fields should an index version record contain?

Strong answer: build and validate a candidate index separately, then switch traffic only after quality gates pass.

Follow-up: what metrics would you watch after switching?

Strong answer: re-running the same pipeline should not create duplicate chunks, orphan records, or inconsistent index state.

Follow-up: how do chunk ids help?

Strong answer: run id, mode, source counts, changed sources, failures, stage versions, chunk counts, index version, quality metrics, publish decision, rollback target.

Follow-up: what should a failure record include?

Strong answer: check metadata completeness, parse quality, golden retrieval tests, access tests, stale-source checks, failed critical sources, cost, and latency.

Follow-up: what should happen if golden queries fail?

Strong answer: parser, cleaner, chunking, metadata schema, embedding model, vector-store config, or source authority/access changes may require re-indexing.

Follow-up: how do you avoid unnecessary full re-indexes?

Strong answer: they keep repeatedly failing records visible with stage, reason, owner, retry state, and next action instead of losing them in logs.

Follow-up: when should a failed source block the whole release?