Complete API Surface
CrawlKit exposes the full production platform in one API: 383 operations, 315 paths, 54 capability domains, and 532 typed schemas. This page is generated from /openapi.json so humans and agents can see the whole system without hunting.
No hidden platform surface. The API reference now includes Arrow/DataFusion query, ETL, workflow templates, pipeline internals, spider AI builder, crawl frontier, artifacts, lineage, quality, schema registry, OpenRate, telemetry, governance, healing, incidents, synthetic data, social, SEO, and every mounted production domain.
Most advanced surfaces first
Arrow + DataFusion data engineering
Register Parquet, query Arrow-native tables, export results, and move datasets through ETL stages.
Durable workflows
Reusable workflow templates, operation discovery, runs, cancellation, pipeline schedules, worker progress, and lineage.
Spider management
AI spider builder, selector suggestions, repair, factory batches, test runs, health, crawl jobs, frontiers, and proxy strategy.
Scalar reference
Interactive request/response docs for all 383 operations and 532 schemas.
All API domains
| Domain | Representative operations |
|---|---|
| Alerts 5 operations |
GET /api/v1/alertsPUT /api/v1/alerts/:id/dismissGET /api/v1/alerts/insightsGET /api/v1/alerts/settingsPUT /api/v1/alerts/settings |
| Clients 10 operations |
GET /api/v1/clientsPOST /api/v1/clientsDELETE /api/v1/clients/:client_idGET /api/v1/clients/:client_idGET /api/v1/domainsPOST /api/v1/domainsDELETE /api/v1/domains/:domain_idGET /api/v1/domains/:domain_id/status |
| Health 2 operations |
GET /api/v1/health/servicesService registry health GET /healthHealth check |
| Auth 6 operations |
POST /api/v1/auth/tenants/provisionProvision tenant GET /api/v1/auth/tokensList tenant API tokens POST /api/v1/auth/tokensCreate tenant API token DELETE /api/v1/auth/tokens/{token_id}Delete tenant API token GET /api/v1/auth/usageGet usage summary POST /api/v1/auth/validateValidate API key |
| GSC 12 operations |
POST /api/v1/gsc/analyticsSearch analytics POST /api/v1/gsc/auth/callbackOAuth callback POST /api/v1/gsc/auth/urlGet OAuth URL POST /api/v1/gsc/indexSubmit URL for indexing POST /api/v1/gsc/index/removeRequest URL removal POST /api/v1/gsc/inspectInspect URL POST /api/v1/gsc/inspect/analyzeInspect and analyze URL POST /api/v1/gsc/inspect/batchBatch inspect URLs |
| Analysis 8 operations |
POST /api/v1/analysis/quality-pipelineTrigger quality pipeline workflow POST /api/v1/audit/runRun authenticated audit with persistence GET /api/v1/auditsGET /api/v1/audits/:audit_idPOST /api/v1/checkPOST /api/v1/check-urlPOST /api/v1/reportGET /api/v1/status |
| DNS 4 operations |
POST /api/v1/dns/auditPOST /api/v1/dns/email-authCheck email authentication POST /api/v1/dns/redirectsTrace redirect chain POST /api/v1/dns/sslValidate SSL/TLS certificate |
| Profile 3 operations |
GET /api/v1/profilePUT /api/v1/profilePOST /api/v1/profile/generate |
| Keywords 6 operations |
GET /api/v1/keywordsPUT /api/v1/keywordsPOST /api/v1/keywords/enrichEnrich keywords with volume/difficulty POST /api/v1/keywords/import/csvPOST /api/v1/keywords/import/gscPOST /api/v1/keywords/recommend |
| Landscape 2 operations |
GET /api/v1/landscapePUT /api/v1/landscape/competitors |
| Content 9 operations |
POST /api/v1/briefGET /api/v1/briefsDELETE /api/v1/briefs/:brief_idGET /api/v1/briefs/:brief_idPUT /api/v1/briefs/:brief_idGET /api/v1/competitorsPOST /api/v1/content/extractExtract article content from a URL POST /api/v1/links |
| Feedback 3 operations |
POST /api/v1/feedback/recordRecord feedback event GET /api/v1/feedback/statsGet feedback stats GET /api/v1/feedback/statusGet pending verifications |
| IndexNow 3 operations |
POST /api/v1/indexnow/key/generateGenerate IndexNow key GET /api/v1/indexnow/key/verifyVerify IndexNow key POST /api/v1/indexnow/submitSubmit URLs via IndexNow |
| Sites 5 operations |
GET /api/v1/sitesPOST /api/v1/sitesPUT /api/v1/sitesDELETE /api/v1/sites/:site_idGET /api/v1/sites/:site_id |
| Dashboard 1 operations |
GET /api/v1/dashboard |
| Audit 1 operations |
POST /api/v1/auditReal-time site audit (SSE) |
| Datasets 12 operations |
GET /api/v1/datasetsList datasets POST /api/v1/datasetsCreate dataset DELETE /api/v1/datasets/:dataset_idDelete dataset GET /api/v1/datasets/:dataset_idGet dataset PUT /api/v1/datasets/:dataset_idUpdate dataset POST /api/v1/datasets/:dataset_id/exportExport dataset POST /api/v1/datasets/:dataset_id/importImport data GET /api/v1/datasets/:dataset_id/rowsList rows |
| Spiders 13 operations |
GET /api/v1/spidersList spiders POST /api/v1/spidersCreate spider DELETE /api/v1/spiders/:spider_idArchive spider GET /api/v1/spiders/:spider_idGet spider PUT /api/v1/spiders/:spider_idUpdate spider POST /api/v1/spiders/:spider_id/exportExport spider GET /api/v1/spiders/:spider_id/healthSpider health POST /api/v1/spiders/:spider_id/repairRepair spider selectors |
| Enrichment 3 operations |
POST /api/v1/enrichment/enrichEnrich rows GET /api/v1/enrichment/jobs/:job_idJob status GET /api/v1/enrichment/sourcesList sources |
| Inspection 3 operations |
POST /api/v1/inspection/htmlAnalyze HTML POST /api/v1/inspection/selectorsFind selectors POST /api/v1/inspection/urlInspect URL |
| Pipelines 17 operations |
GET /api/v1/pipelinesList pipelines POST /api/v1/pipelinesCreate pipeline DELETE /api/v1/pipelines/:pipeline_idDelete pipeline GET /api/v1/pipelines/:pipeline_idGet pipeline PUT /api/v1/pipelines/:pipeline_idUpdate pipeline POST /api/v1/pipelines/:pipeline_id/runRun pipeline GET /api/v1/pipelines/:pipeline_id/runsList runs GET /api/v1/pipelines/:pipeline_id/scheduleSchedule status |
| Queue 6 operations |
POST /api/v1/queue/dequeueDequeue items POST /api/v1/queue/enqueueEnqueue item GET /api/v1/queue/itemsList queue items POST /api/v1/queue/purgePurge queue POST /api/v1/queue/retryRetry queue item GET /api/v1/queue/statsQueue stats |
| Crawl 11 operations |
GET /api/v1/crawl/frontier/:job_idGet frontier stats GET /api/v1/crawl/jobsList crawl jobs POST /api/v1/crawl/jobsCreate crawl job GET /api/v1/crawl/jobs/:job_idGet crawl job POST /api/v1/crawl/jobs/:job_id/cancelCancel crawl job POST /api/v1/crawl/jobs/:job_id/pausePause crawl job POST /api/v1/crawl/jobs/:job_id/resumeResume crawl job GET /api/v1/crawl/proxiesList proxies |
| Content Workspace 12 operations |
POST /api/v1/content/ai/generateAI content generation POST /api/v1/content/ai/rewriteAI content rewrite GET /api/v1/content/calendarGet publishing calendar GET /api/v1/content/documentsList content documents POST /api/v1/content/documentsCreate content document GET /api/v1/content/documents/:id/approval-statusGet document approval status POST /api/v1/content/documents/:id/approveApprove document POST /api/v1/content/documents/:id/rejectReject document approval |
| Intelligence 10 operations |
POST /api/v1/intelligence/causalEstimate causal effect of SEO intervention POST /api/v1/intelligence/enrichEnrich domain with firmographics POST /api/v1/intelligence/forecastRevenue forecast with SEO assumptions GET /api/v1/intelligence/graphQuery knowledge graph POST /api/v1/intelligence/graph/impactAnalyze downstream impact of a node POST /api/v1/intelligence/graph/propagateSimulate signal propagation through graph POST /api/v1/intelligence/graph/root-causeTrace root causes of an anomaly GET /api/v1/intelligence/insightsAI-generated strategic insights |
| Leads 11 operations |
GET /api/v1/campaignsList campaigns POST /api/v1/campaignsCreate a campaign GET /api/v1/campaigns/:idGet a campaign POST /api/v1/campaigns/:id/advanceAdvance leads in a campaign GET /api/v1/leadsList leads POST /api/v1/leadsCreate a lead DELETE /api/v1/leads/:idDelete a lead GET /api/v1/leads/:idGet a lead |
| Pipeline 6 operations |
GET /api/v1/pipelineGet pipeline overview GET /api/v1/pipeline/attributionGet SEO attribution data GET /api/v1/pipeline/dealsList deals POST /api/v1/pipeline/dealsCreate a deal PUT /api/v1/pipeline/deals/:idUpdate a deal GET /api/v1/pipeline/forecastGet pipeline forecast |
| Scoring 9 operations |
POST /api/v1/scoring/batchPOST /api/v1/scoring/comparePOST /api/v1/scoring/computePOST /api/v1/scoring/explainGET /api/v1/scoring/historyGET /api/v1/scoring/modelsPUT /api/v1/scoring/models/{model_type}GET /api/v1/scoring/results |
| Measurement 9 operations |
GET /api/v1/measurement/attributionsPOST /api/v1/measurement/calibrateGET /api/v1/measurement/calibration-historyGET /api/v1/measurement/dashboardGET /api/v1/measurement/experimentsPOST /api/v1/measurement/experimentsGET /api/v1/measurement/experiments/{id}POST /api/v1/measurement/experiments/{id}/cancel |
| Social - Reddit 16 operations |
GET /api/v1/social/reddit/authors/:author/commentsFetch recent comments authored by a Reddit user GET /api/v1/social/reddit/authors/:author/postsFetch recent posts authored by a Reddit user POST /api/v1/social/reddit/discoverDiscover relevant Reddit communities by keywords GET /api/v1/social/reddit/mentionsList Reddit mentions GET /api/v1/social/reddit/mentions/:mention_idGet a Reddit mention GET /api/v1/social/reddit/monitorsList Reddit monitors POST /api/v1/social/reddit/monitorsCreate a Reddit monitor DELETE /api/v1/social/reddit/monitors/:monitor_idDelete a Reddit monitor |
| Social - LinkedIn 8 operations |
GET /api/v1/social/linkedin/mentionsList LinkedIn mentions GET /api/v1/social/linkedin/mentions/:mention_idGet a LinkedIn mention GET /api/v1/social/linkedin/monitorsList LinkedIn monitors POST /api/v1/social/linkedin/monitorsCreate a LinkedIn monitor DELETE /api/v1/social/linkedin/monitors/:monitor_idDelete a LinkedIn monitor GET /api/v1/social/linkedin/monitors/:monitor_idGet a LinkedIn monitor PUT /api/v1/social/linkedin/monitors/:monitor_idUpdate a LinkedIn monitor POST /api/v1/social/linkedin/monitors/:monitor_id/pollPoll a LinkedIn monitor |
| Social - Channels 1 operations |
GET /api/v1/social/:platform/channels/:channel_id/profileGet a channel profile for any supported social platform |
| Usage 1 operations |
GET /api/v1/usageGet the calling account's API usage, bucketed by key, endpoint group and day |
| Graph 1 operations |
GET /api/v1/graph/timelineGet a chronological event timeline for an entity in the temporal knowledge graph |
| Telemetry 22 operations |
GET /api/v1/telemetry/dashboardGET /api/v1/telemetry/destinationsPOST /api/v1/telemetry/destinationsDELETE /api/v1/telemetry/destinations/{id}PUT /api/v1/telemetry/destinations/{id}GET /api/v1/telemetry/errorsGET /api/v1/telemetry/eventsPOST /api/v1/telemetry/funnel |
| Artifacts 6 operations |
GET /api/v1/artifactsList artifacts POST /api/v1/artifactsRegister artifact DELETE /api/v1/artifacts/:artifact_idDelete artifact GET /api/v1/artifacts/:artifact_idGet artifact POST /api/v1/artifacts/:artifact_id/downloadDownload artifact GET /api/v1/artifacts/:artifact_id/urlGet artifact URL |
| Asset Definitions 5 operations |
GET /api/v1/assets/definitionsPOST /api/v1/assets/definitionsDELETE /api/v1/assets/definitions/{id}GET /api/v1/assets/definitions/{id}PUT /api/v1/assets/definitions/{id} |
| Lineage 7 operations |
GET /api/v1/lineage/assetsPOST /api/v1/lineage/assetsGET /api/v1/lineage/assets/{asset_id}GET /api/v1/lineage/assets/{asset_id}/graphGET /api/v1/lineage/consentPOST /api/v1/lineage/consentPOST /api/v1/lineage/edges |
| Observatory 4 operations |
GET /api/v1/observatory/anomaliesPOST /api/v1/observatory/comparePOST /api/v1/observatory/profileGET /api/v1/observatory/profiles/{dataset_id} |
| ETL 14 operations |
POST /api/v1/etl/compressCompress or decompress POST /api/v1/etl/fetchFetch remote artifact POST /api/v1/etl/loadBulk load Parquet to PostgreSQL POST /api/v1/etl/parseParse JSON/CSV to Parquet GET /api/v1/etl/pipelinesList ETL pipelines POST /api/v1/etl/pipelinesCreate ETL pipeline DELETE /api/v1/etl/pipelines/:pipeline_idDelete ETL pipeline GET /api/v1/etl/pipelines/:pipeline_idGet ETL pipeline |
| Query 10 operations |
POST /api/v1/queryExecute a SQL query over Parquet files POST /api/v1/query/arrowArrow-native query POST /api/v1/query/describeDescribe schema of a Parquet file POST /api/v1/query/exportExport query results POST /api/v1/query/hybridHybrid SQL query POST /api/v1/query/register-tableRegister DataFusion table POST /api/v1/query/register-viewRegister Parquet view POST /api/v1/query/schemaGet view schema |
| SchemaRegistry 4 operations |
GET /api/v1/schemasPOST /api/v1/schemasPOST /api/v1/schemas/validateGET /api/v1/schemas/{name}/versions |
| ScheduledAudits 5 operations |
GET /api/v1/scheduled-auditsPOST /api/v1/scheduled-auditsDELETE /api/v1/scheduled-audits/{id}GET /api/v1/scheduled-audits/{id}PUT /api/v1/scheduled-audits/{id} |
| SEO Reports 2 operations |
GET /api/v1/seo/reportsGET /api/v1/seo/reports/{report_id} |
| Cost Attribution 3 operations |
POST /api/v1/cost/recordGET /api/v1/cost/recordsGET /api/v1/cost/summary |
| Healing 6 operations |
POST /api/v1/healing/auto-healAuto-heal pipeline POST /api/v1/healing/diagnoseDiagnose pipeline GET /api/v1/healing/patchesList patches GET /api/v1/healing/patches/:idGet patch POST /api/v1/healing/patches/:id/applyApply patch POST /api/v1/healing/patches/:id/rollbackRollback patch |
| Governance 6 operations |
GET /api/v1/governance/policiesList governance policies POST /api/v1/governance/policiesCreate governance policy GET /api/v1/governance/policies/{id}Get governance policy POST /api/v1/governance/policies/{id}/commentComment on governance policy POST /api/v1/governance/policies/{id}/reviewReview governance policy GET /api/v1/governance/queueGet governance review queue |
| Incidents 7 operations |
GET /api/v1/incidentsList incidents POST /api/v1/incidentsCreate incident GET /api/v1/incidents/{id}Get incident PUT /api/v1/incidents/{id}Update incident POST /api/v1/incidents/{id}/resolveResolve incident GET /api/v1/incidents/{id}/timelineGet incident timeline POST /api/v1/incidents/{id}/triageTriage incident |
| Synthetic 6 operations |
POST /api/v1/synthetic/generateGET /api/v1/synthetic/guardrails/{dataset_id}GET /api/v1/synthetic/jobsGET /api/v1/synthetic/jobs/{id}GET /api/v1/synthetic/operationsPOST /api/v1/synthetic/verify |
| Workflow Templates 13 operations |
GET /api/v1/workflow-templatesList workflow templates POST /api/v1/workflow-templatesCreate workflow template DELETE /api/v1/workflow-templates/:idDelete (archive) workflow template GET /api/v1/workflow-templates/:idGet template by ID PUT /api/v1/workflow-templates/:idUpdate workflow template POST /api/v1/workflow-templates/:id/runRun workflow template GET /api/v1/workflow-templates/by-name/:nameGet template by name GET /api/v1/workflow-templates/operationsList registered operations |
| OpenRate 4 operations |
GET /api/v1/openrate/rates/compareCompare healthcare rates GET /api/v1/openrate/rates/discrepanciesFind pricing discrepancies GET /api/v1/openrate/rates/pipeline-statusOpenRate pipeline status GET /api/v1/openrate/rates/searchSearch healthcare rates |
| OpenRate Admin 18 operations |
POST /api/v1/openrate/admin/billing-codesUpsert billing codes POST /api/v1/openrate/admin/geoUpsert geographic references POST /api/v1/openrate/admin/hospitalsUpsert hospitals POST /api/v1/openrate/admin/payersUpsert payers GET /api/v1/openrate/admin/pipelinesList OpenRate runs GET /api/v1/openrate/admin/pipelines/:run_idGet OpenRate run POST /api/v1/openrate/admin/pipelines/bootstrapTrigger OpenRate bootstrap POST /api/v1/openrate/admin/pipelines/cms-discoveryTrigger CMS source discovery |
| Quality 8 operations |
POST /api/v1/quality/classifyClassify text content using the taxonomy engine POST /api/v1/quality/decontaminateCheck texts for benchmark contamination POST /api/v1/quality/dedupPOST /api/v1/quality/filterPOST /api/v1/quality/gateRun structured quality gate GET /api/v1/quality/profiles/{dataset_id}POST /api/v1/quality/scoreGET /api/v1/quality/taxonomyGet the taxonomy tree |
Agent and LLM entrypoints
/openapi.json— complete OpenAPI 3.1 machine contract./llms.txt— compact LLM navigation file./llms-full.txt— generated endpoint inventory grouped by domain.- Integrations — includes REST, MCP-compatible clients, and the CrawlKit Pi extension as an agent interface.