Tutorials

These tutorials stay in the docs. The difference is framing: each one now maps to the same CrawlKit backbone — source, run/job, dataset, schema, quality, lineage, policy, query/serve, and measurement.

Recommended path: read Platform Model, skim Complete API Surface, then run the Data Pipeline tutorial before the applied SEO/content tutorials.

⚙

Data Pipeline — core backbone tutorial

Build the core CrawlKit loop: dataset schema, spider, crawl job, enrichment, pipeline DAG, quality, query/export, and lineage. This is the best hands-on introduction to Acquire → Extract → Refine → Serve → Prove.

🔍

Site Audit — SEO/GEO lens

Run an SEO audit as an applied lens over acquisition: crawl pages, inspect issues, generate remediation evidence, and connect outputs to reports, GSC, and scheduled monitoring.

📈

Keyword Tracking — search visibility lens

Use GSC analytics and keyword enrichment as a refinement and measurement layer over URL/page/content datasets.

✍

Content Pipeline — downstream content workflow

Create briefs, drafts, schema markup, internal links, publishing workflows, approvals, and IndexNow submissions as downstream assets linked to source evidence.

New core guides to pair with tutorials

Arrow + DataFusion — ETL, Parquet, query, export, quality, lineage.
Durable Workflows — templates, operation registry, schedules, runs, lineage.
Spiders & Scraping — AI builder, deterministic extraction, crawl jobs, frontier.

Tutorial-to-platform map

Tutorial	Platform layer	Backbone evidence
Data Pipeline	Acquire, Extract, Refine, Serve, Prove	Dataset, spider, crawl job, enrichment job, pipeline run, export, lineage.
Site Audit	SEO/GEO applied lens	Web source, crawl/audit run, page issues, remediation, report, scheduled audit.
Keyword Tracking	Search visibility refinement/measurement	GSC connection, keyword dataset, analytics rows, recommendations, measurement history.
Content Pipeline	Downstream content workflow	Brief, document asset, approvals, schema markup, internal links, IndexNow, performance feedback.

API basics

All tutorials use the CrawlKit REST API at https://api.crawlkit.app/api/v1/. Authenticate with a Bearer token:

curl -X POST https://api.crawlkit.app/api/v1/check-url \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://crawlkit.app"}'