Site Audit Workflow

Run a complete SEO audit on any website — from a quick single-URL check to a full crawl with issue detection, framework-specific fixes, and continuous monitoring.

Time to complete: ~15 minutes. You will need a CrawlKit API key and a website to audit.

Step 1: Quick URL Check

Start with a single URL to see what CrawlKit detects. The /check-url endpoint crawls a page, extracts metadata, runs all 6 analyzers (redirects, metadata, canonical, content, links, status codes), and returns issues with root cause analysis.

curl

curl -X POST https://api.crawlkit.app/api/v1/check-url \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://crawlkit.app",
    "js_rendering": false
  }'

JavaScript / TypeScript

const response = await fetch("https://api.crawlkit.app/api/v1/check-url", {
  method: "POST",
  headers: {
    "Authorization": "Bearer ck_live_YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://crawlkit.app",
    js_rendering: false,
  }),
});

const result = await response.json();
console.log(`Score: ${result.score}/100`);
console.log(`Issues found: ${result.issues.length}`);

Response

{
  "score": 72,
  "url": "https://crawlkit.app",
  "issues": [
    {
      "category": "missing_meta_description",
      "severity": "high",
      "message": "Page is missing a meta description",
      "affected_urls": ["https://crawlkit.app"],
      "evidence": ["No <meta name=\"description\"> tag found in <head>"]
    },
    {
      "category": "thin_content",
      "severity": "medium",
      "message": "Page has thin content (127 words)",
      "affected_urls": ["https://crawlkit.app"],
      "evidence": ["Word count: 127 (minimum recommended: 300)"]
    }
  ],
  "remediations": [
    {
      "issue_ids": ["missing_meta_description"],
      "description": "Add a meta description between 150-160 characters",
      "effort": "trivial",
      "file_changes": [
        {
          "path": "app/layout.tsx",
          "find": "export const metadata: Metadata = {",
          "replace": "export const metadata: Metadata = {\n  description: \"Your compelling meta description here (150-160 chars)\","
        }
      ]
    }
  ],
  "coaching_context": {
    "metadata_summary": {
      "has_title": true,
      "has_description": false,
      "has_h1": true,
      "has_canonical": false,
      "word_count": 127,
      "internal_link_count": 3,
      "external_link_count": 1
    }
  }
}

Step 2: Run a Full Site Audit

For a comprehensive view, run a full site audit. This crawls multiple pages, builds a link graph, detects orphan pages, and calculates a site-wide health score. The authenticated /audit/run endpoint persists results to your account for historical tracking.

curl

curl -X POST https://api.crawlkit.app/api/v1/audit/run \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "site_url": "https://crawlkit.app",
    "max_pages": 50,
    "js_rendering": false,
    "include_link_analysis": true
  }'

JavaScript / TypeScript

const audit = await fetch("https://api.crawlkit.app/api/v1/audit/run", {
  method: "POST",
  headers: {
    "Authorization": "Bearer ck_live_YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    site_url: "https://crawlkit.app",
    max_pages: 50,
    js_rendering: false,
    include_link_analysis: true,
  }),
});

const report = await audit.json();
console.log(`Site score: ${report.score}/100`);
console.log(`Pages crawled: ${report.crawl_stats.pages_crawled}`);
console.log(`Total issues: ${report.issues.length}`);

Response

{
  "audit_id": "aud_8f3k2j1m4n5p",
  "score": 64,
  "site": {
    "url": "https://crawlkit.app",
    "framework": "nextjs_app_router"
  },
  "crawl_stats": {
    "pages_crawled": 47,
    "pages_with_issues": 23,
    "crawl_duration_secs": 12.4,
    "status_codes": { "200": 42, "301": 3, "404": 2 }
  },
  "issues": [
    {
      "category": "orphan_page",
      "severity": "high",
      "message": "5 pages have no internal links pointing to them",
      "affected_urls": [
        "https://docs.crawlkit.app/changelog.html",
        "https://docs.crawlkit.app/guides/workflows.html"
      ]
    },
    {
      "category": "redirect_chain",
      "severity": "medium",
      "message": "Redirect chain with 3 hops detected",
      "affected_urls": ["https://docs.crawlkit.app/changelog.html"]
    }
  ],
  "link_health": {
    "orphan_count": 5,
    "avg_links_per_page": 4.2,
    "max_depth": 6,
    "gini_coefficient": 0.72,
    "shannon_entropy": 0.58,
    "targets_met": {
      "no_orphans": false,
      "max_depth_4": false,
      "avg_links_5": false,
      "gini_below_0_6": false
    }
  }
}

Step 3: Review Issues by Severity

Issues are returned with a severity level. Prioritize critical and high issues first — they have the largest impact on your SEO score.

Severity	Score Penalty	Example Categories
Critical	-20 points each	Noindex on important pages, server errors (5xx), blocked by robots.txt
High	-12 points each	Missing meta description, orphan pages, broken internal links (404)
Medium	-6 points each	Thin content, redirect chains, canonical mismatches
Low	-3 points each	Missing Open Graph tags, title length warnings
Info	-1 point each	Missing alt text, response time warnings

The score is calculated by starting at 100 and applying saturating subtractions for each issue. A score of 80+ is healthy, 60-79 needs attention, and below 60 requires urgent fixes.

Step 4: Generate Framework-Specific Fixes

CrawlKit detects your framework (Next.js App Router, Pages Router, Nuxt, Gatsby, and more) and generates code fixes that target your specific setup. The remediations array in every response includes file_changes with find/replace diffs.

Example: Next.js App Router fix for missing meta description

{
  "issue_ids": ["missing_meta_description"],
  "description": "Export metadata object with description in your layout",
  "effort": "trivial",
  "framework": "nextjs_app_router",
  "file_changes": [
    {
      "path": "app/layout.tsx",
      "find": "export const metadata: Metadata = {",
      "replace": "export const metadata: Metadata = {\n  description: \"Build data pipelines that turn any website into structured data. CrawlKit API docs.\","
    }
  ],
  "commands": []
}

Example: Nuxt 3 fix for missing canonical

{
  "issue_ids": ["missing_canonical"],
  "description": "Add useHead() with canonical link in your page component",
  "effort": "trivial",
  "framework": "nuxt3",
  "file_changes": [
    {
      "path": "pages/index.vue",
      "find": "<script setup>",
      "replace": "<script setup>\nuseHead({\n  link: [{ rel: 'canonical', href: 'https://crawlkit.app/' }]\n})"
    }
  ]
}

Step 5: Set Up Monitoring with GSC Integration

Connect Google Search Console to verify that your fixes actually improve indexing. CrawlKit tracks the full lifecycle: detect issue, apply fix, verify via GSC.

5a. Get OAuth URL

curl -X POST https://api.crawlkit.app/api/v1/gsc/auth/url \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "redirect_uri": "https://your-app.com/callback",
    "scopes": ["webmasters.readonly", "indexing"]
  }'

Response

{
  "authorization_url": "https://accounts.google.com/o/oauth2/v2/auth?client_id=...&scope=...&redirect_uri=..."
}

5b. Exchange OAuth Code

curl -X POST https://api.crawlkit.app/api/v1/gsc/auth/callback \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "code": "4/0OAUTH_CODE_FROM_REDIRECT",
    "redirect_uri": "https://your-app.com/callback"
  }'

5c. Inspect a URL via GSC

curl -X POST https://api.crawlkit.app/api/v1/gsc/inspect \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "site_url": "https://crawlkit.app",
    "url": "https://docs.crawlkit.app/guides/spiders.html"
  }'

Response

{
  "url": "https://docs.crawlkit.app/guides/spiders.html",
  "index_status": {
    "coverage_state": "SubmittedAndIndexed",
    "indexing_state": "IndexingAllowed",
    "last_crawl_time": "2026-03-10T14:22:00Z",
    "page_fetch": { "state": "Successful", "http_status": 200 },
    "robots_txt": { "verdict": "Allowed" },
    "canonical": {
      "user_canonical": "https://docs.crawlkit.app/guides/spiders.html",
      "google_canonical": "https://docs.crawlkit.app/guides/spiders.html"
    }
  }
}

Step 6: Verify Fixes with Feedback Loops

CrawlKit's feedback system tracks every fix through a verification state machine. After you apply a fix, record it and CrawlKit will verify via GSC at 24h, 48h, and 7d intervals.

6a. Record a fix

curl -X POST https://api.crawlkit.app/api/v1/feedback/record \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "site_url": "https://crawlkit.app",
    "event_type": "fix_applied",
    "loop_type": "meta_description",
    "url": "https://docs.crawlkit.app/guides/spiders.html",
    "details": {
      "fix": "Added meta description to app/blog/[slug]/page.tsx",
      "issue_category": "missing_meta_description"
    }
  }'

Response

{
  "loop_id": "fl_9a2b3c4d5e",
  "status": "fix_applied",
  "url": "https://docs.crawlkit.app/guides/spiders.html",
  "next_verification": "2026-03-14T10:30:00Z",
  "state_machine": {
    "current": "fix_applied",
    "next": "verify_24h",
    "terminal_states": ["resolved", "escalated", "abandoned"]
  }
}

6b. Check verification status

curl https://api.crawlkit.app/api/v1/feedback/status?site_url=https://crawlkit.app \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY"

Response

{
  "pending_verifications": [
    {
      "loop_id": "fl_9a2b3c4d5e",
      "url": "https://docs.crawlkit.app/guides/spiders.html",
      "status": "verify_24h",
      "loop_type": "meta_description",
      "next_check": "2026-03-14T10:30:00Z"
    }
  ],
  "recently_resolved": [
    {
      "loop_id": "fl_7x8y9z0a1b",
      "url": "https://crawlkit.app",
      "status": "resolved",
      "resolution_time_hours": 52
    }
  ]
}

6c. View overall stats

curl https://api.crawlkit.app/api/v1/feedback/stats?site_url=https://crawlkit.app \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY"

{
  "total_loops": 34,
  "resolved": 28,
  "pending": 4,
  "escalated": 1,
  "abandoned": 1,
  "success_rate": 0.82,
  "avg_resolution_hours": 48.3,
  "by_category": {
    "missing_meta_description": { "total": 12, "resolved": 11 },
    "orphan_page": { "total": 8, "resolved": 6 },
    "thin_content": { "total": 5, "resolved": 4 }
  }
}

What's Next

Set up keyword tracking to monitor your rankings
Build a data pipeline to collect competitor intelligence
Explore the Analysis API reference for advanced options