Authentication

CrawlKit supports two authentication methods: API keys for programmatic access and session JWTs for dashboard-integrated workflows. Both use the Authorization: Bearer header.

Authentication Methods

API Key (Recommended)

API keys are the simplest way to authenticate. Create one in the CrawlKit dashboard under Settings → API Keys. Keys start with ck_live_ for production and ck_test_ for local development.

• Scoped to your account — all resources created with the key belong to your account.
• Keys are hashed (SHA-256) before storage — CrawlKit never stores your raw key.
• Revoke keys instantly from the dashboard if compromised.

Session JWT

If you are building on top of the CrawlKit web dashboard, the frontend uses session JWTs. The JWT is extracted server-side and forwarded to the API. This is primarily for internal use by the dashboard app.

• JWT contains the user ID, which maps to CrawlKit tenant accounts through the account membership layer.
• Team-account requests can select the active tenant with x-crawlkit-account-id or x-account-id.
• Supports team accounts with roles: owner, admin, editor, viewer.
• Tokens expire and are automatically refreshed by the auth client.

Tenant API Key Management

CrawlKit exposes tenant-scoped API token endpoints for account owners and admins. Token records are stored as SHA-256 hashes; the raw token is returned only once at creation time.

Create a token

curl -X POST https://api.crawlkit.app/api/v1/auth/tokens \
  -H "Authorization: Bearer YOUR_DASHBOARD_OR_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production ingestion",
    "environment": "live",
    "scopes": ["datasets:read", "datasets:write"]
  }'

Response:

{
  "token": "ck_live_...",
  "token_info": {
    "id": "f3b0b3d8-...",
    "account_id": "9a8f7c6d-...",
    "name": "Production ingestion",
    "key_prefix": "ck_live_abc12345",
    "scopes": ["datasets:read", "datasets:write"],
    "expires_at": null,
    "last_used_at": null,
    "created_at": "2026-06-03T00:00:00Z"
  }
}

List and revoke tokens

curl https://api.crawlkit.app/api/v1/auth/tokens \
  -H "Authorization: Bearer YOUR_DASHBOARD_OR_ADMIN_TOKEN"

curl -X DELETE https://api.crawlkit.app/api/v1/auth/tokens/TOKEN_ID \
  -H "Authorization: Bearer YOUR_DASHBOARD_OR_ADMIN_TOKEN"

For dashboard sessions with access to multiple team accounts, pass x-crawlkit-account-id or an account_id query/body field to target a specific tenant. Empty scopes means full tenant access for backwards compatibility with legacy keys.

Code Samples

curl

curl -X POST https://api.crawlkit.app/api/v1/check-url \
  -H "Authorization: Bearer ck_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://crawlkit.app"}'

JavaScript / TypeScript

const CRAWLKIT_API_KEY = process.env.CRAWLKIT_API_KEY;
const BASE_URL = "https://api.crawlkit.app/api/v1";

async function checkUrl(url: string) {
  const response = await fetch(`${BASE_URL}/check-url`, {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${CRAWLKIT_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ url }),
  });

  if (!response.ok) {
    const error = await response.json();
    throw new Error(`CrawlKit API error: ${error.message}`);
  }

  return response.json();
}

const result = await checkUrl("https://crawlkit.app");
console.log(`SEO Score: ${result.score}/100`);

Python

import os
import requests

CRAWLKIT_API_KEY = os.environ["CRAWLKIT_API_KEY"]
BASE_URL = "https://api.crawlkit.app/api/v1"

def check_url(url: str) -> dict:
    response = requests.post(
        f"{BASE_URL}/check-url",
        headers={
            "Authorization": f"Bearer {CRAWLKIT_API_KEY}",
            "Content-Type": "application/json",
        },
        json={"url": url},
    )
    response.raise_for_status()
    return response.json()

result = check_url("https://crawlkit.app")
print(f"SEO Score: {result['score']}/100")

Rust

use reqwest::Client;
use serde_json::json;

let client = Client::new();
let api_key = std::env::var("CRAWLKIT_API_KEY")
    .expect("CRAWLKIT_API_KEY must be set");

let response = client
    .post("https://api.crawlkit.app/api/v1/check-url")
    .bearer_auth(&api_key)
    .json(&json!({"url": "https://crawlkit.app"}))
    .send()
    .await?;

let result: serde_json::Value = response.json().await?;
println!("SEO Score: {}/100", result["score"]);

Subscription Tiers

Your API key inherits the subscription tier of your account. Higher tiers unlock more endpoints and higher rate limits.

Validating Your Key

You can verify that an API key is valid and check its tenant, tier, scopes, and usage with the /auth/validate endpoint:

curl -X POST https://api.crawlkit.app/api/v1/auth/validate \
  -H "Content-Type: application/json" \
  -d '{"api_key":"ck_live_YOUR_API_KEY"}'

{
  "valid": true,
  "account_id": "9a8f7c6d-...",
  "tier": "pro",
  "scopes": ["datasets:read", "datasets:write"],
  "usage": {
    "checks_today": 42,
    "checks_limit": 1000,
    "audits_this_week": 7,
    "audits_limit": 100
  }
}

Local Development

When running CrawlKit locally via make dev-up, use the pre-seeded development API key:

# Local development API key (seeded in the dev database)
export CRAWLKIT_API_KEY="ck_test_dev_local_crawlkit_2024"

# Local API runs on port 3001
curl -X POST http://localhost:3001/api/v1/check-url \
  -H "Authorization: Bearer $CRAWLKIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://crawlkit.app"}'

Security Best Practices

• Store API keys in environment variables or a secrets manager — never commit them to source control.
• Use separate keys for development, staging, and production environments.
• Rotate keys periodically and revoke any keys that may have been exposed.
• Set the most restrictive permissions needed for each key.
• Monitor the /auth/usage endpoint for unexpected usage patterns.