# API Keys
API keys authenticate your requests to the API. You manage them entirely from the dashboard: set a category, restrict usage with rolling quotas, add an expiry date, and monitor activity per key through request logs.
For how to include a key in a request, see the [Introduction](/).
Creating a key [#creating-a-key]
Open the form [#open-the-form]
Go to [dashboard → API Keys → New](https://app.browsersolver.com).
Fill in the fields [#fill-in-the-fields]
Give the key an **alias** (unique label for your own reference), pick a **category**, and optionally configure quotas and an expiry date.
Copy the secret [#copy-the-secret]
After saving, the secret is shown **once**. Copy it immediately and store it in an environment variable. It cannot be retrieved again.
Key fields [#key-fields]
| Field | Description |
| ------------------ | ------------------------------------------------------------------------------------------------ |
| **Alias** | Unique label per account. Shows up in logs and dashboard filters. |
| **Category** | Organizational tag (see [Categories](#categories)). Has no effect on API behavior. |
| **Active** | Enables or disables the key. Inactive keys are rejected immediately with `401 Inactive API Key`. |
| **Expires** | Optional date after which the key stops working. Leave blank for no expiry. |
| **Request quotas** | Optional per-key caps on rolling time windows (see [Per-key quotas](#per-key-quotas)). |
Categories [#categories]
Categories tag keys for your own organization. They appear in dashboard filters and analytics but do not change how the API processes a request.
| Category | Typical use |
| ------------- | --------------------------------------------------- |
| `production` | Live traffic, customer-facing systems |
| `development` | Local development and testing |
| `staging` | Pre-production and CI environments |
| Custom | Any free-text label, e.g. `sandbox`, `partner-acme` |
Keep one key per environment so you can revoke or rotate it independently.
Per-key quotas [#per-key-quotas]
Each key can have independent hard caps on request volume, enforced on rolling windows (not calendar boundaries):
| Quota | Window |
| ----------- | ----------------------------------------------------- |
| **Total** | All-time: the key is blocked permanently once reached |
| **Daily** | Last 24 hours (rolling, not midnight-to-midnight) |
| **Weekly** | Last 7 days (rolling) |
| **Monthly** | Last 30 days (rolling) |
Leave a field blank to apply no cap for that window. When a quota is reached, the API returns `401 Rate Limit Exceeded` with the quota name in the message:
```json
{ "error": "Rate limit exceeded: quota total" }
```
```json
{ "error": "Rate limit exceeded: quota daily" }
```
```json
{ "error": "Rate limit exceeded: quota weekly" }
```
```json
{ "error": "Rate limit exceeded: quota monthly" }
```
Quota counts are aggregated from request logs and cached for up to 24 hours. Enforcement is accurate within that window, not to the exact request.
Per-key quotas are distinct from **plan-level rate limits** (per-second and per-minute caps that apply account-wide). Plan limits return `429 Too Many Requests`. See [Error Handling](/error-handling) for details.
Expiry [#expiry]
Set an expiry date on keys used in scripts, CI pipelines, or partner integrations. Once the date passes, the API returns:
```json
{ "error": "Expired API Key" }
```
HTTP status `401`. Create and deploy a replacement key before the expiry date to avoid any downtime.
Request logs [#request-logs]
Every request made with a key is recorded under **API Keys → Logs** in the dashboard. Each entry shows:
| Field | Description |
| -------- | -------------------------------- |
| Endpoint | HTTP method and path |
| Status | Response status code |
| Duration | Time to first byte (ms) |
| IP | Caller IP address |
| Credits | Credits charged for this request |
Use logs to audit activity per key, debug unexpected errors, or identify traffic from compromised keys.
Deactivating vs. deleting [#deactivating-vs-deleting]
| Action | Effect |
| -------------- | ---------------------------------------------------------------------------------------------------- |
| **Deactivate** | Key is blocked immediately. Usage history and settings are kept. Can be reactivated. |
| **Delete** | Permanently removes the key and its configuration. Only possible if the key has **never been used**. |
Prefer deactivation over deletion. If a key has request history, it cannot be deleted. The history is retained for billing and auditing.
Rotating a key [#rotating-a-key]
Create the replacement [#create-the-replacement]
Go to [dashboard → API Keys → New](https://app.browsersolver.com) and create a new key with the same category and quota settings. Give it a new alias to distinguish it.
Deploy the new key [#deploy-the-new-key]
Update the environment variable in every system that uses the old key and redeploy.
Deactivate the old key [#deactivate-the-old-key]
Set the old key to **inactive** in the dashboard. This blocks any remaining requests without removing its history.
Error reference [#error-reference]
| Error | Status | Cause |
| ------------------------------------------------ | ------ | ---------------------------------------------- |
| `Invalid API Key` | 401 | Key not found or malformed |
| `Inactive API Key` | 401 | Key is disabled |
| `Expired API Key` | 401 | Key has passed its expiry date |
| `Rate limit exceeded: quota …` | 401 | Per-key rolling quota reached |
| `exceeded the … rate limit on your subscription` | 429 | Plan-level per-second or per-minute cap |
| `IP temporarily blocked` | 403 | Too many failed auth attempts from the same IP |
***
See also: [Best Practices](/best-practices) for security guidelines and [Glossary](/glossary) for term definitions.
# Auto Top-Up
Never run out of credits mid-workflow. Auto Top-Up monitors your credit balance and purchases a credit pack on your behalf the moment your balance drops below a configurable threshold.
How it works [#how-it-works]
Set a threshold [#set-a-threshold]
Choose the percentage of your remaining credits at which the automatic purchase should trigger. When your balance falls below this level, Auto Top-Up activates.
Pick a credit pack [#pick-a-credit-pack]
Select the one-time credit pack to purchase automatically. The pack is charged to the payment method on file in your account.
Credits are added instantly [#credits-are-added-instantly]
Once the purchase completes, the credits are added to your balance immediately. No downtime, no failed requests.
Enabling Auto Top-Up [#enabling-auto-top-up]
1. Go to **[Settings → Subscription](https://app.browsersolver.com)** in your dashboard.
2. Open the **Auto Top-Up** section.
3. Toggle **Enable** and choose a credit pack and a threshold percentage.
4. Click **Save** to activate.
Auto Top-Up is only available on paid plans. Users on the Free plan cannot enable automatic purchases.
Configuration options [#configuration-options]
| Option | Description |
| ----------------- | ------------------------------------------------------------------------------------------------------ |
| **Credit pack** | The one-time credit bundle purchased automatically when the threshold is reached. |
| **Threshold (%)** | The percentage of your remaining credits that triggers the top-up. Accepts values between 10% and 30%. |
Billing [#billing]
Automatic purchases are charged to the payment method associated with your account. You will receive an invoice by email for each automatic purchase, just like a manual credit purchase.
Make sure your payment method is up to date. If a charge fails, Auto Top-Up is automatically **disabled** on your account to prevent repeated failed attempts. You will need to re-enable it manually once your payment method is updated.
Built-in protections [#built-in-protections]
Auto Top-Up includes two safeguards to prevent unexpected charges:
| Protection | How it works |
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Threshold-only trigger** | A purchase only fires when your remaining balance drops below the configured percentage. It never runs on a fixed schedule or at a specific time. |
| **72-hour cooldown** | After a top-up completes, the system waits at least 72 hours before it can trigger again, even if your balance drops below the threshold a second time during that window. |
| **Auto-disable on failure** | If the Stripe charge fails, Auto Top-Up is automatically disabled to protect your account from repeated failed payment attempts. |
Frequently asked questions [#frequently-asked-questions]
Will I be charged immediately when I enable Auto Top-Up? [#will-i-be-charged-immediately-when-i-enable-auto-top-up]
No. Enabling Auto Top-Up does not trigger a purchase. The charge only fires when your balance actually drops below the configured threshold.
Can I be charged multiple times in a short period? [#can-i-be-charged-multiple-times-in-a-short-period]
No. A built-in 72-hour cooldown prevents more than one automatic purchase from firing within any 72-hour window, regardless of how your balance fluctuates.
Can I disable Auto Top-Up at any time? [#can-i-disable-auto-top-up-at-any-time]
Yes. Toggle the **Enable** switch off in **Settings → Subscription → Auto Top-Up** and save. No further automatic purchases will occur.
What happens if my payment fails? [#what-happens-if-my-payment-fails]
Auto Top-Up is automatically disabled on your account. You will need to update your payment method and re-enable the feature manually from your subscription settings.
Is Auto Top-Up available on the Free plan? [#is-auto-top-up-available-on-the-free-plan]
No. You need an active paid subscription to enable automatic credit top-ups.
# Best Practices
Follow these guidelines to build integrations that are reliable, cost-efficient, and easy to maintain.
Credit efficiency [#credit-efficiency]
Check your balance before large batches [#check-your-balance-before-large-batches]
Before triggering a high-volume operation, verify you have enough credits. The [Usage](/usage) endpoint is free and returns your current balance in real time.
```bash
curl "https://api.browsersolver.com/v1/usage" \
-H "x-api-key: YOUR_API_KEY"
```
Enable [Auto Top-Up](/auto-top-up) to automatically refill your balance when it drops below a threshold, so your workflows keep running without interruption.
Avoid redundant calls [#avoid-redundant-calls]
Cache results on your side whenever the underlying data is unlikely to change between requests. Re-fetching identical inputs wastes credits.
```typescript
const cache = new Map()
async function fetchCached(key: string, fetcher: () => Promise) {
if (cache.has(key)) return cache.get(key)
const result = await fetcher()
cache.set(key, result)
return result
}
```
Only call what you need [#only-call-what-you-need]
Each endpoint has a defined credit cost listed on the [Credits](/credits) page. Prefer lighter endpoints when a full response is not required.
***
Authentication [#authentication]
Store keys in environment variables [#store-keys-in-environment-variables]
Never hardcode your API key in source code. Use environment variables and load them at runtime.
```typescript
const apiKey = process.env.BROWSERSOLVER_API_KEY
if (!apiKey) throw new Error("API key is not set")
```
```python
import os
api_key = os.environ["BROWSERSOLVER_API_KEY"]
```
```php
$apiKey = getenv('BROWSERSOLVER_API_KEY');
```
```go
apiKey := os.Getenv("BROWSERSOLVER_API_KEY")
```
Use separate keys per environment [#use-separate-keys-per-environment]
Create distinct API keys for development, staging, and production in your [dashboard](https://app.browsersolver.com). This lets you revoke or rotate a compromised key without affecting other environments.
| Environment | Recommended category |
| ------------ | -------------------- |
| Local dev | `development` |
| CI / staging | `development` |
| Production | `production` |
Rotate keys periodically [#rotate-keys-periodically]
Treat API keys like passwords. Set an expiry date on sensitive keys and rotate them on a regular schedule from the API Keys section of your account settings.
Never expose your API key in client-side code, browser requests, or public repositories. Always proxy calls through your own backend.
***
Error handling [#error-handling]
Always check the HTTP status code [#always-check-the-http-status-code]
Do not assume a request succeeded. Every response should be checked against its HTTP status code before processing the body. See the [Error Handling](/error-handling) guide for a full breakdown.
Retry on transient errors [#retry-on-transient-errors]
Server errors (`500`) and network timeouts are transient. Implement exponential backoff to retry automatically:
```typescript
async function fetchWithRetry(url: string, options: RequestInit, maxRetries = 3) {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
try {
const res = await fetch(url, options)
if (res.status === 500 && attempt < maxRetries) {
await new Promise(r => setTimeout(r, 500 * 2 ** attempt))
continue
}
return res
} catch {
if (attempt === maxRetries) throw new Error("Max retries reached")
await new Promise(r => setTimeout(r, 500 * 2 ** attempt))
}
}
}
```
Never retry on 402 without topping up [#never-retry-on-402-without-topping-up]
A `402 Payment Required` response means your balance is empty. Retrying the same call immediately will not help. Add credits from your [dashboard](https://app.browsersolver.com) or enable [Auto Top-Up](/auto-top-up).
***
Rate limits [#rate-limits]
Spread requests over time [#spread-requests-over-time]
If you need to send a large number of calls, distribute them across time rather than firing them all at once. A simple delay between iterations prevents hitting rate limits.
```typescript
const DELAY_MS = 100 // adjust based on your plan
for (const item of items) {
await processItem(item)
await new Promise(r => setTimeout(r, DELAY_MS))
}
```
Monitor the 401 Rate Limit Exceeded response [#monitor-the-401-rate-limit-exceeded-response]
When rate-limited, the API returns `401` with status `Rate Limit Exceeded`. Back off and wait before retrying. Consider upgrading your plan if you consistently hit limits.
***
Security [#security]
Never make API calls from the browser [#never-make-api-calls-from-the-browser]
Your API key would be visible to anyone who inspects the network tab. Always route calls through your own server.
```
Browser → Your backend → BrowserSolver API
```
Validate and sanitize inputs [#validate-and-sanitize-inputs]
If your application accepts user-supplied parameters that are passed to the API (URLs, search queries, etc.), sanitize them to prevent injection or unexpected behavior.
***
Monitoring [#monitoring]
Track credit consumption [#track-credit-consumption]
Query the [Usage](/usage) endpoint on a schedule (e.g. daily) and alert when consumption exceeds a threshold. This prevents surprises at the end of a billing period.
| Metric | Why it matters |
| --------------------------- | -------------------------------- |
| `remaining` | Warns you before credits run out |
| `subscription.percent_used` | Tracks your plan utilization |
| `credits.remaining` | Monitors extra credit balance |
Log request context [#log-request-context]
Store the HTTP status code and a timestamp for every API call. This makes it easy to audit credit consumption and debug failures after the fact.
```typescript
async function apiCall(endpoint: string, params: Record) {
const url = new URL(`https://api.browsersolver.com${endpoint}`)
Object.entries(params).forEach(([k, v]) => url.searchParams.set(k, v))
const res = await fetch(url.toString(), {
headers: { "x-api-key": process.env.API_KEY! },
})
console.log(JSON.stringify({
endpoint,
status: res.status,
billed: res.status === 200 || res.status === 201,
timestamp: new Date().toISOString(),
}))
return res
}
```
***
Checklist [#checklist]
Use this checklist before going to production:
* [ ] API key stored in an environment variable, not in source code
* [ ] Separate keys for development and production
* [ ] HTTP status codes checked on every response
* [ ] Retry logic with exponential backoff for `500` errors
* [ ] No retries on `402` without adding credits first
* [ ] Rate limit handling in place
* [ ] Credit balance monitored and alerts configured
* [ ] Auto Top-Up enabled (or a manual top-up process defined)
* [ ] All API calls proxied through your backend
***
See also: [Error Handling](/error-handling) for a complete list of status codes and [Glossary](/glossary) for term definitions.
# Credits
Each API call consumes credits from your account balance. The table below lists every endpoint and its cost.
You are charged for successful requests: `200` for synchronous calls and `201` for async jobs. `404` billing is endpoint-specific (check each endpoint page). All other responses (4xx, 5xx) are free.
| Endpoint | Method | Path | Credits |
| --------------- | ------ | ----------- | ------- |
| [Usage](/usage) | `GET` | `/v1/usage` | 0 |
# Error Handling
Every request to the BrowserSolver API returns a standard HTTP status code. This guide covers what each code means, when credits are consumed, and how to handle failures in your integration.
Response format [#response-format]
Successful responses return a JSON object with the data for that endpoint. Error responses follow a consistent structure:
```json
{
"error": "Human-readable description of what went wrong"
}
```
Always check the HTTP status code **before** reading the body.
***
Status codes [#status-codes]
200 Success [#200-success]
**Billed.** The request completed successfully. Credits are deducted from your balance.
```typescript
const res = await fetch(url, { headers: { "x-api-key": API_KEY } })
if (res.status === 200) {
const data = await res.json()
// process data
}
```
201 Job Created [#201-job-created]
**Billed.** An asynchronous job was created and accepted. Credits are deducted immediately when the job is queued.
202 Accepted (async) [#202-accepted-async]
**Not billed yet.** The request was accepted and queued for processing. Use the returned job ID to poll for the result. Credits are only billed when the job completes successfully.
400 Bad Request [#400-bad-request]
**Not billed.** Your request contains invalid or missing parameters.
**Common causes:**
* A required query parameter is missing
* A parameter value has the wrong type or format
* The request body is malformed
**What to do:** Re-read the endpoint documentation and verify every required parameter. Log the full request URL to spot typos.
```typescript
if (res.status === 400) {
const { error } = await res.json()
console.error("Bad request:", error)
// do not retry, fix the parameters first
}
```
401 Unauthorized [#401-unauthorized]
**Not billed.** Returned for authentication issues **and** rate limiting. Check the error message to distinguish between them.
Your `x-api-key` header is missing or contains an unrecognized value.
**Fix:** Copy your key from your [dashboard → API Keys](https://app.browsersolver.com) and make sure the `x-api-key` header is present on every request.
The key exists but has been deactivated.
**Fix:** Go to your [dashboard → API Keys](https://app.browsersolver.com) and activate the key, or generate a new one.
The key has passed its expiry date.
**Fix:** Generate a new key or extend the expiry from your dashboard.
You have sent too many requests in a short window.
**Fix:** Back off and retry after a delay. If you hit this regularly, consider upgrading your plan.
```typescript
if (res.status === 401) {
const { error } = await res.json()
if (error.toLowerCase().includes("rate limit")) {
await new Promise(r => setTimeout(r, 2000))
// retry
}
}
```
402 Payment Required [#402-payment-required]
**Not billed.** Your credit balance is empty. The request was not executed.
**What to do:**
1. Top up your balance from your [dashboard](https://app.browsersolver.com).
2. Or enable [Auto Top-Up](/auto-top-up) to prevent this from happening again.
Auto Top-Up only triggers when your balance drops below the configured threshold. It never runs on a fixed schedule or at a specific time. A built-in **72-hour cooldown** also prevents multiple charges in a short period: once a top-up fires, it cannot trigger again for 72 hours regardless of how your balance moves. If a charge fails, Auto Top-Up is automatically disabled to protect your account.
Do not retry a `402` response immediately. The call will fail again until you add credits. Set up an alert so your team is notified when this happens.
```typescript
if (res.status === 402) {
// alert your team or trigger an auto top-up flow
throw new Error("Insufficient credits. Top up at https://app.browsersolver.com")
}
```
403 Forbidden [#403-forbidden]
**Not billed.** Your key does not have permission to access this endpoint.
**What to do:** Verify that your key's permissions cover the endpoint you are calling. Contact support if you believe this is a mistake.
404 Not Found [#404-not-found]
**May be billed** (endpoint-specific). The resource was not found. Check the individual endpoint documentation to confirm whether this response is charged.
```typescript
if (res.status === 404) {
// handle "no result" gracefully, not as a fatal error
return null
}
```
500 Internal Server Error [#500-internal-server-error]
**Not billed.** An unexpected error occurred on our side.
**What to do:** Retry with exponential backoff. If the issue persists, contact support.
```typescript
async function fetchWithRetry(url: string, options: RequestInit, maxRetries = 3) {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
const res = await fetch(url, options)
if (res.status !== 500 || attempt === maxRetries) return res
const delay = 500 * 2 ** attempt
console.warn(`500 error, retrying in ${delay}ms (attempt ${attempt + 1}/${maxRetries})`)
await new Promise(r => setTimeout(r, delay))
}
}
```
***
Complete error handler [#complete-error-handler]
A single function that covers every status code:
```typescript
interface ApiResult {
data: T | null
error: string | null
status: number
billed: boolean
}
async function apiCall(url: string, apiKey: string): Promise> {
const res = await fetch(url, {
headers: { "x-api-key": apiKey },
})
// 200 and 201 are always billed; 404 billing is endpoint-specific
const billed = res.status === 200 || res.status === 201
if (res.status === 200) {
return { data: await res.json() as T, error: null, status: 200, billed }
}
const body = await res.json().catch(() => ({ error: "Unknown error" }))
const error = body?.error ?? "Unknown error"
switch (res.status) {
case 400:
console.error("[400] Bad request, fix your parameters:", error)
break
case 401:
if (error.toLowerCase().includes("rate limit")) {
console.warn("[401] Rate limit, back off and retry")
} else {
console.error("[401] Auth error, check your API key:", error)
}
break
case 402:
console.error("[402] No credits remaining. Top up at https://app.browsersolver.com")
break
case 403:
console.error("[403] Forbidden, check your key permissions")
break
case 404:
console.warn("[404] Not found, no result for this request")
break
case 500:
console.error("[500] Server error, retry with backoff")
break
default:
console.error(`[${res.status}] Unexpected status:`, error)
}
return { data: null, error, status: res.status, billed }
}
```
***
Retry strategy [#retry-strategy]
| Status | Retry? | Strategy |
| ------------------ | ------ | ------------------------------------ |
| `400` | No | Fix the request first |
| `401` (rate limit) | Yes | Wait 2–5 s, then retry |
| `401` (auth) | No | Fix the API key |
| `402` | No | Add credits first |
| `403` | No | Check permissions |
| `404` | No | Handle as empty result |
| `500` | Yes | Exponential backoff (max 3 attempts) |
***
Billing summary [#billing-summary]
| Status | Billed |
| --------------- | ------------------------------------------- |
| `200` | Yes |
| `201` | Yes |
| `404` | Endpoint-specific (check the endpoint docs) |
| All other codes | No |
See the [Credits](/credits) page for a full cost breakdown per endpoint. For broader integration advice, see [Best Practices](/best-practices).
# Glossary
A reference for the terms you will encounter in the documentation, dashboard, and API responses.
***
Account [#account]
See [Tenant](#tenant).
***
API key [#api-key]
A secret string used to authenticate API requests. Each key belongs to a tenant and has an alias, category, optional quotas, and an optional expiry date. Passed via the `x-api-key` header or `x_api_key` query parameter.
See [API Keys](/api-keys) for full details.
***
Alias [#alias]
The human-readable name of an API key, unique within a tenant. Used to identify keys in the dashboard and analytics.
***
Auto Top-Up [#auto-top-up]
A feature that automatically purchases a credit pack when your remaining balance drops below a configured threshold. Protected by a 72-hour cooldown between purchases. Requires an active paid subscription.
See [Auto Top-Up](/auto-top-up) for full details.
***
Billing period [#billing-period]
The active subscription window used to measure quota consumption. Determined by the `currentPeriodStart` and `currentPeriodEnd` fields from Stripe. If no active subscription is found, the billing period defaults to the current calendar month.
***
Category [#category]
A label on an API key (`production`, `development`, `staging`, or custom). Used for filtering and organization in the dashboard. Does not change how the API processes the request.
***
Credit [#credit]
The unit of API consumption. Each billable request deducts a number of credits determined by the endpoint's cost. Credits are stored as ledger entries: a negative amount means credits were granted, a positive amount means they were consumed.
See [Credits](/credits) for the cost per endpoint.
***
Extra credits [#extra-credits]
Credits purchased separately from a subscription (via one-time packs or Auto Top-Up). They are drawn from when the subscription quota is exhausted. Also includes onboarding grants. Distinct from the subscription quota because they do not reset at the end of a billing period.
***
Per-key quota [#per-key-quota]
An optional request limit set on a specific API key. Enforced on rolling time windows: all-time total, last 24 hours, last 7 days, and last 30 days. Independent of the subscription quota.
***
Plan [#plan]
A subscription product (e.g. "Starter", "Pro") that defines a set of features: API call quota, rate limits, access to extra credits, and more. A tenant can hold multiple active plan products simultaneously.
***
Quota [#quota]
The maximum number of API calls (weighted by credit cost) allowed within a billing period. Comes from the `api` feature on the active plan. When exhausted, extra credits are used if available; otherwise requests are rejected with `402`.
***
Rate limit [#rate-limit]
There are two separate rate-limiting systems. **Plan limits** (per second / per minute) apply to all keys on an account and are set by the subscription plan. Exceeding them returns `429 Too Many Requests`. **Per-key quotas** (total / daily / weekly / monthly rolling windows) are optional hard caps configured per key. Exceeding them returns `401 Rate Limit Exceeded`. See [API Keys](/api-keys) for details.
***
Subscription [#subscription]
A recurring billing arrangement linking a tenant to one or more plans. Managed through Stripe. Determines the billing period, quota, and features available to the tenant.
***
Tenant [#tenant]
The organizational unit in the platform, equivalent to an account or workspace. All API keys, usage, credits, and subscriptions are scoped to a tenant. Users can belong to multiple tenants. See [Account](#account).
***
Threshold [#threshold]
The percentage of remaining credits at which Auto Top-Up activates. Configurable between 10% and 30%.
# Introduction
Get started with the BrowserSolver API in minutes. Make HTTP requests to our endpoints and receive structured JSON responses.
Quick Start [#quick-start]
Get your API key [#get-your-api-key]
Sign up on your [dashboard](https://app.browsersolver.com) and copy your API key from the **Settings** page.
Make your first request [#make-your-first-request]
Include your key in the `x-api-key` header and call any endpoint:
```bash
curl "https://api.browsersolver.com/v1/usage" \
-H "x-api-key: YOUR_API_KEY"
```
Handle the response [#handle-the-response]
Every successful request returns a JSON object with structured data. See individual endpoint pages for response schemas and examples.
Authentication [#authentication]
All API requests must include an `x-api-key` header. You can find your key in your [dashboard → API Keys](https://app.browsersolver.com/).
```bash
curl -H "x-api-key: xxxx" https://api.browsersolver.com/v1/usage
```
Keep your API key secret. Do not expose it in client-side code or public repositories.
Status Codes [#status-codes]
Billing applies to successful requests: `200` for synchronous calls, `201` for asynchronous jobs. `404` billing varies by endpoint (check each endpoint page). All other 4xx/5xx responses are free.
| Code | Billed | Status | Action |
| ----- | ------ | ------------------- | ------------------------------------------------------------------------------------- |
| `200` | Yes | Successful API Call | No action required. |
| `201` | Yes | Job Created | No action required. |
| `202` | No | Accepted (Async) | The request was accepted and is being processed. Use the job ID to check the status. |
| `400` | No | Bad Request | Verify your parameters and their types. Check the documentation for more information. |
| `401` | No | Invalid API Key | Check your API key (`x-api-key` header or `x_api_key` query string). |
| `401` | No | Inactive API Key | Activate your API key in the API Keys section of your account settings. |
| `401` | No | Expired API Key | Update your API key or generate a new one. |
| `401` | No | Rate Limit Exceeded | Consider upgrading your current plan or contact our sales team. |
| `402` | No | Payment Required | Settle any outstanding invoices to continue using the API. |
| `403` | No | Forbidden | Verify your permissions. Contact us if you believe this is a mistake. |
| `404` | Varies | Not Found | Billing is endpoint-specific. Check the individual endpoint documentation. |
| `500` | No | Internal Error | Retry the action or contact our support team. |
See the [Credits](/credits) page for a full breakdown per endpoint, [API Keys](/api-keys) for key management, and [Best Practices](/best-practices) and [Error Handling](/error-handling) for integration guidance. New to the platform? The [Glossary](/glossary) defines all key terms.
Endpoints [#endpoints]
Browse all available endpoints in the sidebar, or jump directly:
# Status Codes
Billing applies to successful requests: `200` for synchronous calls, `201` for asynchronous jobs, and `404` when the resource is not found (unless specified otherwise in the API documentation).
| Code | Billed | Status | Action |
| ----- | ------ | ------------------- | ------------------------------------------------------------------------------------- |
| `200` | Yes | Successful API Call | No action required. |
| `201` | Yes | Job Created | No action required. |
| `202` | No | Accepted (Async) | The request was accepted and is being processed. Use the job ID to check the status. |
| `400` | No | Bad Request | Verify your parameters and their types. Check the documentation for more information. |
| `401` | No | Invalid API Key | Check your API key (`x-api-key` header or `x_api_key` query string). |
| `401` | No | Inactive API Key | Activate your API key in the API Keys section of your account settings. |
| `401` | No | Expired API Key | Update your API key or generate a new one. |
| `401` | No | Rate Limit Exceeded | Consider upgrading your current plan or contact our sales team. |
| `402` | No | Payment Required | Settle any outstanding invoices to continue using the API. |
| `403` | No | Forbidden | Verify your permissions. Contact us if you believe this is a mistake. |
| `404` | Varies | Not Found | Billing is endpoint-specific. Check the individual endpoint documentation. |
| `500` | No | Internal Error | Retry the action or contact our support team. |
# Usage
The Usage endpoint lets you check your remaining credits, subscription quota, rate limits, and API key details at any time. It does not consume any credits.
This endpoint is free. It never consumes any credits.
Request [#request]
```bash
curl "https://api.browsersolver.com/v1/usage" \
-H "x-api-key: YOUR_API_KEY"
```
Response Fields [#response-fields]
| Field | Type | Description |
| --------------------------- | ----------------- | ------------------------------------------------------------ |
| `remaining` | `integer` | Credits available right now |
| `total_used` | `integer` | Credits consumed in the current billing period |
| `renewal_date` | `string` | Date when the subscription renews (ISO 8601) |
| `period.start` | `string` | Start of the current billing period |
| `period.end` | `string` | End of the current billing period |
| `subscription.used` | `integer` | Subscription calls used |
| `subscription.total` | `integer` | Subscription call quota |
| `subscription.remaining` | `integer` | Subscription calls remaining |
| `subscription.percent_used` | `number` | Percentage of subscription quota consumed |
| `credits.given` | `integer` | Extra credits added to your account |
| `credits.consumed` | `integer` | Extra credits consumed |
| `credits.remaining` | `integer` | Extra credits remaining |
| `rate_limit.per_minute` | `integer \| null` | Requests per minute limit (`null` = unlimited) |
| `rate_limit.per_second` | `integer \| null` | Requests per second limit (`null` = unlimited) |
| `account.name` | `string` | Name of the account |
| `account.slug` | `string` | Account identifier slug |
| `api_key.alias` | `string` | Friendly name of the API key |
| `api_key.active` | `boolean` | Whether the key is currently active |
| `api_key.category` | `string` | Key category (e.g. `development`, `production`) |
| `api_key.expires` | `string \| null` | Expiry date of the key (`null` = never expires) |
| `api_key.quotas` | `object` | Custom per-key quotas: `total`, `daily`, `weekly`, `monthly` |
Example Response [#example-response]
```json
{
"remaining": 450,
"total_used": 1550,
"renewal_date": "2026-04-25",
"period": {
"start": "2026-03-25",
"end": "2026-04-25"
},
"subscription": {
"used": 1550,
"total": 2000,
"remaining": 450,
"percent_used": 77
},
"credits": {
"given": 0,
"consumed": 0,
"remaining": 0,
"percent_used": 0
},
"rate_limit": {
"per_minute": null,
"per_second": null
},
"account": {
"name": "Acme Corp",
"slug": "acme-corp"
},
"api_key": {
"alias": "production",
"active": true,
"category": "production",
"expires": null,
"quotas": {
"total": null,
"daily": null,
"weekly": null,
"monthly": null
}
}
}
```
# Collect Google Images
Overview [#overview]
This playbook shows how to collect image URLs, titles, and dimensions from Google Images for a given query. Each result includes the direct image URL, the page it was found on, its source domain, and pixel dimensions.
Prerequisites [#prerequisites]
* An Autom API key — get one at [app.autom.dev](https://app.autom.dev)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default in most PHP installs).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Search for images [#search-for-images]
Call `GET /v1/google/images` with a `q` parameter.
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.autom.dev/v1/google/images",
headers={"x-api-key": API_KEY},
params={"q": "electric car charging station", "gl": "us", "hl": "en"},
)
data = response.json()
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({ q: "electric car charging station", gl: "us", hl: "en" });
const response = await fetch(`https://api.autom.dev/v1/google/images?${params}`, {
headers: { "x-api-key": API_KEY },
});
const data = await response.json();
```
```php
"electric car charging station", "gl" => "us", "hl" => "en"]);
$ch = curl_init("https://api.autom.dev/v1/google/images?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
```
```go
package main
import (
"encoding/json"
"io"
"net/http"
"net/url"
)
func main() {
params := url.Values{"q": {"electric car charging station"}, "gl": {"us"}, "hl": {"en"}}
req, _ := http.NewRequest("GET", "https://api.autom.dev/v1/google/images?"+params.Encode(), nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var data map[string]any
json.Unmarshal(body, &data)
}
```
```java
import java.net.URI;
import java.net.http.*;
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.autom.dev/v1/google/images?q=electric+car+charging+station&gl=us&hl=en"))
.header("x-api-key", "YOUR_API_KEY")
.GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var body = await client.GetStringAsync(
"https://api.autom.dev/v1/google/images?q=electric+car+charging+station&gl=us&hl=en");
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let data = reqwest::Client::new()
.get("https://api.autom.dev/v1/google/images")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("q", "electric car charging station"), ("gl", "us"), ("hl", "en")])
.send().await?.json::().await?;
println!("{:#?}", data);
Ok(())
}
```
Inspect the image results [#inspect-the-image-results]
Each item in `images` has `url` (direct image), `link` (source page), `title`, `domain`, `source`, `image_width`, and `image_height`.
```python
for img in data.get("images", []):
print(f"[{img['position']}] {img['title']}")
print(f" Image : {img['url']}")
print(f" Source: {img['source']} — {img['image_width']}x{img['image_height']}px\n")
```
```typescript
for (const img of data.images ?? []) {
console.log(`[${img.position}] ${img.title}`);
console.log(` Image : ${img.url}`);
console.log(` Source: ${img.source} — ${img.image_width}x${img.image_height}px\n`);
}
```
```php
foreach ($data["images"] ?? [] as $img) {
echo "[{$img['position']}] {$img['title']}\n";
echo " Image : {$img['url']}\n";
echo " Source: {$img['source']} — {$img['image_width']}x{$img['image_height']}px\n\n";
}
```
```go
import "fmt"
for _, r := range data["images"].([]any) {
img := r.(map[string]any)
fmt.Printf("[%.0f] %s\n Image : %s\n Source: %s — %.0fx%.0fpx\n\n",
img["position"], img["title"], img["url"],
img["source"], img["image_width"], img["image_height"])
}
```
```java
import org.json.*;
var images = new JSONObject(response.body()).getJSONArray("images");
for (int i = 0; i < images.length(); i++) {
var img = images.getJSONObject(i);
System.out.printf("[%d] %s%n Image : %s%n Source: %s — %dx%dpx%n%n",
img.getInt("position"), img.getString("title"),
img.getString("url"), img.getString("source"),
img.getInt("image_width"), img.getInt("image_height"));
}
```
```csharp
using System.Text.Json;
var images = JsonDocument.Parse(body).RootElement.GetProperty("images").EnumerateArray();
foreach (var img in images)
{
Console.WriteLine($"[{img.GetProperty("position")}] {img.GetProperty("title")}");
Console.WriteLine($" Image : {img.GetProperty("url")}");
Console.WriteLine($" Source: {img.GetProperty("source")} — {img.GetProperty("image_width")}x{img.GetProperty("image_height")}px\n");
}
```
```rust
if let Some(images) = data["images"].as_array() {
for img in images {
println!("[{}] {}", img["position"], img["title"].as_str().unwrap_or(""));
println!(" Image : {}", img["url"].as_str().unwrap_or(""));
println!(" Source: {} — {}x{}px\n",
img["source"].as_str().unwrap_or(""), img["image_width"], img["image_height"]);
}
}
```
Build an image catalog filtered by minimum size [#build-an-image-catalog-filtered-by-minimum-size]
Filter out thumbnails and save only high-resolution images.
```python
import csv, requests
API_KEY = "YOUR_API_KEY"
QUERY = "electric car charging station"
MIN_WIDTH = 800
def fetch_images(query: str, pages: int = 2) -> list:
results = []
for page in range(1, pages + 1):
r = requests.get("https://api.autom.dev/v1/google/images",
headers={"x-api-key": API_KEY},
params={"q": query, "gl": "us", "hl": "en", "page": page})
results.extend(r.json().get("images", []))
return results
large = [img for img in fetch_images(QUERY) if img.get("image_width", 0) >= MIN_WIDTH]
with open("image_catalog.csv", "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=["position", "title", "url", "source", "image_width", "image_height"])
writer.writeheader()
writer.writerows(large)
print(f"Saved {len(large)} high-res images to image_catalog.csv")
```
```typescript
import { writeFileSync } from "fs";
const API_KEY = "YOUR_API_KEY";
const MIN_WIDTH = 800;
async function fetchImages(query: string, pages = 2): Promise {
const all: any[] = [];
for (let page = 1; page <= pages; page++) {
const params = new URLSearchParams({ q: query, gl: "us", hl: "en", page: String(page) });
const res = await fetch(`https://api.autom.dev/v1/google/images?${params}`, {
headers: { "x-api-key": API_KEY },
});
all.push(...((await res.json()).images ?? []));
}
return all;
}
const images = await fetchImages("electric car charging station");
const large = images.filter(img => (img.image_width ?? 0) >= MIN_WIDTH);
writeFileSync("image_catalog.json", JSON.stringify(large, null, 2));
console.log(`Saved ${large.length} high-res images to image_catalog.json`);
```
```php
$query, "gl" => "us", "hl" => "en", "page" => $page]);
$ch = curl_init("https://api.autom.dev/v1/google/images?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
$all = array_merge($all, $data["images"] ?? []);
}
return $all;
}
$large = array_filter(fetchImages($apiKey, $query), fn($img) => ($img["image_width"] ?? 0) >= $minWidth);
file_put_contents("image_catalog.json", json_encode(array_values($large), JSON_PRETTY_PRINT));
echo "Saved " . count($large) . " high-res images to image_catalog.json\n";
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strconv"
)
func fetchImages(apiKey, query string, pages int) []map[string]any {
var all []map[string]any
for page := 1; page <= pages; page++ {
params := url.Values{"q": {query}, "gl": {"us"}, "hl": {"en"}, "page": {strconv.Itoa(page)}}
req, _ := http.NewRequest("GET", "https://api.autom.dev/v1/google/images?"+params.Encode(), nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
var data map[string]any
json.Unmarshal(body, &data)
for _, r := range data["images"].([]any) {
all = append(all, r.(map[string]any))
}
}
return all
}
func main() {
images := fetchImages("YOUR_API_KEY", "electric car charging station", 2)
minWidth := float64(800)
var large []map[string]any
for _, img := range images {
if img["image_width"].(float64) >= minWidth {
large = append(large, img)
}
}
b, _ := json.MarshalIndent(large, "", " ")
os.WriteFile("image_catalog.json", b, 0644)
fmt.Printf("Saved %d high-res images to image_catalog.json\n", len(large))
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.util.*;
import org.json.*;
public class Main {
public static void main(String[] args) throws Exception {
var apiKey = "YOUR_API_KEY";
var query = URLEncoder.encode("electric car charging station", StandardCharsets.UTF_8);
var minWidth = 800;
var client = HttpClient.newHttpClient();
var all = new JSONArray();
for (int page = 1; page <= 2; page++) {
var url = "https://api.autom.dev/v1/google/images?q=" + query + "&gl=us&hl=en&page=" + page;
var req = HttpRequest.newBuilder().uri(URI.create(url))
.header("x-api-key", apiKey).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
var imgs = new JSONObject(resp.body()).getJSONArray("images");
for (int i = 0; i < imgs.length(); i++) all.put(imgs.get(i));
}
var large = new JSONArray();
for (int i = 0; i < all.length(); i++) {
var img = all.getJSONObject(i);
if (img.getInt("image_width") >= minWidth) large.put(img);
}
Files.writeString(Path.of("image_catalog.json"), large.toString(2));
System.out.println("Saved " + large.length() + " high-res images to image_catalog.json");
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
var apiKey = "YOUR_API_KEY";
var minWidth = 800;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
var all = new List();
for (int page = 1; page <= 2; page++)
{
var body = await client.GetStringAsync(
$"https://api.autom.dev/v1/google/images?q=electric+car+charging+station&gl=us&hl=en&page={page}");
var json = JsonDocument.Parse(body).RootElement;
foreach (var img in json.GetProperty("images").EnumerateArray())
all.Add(img);
}
var large = all.Where(img => img.GetProperty("image_width").GetInt32() >= minWidth).ToList();
File.WriteAllText("image_catalog.json", JsonSerializer.Serialize(large, new JsonSerializerOptions { WriteIndented = true }));
Console.WriteLine($"Saved {large.Count} high-res images to image_catalog.json");
```
```rust
use reqwest::Client;
use serde_json::Value;
use std::fs;
async fn fetch_images(client: &Client, api_key: &str, query: &str, pages: u32) -> Vec {
let mut all = Vec::new();
for page in 1..=pages {
let data = client.get("https://api.autom.dev/v1/google/images")
.header("x-api-key", api_key)
.query(&[("q", query), ("gl", "us"), ("hl", "en"), ("page", &page.to_string())])
.send().await.unwrap().json::().await.unwrap();
if let Some(imgs) = data["images"].as_array() { all.extend(imgs.clone()); }
}
all
}
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let images = fetch_images(&client, "YOUR_API_KEY", "electric car charging station", 2).await;
let large: Vec<&Value> = images.iter()
.filter(|img| img["image_width"].as_i64().unwrap_or(0) >= 800)
.collect();
fs::write("image_catalog.json", serde_json::to_string_pretty(&large).unwrap()).unwrap();
println!("Saved {} high-res images to image_catalog.json", large.len());
Ok(())
}
```
The `url` field in each result is the direct link to the image file. Always verify you have the rights to use an image before including it in a dataset or product.
# Monitor Google News
Overview [#overview]
This playbook builds a **news monitoring pipeline**: query Google News for a brand name, product, or topic and collect all article titles, sources, and publication dates. Run it on a cron schedule to detect press coverage as soon as it appears.
Prerequisites [#prerequisites]
* An Autom API key — get one at [app.autom.dev](https://app.autom.dev)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default in most PHP installs).
No extra dependencies — uses the `net/http` standard library (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Fetch latest news articles [#fetch-latest-news-articles]
Call `GET /v1/google/news` with the keyword you want to monitor.
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.autom.dev/v1/google/news",
headers={"x-api-key": API_KEY},
params={"q": "OpenAI", "gl": "us", "hl": "en"},
)
data = response.json()
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({ q: "OpenAI", gl: "us", hl: "en" });
const response = await fetch(`https://api.autom.dev/v1/google/news?${params}`, {
headers: { "x-api-key": API_KEY },
});
const data = await response.json();
```
```php
"OpenAI", "gl" => "us", "hl" => "en"]);
$ch = curl_init("https://api.autom.dev/v1/google/news?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
```
```go
package main
import (
"encoding/json"
"io"
"net/http"
"net/url"
)
func main() {
params := url.Values{"q": {"OpenAI"}, "gl": {"us"}, "hl": {"en"}}
req, _ := http.NewRequest("GET", "https://api.autom.dev/v1/google/news?"+params.Encode(), nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var data map[string]any
json.Unmarshal(body, &data)
// use data below
}
```
```java
import java.net.URI;
import java.net.http.*;
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.autom.dev/v1/google/news?q=OpenAI&gl=us&hl=en"))
.header("x-api-key", "YOUR_API_KEY")
.GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var body = await client.GetStringAsync(
"https://api.autom.dev/v1/google/news?q=OpenAI&gl=us&hl=en");
Console.WriteLine(body);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let data = reqwest::Client::new()
.get("https://api.autom.dev/v1/google/news")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("q", "OpenAI"), ("gl", "us"), ("hl", "en")])
.send().await?
.json::().await?;
println!("{:#?}", data);
Ok(())
}
```
Extract and display articles [#extract-and-display-articles]
Each item in `organic_results` has `title`, `link`, `source`, `date`, and `snippet`.
```python
for article in data.get("organic_results", []):
print(f"[{article['date']}] {article['title']}")
print(f" Source : {article['source']}")
print(f" URL : {article['link']}\n")
```
```typescript
for (const article of data.organic_results ?? []) {
console.log(`[${article.date}] ${article.title}`);
console.log(` Source : ${article.source}`);
console.log(` URL : ${article.link}\n`);
}
```
```php
foreach ($data["organic_results"] ?? [] as $article) {
echo "[{$article['date']}] {$article['title']}\n";
echo " Source : {$article['source']}\n";
echo " URL : {$article['link']}\n\n";
}
```
```go
results := data["organic_results"].([]any)
for _, r := range results {
a := r.(map[string]any)
fmt.Printf("[%s] %s\n Source : %s\n URL : %s\n\n",
a["date"], a["title"], a["source"], a["link"])
}
```
```java
import org.json.*;
var json = new JSONObject(response.body());
var results = json.getJSONArray("organic_results");
for (int i = 0; i < results.length(); i++) {
var a = results.getJSONObject(i);
System.out.printf("[%s] %s%n Source : %s%n URL : %s%n%n",
a.getString("date"), a.getString("title"),
a.getString("source"), a.getString("link"));
}
```
```csharp
using System.Text.Json;
var json = JsonDocument.Parse(body);
var results = json.RootElement.GetProperty("organic_results").EnumerateArray();
foreach (var a in results)
{
Console.WriteLine($"[{a.GetProperty("date")}] {a.GetProperty("title")}");
Console.WriteLine($" Source : {a.GetProperty("source")}");
Console.WriteLine($" URL : {a.GetProperty("link")}\n");
}
```
```rust
if let Some(articles) = data["organic_results"].as_array() {
for a in articles {
println!("[{}] {}", a["date"].as_str().unwrap_or(""), a["title"].as_str().unwrap_or(""));
println!(" Source : {}", a["source"].as_str().unwrap_or(""));
println!(" URL : {}\n", a["link"].as_str().unwrap_or(""));
}
}
```
Build a monitoring pipeline with deduplication [#build-a-monitoring-pipeline-with-deduplication]
Store seen article URLs so repeated runs don't produce duplicate alerts.
```python
import json, requests
from pathlib import Path
API_KEY = "YOUR_API_KEY"
KEYWORDS = ["OpenAI", "Anthropic", "Mistral AI"]
SEEN_FILE = Path("seen_articles.json")
def load_seen() -> set:
return set(json.loads(SEEN_FILE.read_text())) if SEEN_FILE.exists() else set()
def save_seen(seen: set) -> None:
SEEN_FILE.write_text(json.dumps(list(seen)))
def fetch_news(query: str) -> list:
r = requests.get("https://api.autom.dev/v1/google/news",
headers={"x-api-key": API_KEY}, params={"q": query, "gl": "us", "hl": "en"})
return r.json().get("organic_results", [])
seen = load_seen()
new_articles = []
for keyword in KEYWORDS:
for article in fetch_news(keyword):
if article["link"] not in seen:
seen.add(article["link"])
new_articles.append({**article, "keyword": keyword})
save_seen(seen)
print(f"Found {len(new_articles)} new article(s):")
for a in new_articles:
print(f" [{a['keyword']}] {a['title']} — {a['source']}")
```
```typescript
import { readFileSync, writeFileSync, existsSync } from "fs";
const API_KEY = "YOUR_API_KEY";
const KEYWORDS = ["OpenAI", "Anthropic", "Mistral AI"];
const SEEN_FILE = "seen_articles.json";
function loadSeen(): Set {
return existsSync(SEEN_FILE)
? new Set(JSON.parse(readFileSync(SEEN_FILE, "utf-8")))
: new Set();
}
async function fetchNews(query: string): Promise {
const params = new URLSearchParams({ q: query, gl: "us", hl: "en" });
const res = await fetch(`https://api.autom.dev/v1/google/news?${params}`, {
headers: { "x-api-key": API_KEY },
});
return (await res.json()).organic_results ?? [];
}
const seen = loadSeen();
const newArticles: any[] = [];
for (const keyword of KEYWORDS) {
for (const article of await fetchNews(keyword)) {
if (!seen.has(article.link)) {
seen.add(article.link);
newArticles.push({ ...article, keyword });
}
}
}
writeFileSync(SEEN_FILE, JSON.stringify([...seen]));
console.log(`Found ${newArticles.length} new article(s):`);
for (const a of newArticles) console.log(` [${a.keyword}] ${a.title} — ${a.source}`);
```
```php
$keyword, "gl" => "us", "hl" => "en"]);
$ch = curl_init("https://api.autom.dev/v1/google/news?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
foreach ($data["organic_results"] ?? [] as $article) {
if (!isset($seen[$article["link"]])) {
$seen[$article["link"]] = true;
$newArticles[] = array_merge($article, ["keyword" => $keyword]);
}
}
}
file_put_contents($seenFile, json_encode(array_keys($seen)));
echo "Found " . count($newArticles) . " new article(s):\n";
foreach ($newArticles as $a) echo " [{$a['keyword']}] {$a['title']} — {$a['source']}\n";
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
)
func fetchNews(apiKey, query string) []map[string]any {
params := url.Values{"q": {query}, "gl": {"us"}, "hl": {"en"}}
req, _ := http.NewRequest("GET", "https://api.autom.dev/v1/google/news?"+params.Encode(), nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var data map[string]any
json.Unmarshal(body, &data)
var out []map[string]any
for _, r := range data["organic_results"].([]any) {
out = append(out, r.(map[string]any))
}
return out
}
func main() {
apiKey := "YOUR_API_KEY"
keywords := []string{"OpenAI", "Anthropic", "Mistral AI"}
seen := map[string]bool{}
if b, err := os.ReadFile("seen_articles.json"); err == nil {
var links []string
json.Unmarshal(b, &links)
for _, l := range links { seen[l] = true }
}
var newCount int
for _, kw := range keywords {
for _, a := range fetchNews(apiKey, kw) {
link := a["link"].(string)
if !seen[link] {
seen[link] = true
fmt.Printf(" [%s] %s — %s\n", kw, a["title"], a["source"])
newCount++
}
}
}
links := make([]string, 0, len(seen))
for l := range seen { links = append(links, l) }
b, _ := json.Marshal(links)
os.WriteFile("seen_articles.json", b, 0644)
fmt.Printf("Found %d new article(s).\n", newCount)
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.util.*;
import org.json.*;
public class Main {
static HttpClient client = HttpClient.newHttpClient();
static String API_KEY = "YOUR_API_KEY";
public static void main(String[] args) throws Exception {
var keywords = List.of("OpenAI", "Anthropic", "Mistral AI");
var seenPath = Path.of("seen_articles.json");
var seen = new HashSet();
if (Files.exists(seenPath)) {
var arr = new JSONArray(Files.readString(seenPath));
for (int i = 0; i < arr.length(); i++) seen.add(arr.getString(i));
}
int newCount = 0;
for (var kw : keywords) {
var q = URLEncoder.encode(kw, StandardCharsets.UTF_8);
var url = "https://api.autom.dev/v1/google/news?q=" + q + "&gl=us&hl=en";
var req = HttpRequest.newBuilder().uri(URI.create(url))
.header("x-api-key", API_KEY).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
var results = new JSONObject(resp.body()).getJSONArray("organic_results");
for (int i = 0; i < results.length(); i++) {
var a = results.getJSONObject(i);
var link = a.getString("link");
if (!seen.contains(link)) {
seen.add(link);
System.out.printf(" [%s] %s — %s%n", kw, a.getString("title"), a.getString("source"));
newCount++;
}
}
}
Files.writeString(seenPath, new JSONArray(seen).toString());
System.out.println("Found " + newCount + " new article(s).");
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
var apiKey = "YOUR_API_KEY";
var keywords = new[] { "OpenAI", "Anthropic", "Mistral AI" };
var seenFile = "seen_articles.json";
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
var seen = File.Exists(seenFile)
? JsonSerializer.Deserialize>(File.ReadAllText(seenFile))!
: new HashSet();
int newCount = 0;
foreach (var keyword in keywords)
{
var url = $"https://api.autom.dev/v1/google/news?q={Uri.EscapeDataString(keyword)}&gl=us&hl=en";
var body = await client.GetStringAsync(url);
var json = JsonDocument.Parse(body).RootElement;
foreach (var a in json.GetProperty("organic_results").EnumerateArray())
{
var link = a.GetProperty("link").GetString()!;
if (seen.Add(link))
{
Console.WriteLine($" [{keyword}] {a.GetProperty("title")} — {a.GetProperty("source")}");
newCount++;
}
}
}
File.WriteAllText(seenFile, JsonSerializer.Serialize(seen));
Console.WriteLine($"Found {newCount} new article(s).");
```
```rust
use reqwest::Client;
use serde_json::Value;
use std::{collections::HashSet, fs, path::Path};
async fn fetch_news(client: &Client, api_key: &str, query: &str) -> Vec {
let data = client
.get("https://api.autom.dev/v1/google/news")
.header("x-api-key", api_key)
.query(&[("q", query), ("gl", "us"), ("hl", "en")])
.send().await.unwrap().json::().await.unwrap();
data["organic_results"].as_array().cloned().unwrap_or_default()
}
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let api_key = "YOUR_API_KEY";
let keywords = ["OpenAI", "Anthropic", "Mistral AI"];
let seen_path = Path::new("seen_articles.json");
let mut seen: HashSet = if seen_path.exists() {
serde_json::from_str::>(&fs::read_to_string(seen_path).unwrap())
.unwrap().into_iter().collect()
} else { HashSet::new() };
let mut new_count = 0;
for keyword in &keywords {
for article in fetch_news(&client, api_key, keyword).await {
let link = article["link"].as_str().unwrap_or("").to_string();
if seen.insert(link) {
println!(" [{}] {} — {}", keyword, article["title"].as_str().unwrap_or(""), article["source"].as_str().unwrap_or(""));
new_count += 1;
}
}
}
let seen_vec: Vec<&String> = seen.iter().collect();
fs::write(seen_path, serde_json::to_string(&seen_vec).unwrap()).unwrap();
println!("Found {new_count} new article(s).");
Ok(())
}
```
Schedule this script with a cron job (e.g. every hour) or a task scheduler to receive continuous coverage monitoring. Combine multiple keywords in one run to minimize credit usage.
# Scrape Google Search Results
Overview [#overview]
In this playbook you will build a script that queries Google Search and extracts structured organic results — positions, titles, URLs, and snippets — for any keyword. A typical use case is **rank tracking**: run this on a schedule to monitor where your pages appear for target keywords.
The endpoint returns up to 10 organic results per page and supports pagination, country (`gl`) and language (`hl`) targeting.
Prerequisites [#prerequisites]
* An Autom API key — get one at [app.autom.dev](https://app.autom.dev)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default in most PHP installs).
No extra dependencies — uses the `net/http` standard library (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Make your first search request [#make-your-first-search-request]
Call `GET /v1/google/search` with the `q` parameter and your API key in the `x-api-key` header.
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.autom.dev/v1/google/search",
headers={"x-api-key": API_KEY},
params={"q": "best python web scraping libraries", "gl": "us", "hl": "en"},
)
data = response.json()
print(data)
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({
q: "best python web scraping libraries",
gl: "us",
hl: "en",
});
const response = await fetch(
`https://api.autom.dev/v1/google/search?${params}`,
{ headers: { "x-api-key": API_KEY } },
);
const data = await response.json();
console.log(data);
```
```php
"best python web scraping libraries",
"gl" => "us",
"hl" => "en",
]);
$ch = curl_init("https://api.autom.dev/v1/google/search?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($data);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
func main() {
apiKey := "YOUR_API_KEY"
params := url.Values{}
params.Set("q", "best python web scraping libraries")
params.Set("gl", "us")
params.Set("hl", "en")
req, _ := http.NewRequest("GET",
"https://api.autom.dev/v1/google/search?"+params.Encode(), nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var data map[string]any
json.Unmarshal(body, &data)
fmt.Println(data)
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
public class Main {
public static void main(String[] args) throws Exception {
var apiKey = "YOUR_API_KEY";
var q = URLEncoder.encode("best python web scraping libraries", StandardCharsets.UTF_8);
var url = "https://api.autom.dev/v1/google/search?q=" + q + "&gl=us&hl=en";
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder()
.uri(URI.create(url))
.header("x-api-key", apiKey)
.GET()
.build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
}
}
```
```csharp
using System.Net.Http;
var apiKey = "YOUR_API_KEY";
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
var url = "https://api.autom.dev/v1/google/search?q=best+python+web+scraping+libraries&gl=us&hl=en";
var response = await client.GetAsync(url);
var body = await response.Content.ReadAsStringAsync();
Console.WriteLine(body);
```
```rust
use reqwest::header;
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = reqwest::Client::new();
let response = client
.get("https://api.autom.dev/v1/google/search")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("q", "best python web scraping libraries"), ("gl", "us"), ("hl", "en")])
.send()
.await?
.json::()
.await?;
println!("{:#?}", response);
Ok(())
}
```
Parse the organic results [#parse-the-organic-results]
The response contains an `organic_results` array. Each entry has `position`, `title`, `link`, `snippet`, `domain`, and `source`.
```python
for result in data["organic_results"]:
print(f"[{result['position']}] {result['title']}")
print(f" URL: {result['link']}")
print(f" {result.get('snippet', '')}")
print()
```
```typescript
for (const result of data.organic_results) {
console.log(`[${result.position}] ${result.title}`);
console.log(` URL: ${result.link}`);
console.log(` ${result.snippet ?? ""}`);
console.log();
}
```
```php
foreach ($data["organic_results"] as $result) {
echo "[{$result['position']}] {$result['title']}\n";
echo " URL: {$result['link']}\n";
echo " " . ($result['snippet'] ?? "") . "\n\n";
}
```
```go
results := data["organic_results"].([]any)
for _, r := range results {
item := r.(map[string]any)
fmt.Printf("[%.0f] %s\n", item["position"], item["title"])
fmt.Printf(" URL: %s\n", item["link"])
fmt.Printf(" %s\n\n", item["snippet"])
}
```
```java
import org.json.JSONArray;
import org.json.JSONObject;
// Add org.json:json to your build tool, or parse manually
var json = new JSONObject(response.body());
var results = json.getJSONArray("organic_results");
for (int i = 0; i < results.length(); i++) {
var r = results.getJSONObject(i);
System.out.printf("[%d] %s%n", r.getInt("position"), r.getString("title"));
System.out.printf(" URL: %s%n%n", r.getString("link"));
}
```
```csharp
using System.Text.Json;
var json = JsonDocument.Parse(body);
var results = json.RootElement.GetProperty("organic_results").EnumerateArray();
foreach (var result in results)
{
Console.WriteLine($"[{result.GetProperty("position")}] {result.GetProperty("title")}");
Console.WriteLine($" URL: {result.GetProperty("link")}");
Console.WriteLine();
}
```
```rust
if let Some(results) = response["organic_results"].as_array() {
for result in results {
println!(
"[{}] {}",
result["position"],
result["title"].as_str().unwrap_or("")
);
println!(" URL: {}", result["link"].as_str().unwrap_or(""));
println!(" {}", result["snippet"].as_str().unwrap_or(""));
println!();
}
}
```
Handle pagination [#handle-pagination]
Use the `page` parameter to fetch subsequent pages. The response includes a `pagination` object with `has_next_page` and `next` page number.
```python
import requests
API_KEY = "YOUR_API_KEY"
QUERY = "best python web scraping libraries"
def fetch_all_results(query: str, max_pages: int = 3) -> list:
all_results = []
page = 1
while page <= max_pages:
response = requests.get(
"https://api.autom.dev/v1/google/search",
headers={"x-api-key": API_KEY},
params={"q": query, "gl": "us", "hl": "en", "page": page},
)
data = response.json()
all_results.extend(data.get("organic_results", []))
if not data.get("pagination", {}).get("has_next_page"):
break
page += 1
return all_results
results = fetch_all_results(QUERY)
for r in results:
print(f"[{r['position']}] {r['title']} — {r['link']}")
```
```typescript
const API_KEY = "YOUR_API_KEY";
async function fetchAllResults(query: string, maxPages = 3) {
const allResults: any[] = [];
let page = 1;
while (page <= maxPages) {
const params = new URLSearchParams({ q: query, gl: "us", hl: "en", page: String(page) });
const res = await fetch(`https://api.autom.dev/v1/google/search?${params}`, {
headers: { "x-api-key": API_KEY },
});
const data = await res.json();
allResults.push(...(data.organic_results ?? []));
if (!data.pagination?.has_next_page) break;
page++;
}
return allResults;
}
const results = await fetchAllResults("best python web scraping libraries");
for (const r of results) {
console.log(`[${r.position}] ${r.title} — ${r.link}`);
}
```
```php
$query, "gl" => "us", "hl" => "en", "page" => $page]);
$ch = curl_init("https://api.autom.dev/v1/google/search?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
$allResults = array_merge($allResults, $data["organic_results"] ?? []);
if (!($data["pagination"]["has_next_page"] ?? false)) break;
$page++;
}
return $allResults;
}
$results = fetchAllResults("YOUR_API_KEY", "best python web scraping libraries");
foreach ($results as $r) {
echo "[{$r['position']}] {$r['title']} — {$r['link']}\n";
}
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
)
func fetchAllResults(apiKey, query string, maxPages int) []map[string]any {
var all []map[string]any
for page := 1; page <= maxPages; page++ {
params := url.Values{"q": {query}, "gl": {"us"}, "hl": {"en"}, "page": {strconv.Itoa(page)}}
req, _ := http.NewRequest("GET", "https://api.autom.dev/v1/google/search?"+params.Encode(), nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
var data map[string]any
json.Unmarshal(body, &data)
if items, ok := data["organic_results"].([]any); ok {
for _, item := range items {
all = append(all, item.(map[string]any))
}
}
pagination, _ := data["pagination"].(map[string]any)
if pagination["has_next_page"] != true {
break
}
}
return all
}
func main() {
results := fetchAllResults("YOUR_API_KEY", "best python web scraping libraries", 3)
for _, r := range results {
fmt.Printf("[%.0f] %s — %s\n", r["position"], r["title"], r["link"])
}
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.util.*;
import org.json.*;
public class Main {
static HttpClient client = HttpClient.newHttpClient();
static String API_KEY = "YOUR_API_KEY";
static List fetchAllResults(String query, int maxPages) throws Exception {
var all = new ArrayList();
var q = URLEncoder.encode(query, StandardCharsets.UTF_8);
for (int page = 1; page <= maxPages; page++) {
var url = "https://api.autom.dev/v1/google/search?q=" + q + "&gl=us&hl=en&page=" + page;
var request = HttpRequest.newBuilder().uri(URI.create(url))
.header("x-api-key", API_KEY).GET().build();
var resp = client.send(request, HttpResponse.BodyHandlers.ofString());
var json = new JSONObject(resp.body());
var results = json.optJSONArray("organic_results");
if (results != null) {
for (int i = 0; i < results.length(); i++) all.add(results.getJSONObject(i));
}
if (!json.optJSONObject("pagination").optBoolean("has_next_page")) break;
}
return all;
}
public static void main(String[] args) throws Exception {
for (var r : fetchAllResults("best python web scraping libraries", 3)) {
System.out.printf("[%d] %s — %s%n", r.getInt("position"), r.getString("title"), r.getString("link"));
}
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
var apiKey = "YOUR_API_KEY";
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
var allResults = new List();
int page = 1, maxPages = 3;
while (page <= maxPages)
{
var url = $"https://api.autom.dev/v1/google/search?q=best+python+web+scraping+libraries&gl=us&hl=en&page={page}";
var response = await client.GetAsync(url);
var body = await response.Content.ReadAsStringAsync();
var json = JsonDocument.Parse(body).RootElement;
foreach (var result in json.GetProperty("organic_results").EnumerateArray())
allResults.Add(result);
var hasNext = json.GetProperty("pagination").GetProperty("has_next_page").GetBoolean();
if (!hasNext) break;
page++;
}
foreach (var r in allResults)
Console.WriteLine($"[{r.GetProperty("position")}] {r.GetProperty("title")} — {r.GetProperty("link")}");
```
```rust
use reqwest::Client;
use serde_json::Value;
async fn fetch_all_results(client: &Client, api_key: &str, query: &str, max_pages: u32) -> Vec {
let mut all_results = Vec::new();
let mut page = 1u32;
while page <= max_pages {
let resp = client
.get("https://api.autom.dev/v1/google/search")
.header("x-api-key", api_key)
.query(&[("q", query), ("gl", "us"), ("hl", "en"), ("page", &page.to_string())])
.send().await.unwrap()
.json::().await.unwrap();
if let Some(results) = resp["organic_results"].as_array() {
all_results.extend(results.clone());
}
if !resp["pagination"]["has_next_page"].as_bool().unwrap_or(false) { break; }
page += 1;
}
all_results
}
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let results = fetch_all_results(&client, "YOUR_API_KEY", "best python web scraping libraries", 3).await;
for r in &results {
println!("[{}] {} — {}", r["position"], r["title"].as_str().unwrap_or(""), r["link"].as_str().unwrap_or(""));
}
Ok(())
}
```
Use the `gl` parameter to target a specific country (e.g. `fr` for France, `de` for Germany) and `hl` for the language of results. This is essential for accurate local rank tracking.
# Analyze a Page with AI
Overview [#overview]
The CaptureKit AI analysis endpoint combines screenshot capture with LLM-powered analysis. Pass any URL and a custom `prompt` — the API returns a structured JSON answer grounded in what the page actually looks like. Use it for **competitor audits**, **UX reviews**, or **automated content QA**.
Prerequisites [#prerequisites]
* A CaptureKit API key — get one at [app.capturekit.dev](https://app.capturekit.dev)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Submit an analysis request [#submit-an-analysis-request]
Call `GET /v1/analyze` with the `url` and `prompt` parameters. The `prompt` shapes the AI's response.
```python
import requests
API_KEY = "YOUR_API_KEY"
PROMPT = "Summarize the main value proposition of this page in 2 sentences. Then list the top 3 CTAs visible above the fold."
response = requests.get(
"https://api.capturekit.dev/v1/analyze",
headers={"x-api-key": API_KEY},
params={"url": "https://stripe.com", "prompt": PROMPT},
)
data = response.json()
print(data)
```
```typescript
const API_KEY = "YOUR_API_KEY";
const PROMPT = "Summarize the main value proposition of this page in 2 sentences. Then list the top 3 CTAs visible above the fold.";
const params = new URLSearchParams({ url: "https://stripe.com", prompt: PROMPT });
const response = await fetch(`https://api.capturekit.dev/v1/analyze?${params}`, {
headers: { "x-api-key": API_KEY },
});
const data = await response.json();
console.log(data);
```
```php
"https://stripe.com", "prompt" => $prompt]);
$ch = curl_init("https://api.capturekit.dev/v1/analyze?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($data);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
func main() {
params := url.Values{
"url": {"https://stripe.com"},
"prompt": {"Summarize the main value proposition in 2 sentences. List the top 3 CTAs above the fold."},
}
req, _ := http.NewRequest("GET", "https://api.capturekit.dev/v1/analyze?"+params.Encode(), nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var data map[string]any
json.Unmarshal(body, &data)
fmt.Println(data)
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
var client = HttpClient.newHttpClient();
var prompt = URLEncoder.encode("Summarize the value proposition in 2 sentences. List the top 3 CTAs above the fold.", StandardCharsets.UTF_8);
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.capturekit.dev/v1/analyze?url=https%3A%2F%2Fstripe.com&prompt=" + prompt))
.header("x-api-key", "YOUR_API_KEY").GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var prompt = Uri.EscapeDataString("Summarize the value proposition in 2 sentences. List the top 3 CTAs above the fold.");
var body = await client.GetStringAsync(
$"https://api.capturekit.dev/v1/analyze?url=https%3A%2F%2Fstripe.com&prompt={prompt}");
Console.WriteLine(body);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let data = reqwest::Client::new()
.get("https://api.capturekit.dev/v1/analyze")
.header("x-api-key", "YOUR_API_KEY")
.query(&[
("url", "https://stripe.com"),
("prompt", "Summarize the value proposition in 2 sentences. List the top 3 CTAs above the fold."),
])
.send().await?.json::().await?;
println!("{:#?}", data);
Ok(())
}
```
Read the AI analysis [#read-the-ai-analysis]
The response includes `analysis` (the AI's answer to your prompt), `screenshot_url`, and metadata like `title` and `description`.
```python
print(f"Page : {data.get('title')}")
print(f"Preview : {data.get('screenshot_url')}")
print()
print("=== AI Analysis ===")
print(data.get("analysis", ""))
```
```typescript
console.log(`Page : ${data.title}`);
console.log(`Preview : ${data.screenshot_url}\n`);
console.log("=== AI Analysis ===");
console.log(data.analysis);
```
```php
echo "Page : {$data['title']}\n";
echo "Preview : {$data['screenshot_url']}\n\n";
echo "=== AI Analysis ===\n";
echo $data["analysis"] ?? "";
```
```go
fmt.Printf("Page : %v\nPreview : %v\n\n=== AI Analysis ===\n%v\n",
data["title"], data["screenshot_url"], data["analysis"])
```
```java
import org.json.*;
var d = new JSONObject(response.body());
System.out.printf("Page : %s%nPreview : %s%n%n=== AI Analysis ===%n%s%n",
d.getString("title"), d.getString("screenshot_url"), d.getString("analysis"));
```
```csharp
using System.Text.Json;
var d = JsonDocument.Parse(body).RootElement;
Console.WriteLine($"Page : {d.GetProperty("title")}");
Console.WriteLine($"Preview : {d.GetProperty("screenshot_url")}\n");
Console.WriteLine("=== AI Analysis ===");
Console.WriteLine(d.GetProperty("analysis"));
```
```rust
println!("Page : {}", data["title"].as_str().unwrap_or(""));
println!("Preview : {}\n", data["screenshot_url"].as_str().unwrap_or(""));
println!("=== AI Analysis ===");
println!("{}", data["analysis"].as_str().unwrap_or(""));
```
Run a competitor audit across multiple pages [#run-a-competitor-audit-across-multiple-pages]
Analyze several competitor landing pages with a consistent prompt and save the results as a structured report.
````python
import json, time, requests
API_KEY = "YOUR_API_KEY"
PROMPT = """Analyze this landing page and return a JSON object with these keys:
- value_proposition (string)
- primary_cta (string)
- target_audience (string)
- pricing_visible (boolean)
- social_proof_types (array of strings: "testimonials", "logos", "stats", etc.)"""
COMPETITORS = [
{"name": "Stripe", "url": "https://stripe.com"},
{"name": "Paddle", "url": "https://paddle.com"},
{"name": "Lemonsqueezy", "url": "https://lemonsqueezy.com"},
]
report = []
for comp in COMPETITORS:
r = requests.get("https://api.capturekit.dev/v1/analyze",
headers={"x-api-key": API_KEY},
params={"url": comp["url"], "prompt": PROMPT})
data = r.json()
analysis_text = data.get("analysis", "{}")
try:
# AI may return the JSON wrapped in markdown fences
raw = analysis_text.strip().removeprefix("```json").removesuffix("```").strip()
analysis = json.loads(raw)
except json.JSONDecodeError:
analysis = {"raw": analysis_text}
report.append({"competitor": comp["name"], "url": comp["url"], **analysis})
print(f"✓ {comp['name']} analyzed")
time.sleep(2)
with open("competitor_audit.json", "w") as f:
json.dump(report, f, indent=2)
print("\nReport saved to competitor_audit.json")
````
````typescript
import { writeFileSync } from "fs";
const API_KEY = "YOUR_API_KEY";
const PROMPT = `Analyze this landing page and return a JSON object with these keys:
- value_proposition (string)
- primary_cta (string)
- target_audience (string)
- pricing_visible (boolean)
- social_proof_types (array of strings)`;
const competitors = [
{ name: "Stripe", url: "https://stripe.com" },
{ name: "Paddle", url: "https://paddle.com" },
{ name: "Lemonsqueezy", url: "https://lemonsqueezy.com" },
];
const report: any[] = [];
for (const comp of competitors) {
const params = new URLSearchParams({ url: comp.url, prompt: PROMPT });
const res = await fetch(`https://api.capturekit.dev/v1/analyze?${params}`, {
headers: { "x-api-key": API_KEY },
});
const data = await res.json();
let analysis: any;
try {
const raw = (data.analysis ?? "{}").replace(/^```json\n?/, "").replace(/```$/, "").trim();
analysis = JSON.parse(raw);
} catch { analysis = { raw: data.analysis }; }
report.push({ competitor: comp.name, url: comp.url, ...analysis });
console.log(`✓ ${comp.name} analyzed`);
await new Promise(r => setTimeout(r, 2000));
}
writeFileSync("competitor_audit.json", JSON.stringify(report, null, 2));
console.log("\nReport saved to competitor_audit.json");
````
````php
"Stripe", "url" => "https://stripe.com"],
["name" => "Paddle", "url" => "https://paddle.com"],
["name" => "Lemonsqueezy", "url" => "https://lemonsqueezy.com"],
];
$report = [];
foreach ($competitors as $comp) {
$params = http_build_query(["url" => $comp["url"], "prompt" => $prompt]);
$ch = curl_init("https://api.capturekit.dev/v1/analyze?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
$raw = preg_replace('/^```json\n?|```$/', '', trim($data["analysis"] ?? "{}"));
$analysis = json_decode($raw, true) ?? ["raw" => $data["analysis"]];
$report[] = array_merge(["competitor" => $comp["name"], "url" => $comp["url"]], $analysis);
echo "✓ {$comp['name']} analyzed\n";
sleep(2);
}
file_put_contents("competitor_audit.json", json_encode($report, JSON_PRETTY_PRINT));
echo "\nReport saved to competitor_audit.json\n";
````
````go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"time"
)
const (
APIKey = "YOUR_API_KEY"
Prompt = "Analyze this landing page and return a JSON object: value_proposition, primary_cta, target_audience, pricing_visible, social_proof_types."
)
type Competitor struct{ Name, URL string }
func analyze(comp Competitor) map[string]any {
params := url.Values{"url": {comp.URL}, "prompt": {Prompt}}
req, _ := http.NewRequest("GET", "https://api.capturekit.dev/v1/analyze?"+params.Encode(), nil)
req.Header.Set("x-api-key", APIKey)
resp, _ := http.DefaultClient.Do(req)
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
var data map[string]any
json.Unmarshal(body, &data)
raw := strings.TrimSpace(fmt.Sprint(data["analysis"]))
raw = strings.TrimPrefix(raw, "```json")
raw = strings.TrimSuffix(raw, "```")
var analysis map[string]any
if err := json.Unmarshal([]byte(strings.TrimSpace(raw)), &analysis); err != nil {
analysis = map[string]any{"raw": raw}
}
analysis["competitor"] = comp.Name
analysis["url"] = comp.URL
return analysis
}
func main() {
competitors := []Competitor{
{"Stripe", "https://stripe.com"},
{"Paddle", "https://paddle.com"},
{"Lemonsqueezy", "https://lemonsqueezy.com"},
}
var report []map[string]any
for _, c := range competitors {
report = append(report, analyze(c))
fmt.Printf("✓ %s analyzed\n", c.Name)
time.Sleep(2 * time.Second)
}
b, _ := json.MarshalIndent(report, "", " ")
os.WriteFile("competitor_audit.json", b, 0644)
fmt.Println("\nReport saved to competitor_audit.json")
}
````
````java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.util.*;
import org.json.*;
public class Main {
static final String API_KEY = "YOUR_API_KEY";
static final String PROMPT = "Analyze this landing page and return a JSON object: value_proposition, primary_cta, target_audience, pricing_visible, social_proof_types.";
public static void main(String[] args) throws Exception {
var client = HttpClient.newHttpClient();
var competitors = List.of(
Map.of("name","Stripe", "url","https://stripe.com"),
Map.of("name","Paddle", "url","https://paddle.com"),
Map.of("name","Lemonsqueezy", "url","https://lemonsqueezy.com"));
var report = new JSONArray();
for (var comp : competitors) {
var encodedUrl = URLEncoder.encode(comp.get("url"), StandardCharsets.UTF_8);
var encodedPrompt = URLEncoder.encode(PROMPT, StandardCharsets.UTF_8);
var req = HttpRequest.newBuilder()
.uri(URI.create("https://api.capturekit.dev/v1/analyze?url=" + encodedUrl + "&prompt=" + encodedPrompt))
.header("x-api-key", API_KEY).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
var data = new JSONObject(resp.body());
var analysisText = data.getString("analysis")
.replaceAll("^```json\\n?","").replaceAll("```$","").strip();
JSONObject analysis;
try { analysis = new JSONObject(analysisText); }
catch (Exception e) { analysis = new JSONObject().put("raw", analysisText); }
analysis.put("competitor", comp.get("name")).put("url", comp.get("url"));
report.put(analysis);
System.out.println("✓ " + comp.get("name") + " analyzed");
Thread.sleep(2000);
}
Files.writeString(Path.of("competitor_audit.json"), report.toString(2));
System.out.println("\nReport saved to competitor_audit.json");
}
}
````
````csharp
using System.Net.Http;
using System.Text.Json;
using System.Text.RegularExpressions;
const string API_KEY = "YOUR_API_KEY";
const string PROMPT = "Analyze this landing page and return a JSON object: value_proposition, primary_cta, target_audience, pricing_visible, social_proof_types.";
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", API_KEY);
var competitors = new[] {
(name: "Stripe", url: "https://stripe.com"),
(name: "Paddle", url: "https://paddle.com"),
(name: "Lemonsqueezy", url: "https://lemonsqueezy.com"),
};
var report = new List
````rust
use reqwest::Client;
use serde_json::{json, Value};
use std::{fs, time::Duration};
use tokio::time::sleep;
const API_KEY: &str = "YOUR_API_KEY";
const PROMPT: &str = "Analyze this landing page and return a JSON object: value_proposition, primary_cta, target_audience, pricing_visible, social_proof_types.";
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let competitors = [("Stripe","https://stripe.com"), ("Paddle","https://paddle.com"), ("Lemonsqueezy","https://lemonsqueezy.com")];
let mut report = Vec::new();
for (name, url) in &competitors {
let data = client.get("https://api.capturekit.dev/v1/analyze")
.header("x-api-key", API_KEY)
.query(&[("url", url), ("prompt", &PROMPT)])
.send().await?.json::().await?;
let analysis_text = data["analysis"].as_str().unwrap_or("{}");
let clean = analysis_text.trim().trim_start_matches("```json").trim_end_matches("```").trim();
let analysis: Value = serde_json::from_str(clean).unwrap_or(json!({ "raw": analysis_text }));
report.push(json!({ "competitor": name, "url": url, "analysis": analysis }));
println!("✓ {} analyzed", name);
sleep(Duration::from_secs(2)).await;
}
fs::write("competitor_audit.json", serde_json::to_string_pretty(&report).unwrap()).unwrap();
println!("\nReport saved to competitor_audit.json");
Ok(())
}
````
Instruct the AI to return structured JSON in your `prompt` — the model will format its output accordingly, making it easy to parse and store results in a database or spreadsheet without additional post-processing.
# Extract Content as Markdown
Overview [#overview]
The CaptureKit content endpoint fetches a webpage, strips navigation, ads, and layout chrome, then returns the main body as **clean Markdown**. Use it for building RAG datasets, indexing documentation, or archiving articles.
Prerequisites [#prerequisites]
* A CaptureKit API key — get one at [app.capturekit.dev](https://app.capturekit.dev)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Fetch the page content [#fetch-the-page-content]
Call `GET /v1/content` with the `url` parameter.
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.capturekit.dev/v1/content",
headers={"x-api-key": API_KEY},
params={"url": "https://stripe.com/docs/payments"},
)
data = response.json()
print(data)
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({ url: "https://stripe.com/docs/payments" });
const response = await fetch(`https://api.capturekit.dev/v1/content?${params}`, {
headers: { "x-api-key": API_KEY },
});
const data = await response.json();
console.log(data);
```
```php
"https://stripe.com/docs/payments"]);
$ch = curl_init("https://api.capturekit.dev/v1/content?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($data);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
func main() {
params := url.Values{"url": {"https://stripe.com/docs/payments"}}
req, _ := http.NewRequest("GET", "https://api.capturekit.dev/v1/content?"+params.Encode(), nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var data map[string]any
json.Unmarshal(body, &data)
fmt.Println(data)
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
var client = HttpClient.newHttpClient();
var target = URLEncoder.encode("https://stripe.com/docs/payments", StandardCharsets.UTF_8);
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.capturekit.dev/v1/content?url=" + target))
.header("x-api-key", "YOUR_API_KEY").GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var target = Uri.EscapeDataString("https://stripe.com/docs/payments");
var body = await client.GetStringAsync($"https://api.capturekit.dev/v1/content?url={target}");
Console.WriteLine(body);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let data = reqwest::Client::new()
.get("https://api.capturekit.dev/v1/content")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("url", "https://stripe.com/docs/payments")])
.send().await?.json::().await?;
println!("{:#?}", data);
Ok(())
}
```
Access the Markdown content [#access-the-markdown-content]
The response includes `markdown`, `title`, `description`, `author`, `published_at`, and `word_count`.
```python
print(f"Title : {data.get('title')}")
print(f"Author : {data.get('author', 'N/A')}")
print(f"Word count : {data.get('word_count')} words")
print()
print("--- Markdown preview ---")
markdown = data.get("markdown", "")
print(markdown[:1000])
```
```typescript
console.log(`Title : ${data.title}`);
console.log(`Author : ${data.author ?? "N/A"}`);
console.log(`Word count : ${data.word_count} words\n`);
console.log("--- Markdown preview ---");
console.log(data.markdown?.slice(0, 1000));
```
```php
echo "Title : {$data['title']}\n";
echo "Author : " . ($data["author"] ?? "N/A") . "\n";
echo "Word count : {$data['word_count']} words\n\n";
echo "--- Markdown preview ---\n";
echo substr($data["markdown"] ?? "", 0, 1000) . "\n";
```
```go
fmt.Printf("Title : %v\nAuthor : %v\nWord count : %v words\n\n",
data["title"], data["author"], data["word_count"])
markdown := data["markdown"].(string)
if len(markdown) > 1000 { markdown = markdown[:1000] }
fmt.Println("--- Markdown preview ---\n" + markdown)
```
```java
import org.json.*;
var d = new JSONObject(response.body());
var markdown = d.getString("markdown");
System.out.printf("Title : %s%nAuthor : %s%nWord count : %d words%n%n",
d.getString("title"), d.optString("author", "N/A"), d.getInt("word_count"));
System.out.println("--- Markdown preview ---");
System.out.println(markdown.substring(0, Math.min(1000, markdown.length())));
```
```csharp
using System.Text.Json;
var d = JsonDocument.Parse(body).RootElement;
var markdown = d.GetProperty("markdown").GetString() ?? "";
Console.WriteLine($"Title : {d.GetProperty("title")}");
Console.WriteLine($"Author : {(d.TryGetProperty("author", out var a) ? a : (object)"N/A")}");
Console.WriteLine($"Word count : {d.GetProperty("word_count")} words\n");
Console.WriteLine("--- Markdown preview ---");
Console.WriteLine(markdown[..Math.Min(1000, markdown.Length)]);
```
```rust
let markdown = data["markdown"].as_str().unwrap_or("");
println!("Title : {}", data["title"].as_str().unwrap_or(""));
println!("Author : {}", data["author"].as_str().unwrap_or("N/A"));
println!("Word count : {} words\n", data["word_count"]);
println!("--- Markdown preview ---");
println!("{}", &markdown[..1000.min(markdown.len())]);
```
Crawl and archive a list of documentation pages [#crawl-and-archive-a-list-of-documentation-pages]
Fetch and save multiple pages as Markdown files for offline search or RAG ingestion.
```python
import os, time, requests
API_KEY = "YOUR_API_KEY"
os.makedirs("docs_archive", exist_ok=True)
PAGES = [
"https://stripe.com/docs/payments",
"https://stripe.com/docs/billing",
"https://stripe.com/docs/connect",
]
for page_url in PAGES:
r = requests.get("https://api.capturekit.dev/v1/content",
headers={"x-api-key": API_KEY}, params={"url": page_url})
data = r.json()
slug = page_url.rstrip("/").split("/")[-1]
path = f"docs_archive/{slug}.md"
with open(path, "w") as f:
f.write(f"# {data.get('title', slug)}\n\n")
f.write(data.get("markdown", ""))
print(f"Saved: {path} ({data.get('word_count', 0)} words)")
time.sleep(1)
print("Done!")
```
```typescript
import { mkdirSync, writeFileSync } from "fs";
const API_KEY = "YOUR_API_KEY";
mkdirSync("docs_archive", { recursive: true });
const PAGES = [
"https://stripe.com/docs/payments",
"https://stripe.com/docs/billing",
"https://stripe.com/docs/connect",
];
for (const pageUrl of PAGES) {
const params = new URLSearchParams({ url: pageUrl });
const res = await fetch(`https://api.capturekit.dev/v1/content?${params}`, {
headers: { "x-api-key": API_KEY },
});
const data = await res.json();
const slug = pageUrl.replace(/\/$/, "").split("/").at(-1)!;
writeFileSync(`docs_archive/${slug}.md`, `# ${data.title ?? slug}\n\n${data.markdown ?? ""}`);
console.log(`Saved: docs_archive/${slug}.md (${data.word_count ?? 0} words)`);
await new Promise(r => setTimeout(r, 1000));
}
console.log("Done!");
```
```php
$pageUrl]);
$ch = curl_init("https://api.capturekit.dev/v1/content?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
$slug = basename(rtrim($pageUrl, "/"));
$path = "docs_archive/{$slug}.md";
file_put_contents($path, "# {$data['title']}\n\n{$data['markdown']}");
echo "Saved: {$path} ({$data['word_count']} words)\n";
sleep(1);
}
echo "Done!\n";
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"path/filepath"
"strings"
"time"
)
func main() {
os.MkdirAll("docs_archive", 0755)
apiKey := "YOUR_API_KEY"
pages := []string{
"https://stripe.com/docs/payments",
"https://stripe.com/docs/billing",
"https://stripe.com/docs/connect",
}
for _, pageURL := range pages {
params := url.Values{"url": {pageURL}}
req, _ := http.NewRequest("GET", "https://api.capturekit.dev/v1/content?"+params.Encode(), nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
var data map[string]any
json.Unmarshal(body, &data)
parts := strings.Split(strings.TrimRight(pageURL, "/"), "/")
slug := parts[len(parts)-1]
path := filepath.Join("docs_archive", slug+".md")
content := fmt.Sprintf("# %v\n\n%v", data["title"], data["markdown"])
os.WriteFile(path, []byte(content), 0644)
fmt.Printf("Saved: %s (%.0f words)\n", path, data["word_count"])
time.Sleep(time.Second)
}
fmt.Println("Done!")
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.util.List;
import org.json.*;
public class Main {
public static void main(String[] args) throws Exception {
var client = HttpClient.newHttpClient();
var apiKey = "YOUR_API_KEY";
Files.createDirectories(Path.of("docs_archive"));
var pages = List.of(
"https://stripe.com/docs/payments",
"https://stripe.com/docs/billing",
"https://stripe.com/docs/connect");
for (var pageUrl : pages) {
var encoded = URLEncoder.encode(pageUrl, StandardCharsets.UTF_8);
var req = HttpRequest.newBuilder()
.uri(URI.create("https://api.capturekit.dev/v1/content?url=" + encoded))
.header("x-api-key", apiKey).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
var data = new JSONObject(resp.body());
var slug = pageUrl.replaceAll("/$","").replaceAll(".*/","");
var content = "# " + data.getString("title") + "\n\n" + data.getString("markdown");
Files.writeString(Path.of("docs_archive/" + slug + ".md"), content);
System.out.printf("Saved: docs_archive/%s.md (%d words)%n", slug, data.getInt("word_count"));
Thread.sleep(1000);
}
System.out.println("Done!");
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
var apiKey = "YOUR_API_KEY";
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
Directory.CreateDirectory("docs_archive");
var pages = new[]
{
"https://stripe.com/docs/payments",
"https://stripe.com/docs/billing",
"https://stripe.com/docs/connect",
};
foreach (var pageUrl in pages)
{
var encoded = Uri.EscapeDataString(pageUrl);
var body = await client.GetStringAsync($"https://api.capturekit.dev/v1/content?url={encoded}");
var data = JsonDocument.Parse(body).RootElement;
var slug = pageUrl.TrimEnd('/').Split('/').Last();
var content = $"# {data.GetProperty("title")}\n\n{data.GetProperty("markdown")}";
File.WriteAllText($"docs_archive/{slug}.md", content);
Console.WriteLine($"Saved: docs_archive/{slug}.md ({data.GetProperty("word_count")} words)");
await Task.Delay(1000);
}
Console.WriteLine("Done!");
```
```rust
use reqwest::Client;
use serde_json::Value;
use std::{fs, path::Path, time::Duration};
use tokio::time::sleep;
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let api_key = "YOUR_API_KEY";
let pages = ["https://stripe.com/docs/payments", "https://stripe.com/docs/billing", "https://stripe.com/docs/connect"];
fs::create_dir_all("docs_archive").unwrap();
for page_url in &pages {
let data = client.get("https://api.capturekit.dev/v1/content")
.header("x-api-key", api_key)
.query(&[("url", page_url)])
.send().await?.json::().await?;
let slug = page_url.trim_end_matches('/').split('/').last().unwrap_or("page");
let path = format!("docs_archive/{}.md", slug);
let content = format!("# {}\n\n{}", data["title"].as_str().unwrap_or(""), data["markdown"].as_str().unwrap_or(""));
fs::write(&path, content).unwrap();
println!("Saved: {} ({} words)", path, data["word_count"]);
sleep(Duration::from_secs(1)).await;
}
println!("Done!");
Ok(())
}
```
The extracted Markdown is ideal as context chunks for a RAG (Retrieval-Augmented Generation) pipeline. Chunk by headings and embed with your preferred vector store for semantic search over any website's documentation.
# Screenshot a Webpage
Overview [#overview]
This playbook shows how to take a screenshot of any public URL and save it to disk. A common use case is **visual regression testing** or generating OG image thumbnails for a link-preview service.
Prerequisites [#prerequisites]
* A CaptureKit API key — get one at [app.capturekit.dev](https://app.capturekit.dev)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Request a screenshot [#request-a-screenshot]
Call `GET /v1/capture` with the `url` parameter. Additional options control the viewport, format, and full-page capture.
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.capturekit.dev/v1/capture",
headers={"x-api-key": API_KEY},
params={
"url": "https://stripe.com",
"format": "png",
"full_page": "true",
"width": "1440",
"height": "900",
},
)
data = response.json()
print(data)
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({
url: "https://stripe.com",
format: "png",
full_page: "true",
width: "1440",
height: "900",
});
const response = await fetch(`https://api.capturekit.dev/v1/capture?${params}`, {
headers: { "x-api-key": API_KEY },
});
const data = await response.json();
console.log(data);
```
```php
"https://stripe.com",
"format" => "png",
"full_page" => "true",
"width" => "1440",
"height" => "900",
]);
$ch = curl_init("https://api.capturekit.dev/v1/capture?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($data);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
func main() {
params := url.Values{
"url": {"https://stripe.com"}, "format": {"png"},
"full_page": {"true"}, "width": {"1440"}, "height": {"900"},
}
req, _ := http.NewRequest("GET", "https://api.capturekit.dev/v1/capture?"+params.Encode(), nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var data map[string]any
json.Unmarshal(body, &data)
fmt.Println(data)
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
var apiKey = "YOUR_API_KEY";
var target = URLEncoder.encode("https://stripe.com", StandardCharsets.UTF_8);
var url = "https://api.capturekit.dev/v1/capture?url=" + target + "&format=png&full_page=true&width=1440&height=900";
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder().uri(URI.create(url))
.header("x-api-key", apiKey).GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var target = Uri.EscapeDataString("https://stripe.com");
var body = await client.GetStringAsync(
$"https://api.capturekit.dev/v1/capture?url={target}&format=png&full_page=true&width=1440&height=900");
Console.WriteLine(body);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let data = reqwest::Client::new()
.get("https://api.capturekit.dev/v1/capture")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("url", "https://stripe.com"), ("format", "png"), ("full_page", "true"), ("width", "1440"), ("height", "900")])
.send().await?.json::().await?;
println!("{:#?}", data);
Ok(())
}
```
Get the image URL from the response [#get-the-image-url-from-the-response]
The API returns a `screenshot_url` with the hosted image — or a base64-encoded `image` field if you set `response_type=base64`.
```python
screenshot_url = data.get("screenshot_url")
print(f"Screenshot ready: {screenshot_url}")
```
```typescript
const { screenshot_url } = data;
console.log(`Screenshot ready: ${screenshot_url}`);
```
```php
$screenshotUrl = $data["screenshot_url"] ?? "";
echo "Screenshot ready: {$screenshotUrl}\n";
```
```go
screenshotURL := data["screenshot_url"].(string)
fmt.Println("Screenshot ready:", screenshotURL)
```
```java
import org.json.*;
var screenshotUrl = new JSONObject(response.body()).getString("screenshot_url");
System.out.println("Screenshot ready: " + screenshotUrl);
```
```csharp
using System.Text.Json;
var screenshotUrl = JsonDocument.Parse(body).RootElement.GetProperty("screenshot_url").GetString();
Console.WriteLine($"Screenshot ready: {screenshotUrl}");
```
```rust
let screenshot_url = data["screenshot_url"].as_str().unwrap_or("");
println!("Screenshot ready: {}", screenshot_url);
```
Download and save the image [#download-and-save-the-image]
Fetch the image bytes from the URL and write them to disk.
```python
filename = "screenshot.png"
with requests.get(screenshot_url, stream=True) as r:
r.raise_for_status()
with open(filename, "wb") as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
print(f"Saved to {filename}")
```
```typescript
import { writeFileSync } from "fs";
const imgResponse = await fetch(screenshotUrl);
const buffer = Buffer.from(await imgResponse.arrayBuffer());
writeFileSync("screenshot.png", buffer);
console.log("Saved to screenshot.png");
```
```php
$fp = fopen("screenshot.png", "wb");
$ch = curl_init($screenshotUrl);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_exec($ch);
curl_close($ch);
fclose($fp);
echo "Saved to screenshot.png\n";
```
```go
import "os"
resp, _ := http.Get(screenshotURL)
defer resp.Body.Close()
file, _ := os.Create("screenshot.png")
defer file.Close()
io.Copy(file, resp.Body)
fmt.Println("Saved to screenshot.png")
```
```java
import java.net.URL;
import java.nio.file.*;
Files.copy(new URL(screenshotUrl).openStream(), Path.of("screenshot.png"), StandardCopyOption.REPLACE_EXISTING);
System.out.println("Saved to screenshot.png");
```
```csharp
var imageBytes = await new HttpClient().GetByteArrayAsync(screenshotUrl);
File.WriteAllBytes("screenshot.png", imageBytes);
Console.WriteLine("Saved to screenshot.png");
```
```rust
use std::{fs::File, io::Write};
let bytes = reqwest::get(screenshot_url).await?.bytes().await?;
let mut file = File::create("screenshot.png").unwrap();
file.write_all(&bytes).unwrap();
println!("Saved to screenshot.png");
```
Use `full_page=false` to capture only the above-the-fold viewport — ideal for social media preview thumbnails where a fixed-height 630×1200 px crop is required.
# Extract Audio from a Video
Overview [#overview]
By passing `quality: "audio"` to the HuntAPI downloader, you get an MP3 extract instead of the full video. This playbook builds a complete audio extraction pipeline including job submission, polling, and saving.
Prerequisites [#prerequisites]
* A HuntAPI key — get one at [app.huntapi.com](https://app.huntapi.com)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Submit an audio extraction job [#submit-an-audio-extraction-job]
Use the same `/v1/video/download` endpoint with `quality=audio`.
```python
import requests
API_KEY = "YOUR_API_KEY"
VIDEO_URL = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
response = requests.get(
"https://api.huntapi.com/v1/video/download",
headers={"x-api-key": API_KEY},
params={"url": VIDEO_URL, "quality": "audio"},
)
data = response.json()
job_id = data["job_id"]
print(f"Audio job submitted: {job_id}")
```
```typescript
const API_KEY = "YOUR_API_KEY";
const VIDEO_URL = "https://www.youtube.com/watch?v=dQw4w9WgXcQ";
const params = new URLSearchParams({ url: VIDEO_URL, quality: "audio" });
const response = await fetch(`https://api.huntapi.com/v1/video/download?${params}`, {
headers: { "x-api-key": API_KEY },
});
const { job_id } = await response.json();
console.log(`Audio job submitted: ${job_id}`);
```
```php
$videoUrl, "quality" => "audio"]);
$ch = curl_init("https://api.huntapi.com/v1/video/download?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
$jobId = $data["job_id"];
echo "Audio job submitted: {$jobId}\n";
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"time"
)
const (
APIKey = "YOUR_API_KEY"
VideoURL = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
)
func get(apiKey, rawURL string) map[string]any {
req, _ := http.NewRequest("GET", rawURL, nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var m map[string]any
json.Unmarshal(body, &m)
return m
}
func main() {
params := url.Values{"url": {VideoURL}, "quality": {"audio"}}
data := get(APIKey, "https://api.huntapi.com/v1/video/download?"+params.Encode())
jobID := data["job_id"].(string)
fmt.Println("Audio job submitted:", jobID)
// polling in next step...
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import org.json.*;
var apiKey = "YOUR_API_KEY";
var videoUrl = URLEncoder.encode("https://www.youtube.com/watch?v=dQw4w9WgXcQ", StandardCharsets.UTF_8);
var client = HttpClient.newHttpClient();
var req = HttpRequest.newBuilder()
.uri(URI.create("https://api.huntapi.com/v1/video/download?url=" + videoUrl + "&quality=audio"))
.header("x-api-key", apiKey).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
var jobId = new JSONObject(resp.body()).getString("job_id");
System.out.println("Audio job submitted: " + jobId);
```
```csharp
using System.Net.Http;
using System.Text.Json;
var apiKey = "YOUR_API_KEY";
var videoUrl = Uri.EscapeDataString("https://www.youtube.com/watch?v=dQw4w9WgXcQ");
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
var body = await client.GetStringAsync($"https://api.huntapi.com/v1/video/download?url={videoUrl}&quality=audio");
var jobId = JsonDocument.Parse(body).RootElement.GetProperty("job_id").GetString()!;
Console.WriteLine($"Audio job submitted: {jobId}");
```
```rust
use reqwest::Client;
use serde_json::Value;
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let api_key = "YOUR_API_KEY";
let video_url = "https://www.youtube.com/watch?v=dQw4w9WgXcQ";
let data = client.get("https://api.huntapi.com/v1/video/download")
.header("x-api-key", api_key)
.query(&[("url", video_url), ("quality", "audio")])
.send().await?.json::().await?;
let job_id = data["job_id"].as_str().unwrap();
println!("Audio job submitted: {}", job_id);
Ok(())
}
```
Poll for completion [#poll-for-completion]
Check the job status every 5 seconds until `status == "done"`.
```python
import time
def wait_for_job(job_id: str) -> dict:
for _ in range(60):
r = requests.get(f"https://api.huntapi.com/v1/job/{job_id}",
headers={"x-api-key": API_KEY})
result = r.json()
status = result["status"]
print(f" [{status}]")
if status == "done":
return result
if status == "error":
raise RuntimeError(result.get("error", "Unknown error"))
time.sleep(5)
raise TimeoutError("Job timed out")
result = wait_for_job(job_id)
download_url = result["download_url"]
```
```typescript
async function waitForJob(jobId: string) {
for (let i = 0; i < 60; i++) {
const res = await fetch(`https://api.huntapi.com/v1/job/${jobId}`, {
headers: { "x-api-key": API_KEY },
});
const result = await res.json();
console.log(` [${result.status}]`);
if (result.status === "done") return result;
if (result.status === "error") throw new Error(result.error ?? "Unknown error");
await new Promise(r => setTimeout(r, 5000));
}
throw new Error("Timeout");
}
const result = await waitForJob(job_id);
const downloadUrl = result.download_url;
```
```php
function waitForJob(string $apiKey, string $jobId): array {
for ($i = 0; $i < 60; $i++) {
$ch = curl_init("https://api.huntapi.com/v1/job/{$jobId}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$result = json_decode(curl_exec($ch), true);
curl_close($ch);
$status = $result["status"] ?? "";
echo " [{$status}]\n";
if ($status === "done") return $result;
if ($status === "error") throw new RuntimeException($result["error"] ?? "Unknown");
sleep(5);
}
throw new RuntimeException("Timeout");
}
$result = waitForJob($apiKey, $jobId);
$downloadUrl = $result["download_url"];
```
```go
func waitForJob(apiKey, jobID string) string {
for i := 0; i < 60; i++ {
data := get(apiKey, "https://api.huntapi.com/v1/job/"+jobID)
status := data["status"].(string)
fmt.Printf(" [%s]\n", status)
if status == "done" { return data["download_url"].(string) }
if status == "error" { panic("Job failed: " + fmt.Sprint(data["error"])) }
time.Sleep(5 * time.Second)
}
panic("Timeout")
}
downloadURL := waitForJob(APIKey, jobID)
```
```java
static String waitForJob(HttpClient client, String apiKey, String jobId) throws Exception {
for (int i = 0; i < 60; i++) {
var req = HttpRequest.newBuilder()
.uri(URI.create("https://api.huntapi.com/v1/job/" + jobId))
.header("x-api-key", apiKey).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
var result = new JSONObject(resp.body());
var status = result.getString("status");
System.out.println(" [" + status + "]");
if ("done".equals(status)) return result.getString("download_url");
if ("error".equals(status)) throw new RuntimeException(result.optString("error"));
Thread.sleep(5000);
}
throw new RuntimeException("Timeout");
}
var downloadUrl = waitForJob(client, apiKey, jobId);
```
```csharp
async Task WaitForJob(HttpClient client, string jobId)
{
for (int i = 0; i < 60; i++)
{
var body = await client.GetStringAsync($"https://api.huntapi.com/v1/job/{jobId}");
var result = JsonDocument.Parse(body).RootElement;
var status = result.GetProperty("status").GetString();
Console.WriteLine($" [{status}]");
if (status == "done") return result.GetProperty("download_url").GetString()!;
if (status == "error") throw new Exception(result.GetProperty("error").GetString());
await Task.Delay(5000);
}
throw new TimeoutException();
}
var downloadUrl = await WaitForJob(client, jobId);
```
```rust
use tokio::time::{sleep, Duration};
async fn wait_for_job(client: &Client, api_key: &str, job_id: &str) -> String {
for _ in 0..60 {
let result = client.get(format!("https://api.huntapi.com/v1/job/{}", job_id))
.header("x-api-key", api_key)
.send().await.unwrap().json::().await.unwrap();
let status = result["status"].as_str().unwrap_or("");
println!(" [{}]", status);
if status == "done" { return result["download_url"].as_str().unwrap().to_string(); }
if status == "error" { panic!("Job failed: {}", result["error"]); }
sleep(Duration::from_secs(5)).await;
}
panic!("Timeout")
}
```
Save the audio file [#save-the-audio-file]
Download the MP3 and save it to disk.
```python
output_file = "audio.mp3"
with requests.get(download_url, stream=True) as r:
r.raise_for_status()
with open(output_file, "wb") as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
print(f"Audio saved to {output_file} ({os.path.getsize(output_file) // 1024} KB)")
```
```typescript
import { createWriteStream } from "fs";
import { Readable } from "stream";
const fileRes = await fetch(downloadUrl);
const writer = createWriteStream("audio.mp3");
Readable.fromWeb(fileRes.body as any).pipe(writer);
await new Promise((resolve, reject) => { writer.on("finish", resolve); writer.on("error", reject); });
console.log("Audio saved to audio.mp3");
```
```php
$fp = fopen("audio.mp3", "wb");
$ch = curl_init($downloadUrl);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_exec($ch);
curl_close($ch);
fclose($fp);
echo "Audio saved to audio.mp3\n";
```
```go
resp, _ := http.Get(downloadURL)
defer resp.Body.Close()
file, _ := os.Create("audio.mp3")
defer file.Close()
io.Copy(file, resp.Body)
fmt.Println("Audio saved to audio.mp3")
```
```java
import java.net.URL;
import java.nio.file.*;
Files.copy(new URL(downloadUrl).openStream(), Path.of("audio.mp3"), StandardCopyOption.REPLACE_EXISTING);
System.out.println("Audio saved to audio.mp3");
```
```csharp
using var audioStream = await new HttpClient().GetStreamAsync(downloadUrl);
using var file = File.Create("audio.mp3");
await audioStream.CopyToAsync(file);
Console.WriteLine("Audio saved to audio.mp3");
```
```rust
use std::{fs::File, io::Write};
let bytes = reqwest::get(download_url).await?.bytes().await?;
let mut file = File::create("audio.mp3").unwrap();
file.write_all(&bytes).unwrap();
println!("Audio saved to audio.mp3");
```
The extracted audio is delivered as an MP3. You can pipe it directly to a transcription service (e.g. OpenAI Whisper or Deepgram) for automatic subtitling or search indexing.
# Download Your First Video
Overview [#overview]
HuntAPI uses an **asynchronous job model**: you submit a URL, receive a `job_id`, then poll until the video is ready. This playbook walks through all three steps and downloads the finished file to disk.
Prerequisites [#prerequisites]
* A HuntAPI key — get one at [app.huntapi.com](https://app.huntapi.com)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json", "stream"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Submit the download job [#submit-the-download-job]
Call `GET /v1/video/download` with the `url` parameter. The response immediately returns a `job_id`.
```python
import requests
API_KEY = "YOUR_API_KEY"
VIDEO_URL = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
response = requests.get(
"https://api.huntapi.com/v1/video/download",
headers={"x-api-key": API_KEY},
params={"url": VIDEO_URL, "quality": "best"},
)
data = response.json()
job_id = data["job_id"]
print(f"Job submitted: {job_id}")
```
```typescript
const API_KEY = "YOUR_API_KEY";
const VIDEO_URL = "https://www.youtube.com/watch?v=dQw4w9WgXcQ";
const params = new URLSearchParams({ url: VIDEO_URL, quality: "best" });
const response = await fetch(`https://api.huntapi.com/v1/video/download?${params}`, {
headers: { "x-api-key": API_KEY },
});
const data = await response.json();
const jobId = data.job_id;
console.log(`Job submitted: ${jobId}`);
```
```php
$videoUrl, "quality" => "best"]);
$ch = curl_init("https://api.huntapi.com/v1/video/download?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
$jobId = $data["job_id"];
echo "Job submitted: {$jobId}\n";
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
const APIKey = "YOUR_API_KEY"
const VideoURL = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
func main() {
params := url.Values{"url": {VideoURL}, "quality": {"best"}}
req, _ := http.NewRequest("GET", "https://api.huntapi.com/v1/video/download?"+params.Encode(), nil)
req.Header.Set("x-api-key", APIKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var data map[string]any
json.Unmarshal(body, &data)
jobID := data["job_id"].(string)
fmt.Println("Job submitted:", jobID)
// continue in next step...
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import org.json.*;
var apiKey = "YOUR_API_KEY";
var videoUrl = URLEncoder.encode("https://www.youtube.com/watch?v=dQw4w9WgXcQ", StandardCharsets.UTF_8);
var url = "https://api.huntapi.com/v1/video/download?url=" + videoUrl + "&quality=best";
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder().uri(URI.create(url))
.header("x-api-key", apiKey).GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
var jobId = new JSONObject(response.body()).getString("job_id");
System.out.println("Job submitted: " + jobId);
```
```csharp
using System.Net.Http;
using System.Text.Json;
var apiKey = "YOUR_API_KEY";
var videoUrl = Uri.EscapeDataString("https://www.youtube.com/watch?v=dQw4w9WgXcQ");
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
var body = await client.GetStringAsync($"https://api.huntapi.com/v1/video/download?url={videoUrl}&quality=best");
var jobId = JsonDocument.Parse(body).RootElement.GetProperty("job_id").GetString()!;
Console.WriteLine($"Job submitted: {jobId}");
```
```rust
use reqwest::Client;
use serde_json::Value;
let client = Client::new();
let api_key = "YOUR_API_KEY";
let video_url = "https://www.youtube.com/watch?v=dQw4w9WgXcQ";
let data = client.get("https://api.huntapi.com/v1/video/download")
.header("x-api-key", api_key)
.query(&[("url", video_url), ("quality", "best")])
.send().await?.json::().await?;
let job_id = data["job_id"].as_str().unwrap();
println!("Job submitted: {}", job_id);
```
Poll until the video is ready [#poll-until-the-video-is-ready]
Check `GET /v1/job/{job_id}` every few seconds. When `status` becomes `"done"`, the `download_url` field contains the file URL.
```python
import time
def wait_for_job(job_id: str, poll_interval: int = 5, timeout: int = 300) -> dict:
start = time.time()
while time.time() - start < timeout:
r = requests.get(f"https://api.huntapi.com/v1/job/{job_id}",
headers={"x-api-key": API_KEY})
result = r.json()
status = result.get("status")
print(f" Status: {status}")
if status == "done":
return result
if status == "error":
raise RuntimeError(f"Job failed: {result.get('error')}")
time.sleep(poll_interval)
raise TimeoutError("Job did not complete within the timeout period.")
result = wait_for_job(job_id)
download_url = result["download_url"]
print(f"Ready! Download URL: {download_url}")
```
```typescript
async function waitForJob(jobId: string, pollMs = 5000, timeoutMs = 300_000) {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
const res = await fetch(`https://api.huntapi.com/v1/job/${jobId}`, {
headers: { "x-api-key": API_KEY },
});
const result = await res.json();
console.log(` Status: ${result.status}`);
if (result.status === "done") return result;
if (result.status === "error") throw new Error(`Job failed: ${result.error}`);
await new Promise(r => setTimeout(r, pollMs));
}
throw new Error("Timeout");
}
const result = await waitForJob(jobId);
const downloadUrl = result.download_url;
console.log(`Ready! Download URL: ${downloadUrl}`);
```
```php
function waitForJob(string $apiKey, string $jobId, int $pollSec = 5, int $timeoutSec = 300): array {
$start = time();
while (time() - $start < $timeoutSec) {
$ch = curl_init("https://api.huntapi.com/v1/job/{$jobId}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$result = json_decode(curl_exec($ch), true);
curl_close($ch);
$status = $result["status"] ?? "";
echo " Status: {$status}\n";
if ($status === "done") return $result;
if ($status === "error") throw new RuntimeException("Job failed: " . ($result["error"] ?? ""));
sleep($pollSec);
}
throw new RuntimeException("Timeout");
}
$result = waitForJob($apiKey, $jobId);
$downloadUrl = $result["download_url"];
echo "Ready! Download URL: {$downloadUrl}\n";
```
```go
import "time"
func waitForJob(apiKey, jobID string) (map[string]any, error) {
deadline := time.Now().Add(5 * time.Minute)
for time.Now().Before(deadline) {
req, _ := http.NewRequest("GET", "https://api.huntapi.com/v1/job/"+jobID, nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
var result map[string]any
json.Unmarshal(body, &result)
status := result["status"].(string)
fmt.Println(" Status:", status)
if status == "done" { return result, nil }
if status == "error" { return nil, fmt.Errorf("job failed: %v", result["error"]) }
time.Sleep(5 * time.Second)
}
return nil, fmt.Errorf("timeout")
}
result, err := waitForJob(APIKey, jobID)
if err != nil { panic(err) }
downloadURL := result["download_url"].(string)
fmt.Println("Ready! Download URL:", downloadURL)
```
```java
import java.time.*;
static JSONObject waitForJob(HttpClient client, String apiKey, String jobId) throws Exception {
var deadline = Instant.now().plusSeconds(300);
while (Instant.now().isBefore(deadline)) {
var req = HttpRequest.newBuilder()
.uri(URI.create("https://api.huntapi.com/v1/job/" + jobId))
.header("x-api-key", apiKey).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
var result = new JSONObject(resp.body());
var status = result.getString("status");
System.out.println(" Status: " + status);
if ("done".equals(status)) return result;
if ("error".equals(status)) throw new RuntimeException("Job failed: " + result.optString("error"));
Thread.sleep(5000);
}
throw new RuntimeException("Timeout");
}
var result = waitForJob(client, apiKey, jobId);
var downloadUrl = result.getString("download_url");
System.out.println("Ready! Download URL: " + downloadUrl);
```
```csharp
async Task WaitForJob(HttpClient client, string jobId)
{
var deadline = DateTime.UtcNow.AddMinutes(5);
while (DateTime.UtcNow < deadline)
{
var body = await client.GetStringAsync($"https://api.huntapi.com/v1/job/{jobId}");
var result = JsonDocument.Parse(body).RootElement;
var status = result.GetProperty("status").GetString();
Console.WriteLine($" Status: {status}");
if (status == "done") return result;
if (status == "error") throw new Exception($"Job failed: {result.GetProperty("error")}");
await Task.Delay(5000);
}
throw new TimeoutException("Job did not complete in time.");
}
var result = await WaitForJob(client, jobId);
var downloadUrl = result.GetProperty("download_url").GetString()!;
Console.WriteLine($"Ready! Download URL: {downloadUrl}");
```
```rust
use tokio::time::{sleep, Duration};
use std::time::Instant;
async fn wait_for_job(client: &Client, api_key: &str, job_id: &str) -> Value {
let deadline = Instant::now() + Duration::from_secs(300);
loop {
assert!(Instant::now() < deadline, "Timeout");
let result = client.get(format!("https://api.huntapi.com/v1/job/{}", job_id))
.header("x-api-key", api_key)
.send().await.unwrap().json::().await.unwrap();
let status = result["status"].as_str().unwrap_or("");
println!(" Status: {}", status);
if status == "done" { return result; }
if status == "error" { panic!("Job failed: {}", result["error"]); }
sleep(Duration::from_secs(5)).await;
}
}
let result = wait_for_job(&client, api_key, job_id).await;
let download_url = result["download_url"].as_str().unwrap();
println!("Ready! Download URL: {}", download_url);
```
Download the video file to disk [#download-the-video-file-to-disk]
Stream the file from the `download_url` and save it locally.
```python
filename = "video.mp4"
with requests.get(download_url, stream=True) as r:
r.raise_for_status()
with open(filename, "wb") as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
print(f"Saved to {filename}")
```
```typescript
import { createWriteStream } from "fs";
import { Readable } from "stream";
const fileResponse = await fetch(downloadUrl);
const writer = createWriteStream("video.mp4");
Readable.fromWeb(fileResponse.body as any).pipe(writer);
await new Promise((resolve, reject) => { writer.on("finish", resolve); writer.on("error", reject); });
console.log("Saved to video.mp4");
```
```php
$fp = fopen("video.mp4", "wb");
$ch = curl_init($downloadUrl);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_exec($ch);
curl_close($ch);
fclose($fp);
echo "Saved to video.mp4\n";
```
```go
import "os"
resp, _ := http.Get(downloadURL)
defer resp.Body.Close()
file, _ := os.Create("video.mp4")
defer file.Close()
io.Copy(file, resp.Body)
fmt.Println("Saved to video.mp4")
```
```java
import java.nio.file.*;
import java.net.URL;
var in = new URL(downloadUrl).openStream();
Files.copy(in, Path.of("video.mp4"), StandardCopyOption.REPLACE_EXISTING);
System.out.println("Saved to video.mp4");
```
```csharp
using var fileStream = File.Create("video.mp4");
using var download = await new HttpClient().GetStreamAsync(downloadUrl);
await download.CopyToAsync(fileStream);
Console.WriteLine("Saved to video.mp4");
```
```rust
use std::io::Write;
use std::fs::File;
let bytes = reqwest::get(download_url).await?.bytes().await?;
let mut file = File::create("video.mp4").unwrap();
file.write_all(&bytes).unwrap();
println!("Saved to video.mp4");
```
You can pass `quality: "best"`, `"1080p"`, `"720p"`, or `"audio"` in the initial request to control the output format before the job is submitted.
# Batch Downloads with Webhooks
Overview [#overview]
Instead of polling, you can pass a `webhook_url` when submitting a job. HuntAPI will POST the result to your URL the moment the download is ready. This playbook shows how to:
1. Submit a batch of jobs with a webhook URL
2. Receive and verify the webhook payload
Prerequisites [#prerequisites]
* A HuntAPI key — get one at [app.huntapi.com](https://app.huntapi.com)
* A publicly reachable HTTP endpoint (use [ngrok](https://ngrok.com) for local testing)
* Install dependencies for your language:
```bash
pip install requests flask
```
```bash
npm install express @types/express
```
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `com.sun.net.httpserver` (built-in, Java 6+).
No extra dependencies — uses ASP.NET minimal API (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
axum = "0.7"
```
Steps [#steps]
Submit a batch of jobs with a webhook URL [#submit-a-batch-of-jobs-with-a-webhook-url]
Pass your publicly accessible endpoint as `webhook_url`. Each job is independent; HuntAPI will call the webhook when it finishes.
```python
import requests
API_KEY = "YOUR_API_KEY"
WEBHOOK_URL = "https://yourserver.example.com/webhooks/huntapi"
URLS = [
"https://www.youtube.com/watch?v=VIDEO_ID_1",
"https://www.youtube.com/watch?v=VIDEO_ID_2",
"https://www.youtube.com/watch?v=VIDEO_ID_3",
]
job_ids = []
for video_url in URLS:
r = requests.get(
"https://api.huntapi.com/v1/video/download",
headers={"x-api-key": API_KEY},
params={"url": video_url, "quality": "best", "webhook_url": WEBHOOK_URL},
)
job_id = r.json()["job_id"]
job_ids.append(job_id)
print(f"Submitted: {job_id}")
print(f"\n{len(job_ids)} jobs submitted. Waiting for webhooks...")
```
```typescript
const API_KEY = "YOUR_API_KEY";
const WEBHOOK_URL = "https://yourserver.example.com/webhooks/huntapi";
const URLS = [
"https://www.youtube.com/watch?v=VIDEO_ID_1",
"https://www.youtube.com/watch?v=VIDEO_ID_2",
"https://www.youtube.com/watch?v=VIDEO_ID_3",
];
const jobIds: string[] = [];
for (const videoUrl of URLS) {
const params = new URLSearchParams({ url: videoUrl, quality: "best", webhook_url: WEBHOOK_URL });
const res = await fetch(`https://api.huntapi.com/v1/video/download?${params}`, {
headers: { "x-api-key": API_KEY },
});
const { job_id } = await res.json();
jobIds.push(job_id);
console.log(`Submitted: ${job_id}`);
}
console.log(`\n${jobIds.length} jobs submitted. Waiting for webhooks...`);
```
```php
$videoUrl, "quality" => "best", "webhook_url" => $webhookUrl]);
$ch = curl_init("https://api.huntapi.com/v1/video/download?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$data = json_decode(curl_exec($ch), true);
curl_close($ch);
$jobIds[] = $data["job_id"];
echo "Submitted: {$data['job_id']}\n";
}
echo count($jobIds) . " jobs submitted. Waiting for webhooks...\n";
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
func submitJob(apiKey, videoURL, webhookURL string) string {
params := url.Values{"url": {videoURL}, "quality": {"best"}, "webhook_url": {webhookURL}}
req, _ := http.NewRequest("GET", "https://api.huntapi.com/v1/video/download?"+params.Encode(), nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var data map[string]any
json.Unmarshal(body, &data)
return data["job_id"].(string)
}
func main() {
apiKey := "YOUR_API_KEY"
webhookURL := "https://yourserver.example.com/webhooks/huntapi"
urls := []string{
"https://www.youtube.com/watch?v=VIDEO_ID_1",
"https://www.youtube.com/watch?v=VIDEO_ID_2",
"https://www.youtube.com/watch?v=VIDEO_ID_3",
}
for _, u := range urls {
jobID := submitJob(apiKey, u, webhookURL)
fmt.Println("Submitted:", jobID)
}
fmt.Printf("%d jobs submitted. Waiting for webhooks...\n", len(urls))
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import org.json.*;
var client = HttpClient.newHttpClient();
var apiKey = "YOUR_API_KEY";
var webhookUrl = "https://yourserver.example.com/webhooks/huntapi";
var urls = new String[]{
"https://www.youtube.com/watch?v=VIDEO_ID_1",
"https://www.youtube.com/watch?v=VIDEO_ID_2",
"https://www.youtube.com/watch?v=VIDEO_ID_3",
};
for (var videoUrl : urls) {
var encoded = URLEncoder.encode(videoUrl, StandardCharsets.UTF_8);
var wh = URLEncoder.encode(webhookUrl, StandardCharsets.UTF_8);
var url = "https://api.huntapi.com/v1/video/download?url=" + encoded + "&quality=best&webhook_url=" + wh;
var req = HttpRequest.newBuilder().uri(URI.create(url))
.header("x-api-key", apiKey).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
var jobId = new JSONObject(resp.body()).getString("job_id");
System.out.println("Submitted: " + jobId);
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
var apiKey = "YOUR_API_KEY";
var webhookUrl = Uri.EscapeDataString("https://yourserver.example.com/webhooks/huntapi");
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
var urls = new[]
{
"https://www.youtube.com/watch?v=VIDEO_ID_1",
"https://www.youtube.com/watch?v=VIDEO_ID_2",
"https://www.youtube.com/watch?v=VIDEO_ID_3",
};
foreach (var videoUrl in urls)
{
var encoded = Uri.EscapeDataString(videoUrl);
var body = await client.GetStringAsync(
$"https://api.huntapi.com/v1/video/download?url={encoded}&quality=best&webhook_url={webhookUrl}");
var jobId = JsonDocument.Parse(body).RootElement.GetProperty("job_id").GetString();
Console.WriteLine($"Submitted: {jobId}");
}
```
```rust
use reqwest::Client;
use serde_json::Value;
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let api_key = "YOUR_API_KEY";
let webhook_url = "https://yourserver.example.com/webhooks/huntapi";
let urls = [
"https://www.youtube.com/watch?v=VIDEO_ID_1",
"https://www.youtube.com/watch?v=VIDEO_ID_2",
"https://www.youtube.com/watch?v=VIDEO_ID_3",
];
for video_url in &urls {
let data = client.get("https://api.huntapi.com/v1/video/download")
.header("x-api-key", api_key)
.query(&[("url", video_url), ("quality", &"best"), ("webhook_url", &webhook_url)])
.send().await?.json::().await?;
println!("Submitted: {}", data["job_id"].as_str().unwrap_or(""));
}
Ok(())
}
```
Build a webhook receiver [#build-a-webhook-receiver]
HuntAPI will POST a JSON body to your endpoint with `job_id`, `status`, and `download_url` when the job completes.
```python
from flask import Flask, request, jsonify
app = Flask(__name__)
@app.route("/webhooks/huntapi", methods=["POST"])
def huntapi_webhook():
payload = request.get_json()
job_id = payload.get("job_id")
status = payload.get("status")
download_url = payload.get("download_url")
print(f"Webhook received — Job: {job_id}, Status: {status}")
if status == "done" and download_url:
# Trigger your download or further processing here
print(f" Download URL: {download_url}")
return jsonify({"received": True}), 200
if __name__ == "__main__":
app.run(port=3000)
```
```typescript
import express from "express";
const app = express();
app.use(express.json());
app.post("/webhooks/huntapi", (req, res) => {
const { job_id, status, download_url } = req.body;
console.log(`Webhook received — Job: ${job_id}, Status: ${status}`);
if (status === "done" && download_url) {
// Trigger your download or further processing here
console.log(` Download URL: ${download_url}`);
}
res.json({ received: true });
});
app.listen(3000, () => console.log("Webhook listener on :3000"));
```
```php
true]);
```
```go
package main
import (
"encoding/json"
"fmt"
"net/http"
)
func main() {
http.HandleFunc("/webhooks/huntapi", func(w http.ResponseWriter, r *http.Request) {
var payload map[string]any
json.NewDecoder(r.Body).Decode(&payload)
jobID := payload["job_id"]
status := payload["status"]
downloadURL := payload["download_url"]
fmt.Printf("Webhook received — Job: %v, Status: %v\n", jobID, status)
if status == "done" && downloadURL != nil {
fmt.Println(" Download URL:", downloadURL)
// Trigger download or further processing here
}
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{"received":true}`))
})
fmt.Println("Webhook listener on :3000")
http.ListenAndServe(":3000", nil)
}
```
```java
import com.sun.net.httpserver.*;
import java.net.InetSocketAddress;
import org.json.*;
public class WebhookServer {
public static void main(String[] args) throws Exception {
var server = HttpServer.create(new InetSocketAddress(3000), 0);
server.createContext("/webhooks/huntapi", exchange -> {
var body = exchange.getRequestBody().readAllBytes();
var payload = new JSONObject(new String(body));
var jobId = payload.optString("job_id");
var status = payload.optString("status");
var downloadUrl = payload.optString("download_url");
System.out.printf("Webhook received — Job: %s, Status: %s%n", jobId, status);
if ("done".equals(status) && !downloadUrl.isEmpty()) {
System.out.println(" Download URL: " + downloadUrl);
}
var resp = "{\"received\":true}".getBytes();
exchange.sendResponseHeaders(200, resp.length);
exchange.getResponseBody().write(resp);
exchange.close();
});
server.start();
System.out.println("Webhook listener on :3000");
}
}
```
```csharp
using System.Text.Json;
var app = WebApplication.Create();
app.MapPost("/webhooks/huntapi", async (HttpContext ctx) =>
{
using var reader = new StreamReader(ctx.Request.Body);
var body = await reader.ReadToEndAsync();
var payload = JsonDocument.Parse(body).RootElement;
var jobId = payload.GetProperty("job_id").GetString();
var status = payload.GetProperty("status").GetString();
var downloadUrl = payload.TryGetProperty("download_url", out var d) ? d.GetString() : null;
Console.WriteLine($"Webhook received — Job: {jobId}, Status: {status}");
if (status == "done" && downloadUrl != null)
Console.WriteLine($" Download URL: {downloadUrl}");
return Results.Json(new { received = true });
});
Console.WriteLine("Webhook listener on :3000");
app.Run("http://0.0.0.0:3000");
```
```rust
use axum::{extract::Json as AxumJson, routing::post, Router};
use serde_json::{json, Value};
async fn huntapi_webhook(AxumJson(payload): AxumJson) -> AxumJson {
let job_id = payload["job_id"].as_str().unwrap_or("");
let status = payload["status"].as_str().unwrap_or("");
let download_url = payload["download_url"].as_str().unwrap_or("");
println!("Webhook received — Job: {}, Status: {}", job_id, status);
if status == "done" && !download_url.is_empty() {
println!(" Download URL: {}", download_url);
}
AxumJson(json!({ "received": true }))
}
#[tokio::main]
async fn main() {
let app = Router::new().route("/webhooks/huntapi", post(huntapi_webhook));
let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
println!("Webhook listener on :3000");
axum::serve(listener, app).await.unwrap();
}
```
Test locally with ngrok [#test-locally-with-ngrok]
During development, use ngrok to expose your local server to the internet so HuntAPI can reach your webhook.
```bash
# Start your local server first, then:
ngrok http 3000
```
Copy the generated `https://xxxx.ngrok.io` URL and use it as your `webhook_url` parameter.
Your webhook endpoint must return a `2xx` status code within 10 seconds, otherwise HuntAPI will retry the delivery. Make heavy processing asynchronous and acknowledge the webhook immediately.
# Find a Professional Email
Overview [#overview]
This playbook shows how to find and verify a professional email address for a prospect. Use it in outbound sales pipelines to auto-enrich contact records before sending a sequence.
Prerequisites [#prerequisites]
* A Piloterr API key — get one at [app.piloterr.com](https://app.piloterr.com)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Find an email address [#find-an-email-address]
Call `GET /v2/email/finder` with `first_name`, `last_name`, and `domain`.
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.piloterr.com/v2/email/finder",
headers={"x-api-key": API_KEY},
params={"first_name": "Patrick", "last_name": "Collison", "domain": "stripe.com"},
)
result = response.json()
print(result)
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({ first_name: "Patrick", last_name: "Collison", domain: "stripe.com" });
const response = await fetch(`https://api.piloterr.com/v2/email/finder?${params}`, {
headers: { "x-api-key": API_KEY },
});
const result = await response.json();
console.log(result);
```
```php
"Patrick", "last_name" => "Collison", "domain" => "stripe.com"]);
$ch = curl_init("https://api.piloterr.com/v2/email/finder?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$result = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($result);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
func main() {
params := url.Values{
"first_name": {"Patrick"}, "last_name": {"Collison"}, "domain": {"stripe.com"},
}
req, _ := http.NewRequest("GET", "https://api.piloterr.com/v2/email/finder?"+params.Encode(), nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var result map[string]any
json.Unmarshal(body, &result)
fmt.Println(result)
}
```
```java
import java.net.URI;
import java.net.http.*;
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.piloterr.com/v2/email/finder?first_name=Patrick&last_name=Collison&domain=stripe.com"))
.header("x-api-key", "YOUR_API_KEY")
.GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var body = await client.GetStringAsync(
"https://api.piloterr.com/v2/email/finder?first_name=Patrick&last_name=Collison&domain=stripe.com");
Console.WriteLine(body);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let result = reqwest::Client::new()
.get("https://api.piloterr.com/v2/email/finder")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("first_name", "Patrick"), ("last_name", "Collison"), ("domain", "stripe.com")])
.send().await?.json::().await?;
println!("{:#?}", result);
Ok(())
}
```
Inspect the result [#inspect-the-result]
The response includes `email`, `confidence` (0–100), `status` (`valid`, `risky`, `invalid`), and the detected email pattern.
```python
email = result.get("email")
confidence = result.get("confidence")
status = result.get("status")
if email and confidence >= 70:
print(f"✓ Found: {email} (confidence: {confidence}%, status: {status})")
else:
print(f"✗ Email not found or low confidence (status: {status})")
```
```typescript
const { email, confidence, status } = result;
if (email && confidence >= 70) {
console.log(`✓ Found: ${email} (confidence: ${confidence}%, status: ${status})`);
} else {
console.log(`✗ Email not found or low confidence (status: ${status})`);
}
```
```php
$email = $result["email"] ?? null;
$confidence = $result["confidence"] ?? 0;
$status = $result["status"] ?? "unknown";
if ($email && $confidence >= 70) {
echo "✓ Found: {$email} (confidence: {$confidence}%, status: {$status})\n";
} else {
echo "✗ Email not found or low confidence (status: {$status})\n";
}
```
```go
email := result["email"]
confidence := result["confidence"]
status := result["status"]
if email != nil && confidence.(float64) >= 70 {
fmt.Printf("✓ Found: %v (confidence: %.0f%%, status: %v)\n", email, confidence, status)
} else {
fmt.Printf("✗ Email not found or low confidence (status: %v)\n", status)
}
```
```java
import org.json.*;
var r = new JSONObject(response.body());
var email = r.optString("email", null);
var confidence = r.optInt("confidence", 0);
var status = r.optString("status", "unknown");
if (email != null && confidence >= 70) {
System.out.printf("✓ Found: %s (confidence: %d%%, status: %s)%n", email, confidence, status);
} else {
System.out.printf("✗ Email not found or low confidence (status: %s)%n", status);
}
```
```csharp
using System.Text.Json;
var r = JsonDocument.Parse(body).RootElement;
var email = r.TryGetProperty("email", out var e) ? e.GetString() : null;
var confidence = r.TryGetProperty("confidence", out var c) ? c.GetInt32() : 0;
var status = r.TryGetProperty("status", out var s) ? s.GetString() : "unknown";
if (email != null && confidence >= 70)
Console.WriteLine($"✓ Found: {email} (confidence: {confidence}%, status: {status})");
else
Console.WriteLine($"✗ Email not found or low confidence (status: {status})");
```
```rust
let email = result["email"].as_str().unwrap_or("");
let confidence = result["confidence"].as_i64().unwrap_or(0);
let status = result["status"].as_str().unwrap_or("unknown");
if !email.is_empty() && confidence >= 70 {
println!("✓ Found: {} (confidence: {}%, status: {})", email, confidence, status);
} else {
println!("✗ Email not found or low confidence (status: {})", status);
}
```
Enrich a list of prospects from a CSV [#enrich-a-list-of-prospects-from-a-csv]
Load a CSV of prospects, find their emails, and write results back out.
```python
import csv, time, requests
API_KEY = "YOUR_API_KEY"
def find_email(first: str, last: str, domain: str) -> dict:
r = requests.get("https://api.piloterr.com/v2/email/finder",
headers={"x-api-key": API_KEY},
params={"first_name": first, "last_name": last, "domain": domain})
return r.json()
prospects = [
{"first_name": "Patrick", "last_name": "Collison", "domain": "stripe.com"},
{"first_name": "Sam", "last_name": "Altman", "domain": "openai.com"},
{"first_name": "Tobi", "last_name": "Lutke", "domain": "shopify.com"},
]
with open("prospects_enriched.csv", "w", newline="") as f:
fieldnames = ["first_name", "last_name", "domain", "email", "confidence", "status"]
writer = csv.DictWriter(f, fieldnames=fieldnames)
writer.writeheader()
for p in prospects:
result = find_email(p["first_name"], p["last_name"], p["domain"])
writer.writerow({**p, "email": result.get("email", ""), "confidence": result.get("confidence", 0), "status": result.get("status", "")})
time.sleep(0.5)
print("Done! Results saved to prospects_enriched.csv")
```
```typescript
import { createWriteStream } from "fs";
const API_KEY = "YOUR_API_KEY";
const prospects = [
{ first_name: "Patrick", last_name: "Collison", domain: "stripe.com" },
{ first_name: "Sam", last_name: "Altman", domain: "openai.com" },
{ first_name: "Tobi", last_name: "Lutke", domain: "shopify.com" },
];
const rows: string[] = ["first_name,last_name,domain,email,confidence,status"];
for (const p of prospects) {
const params = new URLSearchParams({ first_name: p.first_name, last_name: p.last_name, domain: p.domain });
const res = await fetch(`https://api.piloterr.com/v2/email/finder?${params}`, {
headers: { "x-api-key": API_KEY },
});
const r = await res.json();
rows.push(`${p.first_name},${p.last_name},${p.domain},${r.email ?? ""},${r.confidence ?? 0},${r.status ?? ""}`);
await new Promise(resolve => setTimeout(resolve, 500));
}
import { writeFileSync } from "fs";
writeFileSync("prospects_enriched.csv", rows.join("\n"));
console.log("Done! Results saved to prospects_enriched.csv");
```
```php
"Patrick", "last_name" => "Collison", "domain" => "stripe.com"],
["first_name" => "Sam", "last_name" => "Altman", "domain" => "openai.com"],
["first_name" => "Tobi", "last_name" => "Lutke", "domain" => "shopify.com"],
];
$fp = fopen("prospects_enriched.csv", "w");
fputcsv($fp, ["first_name", "last_name", "domain", "email", "confidence", "status"]);
foreach ($prospects as $p) {
$params = http_build_query(array_merge($p, ["q" => ""]));
$ch = curl_init("https://api.piloterr.com/v2/email/finder?" . http_build_query($p));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$r = json_decode(curl_exec($ch), true);
curl_close($ch);
fputcsv($fp, [$p["first_name"], $p["last_name"], $p["domain"],
$r["email"] ?? "", $r["confidence"] ?? 0, $r["status"] ?? ""]);
usleep(500000);
}
fclose($fp);
echo "Done! Results saved to prospects_enriched.csv\n";
```
```go
package main
import (
"encoding/csv"
"encoding/json"
"io"
"net/http"
"net/url"
"os"
"strconv"
"time"
)
type Prospect struct{ FirstName, LastName, Domain string }
func findEmail(apiKey string, p Prospect) map[string]any {
params := url.Values{"first_name": {p.FirstName}, "last_name": {p.LastName}, "domain": {p.Domain}}
req, _ := http.NewRequest("GET", "https://api.piloterr.com/v2/email/finder?"+params.Encode(), nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var r map[string]any
json.Unmarshal(body, &r)
return r
}
func main() {
apiKey := "YOUR_API_KEY"
prospects := []Prospect{
{"Patrick", "Collison", "stripe.com"},
{"Sam", "Altman", "openai.com"},
{"Tobi", "Lutke", "shopify.com"},
}
f, _ := os.Create("prospects_enriched.csv")
w := csv.NewWriter(f)
w.Write([]string{"first_name", "last_name", "domain", "email", "confidence", "status"})
for _, p := range prospects {
r := findEmail(apiKey, p)
confidence := strconv.FormatFloat(r["confidence"].(float64), 'f', 0, 64)
w.Write([]string{p.FirstName, p.LastName, p.Domain,
r["email"].(string), confidence, r["status"].(string)})
time.Sleep(500 * time.Millisecond)
}
w.Flush()
f.Close()
}
```
```java
import java.net.URI;
import java.net.http.*;
import java.nio.file.*;
import java.util.*;
import org.json.*;
public class Main {
public static void main(String[] args) throws Exception {
var client = HttpClient.newHttpClient();
var apiKey = "YOUR_API_KEY";
var prospects = List.of(
Map.of("first_name","Patrick","last_name","Collison","domain","stripe.com"),
Map.of("first_name","Sam", "last_name","Altman", "domain","openai.com"),
Map.of("first_name","Tobi", "last_name","Lutke", "domain","shopify.com"));
var lines = new ArrayList();
lines.add("first_name,last_name,domain,email,confidence,status");
for (var p : prospects) {
var url = "https://api.piloterr.com/v2/email/finder?first_name=" + p.get("first_name")
+ "&last_name=" + p.get("last_name") + "&domain=" + p.get("domain");
var req = HttpRequest.newBuilder().uri(URI.create(url))
.header("x-api-key", apiKey).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
var r = new JSONObject(resp.body());
lines.add(String.join(",", p.get("first_name"), p.get("last_name"), p.get("domain"),
r.optString("email",""), String.valueOf(r.optInt("confidence")), r.optString("status","")));
Thread.sleep(500);
}
Files.write(Path.of("prospects_enriched.csv"), lines);
System.out.println("Done! Results saved to prospects_enriched.csv");
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var prospects = new[] {
(first: "Patrick", last: "Collison", domain: "stripe.com"),
(first: "Sam", last: "Altman", domain: "openai.com"),
(first: "Tobi", last: "Lutke", domain: "shopify.com"),
};
var lines = new List { "first_name,last_name,domain,email,confidence,status" };
foreach (var p in prospects)
{
var body = await client.GetStringAsync(
$"https://api.piloterr.com/v2/email/finder?first_name={p.first}&last_name={p.last}&domain={p.domain}");
var r = JsonDocument.Parse(body).RootElement;
var email = r.TryGetProperty("email", out var e) ? e.GetString() : "";
var confidence = r.TryGetProperty("confidence", out var c) ? c.GetInt32().ToString() : "0";
var status = r.TryGetProperty("status", out var s) ? s.GetString() : "";
lines.Add($"{p.first},{p.last},{p.domain},{email},{confidence},{status}");
await Task.Delay(500);
}
File.WriteAllLines("prospects_enriched.csv", lines);
Console.WriteLine("Done! Results saved to prospects_enriched.csv");
```
```rust
use reqwest::Client;
use serde_json::Value;
use std::{fs::File, io::Write, time::Duration};
use tokio::time::sleep;
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let api_key = "YOUR_API_KEY";
let prospects = vec![
("Patrick", "Collison", "stripe.com"),
("Sam", "Altman", "openai.com"),
("Tobi", "Lutke", "shopify.com"),
];
let mut file = File::create("prospects_enriched.csv").unwrap();
writeln!(file, "first_name,last_name,domain,email,confidence,status").unwrap();
for (first, last, domain) in &prospects {
let r = client.get("https://api.piloterr.com/v2/email/finder")
.header("x-api-key", api_key)
.query(&[("first_name", first), ("last_name", last), ("domain", domain)])
.send().await?.json::().await?;
writeln!(file, "{},{},{},{},{},{}",
first, last, domain,
r["email"].as_str().unwrap_or(""),
r["confidence"].as_i64().unwrap_or(0),
r["status"].as_str().unwrap_or("")).unwrap();
sleep(Duration::from_millis(500)).await;
}
println!("Done! Results saved to prospects_enriched.csv");
Ok(())
}
```
Only use emails with `confidence >= 70` and `status == "valid"` in cold outreach. Lower-confidence addresses risk bounces that harm your domain reputation.
# Enrich a Lead with LinkedIn
Overview [#overview]
This playbook builds a **CRM enrichment pipeline**: given a company's website domain, fetch its LinkedIn profile and extract structured data like industry, employee count, headquarters, and specialities.
Prerequisites [#prerequisites]
* A Piloterr API key — get one at [app.piloterr.com](https://app.piloterr.com)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default in most PHP installs).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Look up a company by domain [#look-up-a-company-by-domain]
Pass the company's website domain to the `domain` parameter. You can also use a LinkedIn URL via `query`.
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.piloterr.com/v2/linkedin/company/info",
headers={"x-api-key": API_KEY},
params={"domain": "stripe.com"},
)
company = response.json()
print(company)
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({ domain: "stripe.com" });
const response = await fetch(`https://api.piloterr.com/v2/linkedin/company/info?${params}`, {
headers: { "x-api-key": API_KEY },
});
const company = await response.json();
console.log(company);
```
```php
"stripe.com"]);
$ch = curl_init("https://api.piloterr.com/v2/linkedin/company/info?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$company = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($company);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
)
func main() {
req, _ := http.NewRequest("GET",
"https://api.piloterr.com/v2/linkedin/company/info?domain=stripe.com", nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var company map[string]any
json.Unmarshal(body, &company)
fmt.Println(company)
}
```
```java
import java.net.URI;
import java.net.http.*;
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.piloterr.com/v2/linkedin/company/info?domain=stripe.com"))
.header("x-api-key", "YOUR_API_KEY")
.GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var body = await client.GetStringAsync(
"https://api.piloterr.com/v2/linkedin/company/info?domain=stripe.com");
Console.WriteLine(body);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let company = reqwest::Client::new()
.get("https://api.piloterr.com/v2/linkedin/company/info")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("domain", "stripe.com")])
.send().await?.json::().await?;
println!("{:#?}", company);
Ok(())
}
```
Extract key company fields [#extract-key-company-fields]
The response contains `company_name`, `industry`, `staff_count`, `staff_range`, `tagline`, `description`, `headquarter`, and `specialities`.
```python
print(f"Company : {company.get('company_name')}")
print(f"Industry : {company.get('industry')}")
print(f"Employees : {company.get('staff_count')} ({company.get('staff_range')})")
print(f"Website : {company.get('website')}")
hq = company.get("headquarter", {})
print(f"HQ : {hq.get('city')}, {hq.get('country')}")
print(f"Topics : {', '.join(company.get('specialities', [])[:5])}")
```
```typescript
console.log(`Company : ${company.company_name}`);
console.log(`Industry : ${company.industry}`);
console.log(`Employees : ${company.staff_count} (${company.staff_range})`);
console.log(`Website : ${company.website}`);
console.log(`HQ : ${company.headquarter?.city}, ${company.headquarter?.country}`);
console.log(`Topics : ${(company.specialities ?? []).slice(0, 5).join(", ")}`);
```
```php
echo "Company : {$company['company_name']}\n";
echo "Industry : {$company['industry']}\n";
echo "Employees : {$company['staff_count']} ({$company['staff_range']})\n";
echo "HQ : {$company['headquarter']['city']}, {$company['headquarter']['country']}\n";
echo "Topics : " . implode(", ", array_slice($company["specialities"] ?? [], 0, 5)) . "\n";
```
```go
c := company
fmt.Printf("Company : %v\nIndustry : %v\nEmployees : %v (%v)\nWebsite : %v\n",
c["company_name"], c["industry"], c["staff_count"], c["staff_range"], c["website"])
if hq, ok := c["headquarter"].(map[string]any); ok {
fmt.Printf("HQ : %v, %v\n", hq["city"], hq["country"])
}
```
```java
import org.json.*;
var c = new JSONObject(response.body());
System.out.printf("Company : %s%nIndustry : %s%nEmployees : %d (%s)%nWebsite : %s%n",
c.getString("company_name"), c.getString("industry"),
c.getInt("staff_count"), c.getString("staff_range"), c.getString("website"));
var hq = c.optJSONObject("headquarter");
if (hq != null) System.out.printf("HQ : %s, %s%n", hq.getString("city"), hq.getString("country"));
```
```csharp
using System.Text.Json;
var c = JsonDocument.Parse(body).RootElement;
Console.WriteLine($"Company : {c.GetProperty("company_name")}");
Console.WriteLine($"Industry : {c.GetProperty("industry")}");
Console.WriteLine($"Employees : {c.GetProperty("staff_count")} ({c.GetProperty("staff_range")})");
Console.WriteLine($"Website : {c.GetProperty("website")}");
var hq = c.GetProperty("headquarter");
Console.WriteLine($"HQ : {hq.GetProperty("city")}, {hq.GetProperty("country")}");
```
```rust
let c = &company;
println!("Company : {}", c["company_name"].as_str().unwrap_or(""));
println!("Industry : {}", c["industry"].as_str().unwrap_or(""));
println!("Employees : {} ({})", c["staff_count"], c["staff_range"].as_str().unwrap_or(""));
println!("HQ : {}, {}", c["headquarter"]["city"].as_str().unwrap_or(""), c["headquarter"]["country"].as_str().unwrap_or(""));
```
Enrich a batch of leads [#enrich-a-batch-of-leads]
Loop over a list of email domains and build enriched company records ready to push to your CRM.
```python
import json, time, requests
API_KEY = "YOUR_API_KEY"
leads = [
{"email": "alice@stripe.com", "domain": "stripe.com"},
{"email": "bob@notion.so", "domain": "notion.so"},
{"email": "carol@figma.com", "domain": "figma.com"},
]
enriched = []
for lead in leads:
r = requests.get("https://api.piloterr.com/v2/linkedin/company/info",
headers={"x-api-key": API_KEY}, params={"domain": lead["domain"]})
if r.status_code == 200:
c = r.json()
enriched.append({"email": lead["email"], "domain": lead["domain"],
"company": c.get("company_name"), "industry": c.get("industry"),
"employees": c.get("staff_count"), "hq_country": c.get("headquarter", {}).get("country"),
"linkedin_url": c.get("company_url")})
time.sleep(0.5)
with open("enriched_leads.json", "w") as f:
json.dump(enriched, f, indent=2)
print(f"Enriched {len(enriched)} leads → enriched_leads.json")
```
```typescript
import { writeFileSync } from "fs";
const API_KEY = "YOUR_API_KEY";
const leads = [
{ email: "alice@stripe.com", domain: "stripe.com" },
{ email: "bob@notion.so", domain: "notion.so" },
{ email: "carol@figma.com", domain: "figma.com" },
];
const enriched: any[] = [];
for (const lead of leads) {
const params = new URLSearchParams({ domain: lead.domain });
const res = await fetch(`https://api.piloterr.com/v2/linkedin/company/info?${params}`, {
headers: { "x-api-key": API_KEY },
});
if (res.ok) {
const c = await res.json();
enriched.push({ email: lead.email, domain: lead.domain, company: c.company_name,
industry: c.industry, employees: c.staff_count, hq_country: c.headquarter?.country,
linkedin_url: c.company_url });
}
await new Promise(r => setTimeout(r, 500));
}
writeFileSync("enriched_leads.json", JSON.stringify(enriched, null, 2));
console.log(`Enriched ${enriched.length} leads → enriched_leads.json`);
```
```php
"alice@stripe.com", "domain" => "stripe.com"],
["email" => "bob@notion.so", "domain" => "notion.so"],
["email" => "carol@figma.com", "domain" => "figma.com"],
];
$enriched = [];
foreach ($leads as $lead) {
$params = http_build_query(["domain" => $lead["domain"]]);
$ch = curl_init("https://api.piloterr.com/v2/linkedin/company/info?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$c = json_decode(curl_exec($ch), true);
if (curl_getinfo($ch, CURLINFO_HTTP_CODE) === 200) {
$enriched[] = ["email" => $lead["email"], "domain" => $lead["domain"],
"company" => $c["company_name"] ?? null, "industry" => $c["industry"] ?? null,
"employees" => $c["staff_count"] ?? null, "hq_country" => $c["headquarter"]["country"] ?? null];
}
curl_close($ch);
usleep(500000);
}
file_put_contents("enriched_leads.json", json_encode($enriched, JSON_PRETTY_PRINT));
echo "Enriched " . count($enriched) . " leads → enriched_leads.json\n";
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"time"
)
func enrichDomain(apiKey, domain string) map[string]any {
req, _ := http.NewRequest("GET",
"https://api.piloterr.com/v2/linkedin/company/info?domain="+domain, nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var c map[string]any
json.Unmarshal(body, &c)
return c
}
func main() {
apiKey := "YOUR_API_KEY"
leads := []struct{ Email, Domain string }{
{"alice@stripe.com", "stripe.com"},
{"bob@notion.so", "notion.so"},
{"carol@figma.com", "figma.com"},
}
var enriched []map[string]any
for _, lead := range leads {
c := enrichDomain(apiKey, lead.Domain)
hq, _ := c["headquarter"].(map[string]any)
enriched = append(enriched, map[string]any{
"email": lead.Email, "domain": lead.Domain,
"company": c["company_name"], "industry": c["industry"],
"employees": c["staff_count"], "hq_country": hq["country"],
})
time.Sleep(500 * time.Millisecond)
}
b, _ := json.MarshalIndent(enriched, "", " ")
os.WriteFile("enriched_leads.json", b, 0644)
fmt.Printf("Enriched %d leads → enriched_leads.json\n", len(enriched))
}
```
```java
import java.net.URI;
import java.net.http.*;
import java.nio.file.*;
import java.util.*;
import org.json.*;
public class Main {
public static void main(String[] args) throws Exception {
var client = HttpClient.newHttpClient();
var apiKey = "YOUR_API_KEY";
var leads = List.of(
Map.of("email", "alice@stripe.com", "domain", "stripe.com"),
Map.of("email", "bob@notion.so", "domain", "notion.so"),
Map.of("email", "carol@figma.com", "domain", "figma.com"));
var enriched = new JSONArray();
for (var lead : leads) {
var url = "https://api.piloterr.com/v2/linkedin/company/info?domain=" + lead.get("domain");
var req = HttpRequest.newBuilder().uri(URI.create(url))
.header("x-api-key", apiKey).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
if (resp.statusCode() == 200) {
var c = new JSONObject(resp.body());
var hq = c.optJSONObject("headquarter");
enriched.put(new JSONObject()
.put("email", lead.get("email"))
.put("domain", lead.get("domain"))
.put("company", c.optString("company_name"))
.put("industry", c.optString("industry"))
.put("employees", c.optInt("staff_count"))
.put("hq_country", hq != null ? hq.optString("country") : ""));
}
Thread.sleep(500);
}
Files.writeString(Path.of("enriched_leads.json"), enriched.toString(2));
System.out.println("Enriched " + enriched.length() + " leads → enriched_leads.json");
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var leads = new[] {
(email: "alice@stripe.com", domain: "stripe.com"),
(email: "bob@notion.so", domain: "notion.so"),
(email: "carol@figma.com", domain: "figma.com"),
};
var enriched = new List
```rust
use reqwest::Client;
use serde_json::{json, Value};
use std::{fs, time::Duration};
use tokio::time::sleep;
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let api_key = "YOUR_API_KEY";
let leads = vec![("alice@stripe.com", "stripe.com"), ("bob@notion.so", "notion.so"), ("carol@figma.com", "figma.com")];
let mut enriched: Vec = Vec::new();
for (email, domain) in &leads {
let c = client.get("https://api.piloterr.com/v2/linkedin/company/info")
.header("x-api-key", api_key).query(&[("domain", domain)])
.send().await?.json::().await?;
enriched.push(json!({
"email": email, "domain": domain,
"company": c["company_name"], "industry": c["industry"],
"employees": c["staff_count"], "hq_country": c["headquarter"]["country"],
}));
sleep(Duration::from_millis(500)).await;
}
fs::write("enriched_leads.json", serde_json::to_string_pretty(&enriched).unwrap()).unwrap();
println!("Enriched {} leads → enriched_leads.json", enriched.len());
Ok(())
}
```
You can also pass a LinkedIn company URL or username to the `query` parameter instead of a domain — useful when you already have the LinkedIn URL from a scrape or manual research.
# Crawl Any Website
Overview [#overview]
This playbook shows how to fetch the full HTML of any webpage using the Piloterr Website Crawler, then extract specific data from it. A typical use case is **competitor price monitoring**: crawl a product page daily and parse the price from the HTML.
Prerequisites [#prerequisites]
* A Piloterr API key — get one at [app.piloterr.com](https://app.piloterr.com)
* Install dependencies for your language:
```bash
pip install requests beautifulsoup4
```
```bash
npm install node-html-parser
```
`curl` and `DOMDocument` extensions (both enabled by default).
No extra dependencies for the request — uses `net/http` (Go 1.18+). Add `golang.org/x/net/html` for parsing.
No extra dependencies for the request — uses `java.net.http` (Java 11+). Add `org.jsoup:jsoup` for parsing.
No extra dependencies for the request — uses `System.Net.Http` (.NET 6+). Add `HtmlAgilityPack` for parsing.
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Crawl a webpage [#crawl-a-webpage]
Call `GET /v2/website/crawler` with the `query` parameter set to the target URL. The response is the raw HTML string (JSON-encoded).
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.piloterr.com/v2/website/crawler",
headers={"x-api-key": API_KEY},
params={"query": "https://example.com", "allow_redirects": "true"},
)
html = response.json() # returns the HTML as a string
print(html[:500])
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({ query: "https://example.com", allow_redirects: "true" });
const response = await fetch(`https://api.piloterr.com/v2/website/crawler?${params}`, {
headers: { "x-api-key": API_KEY },
});
const html: string = await response.json();
console.log(html.slice(0, 500));
```
```php
"https://example.com", "allow_redirects" => "true"]);
$ch = curl_init("https://api.piloterr.com/v2/website/crawler?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$html = json_decode(curl_exec($ch), true); // HTML string
curl_close($ch);
echo substr($html, 0, 500);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
func main() {
params := url.Values{"query": {"https://example.com"}, "allow_redirects": {"true"}}
req, _ := http.NewRequest("GET", "https://api.piloterr.com/v2/website/crawler?"+params.Encode(), nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var html string
json.Unmarshal(body, &html) // response is a JSON-encoded string
fmt.Println(html[:500])
}
```
```java
import java.net.URI;
import java.net.http.*;
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.piloterr.com/v2/website/crawler?query=https%3A%2F%2Fexample.com&allow_redirects=true"))
.header("x-api-key", "YOUR_API_KEY")
.GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
// Response body is a JSON-encoded string — strip the outer quotes
var html = response.body().replaceAll("^\"|\"$", "")
.replace("\\n", "\n").replace("\\\"", "\"");
System.out.println(html.substring(0, Math.min(500, html.length())));
```
```csharp
using System.Net.Http;
using System.Text.Json;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var raw = await client.GetStringAsync(
"https://api.piloterr.com/v2/website/crawler?query=https%3A%2F%2Fexample.com&allow_redirects=true");
var html = JsonSerializer.Deserialize(raw)!; // response is JSON-encoded string
Console.WriteLine(html[..Math.Min(500, html.Length)]);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let html = reqwest::Client::new()
.get("https://api.piloterr.com/v2/website/crawler")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("query", "https://example.com"), ("allow_redirects", "true")])
.send().await?
.json::().await?;
println!("{}", &html[..500.min(html.len())]);
Ok(())
}
```
Extract data from the HTML [#extract-data-from-the-html]
Parse the HTML to extract specific elements — here we extract the page title and all `
` headings.
```python
from bs4 import BeautifulSoup
soup = BeautifulSoup(html, "html.parser")
title = soup.find("title")
print("Page title:", title.text if title else "N/A")
for h in soup.find_all("h1"):
print("H1:", h.get_text(strip=True))
```
```typescript
import { parse } from "node-html-parser";
const root = parse(html);
const title = root.querySelector("title");
console.log("Page title:", title?.text ?? "N/A");
for (const h of root.querySelectorAll("h1")) {
console.log("H1:", h.text.trim());
}
```
```php
$dom = new DOMDocument();
@$dom->loadHTML($html);
$xpath = new DOMXPath($dom);
$title = $xpath->query("//title")->item(0);
echo "Page title: " . ($title ? $title->textContent : "N/A") . "\n";
foreach ($xpath->query("//h1") as $h) {
echo "H1: " . trim($h->textContent) . "\n";
}
```
```go
import (
"fmt"
"strings"
"golang.org/x/net/html"
)
doc, _ := html.Parse(strings.NewReader(html))
var traverse func(*html.Node)
traverse = func(n *html.Node) {
if n.Type == html.ElementNode && n.Data == "title" && n.FirstChild != nil {
fmt.Println("Page title:", n.FirstChild.Data)
}
if n.Type == html.ElementNode && n.Data == "h1" && n.FirstChild != nil {
fmt.Println("H1:", n.FirstChild.Data)
}
for c := n.FirstChild; c != nil; c = c.NextSibling { traverse(c) }
}
traverse(doc)
```
```java
import org.jsoup.Jsoup;
var doc = Jsoup.parse(html);
System.out.println("Page title: " + doc.title());
doc.select("h1").forEach(h -> System.out.println("H1: " + h.text()));
```
```csharp
using HtmlAgilityPack;
var doc = new HtmlDocument();
doc.LoadHtml(html);
var title = doc.DocumentNode.SelectSingleNode("//title");
Console.WriteLine($"Page title: {title?.InnerText ?? "N/A"}");
foreach (var h in doc.DocumentNode.SelectNodes("//h1") ?? Enumerable.Empty())
Console.WriteLine($"H1: {h.InnerText.Trim()}");
```
```rust
// Minimal regex-based extraction
use regex::Regex; // add regex = "1" to Cargo.toml
let title_re = Regex::new(r"]*>(.*?)").unwrap();
if let Some(cap) = title_re.captures(&html) { println!("Page title: {}", &cap[1]); }
let h1_re = Regex::new(r"]*>(.*?)
").unwrap();
for cap in h1_re.captures_iter(&html) { println!("H1: {}", &cap[1]); }
```
Build a price monitoring script [#build-a-price-monitoring-script]
Crawl a product page and extract the price using a CSS selector.
```python
import requests
from bs4 import BeautifulSoup
API_KEY = "YOUR_API_KEY"
WATCH_URL = "https://www.example-shop.com/product/123"
def crawl(url: str) -> str:
r = requests.get("https://api.piloterr.com/v2/website/crawler",
headers={"x-api-key": API_KEY}, params={"query": url, "allow_redirects": "true"})
return r.json()
def extract_price(html: str) -> str | None:
soup = BeautifulSoup(html, "html.parser")
el = soup.select_one("[data-price], .price, #price")
return el.get_text(strip=True) if el else None
price = extract_price(crawl(WATCH_URL))
print(f"Current price: {price}" if price else "Price element not found.")
```
```typescript
import { parse } from "node-html-parser";
const API_KEY = "YOUR_API_KEY";
const WATCH_URL = "https://www.example-shop.com/product/123";
async function crawl(url: string): Promise {
const params = new URLSearchParams({ query: url, allow_redirects: "true" });
const res = await fetch(`https://api.piloterr.com/v2/website/crawler?${params}`, {
headers: { "x-api-key": API_KEY },
});
return res.json();
}
const html = await crawl(WATCH_URL);
const root = parse(html);
const price = root.querySelector("[data-price], .price, #price")?.text.trim() ?? null;
console.log(price ? `Current price: ${price}` : "Price element not found.");
```
```php
$url, "allow_redirects" => "true"]);
$ch = curl_init("https://api.piloterr.com/v2/website/crawler?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$html = json_decode(curl_exec($ch), true);
curl_close($ch);
return $html;
}
$html = crawl("YOUR_API_KEY", "https://www.example-shop.com/product/123");
$dom = new DOMDocument();
@$dom->loadHTML($html);
$xpath = new DOMXPath($dom);
$price = null;
foreach (["//span[@class='price']", "//*[@id='price']", "//*[@data-price]"] as $sel) {
$nodes = $xpath->query($sel);
if ($nodes->length > 0) { $price = trim($nodes->item(0)->textContent); break; }
}
echo $price ? "Current price: {$price}\n" : "Price element not found.\n";
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"regexp"
)
func crawl(apiKey, pageUrl string) string {
params := url.Values{"query": {pageUrl}, "allow_redirects": {"true"}}
req, _ := http.NewRequest("GET", "https://api.piloterr.com/v2/website/crawler?"+params.Encode(), nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var html string
json.Unmarshal(body, &html)
return html
}
func main() {
html := crawl("YOUR_API_KEY", "https://www.example-shop.com/product/123")
re := regexp.MustCompile(`class="price[^"]*"[^>]*>([^<]+)`)
match := re.FindStringSubmatch(html)
if match != nil {
fmt.Println("Current price:", match[1])
} else {
fmt.Println("Price element not found.")
}
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import org.jsoup.Jsoup;
public class Main {
public static void main(String[] args) throws Exception {
var apiKey = "YOUR_API_KEY";
var watchUrl = URLEncoder.encode("https://www.example-shop.com/product/123", StandardCharsets.UTF_8);
var url = "https://api.piloterr.com/v2/website/crawler?query=" + watchUrl + "&allow_redirects=true";
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder().uri(URI.create(url))
.header("x-api-key", apiKey).GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
// Strip JSON string quotes
var html = response.body().replaceAll("^\"|\"$", "").replace("\\\"", "\"").replace("\\n", "\n");
var doc = Jsoup.parse(html);
var priceEl = doc.selectFirst(".price, #price, [data-price]");
System.out.println(priceEl != null ? "Current price: " + priceEl.text() : "Price element not found.");
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
using HtmlAgilityPack;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var watchUrl = Uri.EscapeDataString("https://www.example-shop.com/product/123");
var raw = await client.GetStringAsync(
$"https://api.piloterr.com/v2/website/crawler?query={watchUrl}&allow_redirects=true");
var html = JsonSerializer.Deserialize(raw)!;
var doc = new HtmlDocument();
doc.LoadHtml(html);
var price = doc.DocumentNode.SelectSingleNode("//*[contains(@class,'price') or @id='price' or @data-price]");
Console.WriteLine(price != null ? $"Current price: {price.InnerText.Trim()}" : "Price element not found.");
```
```rust
use reqwest::Client;
use regex::Regex;
async fn crawl(client: &Client, api_key: &str, url: &str) -> String {
client.get("https://api.piloterr.com/v2/website/crawler")
.header("x-api-key", api_key)
.query(&[("query", url), ("allow_redirects", "true")])
.send().await.unwrap().json::().await.unwrap()
}
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let html = crawl(&client, "YOUR_API_KEY", "https://www.example-shop.com/product/123").await;
let re = Regex::new(r#"class="price[^"]*"[^>]*>([^<]+)"#).unwrap();
match re.captures(&html) {
Some(cap) => println!("Current price: {}", cap[1].trim()),
None => println!("Price element not found."),
}
Ok(())
}
```
Set `allow_redirects=true` to follow HTTP 301/302 redirects automatically — useful for short URLs or e-commerce platforms that redirect product pages.
# Detect Disposable Domains
Overview [#overview]
Disposable email providers let users create temporary inboxes that get deleted after minutes or days. This playbook shows how to query the Veille domain validation endpoint and use the result to block or flag signups at the point of registration.
Prerequisites [#prerequisites]
* A Veille API key — get one at [app.veille.io](https://app.veille.io)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Check a domain [#check-a-domain]
Call `GET /v1/domain` with the `domain` parameter. The response returns whether it is disposable, free, or a custom domain.
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.veille.io/v1/domain",
headers={"x-api-key": API_KEY},
params={"domain": "mailinator.com"},
)
result = response.json()
print(result)
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({ domain: "mailinator.com" });
const response = await fetch(`https://api.veille.io/v1/domain?${params}`, {
headers: { "x-api-key": API_KEY },
});
const result = await response.json();
console.log(result);
```
```php
"mailinator.com"]);
$ch = curl_init("https://api.veille.io/v1/domain?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$result = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($result);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
)
func main() {
req, _ := http.NewRequest("GET", "https://api.veille.io/v1/domain?domain=mailinator.com", nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var result map[string]any
json.Unmarshal(body, &result)
fmt.Println(result)
}
```
```java
import java.net.URI;
import java.net.http.*;
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.veille.io/v1/domain?domain=mailinator.com"))
.header("x-api-key", "YOUR_API_KEY")
.GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var body = await client.GetStringAsync("https://api.veille.io/v1/domain?domain=mailinator.com");
Console.WriteLine(body);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let result = reqwest::Client::new()
.get("https://api.veille.io/v1/domain")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("domain", "mailinator.com")])
.send().await?.json::().await?;
println!("{:#?}", result);
Ok(())
}
```
Read the response fields [#read-the-response-fields]
The response includes `is_disposable`, `is_free`, `is_custom`, `domain`, and `provider` (when identified).
```python
domain = result.get("domain")
is_disposable = result.get("is_disposable")
is_free = result.get("is_free")
provider = result.get("provider", "unknown")
if is_disposable:
print(f"❌ {domain} is a DISPOSABLE email provider ({provider}). Block this signup.")
elif is_free:
print(f"⚠️ {domain} is a free email provider. Consider extra verification.")
else:
print(f"✅ {domain} looks like a custom / business domain. Allow.")
```
```typescript
const { domain, is_disposable, is_free, provider } = result;
if (is_disposable) {
console.log(`❌ ${domain} is DISPOSABLE (${provider ?? "unknown"}). Block this signup.`);
} else if (is_free) {
console.log(`⚠️ ${domain} is a free provider. Consider extra verification.`);
} else {
console.log(`✅ ${domain} looks like a business domain. Allow.`);
}
```
```php
$domain = $result["domain"] ?? "";
$isDisposable = $result["is_disposable"] ?? false;
$isFree = $result["is_free"] ?? false;
$provider = $result["provider"] ?? "unknown";
if ($isDisposable) {
echo "❌ {$domain} is DISPOSABLE ({$provider}). Block this signup.\n";
} elseif ($isFree) {
echo "⚠️ {$domain} is free. Consider extra verification.\n";
} else {
echo "✅ {$domain} looks like a business domain. Allow.\n";
}
```
```go
domain := result["domain"].(string)
isDisposable := result["is_disposable"].(bool)
isFree := result["is_free"].(bool)
provider, _ := result["provider"].(string)
switch {
case isDisposable:
fmt.Printf("❌ %s is DISPOSABLE (%s). Block this signup.\n", domain, provider)
case isFree:
fmt.Printf("⚠️ %s is a free provider.\n", domain)
default:
fmt.Printf("✅ %s is a business domain. Allow.\n", domain)
}
```
```java
import org.json.*;
var r = new JSONObject(response.body());
var domain = r.getString("domain");
var isDisposable = r.getBoolean("is_disposable");
var isFree = r.getBoolean("is_free");
var provider = r.optString("provider", "unknown");
if (isDisposable) System.out.printf("❌ %s is DISPOSABLE (%s). Block.%n", domain, provider);
else if (isFree) System.out.printf("⚠️ %s is free. Extra verification.%n", domain);
else System.out.printf("✅ %s is a business domain. Allow.%n", domain);
```
```csharp
using System.Text.Json;
var r = JsonDocument.Parse(body).RootElement;
var domain = r.GetProperty("domain").GetString();
var isDisposable = r.GetProperty("is_disposable").GetBoolean();
var isFree = r.GetProperty("is_free").GetBoolean();
var provider = r.TryGetProperty("provider", out var p) ? p.GetString() : "unknown";
if (isDisposable)
Console.WriteLine($"❌ {domain} is DISPOSABLE ({provider}). Block.");
else if (isFree)
Console.WriteLine($"⚠️ {domain} is free. Extra verification.");
else
Console.WriteLine($"✅ {domain} is a business domain. Allow.");
```
```rust
let domain = result["domain"].as_str().unwrap_or("");
let is_disposable = result["is_disposable"].as_bool().unwrap_or(false);
let is_free = result["is_free"].as_bool().unwrap_or(false);
let provider = result["provider"].as_str().unwrap_or("unknown");
if is_disposable {
println!("❌ {} is DISPOSABLE ({}). Block.", domain, provider);
} else if is_free {
println!("⚠️ {} is free. Extra verification.", domain);
} else {
println!("✅ {} is a business domain. Allow.", domain);
}
```
Integrate into a signup handler [#integrate-into-a-signup-handler]
Add the domain check to your registration endpoint and return a validation error before the user record is created.
```python
import requests
API_KEY = "YOUR_API_KEY"
def is_disposable_email(email: str) -> bool:
domain = email.split("@")[-1].lower()
r = requests.get("https://api.veille.io/v1/domain",
headers={"x-api-key": API_KEY}, params={"domain": domain})
return r.json().get("is_disposable", False)
def register_user(email: str, password: str) -> dict:
if is_disposable_email(email):
return {"success": False, "error": "Disposable email addresses are not allowed."}
# ... create the user record in your database
return {"success": True, "message": f"Welcome, {email}!"}
print(register_user("alice@mailinator.com", "secret"))
print(register_user("alice@company.com", "secret"))
```
```typescript
const API_KEY = "YOUR_API_KEY";
async function isDisposableEmail(email: string): Promise {
const domain = email.split("@").at(-1)!.toLowerCase();
const params = new URLSearchParams({ domain });
const res = await fetch(`https://api.veille.io/v1/domain?${params}`, {
headers: { "x-api-key": API_KEY },
});
return (await res.json()).is_disposable ?? false;
}
async function registerUser(email: string, password: string) {
if (await isDisposableEmail(email)) {
return { success: false, error: "Disposable email addresses are not allowed." };
}
// ... create user record
return { success: true, message: `Welcome, ${email}!` };
}
console.log(await registerUser("alice@mailinator.com", "secret"));
console.log(await registerUser("alice@company.com", "secret"));
```
```php
$domain]);
$ch = curl_init("https://api.veille.io/v1/domain?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$result = json_decode(curl_exec($ch), true);
curl_close($ch);
return $result["is_disposable"] ?? false;
}
function registerUser(string $apiKey, string $email, string $password): array {
if (isDisposableEmail($apiKey, $email)) {
return ["success" => false, "error" => "Disposable email addresses are not allowed."];
}
// ... create user record
return ["success" => true, "message" => "Welcome, {$email}!"];
}
print_r(registerUser("YOUR_API_KEY", "alice@mailinator.com", "secret"));
print_r(registerUser("YOUR_API_KEY", "alice@company.com", "secret"));
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
)
func isDisposable(apiKey, email string) bool {
domain := strings.ToLower(strings.SplitN(email, "@", 2)[1])
req, _ := http.NewRequest("GET", "https://api.veille.io/v1/domain?domain="+domain, nil)
req.Header.Set("x-api-key", apiKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var result map[string]any
json.Unmarshal(body, &result)
v, _ := result["is_disposable"].(bool)
return v
}
func registerUser(apiKey, email, password string) string {
if isDisposable(apiKey, email) {
return "❌ Disposable email not allowed."
}
return "✅ Welcome, " + email + "!"
}
func main() {
apiKey := "YOUR_API_KEY"
fmt.Println(registerUser(apiKey, "alice@mailinator.com", "secret"))
fmt.Println(registerUser(apiKey, "alice@company.com", "secret"))
}
```
```java
import java.net.URI;
import java.net.http.*;
import org.json.*;
public class Main {
static HttpClient client = HttpClient.newHttpClient();
static String API_KEY = "YOUR_API_KEY";
static boolean isDisposable(String email) throws Exception {
var domain = email.substring(email.indexOf('@') + 1).toLowerCase();
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.veille.io/v1/domain?domain=" + domain))
.header("x-api-key", API_KEY).GET().build();
var resp = client.send(request, HttpResponse.BodyHandlers.ofString());
return new JSONObject(resp.body()).optBoolean("is_disposable", false);
}
static String registerUser(String email, String password) throws Exception {
if (isDisposable(email)) return "❌ Disposable email not allowed.";
return "✅ Welcome, " + email + "!";
}
public static void main(String[] args) throws Exception {
System.out.println(registerUser("alice@mailinator.com", "secret"));
System.out.println(registerUser("alice@company.com", "secret"));
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
var apiKey = "YOUR_API_KEY";
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
async Task IsDisposable(string email)
{
var domain = email.Split('@').Last().ToLower();
var body = await client.GetStringAsync($"https://api.veille.io/v1/domain?domain={domain}");
return JsonDocument.Parse(body).RootElement.GetProperty("is_disposable").GetBoolean();
}
async Task RegisterUser(string email, string password) =>
await IsDisposable(email)
? "❌ Disposable email not allowed."
: $"✅ Welcome, {email}!";
Console.WriteLine(await RegisterUser("alice@mailinator.com", "secret"));
Console.WriteLine(await RegisterUser("alice@company.com", "secret"));
```
```rust
use reqwest::Client;
use serde_json::Value;
async fn is_disposable(client: &Client, api_key: &str, email: &str) -> bool {
let domain = email.split('@').last().unwrap_or("").to_lowercase();
let result = client.get("https://api.veille.io/v1/domain")
.header("x-api-key", api_key)
.query(&[("domain", domain.as_str())])
.send().await.unwrap().json::().await.unwrap();
result["is_disposable"].as_bool().unwrap_or(false)
}
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
let api_key = "YOUR_API_KEY";
for email in ["alice@mailinator.com", "alice@company.com"] {
if is_disposable(&client, api_key, email).await {
println!("❌ {} — disposable. Block.", email);
} else {
println!("✅ {} — looks valid. Allow.", email);
}
}
Ok(())
}
```
Cache results for frequently checked domains (e.g. in Redis with a 24-hour TTL) to avoid redundant API calls. The list of disposable providers rarely changes within a single day.
# Validate Email on Signup
Overview [#overview]
This playbook shows how to run a full email validation before accepting a signup: syntax check, DNS/MX record lookup, and SMTP reachability. The API returns a `risk_score` (0 = clean, 100 = high risk) that you can use to route users to extra verification steps.
Prerequisites [#prerequisites]
* A Veille API key — get one at [app.veille.io](https://app.veille.io)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Validate an email address [#validate-an-email-address]
Call `GET /v1/email` with the `email` parameter.
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.veille.io/v1/email",
headers={"x-api-key": API_KEY},
params={"email": "alice@example.com"},
)
result = response.json()
print(result)
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({ email: "alice@example.com" });
const response = await fetch(`https://api.veille.io/v1/email?${params}`, {
headers: { "x-api-key": API_KEY },
});
const result = await response.json();
console.log(result);
```
```php
"alice@example.com"]);
$ch = curl_init("https://api.veille.io/v1/email?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$result = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($result);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
func main() {
params := url.Values{"email": {"alice@example.com"}}
req, _ := http.NewRequest("GET", "https://api.veille.io/v1/email?"+params.Encode(), nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var result map[string]any
json.Unmarshal(body, &result)
fmt.Println(result)
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
var client = HttpClient.newHttpClient();
var email = URLEncoder.encode("alice@example.com", StandardCharsets.UTF_8);
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.veille.io/v1/email?email=" + email))
.header("x-api-key", "YOUR_API_KEY")
.GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var email = Uri.EscapeDataString("alice@example.com");
var body = await client.GetStringAsync($"https://api.veille.io/v1/email?email={email}");
Console.WriteLine(body);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let result = reqwest::Client::new()
.get("https://api.veille.io/v1/email")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("email", "alice@example.com")])
.send().await?.json::().await?;
println!("{:#?}", result);
Ok(())
}
```
Interpret the result [#interpret-the-result]
Key fields: `is_valid`, `is_deliverable`, `is_disposable`, `is_role_account`, `risk_score`, `did_you_mean` (typo suggestion).
```python
print(f"Valid : {result['is_valid']}")
print(f"Deliverable : {result['is_deliverable']}")
print(f"Disposable : {result['is_disposable']}")
print(f"Role account: {result['is_role_account']}")
print(f"Risk score : {result['risk_score']}/100")
if result.get("did_you_mean"):
print(f"Did you mean: {result['did_you_mean']}?")
```
```typescript
console.log(`Valid : ${result.is_valid}`);
console.log(`Deliverable : ${result.is_deliverable}`);
console.log(`Disposable : ${result.is_disposable}`);
console.log(`Role account: ${result.is_role_account}`);
console.log(`Risk score : ${result.risk_score}/100`);
if (result.did_you_mean) console.log(`Did you mean: ${result.did_you_mean}?`);
```
```php
echo "Valid : " . ($result["is_valid"] ? "true" : "false") . "\n";
echo "Deliverable : " . ($result["is_deliverable"] ? "true" : "false") . "\n";
echo "Disposable : " . ($result["is_disposable"] ? "true" : "false") . "\n";
echo "Risk score : {$result['risk_score']}/100\n";
if (!empty($result["did_you_mean"])) echo "Did you mean: {$result['did_you_mean']}?\n";
```
```go
fmt.Printf("Valid : %v\nDeliverable : %v\nDisposable : %v\nRisk score : %v/100\n",
result["is_valid"], result["is_deliverable"], result["is_disposable"], result["risk_score"])
if typo, ok := result["did_you_mean"].(string); ok && typo != "" {
fmt.Println("Did you mean:", typo+"?")
}
```
```java
import org.json.*;
var r = new JSONObject(response.body());
System.out.printf("Valid : %b%nDeliverable : %b%nDisposable : %b%nRisk score : %d/100%n",
r.getBoolean("is_valid"), r.getBoolean("is_deliverable"),
r.getBoolean("is_disposable"), r.getInt("risk_score"));
if (r.has("did_you_mean")) System.out.println("Did you mean: " + r.getString("did_you_mean") + "?");
```
```csharp
using System.Text.Json;
var r = JsonDocument.Parse(body).RootElement;
Console.WriteLine($"Valid : {r.GetProperty("is_valid").GetBoolean()}");
Console.WriteLine($"Deliverable : {r.GetProperty("is_deliverable").GetBoolean()}");
Console.WriteLine($"Disposable : {r.GetProperty("is_disposable").GetBoolean()}");
Console.WriteLine($"Risk score : {r.GetProperty("risk_score").GetInt32()}/100");
if (r.TryGetProperty("did_you_mean", out var typo) && typo.GetString() is { } t and not "")
Console.WriteLine($"Did you mean: {t}?");
```
```rust
println!("Valid : {}", result["is_valid"]);
println!("Deliverable : {}", result["is_deliverable"]);
println!("Disposable : {}", result["is_disposable"]);
println!("Risk score : {}/100", result["risk_score"]);
if let Some(typo) = result["did_you_mean"].as_str() {
if !typo.is_empty() { println!("Did you mean: {}?", typo); }
}
```
Build a risk-tiered registration flow [#build-a-risk-tiered-registration-flow]
Use `risk_score` to route users: block high-risk addresses, prompt typo corrections, and apply extra friction to role accounts.
```python
import requests
API_KEY = "YOUR_API_KEY"
def validate_email(email: str) -> dict:
r = requests.get("https://api.veille.io/v1/email",
headers={"x-api-key": API_KEY}, params={"email": email})
return r.json()
def register(email: str) -> dict:
v = validate_email(email)
if not v.get("is_valid"):
return {"ok": False, "error": "This email address is invalid."}
if v.get("is_disposable"):
return {"ok": False, "error": "Temporary email addresses are not accepted."}
if v.get("did_you_mean"):
return {"ok": False, "suggestion": f"Did you mean {v['did_you_mean']}?"}
if v.get("risk_score", 0) >= 70:
return {"ok": False, "error": "This email has a high risk score. Please use a different address."}
if v.get("is_role_account"):
# Allow but flag for manual review
return {"ok": True, "flag": "role_account", "message": "Please verify your email."}
return {"ok": True, "message": "Registration successful!"}
for test_email in ["alice@mailinator.com", "info@company.com", "alice@gmial.com", "alice@stripe.com"]:
print(f"{test_email}: {register(test_email)}")
```
```typescript
const API_KEY = "YOUR_API_KEY";
async function validateEmail(email: string) {
const params = new URLSearchParams({ email });
const res = await fetch(`https://api.veille.io/v1/email?${params}`, {
headers: { "x-api-key": API_KEY },
});
return res.json();
}
async function register(email: string) {
const v = await validateEmail(email);
if (!v.is_valid) return { ok: false, error: "Invalid email address." };
if (v.is_disposable) return { ok: false, error: "Temporary emails not accepted." };
if (v.did_you_mean) return { ok: false, suggestion: `Did you mean ${v.did_you_mean}?` };
if (v.risk_score >= 70) return { ok: false, error: "High risk score — please use another address." };
if (v.is_role_account) return { ok: true, flag: "role_account", message: "Please verify your email." };
return { ok: true, message: "Registration successful!" };
}
for (const email of ["alice@mailinator.com", "info@company.com", "alice@gmial.com", "alice@stripe.com"]) {
console.log(email, await register(email));
}
```
```php
$email]);
$ch = curl_init("https://api.veille.io/v1/email?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$result = json_decode(curl_exec($ch), true);
curl_close($ch);
return $result;
}
function register(string $apiKey, string $email): array {
$v = validateEmail($apiKey, $email);
if (!($v["is_valid"] ?? false)) return ["ok" => false, "error" => "Invalid email."];
if ($v["is_disposable"] ?? false) return ["ok" => false, "error" => "Temporary emails not accepted."];
if (!empty($v["did_you_mean"])) return ["ok" => false, "suggestion" => "Did you mean {$v['did_you_mean']}?"];
if (($v["risk_score"] ?? 0) >= 70) return ["ok" => false, "error" => "High risk score."];
if ($v["is_role_account"] ?? false) return ["ok" => true, "flag" => "role_account"];
return ["ok" => true, "message" => "Registration successful!"];
}
foreach (["alice@mailinator.com", "info@company.com", "alice@gmial.com"] as $email) {
print_r(["email" => $email, "result" => register("YOUR_API_KEY", $email)]);
}
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
)
const APIKey = "YOUR_API_KEY"
func validate(email string) map[string]any {
params := url.Values{"email": {email}}
req, _ := http.NewRequest("GET", "https://api.veille.io/v1/email?"+params.Encode(), nil)
req.Header.Set("x-api-key", APIKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var v map[string]any
json.Unmarshal(body, &v)
return v
}
func register(email string) string {
v := validate(email)
if !(v["is_valid"].(bool)) { return "❌ Invalid email." }
if v["is_disposable"].(bool) { return "❌ Disposable not accepted." }
if typo, ok := v["did_you_mean"].(string); ok && typo != "" {
return "⚠️ Did you mean " + typo + "?"
}
if v["risk_score"].(float64) >= 70 { return "❌ High risk score." }
return "✅ Registration successful!"
}
func main() {
for _, email := range []string{"alice@mailinator.com", "info@company.com", "alice@stripe.com"} {
fmt.Printf("%s: %s\n", email, register(email))
}
}
```
```java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import org.json.*;
public class Main {
static HttpClient client = HttpClient.newHttpClient();
static String API_KEY = "YOUR_API_KEY";
static JSONObject validate(String email) throws Exception {
var encoded = URLEncoder.encode(email, StandardCharsets.UTF_8);
var req = HttpRequest.newBuilder()
.uri(URI.create("https://api.veille.io/v1/email?email=" + encoded))
.header("x-api-key", API_KEY).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
return new JSONObject(resp.body());
}
static String register(String email) throws Exception {
var v = validate(email);
if (!v.getBoolean("is_valid")) return "❌ Invalid email.";
if (v.getBoolean("is_disposable")) return "❌ Disposable not accepted.";
if (v.has("did_you_mean")) return "⚠️ Did you mean " + v.getString("did_you_mean") + "?";
if (v.getInt("risk_score") >= 70) return "❌ High risk score.";
return "✅ Registration successful!";
}
public static void main(String[] args) throws Exception {
for (var email : new String[]{"alice@mailinator.com", "info@company.com", "alice@stripe.com"}) {
System.out.printf("%s: %s%n", email, register(email));
}
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
var apiKey = "YOUR_API_KEY";
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
async Task Validate(string email)
{
var encoded = Uri.EscapeDataString(email);
var body = await client.GetStringAsync($"https://api.veille.io/v1/email?email={encoded}");
return JsonDocument.Parse(body).RootElement;
}
async Task Register(string email)
{
var v = await Validate(email);
if (!v.GetProperty("is_valid").GetBoolean()) return "❌ Invalid email.";
if (v.GetProperty("is_disposable").GetBoolean()) return "❌ Disposable not accepted.";
if (v.TryGetProperty("did_you_mean", out var t) && t.GetString() is { Length: > 0 } typo)
return $"⚠️ Did you mean {typo}?";
if (v.GetProperty("risk_score").GetInt32() >= 70) return "❌ High risk score.";
return "✅ Registration successful!";
}
foreach (var email in new[] { "alice@mailinator.com", "info@company.com", "alice@stripe.com" })
Console.WriteLine($"{email}: {await Register(email)}");
```
```rust
use reqwest::Client;
use serde_json::Value;
const API_KEY: &str = "YOUR_API_KEY";
async fn validate(client: &Client, email: &str) -> Value {
client.get("https://api.veille.io/v1/email")
.header("x-api-key", API_KEY)
.query(&[("email", email)])
.send().await.unwrap().json::().await.unwrap()
}
async fn register(client: &Client, email: &str) -> &'static str {
let v = validate(client, email).await;
if !v["is_valid"].as_bool().unwrap_or(false) { return "❌ Invalid email."; }
if v["is_disposable"].as_bool().unwrap_or(false) { return "❌ Disposable not accepted."; }
if v["risk_score"].as_i64().unwrap_or(0) >= 70 { return "❌ High risk score."; }
"✅ Registration successful!"
}
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
for email in ["alice@mailinator.com", "info@company.com", "alice@stripe.com"] {
println!("{}: {}", email, register(&client, email).await);
}
Ok(())
}
```
Role accounts (`admin@`, `info@`, `support@`) are often monitored by multiple people — or not monitored at all. Flag them for manual review rather than blocking outright to avoid losing legitimate B2B signups.
# Validate EU VAT Numbers
Overview [#overview]
EU businesses are exempt from VAT when purchasing from other EU businesses — but only with a valid VAT number. This playbook shows how to validate a VAT number at checkout and retrieve the associated company details (name and address) from the VIES database.
Prerequisites [#prerequisites]
* A Veille API key — get one at [app.veille.io](https://app.veille.io)
* Install dependencies for your language:
```bash
pip install requests
```
No extra dependencies — uses the native `fetch` API (Node 18+).
`curl` extension enabled (on by default).
No extra dependencies — uses `net/http` (Go 1.18+).
No extra dependencies — uses `java.net.http` (Java 11+).
No extra dependencies — uses `System.Net.Http` (.NET 6+).
```toml
# Cargo.toml
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```
Steps [#steps]
Validate a VAT number [#validate-a-vat-number]
Call `GET /v1/vat` with the `vat` parameter. The number should include the country prefix (e.g. `FR12345678901`).
```python
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.veille.io/v1/vat",
headers={"x-api-key": API_KEY},
params={"vat": "FR12345678901"},
)
result = response.json()
print(result)
```
```typescript
const API_KEY = "YOUR_API_KEY";
const params = new URLSearchParams({ vat: "FR12345678901" });
const response = await fetch(`https://api.veille.io/v1/vat?${params}`, {
headers: { "x-api-key": API_KEY },
});
const result = await response.json();
console.log(result);
```
```php
"FR12345678901"]);
$ch = curl_init("https://api.veille.io/v1/vat?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$result = json_decode(curl_exec($ch), true);
curl_close($ch);
print_r($result);
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
)
func main() {
req, _ := http.NewRequest("GET", "https://api.veille.io/v1/vat?vat=FR12345678901", nil)
req.Header.Set("x-api-key", "YOUR_API_KEY")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var result map[string]any
json.Unmarshal(body, &result)
fmt.Println(result)
}
```
```java
import java.net.URI;
import java.net.http.*;
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder()
.uri(URI.create("https://api.veille.io/v1/vat?vat=FR12345678901"))
.header("x-api-key", "YOUR_API_KEY")
.GET().build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
```
```csharp
using System.Net.Http;
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", "YOUR_API_KEY");
var body = await client.GetStringAsync("https://api.veille.io/v1/vat?vat=FR12345678901");
Console.WriteLine(body);
```
```rust
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let result = reqwest::Client::new()
.get("https://api.veille.io/v1/vat")
.header("x-api-key", "YOUR_API_KEY")
.query(&[("vat", "FR12345678901")])
.send().await?.json::().await?;
println!("{:#?}", result);
Ok(())
}
```
Read the company details [#read-the-company-details]
A valid number returns `is_valid: true`, the `company_name`, `company_address`, and `country_code`.
```python
if result.get("is_valid"):
print(f"✅ Valid VAT number")
print(f" Company : {result.get('company_name')}")
print(f" Address : {result.get('company_address')}")
print(f" Country : {result.get('country_code')}")
else:
print(f"❌ Invalid VAT number: {result.get('error', 'Not found in VIES')}")
```
```typescript
if (result.is_valid) {
console.log("✅ Valid VAT number");
console.log(` Company : ${result.company_name}`);
console.log(` Address : ${result.company_address}`);
console.log(` Country : ${result.country_code}`);
} else {
console.log(`❌ Invalid VAT number: ${result.error ?? "Not found in VIES"}`);
}
```
```php
if ($result["is_valid"] ?? false) {
echo "✅ Valid VAT number\n";
echo " Company : {$result['company_name']}\n";
echo " Address : {$result['company_address']}\n";
echo " Country : {$result['country_code']}\n";
} else {
echo "❌ Invalid VAT number: " . ($result["error"] ?? "Not found in VIES") . "\n";
}
```
```go
if result["is_valid"].(bool) {
fmt.Printf("✅ Valid VAT number\n Company : %v\n Address : %v\n Country : %v\n",
result["company_name"], result["company_address"], result["country_code"])
} else {
errMsg := "Not found in VIES"
if e, ok := result["error"].(string); ok { errMsg = e }
fmt.Println("❌ Invalid VAT number:", errMsg)
}
```
```java
import org.json.*;
var r = new JSONObject(response.body());
if (r.getBoolean("is_valid")) {
System.out.printf("✅ Valid VAT number%n Company : %s%n Address : %s%n Country : %s%n",
r.getString("company_name"), r.getString("company_address"), r.getString("country_code"));
} else {
System.out.println("❌ Invalid VAT number: " + r.optString("error", "Not found in VIES"));
}
```
```csharp
using System.Text.Json;
var r = JsonDocument.Parse(body).RootElement;
if (r.GetProperty("is_valid").GetBoolean())
{
Console.WriteLine("✅ Valid VAT number");
Console.WriteLine($" Company : {r.GetProperty("company_name")}");
Console.WriteLine($" Address : {r.GetProperty("company_address")}");
Console.WriteLine($" Country : {r.GetProperty("country_code")}");
}
else
{
var error = r.TryGetProperty("error", out var e) ? e.GetString() : "Not found in VIES";
Console.WriteLine($"❌ Invalid VAT number: {error}");
}
```
```rust
if result["is_valid"].as_bool().unwrap_or(false) {
println!("✅ Valid VAT number");
println!(" Company : {}", result["company_name"].as_str().unwrap_or(""));
println!(" Address : {}", result["company_address"].as_str().unwrap_or(""));
println!(" Country : {}", result["country_code"].as_str().unwrap_or(""));
} else {
let err = result["error"].as_str().unwrap_or("Not found in VIES");
println!("❌ Invalid VAT number: {}", err);
}
```
Build a B2B checkout VAT handler [#build-a-b2b-checkout-vat-handler]
Strip VAT from the order total when a verified EU business VAT number is provided.
```python
import requests
API_KEY = "YOUR_API_KEY"
VAT_RATE = 0.20 # 20% VAT
def lookup_vat(number: str) -> dict:
r = requests.get("https://api.veille.io/v1/vat",
headers={"x-api-key": API_KEY}, params={"vat": number})
return r.json()
def calculate_checkout(subtotal: float, vat_number: str | None = None) -> dict:
vat_amount = subtotal * VAT_RATE
vat_info = None
if vat_number:
result = lookup_vat(vat_number)
if result.get("is_valid"):
vat_amount = 0 # B2B reverse charge — no VAT charged
vat_info = {"company": result["company_name"], "country": result["country_code"]}
else:
return {"error": "The VAT number you provided is not valid."}
total = subtotal + vat_amount
return {"subtotal": subtotal, "vat": vat_amount, "total": total, "company": vat_info}
print(calculate_checkout(100.00))
print(calculate_checkout(100.00, vat_number="FR12345678901"))
print(calculate_checkout(100.00, vat_number="INVALID123"))
```
```typescript
const API_KEY = "YOUR_API_KEY";
const VAT_RATE = 0.20;
async function lookupVat(number: string) {
const params = new URLSearchParams({ vat: number });
const res = await fetch(`https://api.veille.io/v1/vat?${params}`, {
headers: { "x-api-key": API_KEY },
});
return res.json();
}
async function calculateCheckout(subtotal: number, vatNumber?: string) {
let vatAmount = subtotal * VAT_RATE;
let companyInfo: any = null;
if (vatNumber) {
const result = await lookupVat(vatNumber);
if (result.is_valid) {
vatAmount = 0;
companyInfo = { company: result.company_name, country: result.country_code };
} else {
return { error: "The VAT number you provided is not valid." };
}
}
return { subtotal, vat: vatAmount, total: subtotal + vatAmount, company: companyInfo };
}
console.log(await calculateCheckout(100));
console.log(await calculateCheckout(100, "FR12345678901"));
console.log(await calculateCheckout(100, "INVALID123"));
```
```php
$number]);
$ch = curl_init("https://api.veille.io/v1/vat?{$params}");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["x-api-key: {$apiKey}"]);
$result = json_decode(curl_exec($ch), true);
curl_close($ch);
return $result;
}
function calculateCheckout(string $apiKey, float $subtotal, ?string $vatNumber = null): array {
$vatAmount = $subtotal * VAT_RATE;
$company = null;
if ($vatNumber) {
$result = lookupVat($apiKey, $vatNumber);
if ($result["is_valid"] ?? false) {
$vatAmount = 0;
$company = ["company" => $result["company_name"], "country" => $result["country_code"]];
} else {
return ["error" => "Invalid VAT number."];
}
}
return ["subtotal" => $subtotal, "vat" => $vatAmount, "total" => $subtotal + $vatAmount, "company" => $company];
}
print_r(calculateCheckout("YOUR_API_KEY", 100.0));
print_r(calculateCheckout("YOUR_API_KEY", 100.0, "FR12345678901"));
```
```go
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
)
const APIKey = "YOUR_API_KEY"
const VATRate = 0.20
func lookupVAT(number string) map[string]any {
req, _ := http.NewRequest("GET", "https://api.veille.io/v1/vat?vat="+number, nil)
req.Header.Set("x-api-key", APIKey)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var r map[string]any
json.Unmarshal(body, &r)
return r
}
func calculateCheckout(subtotal float64, vatNumber string) map[string]any {
vatAmount := subtotal * VATRate
var company map[string]any
if vatNumber != "" {
result := lookupVAT(vatNumber)
if result["is_valid"].(bool) {
vatAmount = 0
company = map[string]any{"company": result["company_name"], "country": result["country_code"]}
} else {
return map[string]any{"error": "Invalid VAT number."}
}
}
return map[string]any{"subtotal": subtotal, "vat": vatAmount, "total": subtotal + vatAmount, "company": company}
}
func main() {
fmt.Println(calculateCheckout(100, ""))
fmt.Println(calculateCheckout(100, "FR12345678901"))
fmt.Println(calculateCheckout(100, "INVALID123"))
}
```
```java
import java.net.URI;
import java.net.http.*;
import org.json.*;
public class Main {
static HttpClient client = HttpClient.newHttpClient();
static String API_KEY = "YOUR_API_KEY";
static double VAT_RATE = 0.20;
static JSONObject lookupVat(String number) throws Exception {
var req = HttpRequest.newBuilder()
.uri(URI.create("https://api.veille.io/v1/vat?vat=" + number))
.header("x-api-key", API_KEY).GET().build();
var resp = client.send(req, HttpResponse.BodyHandlers.ofString());
return new JSONObject(resp.body());
}
static JSONObject calculateCheckout(double subtotal, String vatNumber) throws Exception {
double vatAmount = subtotal * VAT_RATE;
JSONObject company = null;
if (vatNumber != null && !vatNumber.isEmpty()) {
var result = lookupVat(vatNumber);
if (result.getBoolean("is_valid")) {
vatAmount = 0;
company = new JSONObject().put("company", result.getString("company_name"))
.put("country", result.getString("country_code"));
} else {
return new JSONObject().put("error", "Invalid VAT number.");
}
}
return new JSONObject().put("subtotal", subtotal).put("vat", vatAmount)
.put("total", subtotal + vatAmount).put("company", company);
}
public static void main(String[] args) throws Exception {
System.out.println(calculateCheckout(100, null));
System.out.println(calculateCheckout(100, "FR12345678901"));
System.out.println(calculateCheckout(100, "INVALID123"));
}
}
```
```csharp
using System.Net.Http;
using System.Text.Json;
const double VAT_RATE = 0.20;
var apiKey = "YOUR_API_KEY";
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", apiKey);
async Task LookupVat(string number)
{
var body = await client.GetStringAsync($"https://api.veille.io/v1/vat?vat={number}");
return JsonDocument.Parse(body).RootElement;
}
async Task CalculateCheckout(double subtotal, string? vatNumber = null)
{
double vatAmount = subtotal * VAT_RATE;
string? company = null;
if (vatNumber != null)
{
var result = await LookupVat(vatNumber);
if (result.GetProperty("is_valid").GetBoolean())
{ vatAmount = 0; company = result.GetProperty("company_name").GetString(); }
else return """{"error":"Invalid VAT number."}""";
}
return $"""{{ "subtotal": {subtotal}, "vat": {vatAmount}, "total": {subtotal + vatAmount}, "company": "{company}" }}""";
}
Console.WriteLine(await CalculateCheckout(100));
Console.WriteLine(await CalculateCheckout(100, "FR12345678901"));
Console.WriteLine(await CalculateCheckout(100, "INVALID123"));
```
```rust
use reqwest::Client;
use serde_json::{json, Value};
const VAT_RATE: f64 = 0.20;
const API_KEY: &str = "YOUR_API_KEY";
async fn lookup_vat(client: &Client, number: &str) -> Value {
client.get("https://api.veille.io/v1/vat")
.header("x-api-key", API_KEY)
.query(&[("vat", number)])
.send().await.unwrap().json::().await.unwrap()
}
async fn calculate_checkout(client: &Client, subtotal: f64, vat_number: Option<&str>) -> Value {
let mut vat_amount = subtotal * VAT_RATE;
let mut company = json!(null);
if let Some(number) = vat_number {
let result = lookup_vat(client, number).await;
if result["is_valid"].as_bool().unwrap_or(false) {
vat_amount = 0.0;
company = json!({"company": result["company_name"], "country": result["country_code"]});
} else {
return json!({ "error": "Invalid VAT number." });
}
}
json!({ "subtotal": subtotal, "vat": vat_amount, "total": subtotal + vat_amount, "company": company })
}
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = Client::new();
println!("{}", calculate_checkout(&client, 100.0, None).await);
println!("{}", calculate_checkout(&client, 100.0, Some("FR12345678901")).await);
println!("{}", calculate_checkout(&client, 100.0, Some("INVALID123")).await);
Ok(())
}
```
Always store the validated company name and address alongside the order record. EU tax authorities require proof that the reverse-charge mechanism was correctly applied.