Frequently Asked Questions

To get an API key, you need to create an account and log in to the web interface. Once logged in, visit your Dashboard where you can create a new API key. Simply click the "Create API Key" button, give it a name (optional), and your key will be displayed once. Make sure to save it securely as you won't be able to view it again.

If you don't have an account yet, please contact an administrator to create one for you.

You can provide your API key using either of these methods:

• X-API-Key header: X-API-Key: your-api-key-here
• Authorization Bearer header: Authorization: Bearer your-api-key-here

Example using curl:

curl -H "X-API-Key: your-api-key-here" \
  https://api.example.com/api/v1/audible/books/B002V1OF70

For security and ease of development, requests from private IP addresses do not require API keys. This includes:

• localhost (127.0.0.1)
• Private network ranges: 192.168.x.x, 10.x.x.x, and 172.16-31.x.x

This allows you to test the API locally without setting up authentication, but remember that production requests from public IPs will always require a valid API key.

API keys expire after 2 weeks for security reasons. To create a new one:

1. Log in to your account
2. Go to your Dashboard
3. Click "Create API Key"
4. Give it a name (optional) and save the key securely

You can have multiple active API keys at once, so you can create a new one before your current key expires to avoid any service interruption.

API keys are valid for 2 weeks (14 days) from the date of creation. After expiration, you'll need to create a new API key to continue using the API.

For security reasons, API keys are only displayed once when created. If you lose your key, you have two options:

1. Create a new key: Log in to your Dashboard and create a new API key. You can revoke the old one if needed.
2. Revoke and replace: If you suspect your key was compromised, revoke it immediately from your Dashboard and create a new one.

Yes! You can revoke any of your API keys at any time from your Dashboard. Simply click the "Revoke" button next to the key you want to disable. Once revoked, that key will no longer work for authentication, and you'll need to create a new one.

Unauthorized errors (401) can occur for several reasons:

• Missing API key: You're not including the API key in your request headers
• Invalid API key: The key is incorrect or has been revoked
• Expired key: Your API key has passed its 2-week expiration date
• Incorrect header format: Make sure you're using either X-API-Key or Authorization: Bearer headers

Check your Dashboard to verify your API keys are active and not expired.

Yes, you can use the same API key from any IP address. There are no IP-based restrictions on API keys. However, rate limiting applies per API key regardless of which IP address is making the request.

The default rate limit is 10 requests per hour per API key. This means you can make up to 10 API calls within any 60-minute window.

Administrators can configure custom rate limits for specific users. If you need a higher rate limit, please contact an administrator.

When you exceed the rate limit, you'll receive a 429 (Too Many Requests) error with a Retry-After header indicating when you can make more requests.

Options to resolve this:

• Wait: The rate limit window resets automatically after the time period expires
• Create additional API keys: Each API key has its own rate limit, so multiple keys can increase your total capacity
• Optimize your requests: Use the cache effectively and avoid unnecessary force refreshes
• Contact support: If you need a higher limit for legitimate use, administrators can increase your rate limit

When you receive a rate limit error (429), the response includes information about your remaining requests and when the limit resets. The response body will contain a remaining field and a retryAfter value indicating seconds until reset.

Additionally, the Retry-After HTTP header specifies when you can make your next request.

Administrators can configure custom rate limits for users. If you have a legitimate need for higher rate limits, please contact an administrator to discuss your requirements. Custom limits are configured on a per-user basis and are separate from the default API key limits.

Each API key has its own independent rate limit. If you have multiple API keys, each one can make up to 10 requests per hour (or your custom limit). This means if you have 3 API keys, you could theoretically make up to 300 requests per hour by distributing requests across them.

Note: Rate limiting uses Redis for fast tracking with MySQL fallback, ensuring accurate limits even if one system is unavailable.

Rate limiting applies per API key, not per user. However, users can have custom rate limits configured by administrators that apply to all of that user's API keys. If no custom limit is set, the default limit (10 requests/hour) applies to each API key independently.

The API provides the following main endpoints:

• GET /api/v1/audible/books/{asin} - Get Audible book metadata by ASIN
• GET /api/v1/amazon/books/{asin} - Get Amazon retail book metadata (uses async job processing)
• POST /api/v1/amazon/books/{asin}/refresh - Force refresh Amazon book metadata (always async)
• GET /api/v1/jobs/{jobId} - Check status of background jobs
• GET /api/v1/amazon/books/series/{asin} - Get Amazon series metadata with all books
• GET /api/v1/authors/{authorId} - Get author information and bibliography
• GET /api/v1/search - Search for books and authors with pagination

For complete documentation with all parameters and response formats, visit /docs.

ASIN (Amazon Standard Identification Number) is a 10-character alphanumeric unique identifier for products on Amazon and Audible. For Audible books, ASINs typically start with 'B' (e.g., B002V1OF70).

You can find the ASIN in several places:

• On the Audible product page URL: audible.com/pd/BOOK-NAME/B002V1OF70
• In the product details section on the book's page
• In the search results from our API

Use the search endpoint: GET /api/v1/search?query=your+search+term

Additional parameters:

• type - Filter by 'book' or 'author' (default: 'book')
• region - Audible region (e.g., 'us', 'uk', default: 'us')
• page - Page number for pagination (default: 1)
• per_page - Results per page (1-10, default: 20)
• force - Force cache refresh (default: false)
• amazon_search_type - Amazon fallback search type ('books', 'audible', 'kindle')

Example: /api/v1/search?query=Stephen+King&type=author&page=1&per_page=20

The API supports multiple Audible storefront regions including (but not limited to):

• us - United States (default)
• uk - United Kingdom
• And other regional Audible marketplaces

Specify the region using the region query parameter. Different regions may have different book availability and pricing.

Audible endpoint (/api/v1/audible/books/{asin}):

• Fast response time (cached, typically < 1 second)
• Returns audiobook-specific metadata
• Includes narrator information, listening length, etc.
• Always returns immediately (200 response)

Amazon endpoint (/api/v1/amazon/books/{asin}):

• Uses async job processing for scraping
• If cached and fresh: Returns immediately (200 response)
• If uncached or stale: Returns 202 with a jobId for polling
• Jobs typically take 10-20 seconds to complete
• Uses headless Playwright browser for scraping
• Returns retail book metadata including ISBN, pages, physical book details
• Useful when you need physical book information or when Audible doesn't have the title

See "How does the Amazon async job system work?" for details on polling job status.

Amazon endpoints use async job processing because scraping requires a headless Playwright browser, which takes time. The process involves:

• Launching a browser instance
• Navigating to the page (with 2-5 second waits to appear more human-like)
• Waiting for content to load
• Extracting metadata
• Rotating user agents and handling anti-bot measures

Jobs typically take 10-20 seconds to complete. When you request uncached data, you'll receive a 202 response with a jobId. You need to poll the /api/v1/jobs/:jobId endpoint every 10-20 seconds until the job completes.

Results are cached for 6 months, so subsequent requests for the same ASIN will return immediately (200 response) if the cache is fresh.

Yes! There are two ways to force a cache refresh:

1. Query parameter: Add force=true to any endpoint. For Amazon endpoints, this will enqueue a job and return 202 with a jobId.
2. Refresh endpoint: Use POST /api/v1/amazon/books/{asin}/refresh which always enqueues a job and returns 202.

Example: /api/v1/audible/books/B002V1OF70?force=true

For Amazon endpoints with force=true, you'll need to poll the job status endpoint to get the result. Jobs typically take 10-20 seconds.

Note: Force refreshes count against your rate limit and will be slower. Use sparingly and only when you need the most up-to-date information.

Data is cached for 6 months (155,520,000 milliseconds). This long cache duration ensures fast responses while keeping data relatively fresh. The cache is automatically refreshed in the background by a scheduled job that runs nightly at 3 AM, refreshing a batch of older entries each time.

The search endpoint supports pagination with the following parameters:

• page - Page number (minimum: 1, default: 1)
• per_page - Number of results per page (minimum: 1, maximum: 10, default: 20)

The response includes a pagination object with:

• page - Current page number
• per_page - Results per page
• total - Total number of results
• total_pages - Total number of pages

Amazon book endpoints use background job processing to handle the time-consuming scraping process. Here's how it works:

1. Request the book: GET /api/v1/amazon/books/{asin}
2. Check cache: If cached and fresh, return immediately (200)
3. If uncached/stale: Enqueue a background job and return 202 with a jobId
4. Poll for status: Use GET /api/v1/jobs/{jobId} to check job status
5. Wait and retry: Jobs take 10-20 seconds. Poll every 10-20 seconds until complete
6. Get result: When status is "success", the result field contains the book data

Job statuses:

• pending - Job is waiting to be processed
• running - Job is currently being processed
• success - Job completed successfully (check result field)
• failed - Job failed (check error field for details)

Important: You may need to query the job status endpoint multiple times, waiting 10-20 seconds between requests, until the job completes.

After receiving a 202 response with a jobId, follow these steps:

1. Extract the jobId from the 202 response
2. Wait 10-20 seconds (use the pollAfterMs value as a guide, but wait longer for first poll)
3. Query GET /api/v1/jobs/{jobId}
4. Check the status field:
   • If pending or running: Wait another 10-20 seconds and query again
   • If success: The result field contains your book data
   • If failed: Check the error field for failure details
5. Repeat step 4 until status is success or failed

Example polling pattern:

# Request book
GET /api/v1/amazon/books/B08FDFB754
→ 202 { "jobId": "Byrsbi-B08FDFB754", ... }

# Wait 15 seconds, then check status
GET /api/v1/jobs/Byrsbi-B08FDFB754
→ 202 { "status": "running", ... }

# Wait another 15 seconds, check again
GET /api/v1/jobs/Byrsbi-B08FDFB754
→ 200 { "status": "success", "result": { ... } }

Tip: Most jobs complete within 10-20 seconds, but some may take longer. Be patient and continue polling.

GET /api/v1/amazon/books/{asin}:

• Checks cache first - returns immediately (200) if cached and fresh
• Only enqueues a job if cache is stale or missing
• Returns 202 with jobId when job is enqueued
• Use this for normal requests - it's efficient and uses cache when possible

POST /api/v1/amazon/books/{asin}/refresh:

• Always enqueues a job, bypassing cache completely
• Always returns 202 with jobId
• Use this when you specifically need fresh data and want to bypass cache
• Equivalent to GET ...?force=true but more explicit

Both endpoints require polling the job status endpoint to get results when a job is enqueued.

The API uses a two-tier caching system:

1. MySQL database: Persistent cache storing all scraped data with a 6-month TTL
2. Redis: Used primarily for rate limiting, but can also cache frequently accessed data for even faster responses

When you make a request, the API checks the cache first. If cached data exists and hasn't expired, it's returned immediately. If not, the API scrapes fresh data, stores it in the cache, and returns it to you.

If you're getting data that seems outdated, it's likely because the cache hasn't been refreshed yet. The cache has a 6-month TTL, and background refresh runs nightly, processing a batch of entries each time.

To get fresh data immediately, use the force=true query parameter. This bypasses the cache and fetches the latest information from the source.

Check the response metadata - it includes cachedAt, expiresAt, and stale fields to help you understand the data's age.

Cache entries are automatically refreshed when API endpoints detect that they have expired. When you request data that has expired, the system will fetch fresh data from the source and update the cache. This ensures that frequently accessed data stays up-to-date while minimizing unnecessary refreshes.

Yes, when you use force=true, the API completely bypasses the cache and fetches fresh data from the source (Audible or Amazon). However, the new data is then stored in the cache for future requests. This ensures that even after a force refresh, subsequent requests benefit from the updated cache.

404 errors typically mean the ASIN doesn't exist or isn't available in the requested region. Common causes:

• The ASIN is invalid or incorrectly formatted
• The book isn't available in the specified Audible region
• The product has been removed or is no longer listed
• You're using an Amazon ASIN with the Audible endpoint (they may differ)

Try using the Amazon endpoint (/api/v1/amazon/books/{asin}) or search for the book to find the correct ASIN.

With the async job system, timeouts are less common because Amazon endpoints return immediately (202) when a job is enqueued. However, if you experience issues:

• For job polling: Ensure your client can handle 202 responses and has appropriate timeout settings for polling
• Job failures: Check the job status endpoint - if status is "failed", the error field will contain details
• Long-running jobs: Jobs typically take 10-20 seconds. If a job is still "running" after 30+ seconds, continue polling - it may just be taking longer
• Network issues: Verify network connectivity when polling job status
• Amazon blocking: Check if Amazon is blocking requests (captcha, rate limiting) - this will show in the job error
• Use cache: Try using cached data first (omit force=true) to avoid job processing

Audible endpoints should respond much faster (typically under 1 second) as they use simple HTTP requests and don't use job processing.

The API follows RFC7807 Problem Details for HTTP APIs standard. Error responses include:

• type - URI identifying the problem type (usually "about:blank")
• title - Short summary of the problem
• status - HTTP status code (400, 404, 429, etc.)
• detail - Human-readable explanation
• instance - URI of the specific request that failed

Example error response:

{
  "type": "about:blank",
  "title": "Not Found",
  "status": 404,
  "detail": "Book with ASIN B002V1OF70 not found",
  "instance": "/api/v1/audible/books/B002V1OF70"
}

If you suspect the data is incorrect:

1. Check the metadata in the response - verify the cachedAt date to see how old the data is
2. Try using force=true to fetch fresh data from the source
3. Verify the ASIN is correct and corresponds to the right book
4. Check if the source (Audible/Amazon) has updated their data
5. Contact support if the issue persists - the scraper may need updates for changes in the source website structure

Audible book responses include:

• Title, authors, narrators
• Description and categories
• Cover image URL
• Release date, publisher
• Listening length
• Ratings and reviews
• Price information

Amazon book responses add:

• ISBN-10 and ISBN-13
• Page count
• Edition format (hardcover, paperback, etc.)
• Physical book dimensions

See the full documentation at /docs for complete response schemas.

Yes! ISBN information is available through the Amazon endpoint or in search results that include Amazon data. The response includes both ISBN-10 and ISBN-13 when available. Some search results may also include ISBN data if the book has been previously scraped from Amazon and stored in the database.

The search endpoint first searches Audible, then falls back to Amazon if no results are found. Results include a source field indicating whether they came from 'audible' or 'amazon'.

Audible results include narrator information and audiobook-specific metadata.

Amazon results include price, rating details, product URLs, and may include ISBN and page count if available. Amazon results may lack detailed descriptions compared to Audible.

Results can also be enhanced with amazonData when Amazon metadata exists in the database, combining the best of both sources.

Yes! Use the amazon_search_type parameter when searching. Options include:

• books - Physical books (default Amazon fallback)
• audible - Audiobooks on Amazon
• kindle - Kindle ebooks

Note: This parameter only applies when Audible returns no results and the search falls back to Amazon.

The API supports multiple Audible storefront regions via the region parameter. Different regions have different book catalogs, pricing, and availability. Supported regions include 'us' (default), 'uk', and other regional marketplaces.

For Amazon endpoints, use the marketplace parameter to specify the domain (e.g., 'www.amazon.com', 'www.amazon.co.uk').

The data accuracy depends on the source (Audible or Amazon). The API scrapes directly from these sites, so accuracy is generally very high. However:

• Data may become stale if cached for a long time (use force=true for fresh data)
• Source websites may update their data structure, requiring scraper updates
• Some fields may be missing if not available on the source page

The response metadata includes cache information so you can verify data freshness.

Password changes must be handled by an administrator. Please contact an administrator to request a password reset. There's also a script available (scripts/change-admin-password.js) that administrators can use to change passwords.

Account deletion must be performed by an administrator. Please contact an administrator if you need your account removed. When an account is deleted, all associated API keys are automatically revoked and removed.

Yes! You can create multiple API keys from your Dashboard. This is useful for:

• Separating different applications or environments
• Increasing your total rate limit capacity
• Organizing access by project or service

Each API key has its own independent rate limit, and you can revoke them individually.

Regular users can:

• Create and manage their own API keys
• View their dashboard
• Use the API with their API keys

Admin users can do everything regular users can, plus:

• Create and delete user accounts
• View all users and their API keys
• Configure custom rate limits for users
• Revoke any user's API keys

Currently, usage statistics are available through the API response headers and rate limit error messages. The Dashboard shows when each API key was last used, but detailed usage analytics are not yet available in the web interface. API key usage is tracked in the database for administrative purposes.

Successful responses follow this structure:

{
  "data": { /* entity data */ },
  "metadata": {
    "source": "cache" | "scraper",
    "region": "us",
    "cachedAt": "2024-01-01T00:00:00Z",
    "expiresAt": "2024-07-01T00:00:00Z",
    "stale": false
  }
}

Search responses add a pagination object. Error responses use RFC7807 Problem Details format. See /docs for complete schemas.

No, the API does not currently support webhooks. You'll need to poll the API endpoints to check for updates. Use the cache effectively and only force refresh when necessary to avoid hitting rate limits.

Yes! The API can be run locally. See the README for setup instructions. You'll need:

• Node.js 18+
• MySQL database
• Redis (optional, but recommended)
• Python 3 with Playwright (for Amazon scraping)

Or use Docker Compose for a complete stack: docker compose up --build

Server requirements:

• Node.js 18.0.0 or higher
• MySQL 8.0+ for persistent cache
• Redis 7+ (optional but recommended for rate limiting)
• Python 3 with Playwright for Amazon scraping
• Sufficient memory for headless browser instances

See docs/environment.md for configuration details.

Docker Compose setup is straightforward:

1. Ensure Docker and Docker Compose are installed
2. Copy env.example to .env and configure as needed
3. Run: docker compose up --build

This starts three services:

• api - Express server on port 3000
• mysql - MySQL 8 with persistent volume
• redis - Redis 7 with persistent volume

See docs/environment.md for the complete list. Key variables include:

• PORT - Server port (default: 3000)
• AUDIBLE_REGION - Default region (default: us)
• DB_* - MySQL connection settings
• REDIS_URL - Redis connection
• RATE_LIMIT_MAX_REQUESTS - Rate limit (default: 10/hour)
• CACHE_MAX_AGE_MS - Cache TTL (default: 6 months)
• AMAZON_* - Amazon scraper configuration

The Amazon scraper uses several techniques:

• Headless Playwright browser: More realistic than simple HTTP requests
• User agent rotation: Rotates between realistic browser user agents
• Randomized waits: Waits 2-5 seconds between actions to appear human
• Resource blocking: Blocks unnecessary fonts/images but allows first-party content
• Geolocation randomization: Retries with randomized timezone/geolocation on captcha
• Cookie support: Can accept session cookies via query parameter
• Proxy support: Can use HTTP/HTTPS proxies (including residential proxies)

Yes! You can configure proxies via:

• AMAZON_PROXY environment variable
• HTTP_PROXY / HTTPS_PROXY environment variables
• proxy query parameter on the Amazon endpoint

Format: http://USER:PASS@host:port or http://host:port

Residential proxies work best for avoiding captchas. Pair with session cookies for best results.

MySQL unavailability: The API cannot cache or retrieve cached data. Requests will still work but will always hit the source (slower, uses rate limits). The API will attempt to store new data when MySQL becomes available again.

Redis unavailability: Rate limiting falls back to MySQL database queries. This is slower but ensures rate limits still work. When Redis is available again, the system will automatically resume using it for faster rate limit checks.

Browser mode (default): Uses headless Playwright with Chromium. More realistic, better at handling JavaScript and avoiding detection, but slower (30-90 seconds).

Requests mode: Uses simple HTTP requests. Faster but more likely to trigger captchas. Use via transport=requests query parameter or AMAZON_TRANSPORT=requests environment variable.

Browser mode is recommended for production use.

Common reasons for failure:

• Captcha challenges: Amazon detected automated access
• Rate limiting: Too many requests from your IP
• Invalid ASIN: The ASIN doesn't exist or isn't a book
• Network issues: Timeout or connectivity problems
• Page structure changes: Amazon updated their HTML (requires scraper update)
• Geographic restrictions: Product not available in your region

Always check HTTP status codes:

• 200: Success - parse the response data
• 202: Accepted - job enqueued (Amazon endpoints). Extract jobId and poll for status
• 400: Bad Request - check your parameters
• 401: Unauthorized - verify your API key
• 404: Not Found - the resource doesn't exist
• 429: Rate Limited - respect the Retry-After header
• 500: Server Error - retry with exponential backoff

For 202 responses from Amazon endpoints, you'll need to poll the job status endpoint. Error responses follow RFC7807 format with title, detail, and status fields. Log these for debugging.

Yes, but consider the cache duration carefully:

• The API already caches responses for 6 months
• You can cache responses on your end for shorter periods (e.g., 1 hour, 1 day) to reduce API calls
• Check the metadata.expiresAt field to know when to refresh
• Use force=true only when you need absolutely fresh data

This helps you stay within rate limits while keeping data reasonably fresh.

There are two types of polling to consider:

1. Polling for new/updated books (data freshness):

• With the 6-month cache TTL and background refresh, you typically don't need frequent polling
• For most use cases: Poll daily or weekly
• For critical updates: Check the metadata.cachedAt to see when data was last refreshed
• Use the cache effectively - don't use force=true on every request
• Respect rate limits - spread requests over time rather than making bursts

2. Polling for job status (Amazon endpoints):

• When you receive a 202 response with a jobId, you need to poll the job status endpoint
• Wait 10-20 seconds between polling requests
• Continue polling until status is "success" or "failed"
• Most jobs complete within 10-20 seconds, but some may take longer
• You may need to query the job status endpoint 2-3 times before completion

Remember: Each API key has 10 requests/hour limit. Plan your polling accordingly.

When you receive a 429 error:

1. Check the Retry-After HTTP header (specifies seconds until reset)
2. Wait at least that long before retrying
3. Implement exponential backoff for retries
4. Consider using multiple API keys to distribute load
5. Optimize your requests (use cache, avoid unnecessary force refreshes)

Never ignore rate limit headers - retrying immediately will just waste requests and potentially get your key temporarily blocked.

No! Use force=true or the refresh endpoint sparingly:

• Only when you absolutely need the freshest data
• For debugging or testing
• After you know data has changed on the source

Force refreshes:

• Require job processing (Amazon endpoints return 202, you must poll for results)
• Take 10-20 seconds to complete (you'll need to poll the job status endpoint)
• Count against your rate limit
• Put unnecessary load on source websites

The cache is automatically refreshed in the background, so force refreshes are rarely needed in production.