Overview

TokenFlow takes messy inputs (PDFs, Word docs, spreadsheets, audio files, web pages) and returns structured text your systems can actually use. It runs as a hosted API at https://tokenflow.fly.dev.

Key Use Cases

  • Document ingestion — Drop a PDF or DOCX into your agent; get structured markdown without writing a parser

  • Spreadsheet-to-text — Convert XLSX/CSV to readable tables or structured data

  • Audio transcription — WAV, MP3, OGG, FLAC, M4A → timestamped text

  • URL extraction — Paste a link; get the article content stripped of ads and nav

  • Archive handling — Unzip and process contents in one call

Quick Start

curl -X POST "https://tokenflow.fly.dev/convert" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F "file=@document.pdf" \
  -F "output_format=markdown"

Response:

{
  "markdown": "# Document Title\n\nFirst paragraph..."
}

Supported Platforms

  • OpenClaw

  • Hermes

  • Any system that can make HTTP POST requests

Authentication

TokenFlow uses Bearer token authentication. Every request (except /health) requires an Authorization header.

Getting an API Key

  1. Sign up at tokenflow.ironlion.cc

  2. Copy your API key from the dashboard

  3. Set it as TOKENFLOW_API_KEY in your environment or skill config

Using the API Key

cURL

curl -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  "https://tokenflow.fly.dev/usage"

JavaScript

const response = await fetch('https://tokenflow.fly.dev/convert', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${apiKey}`
  },
  body: formData
});

Python

import requests

headers = {"Authorization": f"Bearer {api_key}"}
response = requests.post(
    "https://tokenflow.fly.dev/convert",
    headers=headers,
    files={"file": open("doc.pdf", "rb")},
    data={"output_format": "markdown"}
)

Error: Invalid Key

If your key is missing or invalid, you'll get:

{
  "error": "Unauthorized",
  "status": 401
}

Check your key in the dashboard. Keys are scoped to your account and usage quota.

API Endpoints

Base URL: https://tokenflow.fly.dev

All endpoints except /health require Bearer authentication.

POST /convert

Convert a file to structured text.

Request

curl -X POST "https://tokenflow.fly.dev/convert" \
  -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  -F "file=@document.pdf" \
  -F "output_format=markdown"

Parameters

  • file (File, required) — The file to convert

  • output_format (String, optional) — "markdown" (default) or "html"

Response (200 OK) — Single File

{
  "markdown": "# Document Title\n\nContent here..."
}

Response (HTML format) — Single File

{
  "html": "<h1>Document Title</h1><p>Content here...</p>"
}

Response (ZIP Archive)

When converting a ZIP file, the response contains per-file results:

{
  "filename": "archive.zip",
  "type": "zip",
  "results": [
    {
      "filename": "report.pdf",
      "status": "success",
      "markdown": "# Report\n\nContent...",
      "input_tokens": 15000,
      "output_tokens": 8200,
      "output_chars": 32800,
      "meaningful": true
    },
    {
      "filename": "data.exe",
      "status": "skipped",
      "reason": "unsupported"
    },
    {
      "filename": "notes.docx",
      "status": "skipped",
      "reason": "quota_exceeded"
    }
  ],
  "total_files": 3,
  "converted_files": 1,
  "processing_time_ms": 2450
}

Each successful conversion inside a ZIP counts individually against your monthly quota. Quota exhaustion mid-archive skips remaining files with "reason": "quota_exceeded". Unsupported files (including nested ZIPs) are skipped with "reason": "unsupported".

Error Responses

  • 400 — Unsupported format or bad request

  • 401 — Invalid or missing API key

  • 413 — File too large

  • 429 — Quota exceeded or rate limited

  • 500 — Server error

POST /convert/url

Convert a web page to structured text.

Request

curl -X POST "https://tokenflow.fly.dev/convert/url" \
  -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/article"}'

Parameters

  • url (String, required) — The URL to extract content from

  • output_format (String, optional) — "markdown" (default) or "html"

Response

{
  "markdown": "# Article Title\n\nArticle content..."
}

POST /convert/preview

Get sheet names from an XLSX file before converting. Useful when you want to let users pick which sheets to process.

Request

curl -X POST "https://tokenflow.fly.dev/convert/preview" \
  -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  -F "file=@data.xlsx"

Response

{
  "sheets": ["Sheet1", "Sheet2", "Revenue 2024"]
}

GET /health

Check if the service is up. No auth required.

Request

curl "https://tokenflow.fly.dev/health"

Response

{
  "status": "ok"
}

GET /usage

Check your current usage and quota.

Request

curl -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  "https://tokenflow.fly.dev/usage"

Response

{
  "used": 1247,
  "quota": 5000,
  "reset_date": "2026-06-01"
}

Supported File Formats

TokenFlow handles these file types out of the box. No plugins, no extra configuration.

Documents

  • .pdf — PDF documents, text extraction with layout preservation

  • .docx — Microsoft Word documents

  • .pptx — Microsoft PowerPoint presentations

  • .html, .htm — HTML web pages (local files)

Spreadsheets

  • .xlsx — Microsoft Excel (multi-sheet supported)

  • .xls — Legacy Excel format

  • .csv — Comma-separated values

Note on XLSX: Use /convert/preview to list sheet names, then pass sheets="all" or specific sheet names.

Data Files

  • .json — JSON, pretty-printed and validated

  • .xml — XML, structured with tags preserved

  • .txt — Plain text, passed through with normalization

  • .md — Markdown, returned as-is

Audio

  • .wav — WAV audio → transcription

  • .mp3 — MP3 audio → transcription

  • .ogg — OGG audio → transcription

  • .flac — FLAC audio → transcription

  • .m4a — M4A audio → transcription

Audio files are transcribed to text. Speaker detection is not supported.

Archives

  • .zip — ZIP archives. Each supported file is extracted and converted individually. Unsupported files and nested ZIPs are skipped. Each successful conversion counts against your quota.

Output Formats

The API returns content in one of two formats:

  • markdown (default) — Clean, structured markdown. Best for agents and LLM consumption.

  • html — Structured HTML. Use when you need markup for web rendering.

Set output_format on any conversion request:

-F "output_format=markdown"

Error Handling

TokenFlow fails fast and tells you why. Here's what each error means and what to do about it.

HTTP Status Codes

400 — Bad Request - Unsupported file format or malformed request. Check the file extension against Supported Formats. - Nested ZIPs not supported. Extract nested archives before uploading.

401 — Unauthorized - Invalid or missing API key. Verify your TOKENFLOW_API_KEY is set and active.

413 — Payload Too Large - File exceeds size limit. Compress, split, or reduce the file size.

429 — Too Many Requests - Quota exceeded or rate limited. Check /usage — upgrade or wait for reset.

500 — Server Error - Something broke on our end. Retry once; if persistent, open an issue.

504 — Gateway Timeout - Conversion took too long. Retry with a smaller file or simpler format.

Retry Behavior

The TokenFlow skill (OpenClaw / Hermes) retries automatically:

  • Rate limits (429): Waits 1 second, retries once

  • Timeouts: Retries up to maxRetries (default: 1, configurable in skill config)

  • 5xx errors: Retries once, then falls back

Fallback Behavior

When conversion fails, the skill can either:

  • use_file (default) — Pass the original file through unconverted. Log a warning.

  • fail — Report the error to the user and stop.

Set this in config.json:

{
  "fallbackBehavior": "use_file"
}

Common Issues

"Empty result" The file converted but returned no extractable text. Usually means: - Scanned PDF without OCR text layer - Corrupted or password-protected file - Audio file with no detectable speech

"File type not supported" Extension isn't in the supported list. If you think it should be, open an issue.

"Quota exceeded" You've hit your monthly limit. Check /usage to see your reset date.

Reporting Bugs

From the TokenFlow web interface: click the Support button (bottom-right corner) to open the help modal. Describe what happened, pick a category, and submit — tickets are created directly from the UI.

From the API: include these details in your report:

  1. The file type and size

  2. The exact error message

  3. Your request ID (from response headers)

For account or billing issues, email tokenflow@ironlioninternational.com.

SDK & Integration

TokenFlow integrates with agent frameworks and works with any HTTP client.

OpenClaw

Install the skill:

openclaw skills install tokenflow

Or manually copy to ~/.openclaw/skills/tokenflow/.

Configure in ~/.openclaw/skills/tokenflow/config.json:

{
  "apiKey": "your-api-key",
  "apiUrl": "https://tokenflow.fly.dev",
  "filetypeBehavior": {
    "docx": { "action": "convert", "askEachTime": false },
    "pdf":  { "action": "convert", "askEachTime": false },
    "xlsx": { "action": "convert", "askEachTime": true }
  },
  "outputFormat": "markdown",
  "maxRetries": 1,
  "fallbackBehavior": "use_file"
}

When a user attaches a supported file, OpenClaw automatically calls TokenFlow based on filetypeBehavior rules.

Hermes

Copy the skill directory to ~/.hermes/skills/tokenflow/.

Configure in ~/.hermes/skills/tokenflow/config.json (same schema as OpenClaw).

Hermes agents load the skill automatically on startup if present in the skills directory.

JavaScript / Node.js

const fs = require('fs');
const FormData = require('form-data');
const fetch = require('node-fetch');

async function convertFile(filePath, apiKey) {
  const form = new FormData();
  form.append('file', fs.createReadStream(filePath));
  form.append('output_format', 'markdown');

  const response = await fetch('https://tokenflow.fly.dev/convert', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${apiKey}`,
      ...form.getHeaders()
    },
    body: form
  });

  const data = await response.json();
  // For ZIP archives, data.type === 'zip' and data.results is an array
  if (data.type === 'zip') {
    return data.results.filter(r => r.status === 'success').map(r => r.markdown).join('\n\n');
  }
  return data.markdown;
}

Python

import requests

def convert_file(file_path, api_key, output_format="markdown"):
    url = "https://tokenflow.fly.dev/convert"
    headers = {"Authorization": f"Bearer {api_key}"}

    with open(file_path, "rb") as f:
        files = {"file": f}
        data = {"output_format": output_format}
        response = requests.post(url, headers=headers, files=files, data=data)

    response.raise_for_status()
    return response.json()["markdown"]

# Convert a URL
def convert_url(target_url, api_key):
    url = "https://tokenflow.fly.dev/convert/url"
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json"
    }
    response = requests.post(url, headers=headers, json={"url": target_url})
    response.raise_for_status()
    return response.json()["markdown"]

cURL Examples

Convert a PDF:

curl -X POST "https://tokenflow.fly.dev/convert" \
  -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  -F "file=@report.pdf" \
  -F "output_format=markdown"

Convert a URL:

curl -X POST "https://tokenflow.fly.dev/convert/url" \
  -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

Preview XLSX sheets:

curl -X POST "https://tokenflow.fly.dev/convert/preview" \
  -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  -F "file=@data.xlsx"

Check usage:

curl -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  "https://tokenflow.fly.dev/usage"

Changelog

2.1.0 — Current

  • Added /convert/url endpoint for web page extraction

  • Added /convert/preview endpoint for XLSX sheet discovery

  • Added /usage endpoint for quota tracking

  • Expanded audio support: WAV, MP3, OGG, FLAC, M4A

  • Added ZIP archive handling

  • Configurable fallbackBehavior: use_file or fail

  • Configurable maxRetries: 0–3

2.0.0

  • Migrated API to https://tokenflow.fly.dev

  • Added Bearer token authentication

  • Added HTML output format option

  • Restructured skill config with filetypeBehavior per extension

1.0.0

  • Initial release

  • PDF, DOCX, PPTX, XLSX, CSV, JSON, XML, TXT, MD support

  • Markdown output only

  • Basic error handling