diff --git a/browsers/extensions.mdx b/browsers/extensions.mdx index 9b3189b..e85c9a8 100644 --- a/browsers/extensions.mdx +++ b/browsers/extensions.mdx @@ -97,3 +97,54 @@ kernel browsers extensions upload ./my-extension Note that this will restart the browser process and break any connections to the browser CDP URL. + +## Extensions requiring enterprise policies + +For a complete list of available extension settings and policies, refer to the [Chrome Enterprise Policy documentation](https://chromeenterprise.google/policies/extension-settings/). + +### Uploading extensions requiring enterprise policies + +Some Chrome extensions require elevated permissions that Chrome will only grant when the extension is installed via enterprise policies. These extensions cannot be loaded with the standard `--load-extension` flag and require special handling. + +### What are enterprise policy extensions? + +Extensions that require enterprise policies typically: + +- Use permissions like `webRequestBlocking` or `webRequest` with blocking capabilities +- Need to intercept and modify network requests before they're sent +- Require installation via Chrome's `ExtensionInstallForcelist` policy + +Common examples include extensions for network filtering, request signing, or advanced content modification. + +### Required files for upload + +When uploading an extension that requires enterprise policies to Kernel, your extension directory or zip file must include: + +1. **Extension source files** - Your `manifest.json` and all extension code (background scripts, content scripts, etc.) +2. **`update.xml`** - A Chrome update manifest that points to the `.crx` file location +3. **`.crx` file** - The signed and packed extension file + + +The `.crx` file and `update.xml` are required for Kernel to serve the extension via Chrome's `ExtensionInstallForcelist` policy. If you're deploying extensions from the Chrome Web Store via `ExtensionInstallForcelist`, these files are optional since Chrome uses the Web Store's default update URL. + + +### Automatic detection and validation + +Kernel automatically detects extensions that require enterprise policies by analyzing the `manifest.json` file during upload. No manual configuration is needed. + +**Detection process:** +1. You upload an extension via CLI or API +2. Kernel scans the `manifest.json` for permissions like `webRequestBlocking` +3. If enterprise policies are required, Kernel validates the required files are present + +### How it works + +Once you successfully upload an enterprise policy extension, Kernel handles the rest automatically: + +1. **Upload** - You upload your extension with all required files +2. **Detection** - Kernel detects the enterprise policy requirement from the manifest +3. **Policy configuration** - Extension is automatically added to `ExtensionInstallForcelist` +4. **File serving** - The kernel-images server serves update files at `http://127.0.0.1:10001/extensions/{extension-id}/update.xml` +5. **Installation** - Chrome installs the extension via enterprise policy when the browser starts + +No additional HTTP server or manual policy configuration is needed. The extension works seamlessly in any browser session that it's uploaded to. diff --git a/docs.json b/docs.json index dedd1b8..901c0e9 100644 --- a/docs.json +++ b/docs.json @@ -136,7 +136,8 @@ "integrations/vercel/ai-sdk", "integrations/vercel/marketplace" ] - } + }, + "integrations/web-bot-auth" ] }, { @@ -182,7 +183,8 @@ "reference/cli/create", "reference/cli/auth", "reference/cli/browsers", - "reference/cli/apps" + "reference/cli/apps", + "reference/cli/extensions" ] }, { diff --git a/integrations/overview.mdx b/integrations/overview.mdx index 398a349..6db2a01 100644 --- a/integrations/overview.mdx +++ b/integrations/overview.mdx @@ -28,7 +28,8 @@ Kernel provides detailed guides for popular agent frameworks: - **[Notte](/integrations/notte)** - AI agent framework for browser automation - **[Val Town](/integrations/valtown)** - Serverless function runtime - **[Vercel](https://github.com/onkernel/vercel-template)** - Deploy browser automations to Vercel +- **[Web Bot Authentication](/integrations/web-bot-auth)** - Create signed Chrome extensions for web bot authentication ## Custom Integrations -Kernel works with any tool that supports CDP. Check out our [browser creation guide](/browsers/create-a-browser) to learn how to connect any other agent framework. \ No newline at end of file +Kernel works with any tool that supports CDP. Check out our [browser creation guide](/browsers/create-a-browser) to learn how to connect any other agent framework. diff --git a/integrations/web-bot-auth.mdx b/integrations/web-bot-auth.mdx new file mode 100644 index 0000000..f111c75 --- /dev/null +++ b/integrations/web-bot-auth.mdx @@ -0,0 +1,172 @@ +--- +title: "Web Bot Auth" +description: "Cryptographically sign browser requests with Cloudflare's Web Bot Auth" +--- + +[Web Bot Auth](https://github.com/cloudflare/web-bot-auth) is Cloudflare's implementation of cryptographic authentication for automated web agents. It uses [RFC 9421 HTTP Message Signatures](https://datatracker.ietf.org/doc/html/rfc9421) to sign outgoing HTTP requests, allowing websites to verify the identity and integrity of bot traffic. + +By integrating Web Bot Auth with Kernel, your browser automations can cryptographically prove their identity to websites that support signature verification. + +## How it works + +Web Bot Auth works via a Chrome extension that intercepts all outgoing HTTP requests and adds cryptographic signature headers: + +- **`Signature`**: The RFC 9421 signature of the request +- **`Signature-Input`**: Metadata about how the signature was created + +Websites can verify these signatures against your public key to confirm the request came from your authenticated agent. + +## Quick Start with Test Key + +The fastest way to get started is using Cloudflare's RFC9421 test key, which works with their [test verification site](https://http-message-signatures-example.research.cloudflare.com/). + +### 1. Build the extension + +Use the Kernel CLI to build the Web Bot Auth extension: + +```bash +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --upload my-web-bot-auth +``` + +This command: +- Downloads Cloudflare's web-bot-auth browser extension source +- Builds it with the default RFC9421 test key +- Uploads it to Kernel as `my-web-bot-auth` + + +The build command requires Node.js and npm to be installed on your system. + + +### 2. Create a browser with the extension + + +```bash CLI +# Create a browser with the web-bot-auth extension +kernel browsers create --extension my-web-bot-auth + +# The command outputs the browser ID and live view URL +# Open the live view URL in your browser, then navigate to: +# https://http-message-signatures-example.research.cloudflare.com/ +``` + +```typescript TypeScript +import { Kernel } from "@onkernel/sdk"; +import { chromium } from "playwright"; + +const kernel = new Kernel(); + +// Create browser with web-bot-auth extension +const browser = await kernel.browsers.create({ + extensions: [{ name: "my-web-bot-auth" }], +}); + +// Connect via Playwright +const pw = await chromium.connectOverCDP(browser.browser_url); +const context = pw.contexts()[0]; +const page = context?.pages()[0] || await context.newPage(); + +// Navigate to a page - requests will be automatically signed +await page.goto("https://http-message-signatures-example.research.cloudflare.com/"); +``` + +```python Python +from kernel import Kernel +from playwright.sync_api import sync_playwright + +kernel = Kernel() + +# Create browser with web-bot-auth extension +browser = kernel.browsers.create(extensions=[{"name": "my-web-bot-auth"}]) + +# Connect via Playwright +with sync_playwright() as p: + pw = p.chromium.connect_over_cdp(browser.browser_url) + context = pw.contexts[0] + page = context.pages[0] if context.pages else context.new_page() + + # Navigate to a page - requests will be automatically signed + page.goto("https://http-message-signatures-example.research.cloudflare.com/") +``` + + + +### 3. Verify it's working + +Navigate to Cloudflare's test site to verify your signatures are being accepted: + +``` +https://http-message-signatures-example.research.cloudflare.com/ +``` + +This site validates requests signed with the RFC9421 test key and shows whether the signature was verified successfully. + +## Using Your Own Keys + +For production use, you'll want to use your own signing keys instead of the test key. + +### 1. Generate an Ed25519 key pair + +Create a JWK file with your Ed25519 private key. The key must include both the public (`x`) and private (`d`) components: + +```json my-key.jwk +{ + "kty": "OKP", + "crv": "Ed25519", + "x": "YOUR_PUBLIC_KEY_BASE64URL", + "d": "YOUR_PRIVATE_KEY_BASE64URL" +} +``` + + +See [Cloudflare's web-bot-auth documentation](https://github.com/cloudflare/web-bot-auth) for tools to generate Ed25519 key pairs. + + +### 2. Build with your key + +```bash +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --key ./my-key.jwk --upload my-web-bot-auth +``` + +### 3. Host your public key + +For websites to verify your signatures, you need to host your public key at a well-known URL. Create a key directory at: + +``` +https://yourdomain.com/.well-known/http-message-signatures-directory +``` + +The directory should contain your public keys in JWKS format: + +```json +{ + "keys": [ + { + "kty": "OKP", + "crv": "Ed25519", + "x": "YOUR_PUBLIC_KEY_BASE64URL", + "kid": "YOUR_KEY_ID" + } + ], + "purpose": "your-bot-purpose" +} +``` + +### 4. Register with Cloudflare (optional) + +If you want Cloudflare-protected sites to recognize your bot, you can register your key directory with Cloudflare: + +1. Log into the Cloudflare dashboard +2. Navigate to **Manage Account > Configurations** +3. Select the **Bot Submission Form** tab +4. Choose **Request Signature** as the verification method +5. Enter your key directory URL + +See [Cloudflare's Web Bot Auth documentation](https://developers.cloudflare.com/bots/reference/bot-verification/web-bot-auth/) for complete details. + +## References + +- [Web Bot Auth GitHub Repository](https://github.com/cloudflare/web-bot-auth) +- [Cloudflare Web Bot Auth Documentation](https://developers.cloudflare.com/bots/reference/bot-verification/web-bot-auth/) +- [RFC 9421 - HTTP Message Signatures](https://datatracker.ietf.org/doc/html/rfc9421) +- [Cloudflare Test Verification Site](https://http-message-signatures-example.research.cloudflare.com/) +- [Web Bot Auth Architecture Draft](https://thibmeu.github.io/http-message-signatures-directory/draft-meunier-web-bot-auth-architecture.html) diff --git a/reference/cli.mdx b/reference/cli.mdx index 364d175..90ec699 100644 --- a/reference/cli.mdx +++ b/reference/cli.mdx @@ -37,6 +37,9 @@ kernel --version Deploy apps, invoke actions, and stream logs. + + Upload, download, and build browser extensions. + Install Kernel MCP server configuration for AI tools. diff --git a/reference/cli/extensions.mdx b/reference/cli/extensions.mdx new file mode 100644 index 0000000..c16a44f --- /dev/null +++ b/reference/cli/extensions.mdx @@ -0,0 +1,131 @@ +--- +title: "Extensions" +--- + +Manage browser extensions for use with Kernel browsers. + +## Extension Management + +### `kernel extensions list` + +List all extensions in your organization. + +**Output includes:** +- Extension ID +- Name (if assigned) +- Created date +- Size in bytes +- Last used date + +**Example:** + +```bash +kernel extensions list +``` + +### `kernel extensions upload ` + +Upload an unpacked browser extension directory or zip file. + +| Flag | Description | +|------|-------------| +| `--name ` | Optional unique name for the extension. Must be unique within your organization. | + +**Example:** + +```bash +# Upload with auto-generated ID +kernel extensions upload ./my-extension + +# Upload with custom name +kernel extensions upload ./my-extension --name my-custom-extension +``` + + +Extensions requiring enterprise policies must include `update.xml` and `.crx` files. See [Extensions requiring enterprise policies](/browsers/extensions#uploading-extensions-requiring-enterprise-policies) for details. + + +### `kernel extensions download ` + +Download an extension archive by ID or name. + +| Flag | Description | +|------|-------------| +| `--to ` | Output path for the downloaded zip file. | + +**Example:** + +```bash +# Download to current directory +kernel extensions download my-extension + +# Download to specific path +kernel extensions download my-extension --to ./downloads/extension.zip +``` + +### `kernel extensions delete ` + +Delete an extension by ID or name. + +| Flag | Description | +|------|-------------| +| `--yes`, `-y` | Skip confirmation prompt. | + +**Example:** + +```bash +# Delete with confirmation +kernel extensions delete my-extension + +# Delete without confirmation +kernel extensions delete my-extension --yes +``` + +### `kernel extensions download-web-store ` + +Download and unpack an extension directly from the Chrome Web Store. + +| Flag | Description | +|------|-------------| +| `--to ` | Output directory for the unpacked extension. | +| `--os ` | Target OS: `mac`, `win`, or `linux` (default: `linux`). | + +**Example:** + +```bash +kernel extensions download-web-store \ + https://chromewebstore.google.com/detail/extension-id \ + --to ./my-extension +``` + +After downloading, you can upload the extension to Kernel: + +```bash +kernel extensions upload ./my-extension --name my-extension +``` + +### `kernel extensions build-web-bot-auth` +Build Cloudflare's [Web Bot Auth](/integrations/web-bot-auth) browser extension for signing HTTP requests with RFC 9421 signatures. + +| Flag | Description | +|------|-------------| +| `--to ` | Output directory for the built extension (required). | +| `--key ` | Path to JWK file with Ed25519 signing key (defaults to RFC9421 test key). | +| `--upload ` | Upload the extension to Kernel with specified name (e.g., --upload my-web-bot-auth). | + +**Examples:** + +```bash +# Build with default test key +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext + +# Build with custom key +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --key ./my-key.jwk + +# Build with custom name and upload +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --upload my-web-bot-auth +``` + + +This command requires Node.js and npm to be installed on your system. + diff --git a/testing/profile-loading-performance.md b/testing/profile-loading-performance.md new file mode 100644 index 0000000..8d6fd9e --- /dev/null +++ b/testing/profile-loading-performance.md @@ -0,0 +1,187 @@ +# Profile Loading Performance Testing Guide + +## Overview + +This guide documents the process for testing profile loading performance with Google login across SaaS sites. The goal is to understand if large profiles cause slow browser startup times and whether origin filtering could improve performance. + +## Background + +Some customers use Kernel to manage multiple SaaS accounts by: +1. Creating a synthetic Google account +2. Logging into Google once in a browser +3. Using "Login with Google" across many SaaS products +4. Saving all state in a single mega-profile +5. Reusing this profile across sessions + +This can result in large profiles with many origins and cookies. This test measures the performance impact. + +## Test Setup + +### Prerequisites + +1. **Kernel API Key**: Set `KERNEL_API_KEY` environment variable +2. **Google Account**: A test Google account for logging in +3. **Node.js/Bun**: To run the test script + +### Running the Test + +```bash +cd kernel/packages/synthetic-tests +export KERNEL_API_KEY="your-api-key" +npx tsx src/profile-load-test.ts +``` + +## Test Workflow + +The test performs these steps: + +1. **Create Test Profile** - Creates a new profile with unique name +2. **Create Browser** - Launches browser with `save_changes: true` +3. **Manual Login** - User logs into Google and Notion via live view +4. **Save Profile** - Deletes browser to persist profile data +5. **Download Profile** - Downloads profile via API +6. **Analyze Profile** - Measures size, cookies, origins +7. **Measure Startup** - Creates browsers with profile, measures timing +8. **Origin Analysis** - Calculates filtering potential + +### Manual Steps During Test + +When the test pauses, you need to: + +1. Open the Live View URL displayed in the terminal +2. Navigate to `https://accounts.google.com` +3. Log in to your test Google account +4. Navigate to `https://www.notion.so` +5. Click "Continue with Google" or "Login with Google" +6. Complete the login flow +7. Wait for Notion to load completely +8. Press Enter in the terminal to continue + +## Metrics Collected + +### Profile Analysis + +- **Total Size**: Profile JSON size in KB and bytes +- **Cookie Count**: Total cookies across all domains +- **Origin Count**: Unique origins with stored data +- **Top Origins**: Origins with most cookies/localStorage +- **Domain Breakdown**: Cookies, localStorage, IndexedDB per domain + +### Timing Measurements + +- **Browser Creation Time**: Time from API call to browser ready +- **Navigation Time**: Time to navigate to Notion +- **Total Time**: Combined time for full browser startup +- **Statistics**: Average, median, min, max across 3 iterations + +### Origin Filtering Analysis + +- **Google-related origins/cookies**: Count and percentage +- **Notion-related origins/cookies**: Count and percentage +- **Minimal profile size**: Size if only Google + Notion origins were loaded +- **Size reduction**: Percentage reduction from filtering + +## Interpreting Results + +### Profile Size + +- **< 100 KB**: Small profile, unlikely performance issues +- **100 KB - 1 MB**: Medium profile, minor impact possible +- **1 MB - 10 MB**: Large profile, likely performance impact +- **> 10 MB**: Mega-profile, significant impact expected + +### Browser Startup Time + +- **< 5 seconds**: Fast startup, no optimization needed +- **5-10 seconds**: Moderate startup, optimization may help +- **10-30 seconds**: Slow startup, optimization recommended +- **> 30 seconds**: Very slow, optimization critical + +### Origin Filtering Potential + +- **> 50% reduction**: Strong candidate for origin filtering +- **25-50% reduction**: Moderate benefit from filtering +- **< 25% reduction**: Limited benefit + +## Origin Filtering Implementation + +If testing confirms filtering would help, the recommended approach: + +### 1. API Parameter + +Add optional `origins` parameter to browser creation: + +```typescript +await kernel.browsers.create({ + profile: { + name: 'my-profile', + origins: ['google.com', 'notion.so'], + }, +}); +``` + +### 2. Backend Implementation + +In `kernel/packages/api/lib/profiles/service_cdp.go`, modify `LoadProfileIntoSession`: + +```go +// Filter cookies by allowed origins +if len(allowedOrigins) > 0 { + filteredCookies := filterCookiesByOrigins(state.Cookies, allowedOrigins) + filteredOrigins := filterOriginsByList(state.Origins, allowedOrigins) +} +``` + +### 3. Domain Matching + +Support flexible domain matching: +- `google.com` matches `.google.com`, `accounts.google.com`, etc. +- `notion.so` matches `.notion.so`, `www.notion.so`, etc. + +## Troubleshooting + +### Profile Not Saving + +- Ensure `save_changes: true` is set +- Wait at least 5 seconds after deleting browser +- Verify you created session state during login + +### Browser Startup Timeout + +- Profile may be too large +- Check network connection +- Increase timeout if necessary + +### Download Fails + +- Profile may not have saved data +- Browser must be deleted (not just closed) +- Wait longer for profile persistence + +## Test Results + +### Test Run: [Date] + +**Profile Details**: +- Size: X KB +- Cookies: X +- Origins: X + +**Timing Results**: +- Average startup: X ms +- Median startup: X ms +- Range: X ms + +**Origin Filtering Analysis**: +- Potential size reduction: X% +- Google origins: X +- Notion origins: X + +**Recommendation**: +[Based on results] + +## References + +- [Kernel Profiles Documentation](/browsers/profiles) +- [Profile API Reference](/api-reference/profiles/list-profiles) +- [Browser Creation API](/api-reference/browsers/create-a-browser)