Skip to content

Conversation

@raiden-staging
Copy link
Contributor

@raiden-staging raiden-staging commented Jan 13, 2026

telemetry for neko client

usage

# default | set telemetry endpoint to enable

TELEMETRY_ENDPOINT=https://api.com/telemetry/endpoint \
  IMAGE=kernel-docker WITH_KERNEL_IMAGES_API=true ENABLE_WEBRTC=true \
  ./run-docker.sh

# filter data points | comma-separated fields
# specific events   : error_js,perf_fps
# wildcard matching : error_*

TELEMETRY_CAPTURE='error_*,perf_fps,connection_webrtc_failed' \
  TELEMETRY_ENDPOINT=https://api.com/telemetry/endpoint \
  IMAGE=kernel-docker WITH_KERNEL_IMAGES_API=true ENABLE_WEBRTC=true \
  ./run-docker.sh

data points

#### available data
- session_start: Session initiated with device/browser info
- session_end: Session ended (page unload)
- session_heartbeat: Periodic heartbeat (every 60s)
- app_init: App initialization started
- app_ready: Vue app mounted and ready
- app_visible: Page became visible
- app_hidden: Page became hidden
- app_beforeunload: Page about to unload

- error_js: Uncaught JavaScript error
- error_unhandled_rejection: Unhandled Promise rejection
- error_vue: Vue component error
- error_network: Fetch/XHR/resource loading error
- error_websocket: WebSocket error
- error_webrtc: WebRTC error

- perf_page_load: Navigation timing metrics
- perf_first_contentful_paint: FCP metric
- perf_largest_contentful_paint: LCP metric
- perf_first_input_delay: FID metric
- perf_cumulative_layout_shift: CLS metric
- perf_long_task: Tasks blocking main thread >50ms
- perf_memory: JS heap usage (every 30s)
- perf_fps: Frame rate stats (every 10s)

- connection_websocket_open: WebSocket opened
- connection_websocket_close: WebSocket closed
- connection_websocket_error: WebSocket error
- connection_webrtc_connecting: WebRTC connecting
- connection_webrtc_connected: WebRTC connected
- connection_webrtc_disconnected: WebRTC disconnected
- connection_webrtc_failed: WebRTC connection failed
- connection_webrtc_ice_state: ICE state change
- connection_reconnecting: Reconnection attempt
- connection_timeout: Connection timeout

- action_resolution_change: Resolution changed # triggers on initial resolution setup
- action_login: User logged in # (default neko client->server auth)
- action_logout: User logged out # (default neko client->server auth)

#### disabled in client , unused :
- action_control_request: Control requested
- action_control_release: Control released
- action_fullscreen: Fullscreen toggled
- action_pip: Picture-in-picture toggled
- stream_track_added: Media track added
- stream_play_started: Video playback started
- stream_play_failed: Video playback failed
- stream_quality_change: Stream quality changed
- action_volume_change: Volume/mute changed # audio support not merged yet

api endpoint payload

example from local docker run

{
  "batchId": "mkc3pb62-27jgux5qy",
  "events": [
    {
      "eventId": "mkc3pauc-9bpqwzkcz",
      "eventType": "session_start",
      "severity": "info",
      "timestamp": 1768279070244,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 1,
      "data": {
        "session": {
          "sessionId": "mkc3patd-axs7otq1v",
          "startTime": 1768279070209,
          "pageUrl": "http://localhost:8080/",
          "referrer": "",
          "device": {
            "userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36",
            "platform": "Linux x86_64",
            "language": "en-US",
            "languages": [
              "en-US",
              "en"
            ],
            "cookiesEnabled": true,
            "doNotTrack": null,
            "hardwareConcurrency": 8,
            "maxTouchPoints": 0,
            "deviceMemory": 8,
            "connection": {
              "effectiveType": "4g",
              "downlink": 10,
              "rtt": 50,
              "saveData": false
            }
          },
          "screen": {
            "width": 1536,
            "height": 776,
            "availWidth": 1536,
            "availHeight": 776,
            "colorDepth": 24,
            "pixelRatio": 1,
            "orientation": "landscape-primary"
          },
          "viewport": {
            "width": 981,
            "height": 689
          },
          "browser": {
            "name": "Chrome",
            "version": "143.0",
            "engine": "Blink",
            "webrtcSupported": true,
            "webglSupported": false,
            "cookiesEnabled": true
          },
          "timezone": "UTC",
          "timezoneOffset": 0
        }
      }
    },
    {
      "eventId": "mkc3paud-i307pegcb",
      "eventType": "app_init",
      "severity": "info",
      "timestamp": 1768279070245,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 2
    },
    {
      "eventId": "mkc3pave-lsxjmkwty",
      "eventType": "app_ready",
      "severity": "info",
      "timestamp": 1768279070282,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 3,
      "data": {
        "mountTime": 1768279070282
      }
    },
    {
      "eventId": "mkc3paw9-rpmqfufp8",
      "eventType": "action_login",
      "severity": "info",
      "timestamp": 1768279070313,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 4,
      "data": {
        "displayname": "kernel",
        "timestamp": 1768279070313
      }
    },
    {
      "eventId": "mkc3pax3-4fnm83lm9",
      "eventType": "perf_long_task",
      "severity": "info",
      "timestamp": 1768279070343,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 5,
      "data": {
        "duration": 162,
        "startTime": 155.89999997615814,
        "containerType": "window",
        "containerSrc": "",
        "containerId": "",
        "containerName": ""
      }
    },
    {
      "eventId": "mkc3pay1-e2g78op3s",
      "eventType": "action_resolution_change",
      "severity": "info",
      "timestamp": 1768279070377,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 6,
      "data": {
        "width": 1920,
        "height": 1080,
        "rate": 25,
        "timestamp": 1768279070377
      }
    },
    {
      "eventId": "mkc3pay2-plos0o09d",
      "eventType": "connection_webrtc_connected",
      "severity": "info",
      "timestamp": 1768279070378,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 7,
      "data": {
        "serverInitComplete": true,
        "implicitHosting": true,
        "fileTransferEnabled": false,
        "heartbeatInterval": 10,
        "locksConfigured": 0,
        "timestamp": 1768279070378
      }
    },
    {
      "eventId": "mkc3pazx-7wz2qkm9d",
      "eventType": "perf_page_load",
      "severity": "info",
      "timestamp": 1768279070445,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 8,
      "performance": {
        "domContentLoaded": 313.89999997615814,
        "domComplete": 336.0999999642372,
        "loadEventEnd": 336.30000001192093,
        "ttfb": 3,
        "resourceCount": 8,
        "transferSize": 300,
        "usedJSHeapSize": 6454737,
        "totalJSHeapSize": 8600853,
        "jsHeapSizeLimit": 2248146944
      }
    },
    {
      "eventId": "mkc3pb56-bi5rkbc46",
      "eventType": "connection_webrtc_ice_state",
      "severity": "info",
      "timestamp": 1768279070634,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 9,
      "connection": {
        "state": "checking",
        "iceConnectionState": "checking"
      }
    },
    {
      "eventId": "mkc3pb62-o691807d4",
      "eventType": "connection_webrtc_ice_state",
      "severity": "info",
      "timestamp": 1768279070666,
      "sessionId": "mkc3patd-axs7otq1v",
      "sequenceNumber": 10,
      "connection": {
        "state": "connected",
        "iceConnectionState": "connected"
      }
    }
  ],
  "session": {
    "sessionId": "mkc3patd-axs7otq1v",
    "startTime": 1768279070209,
    "pageUrl": "http://localhost:8080/",
    "referrer": "",
    "device": {
      "userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36",
      "platform": "Linux x86_64",
      "language": "en-US",
      "languages": [
        "en-US",
        "en"
      ],
      "cookiesEnabled": true,
      "doNotTrack": null,
      "hardwareConcurrency": 8,
      "maxTouchPoints": 0,
      "deviceMemory": 8,
      "connection": {
        "effectiveType": "4g",
        "downlink": 10,
        "rtt": 50,
        "saveData": false
      }
    },
    "screen": {
      "width": 1536,
      "height": 776,
      "availWidth": 1536,
      "availHeight": 776,
      "colorDepth": 24,
      "pixelRatio": 1,
      "orientation": "landscape-primary"
    },
    "viewport": {
      "width": 981,
      "height": 689
    },
    "browser": {
      "name": "Chrome",
      "version": "143.0",
      "engine": "Blink",
      "webrtcSupported": true,
      "webglSupported": false,
      "cookiesEnabled": true
    },
    "timezone": "UTC",
    "timezoneOffset": 0
  },
  "sentAt": 1768279070666,
  "retryCount": 0
}

next

  • setup kernel prod api telemetry endpoint
  • sync kernel prod api generated session_ids with client telemetry session_ids

[ @rgarcia ]


Note

Adds a production-ready telemetry system with runtime config and broad instrumentation across the client.

  • New telemetry module (src/telemetry/*) with service, types, and collectors: errors, performance (incl. Core Web Vitals/FPS/memory), and connection
  • Vue plugin plugins/telemetry initializes telemetry early, exposes $telemetry, and installs collectors; installTelemetryConnectionCollector hooks into NekoClient after init
  • Instrumentation: video.vue emits telemetry for playback, volume, PiP, fullscreen, and stream events; neko/index.ts tracks system init, disconnects, errors, and resolution changes; plugins/neko.ts forwards client errors/info/debug to telemetry and sets server info
  • Runtime config: load telemetry-config.js from public/index.html (populated at startup) with only two flags: TELEMETRY_ENDPOINT (enables telemetry) and TELEMETRY_CAPTURE (filter); query params supported for testing
  • Container integration: wrapper.sh generates telemetry-config.js; run-docker.sh and run-unikernel.sh pass TELEMETRY_* env vars
  • Build-time support: vue.config.js defines VUE_APP_TELEMETRY_* for optional build-time configuration
  • App bootstrap: main.ts installs telemetry before other plugins and attaches the connection collector after $client init

Written by Cursor Bugbot for commit 61b0ed5. This will update automatically on new commits. Configure here.

@tembo
Copy link
Contributor

tembo bot commented Jan 13, 2026

A few things to consider in the new telemetry wiring:

  • images/chromium-headful/client/src/plugins/telemetry.ts: query params currently have highest priority and can enable telemetry + send data to an arbitrary telemetry_endpoint. That’s great for local testing, but in prod it’s an easy footgun for data exfiltration (anyone who can influence the URL can redirect telemetry off-domain). I’d gate query-param overrides to dev/localhost and/or validate the endpoint (scheme + allowlist).
  // 3. Query parameters (highest priority)
  // Consider only allowing this in local/dev to avoid arbitrary endpoint overrides in production.
  const isDev = process.env.NODE_ENV !== 'production'
  if (isDev) {
    const urlParams = new URLSearchParams(window.location.search)
    const paramEndpoint = urlParams.get('telemetry_endpoint')
    const paramCapture = urlParams.get('telemetry_capture')

    if (paramEndpoint) endpoint = paramEndpoint
    if (paramCapture) captureSpec = paramCapture
  }
  • images/chromium-headful/wrapper.sh: the generated telemetry-config.js escapes single quotes in endpoint/capture, but the values are also written unescaped into // TELEMETRY_ENDPOINT=... / // TELEMETRY_CAPTURE=... comment lines. A newline (or other control chars) in env could break the JS file and potentially inject code. Even if env is “trusted”, it’s cheap to harden: either drop those comment lines or escape \, \r, \n, and any other JS-special chars consistently.

  • images/chromium-headful/client/src/telemetry/collectors/performance.ts: the severity threshold ordering looks inverted, so >90 will never produce 'error'.

        const severity = usagePercent > 90 ? 'error' : usagePercent > 80 ? 'warning' : 'debug'
  • images/chromium-headful/client/src/telemetry/collectors/performance.ts: startFPSMonitoring() uses a setInterval(...) that isn’t stored/cleared in uninstall(), so it will keep running after teardown.

  • images/chromium-headful/client/src/telemetry/collectors/errors.ts: installUnhandledRejectionHandler() adds an event listener but also calls the existing window.onunhandledrejection handler manually; browsers will also call the property handler, so this can double-invoke the original handler. Also, uninstall() doesn’t remove the unhandledrejection/resource error listeners.

  • images/chromium-headful/client/src/components/video.vue: ended currently emits stream_play_started with { ended: true } which reads like a copy/paste typo (there’s already a stream_play_started tracked on playing). Either track a dedicated “ended” event or drop this.

  • images/chromium-headful/client/src/plugins/neko.ts: parsing log strings (args.join(' ')) to infer connection state/URL is pretty brittle and may generate noisy telemetry (and duplicates what ConnectionCollector is already tracking via client events). If possible, prefer hooking into structured events/fields instead of log text.

  • images/chromium-headful/run-docker.sh and images/chromium-headful/run-unikernel.sh: printing the full telemetry endpoint to stdout could leak secrets if the endpoint ever carries tokens in the URL; consider redacting before logging.

this._video.addEventListener('ended', () => {
this.$accessor.video.setPlayable(false)
getTelemetry().track('stream_play_started', 'info', { ended: true })
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong telemetry event for video ended handler

Medium Severity

The video ended event handler tracks stream_play_started which is semantically incorrect. When a video finishes playing, the telemetry reports that playback started, which produces misleading data. This appears to be a copy-paste error or incorrect event type selection.

Fix in Cursor Fix in Web

const usagePercent = Math.round((memory.usedJSHeapSize / memory.jsHeapSizeLimit) * 100)

// Warn if memory usage is high
const severity = usagePercent > 80 ? 'warning' : usagePercent > 90 ? 'error' : 'debug'
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memory severity threshold order prevents error detection

Medium Severity

The ternary chain usagePercent > 80 ? 'warning' : usagePercent > 90 ? 'error' : 'debug' has incorrect threshold ordering. Since > 80 is checked first, any value above 80 (including 95%) returns 'warning', and the > 90 condition for 'error' is never reached. The thresholds need to be checked in descending order: > 90 first, then > 80.

Fix in Cursor Fix in Web

sampleCount: this.fpsValues.length,
})
}
}, 10000)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FPS reporting interval is never cleared on uninstall

Medium Severity

The setInterval for FPS reporting is never assigned to a variable, so it cannot be cleared in uninstall(). Unlike memoryMonitorId which is properly stored and cleared, this interval continues running indefinitely even after the collector is uninstalled, causing a resource leak and potentially continuing to call telemetry methods on a destroyed service.

Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant