How to set up Telemetry Gateway locally

WARNING: This is a guide intended for development reference.

To learn more about telemetry export, refer to the Sourcegraph adminstrator documentation on telemetry.

Telemetry Gateway is a managed service that ingests events exported from Sourcegraph instances, manipulates them as needed, and exports them to designated Pub/Sub topics or other destinations for processing.

It exposes a gRPC API defined in telemetrygateway/v1, and the service itself is implemented in cmd/telemetry-gateway.

To learn more about the Sourcegraph's new Telemetry framework, refer to the telemetry documentation.

NOTE: In the Sourcegraph application, the new events being exported using internal/telemetry are sometimes loosely referred to as "V2", as it supersedes the existing mechanisms of writing directly to the event_logs database table. The Telemetry Gateway schema, however, is telemetrygateway/v1, as it is the first iteration of the service's API.

Running Telemetry Gateway locally

Exports of telemetry events to a locally running Telemetry Gateway instance that is started as part of sg start and sg start dotcom are enabled by default in development using the TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR environment variable configured in sg.config.yaml. By default, the local Telemetry Gateway instance will simply log any events it receives at debug level without forwarding the events anywhere.

To see the message payloads it would emit in a production environment, configure the log level in sg.config.overwrite.yaml:

commands:
  telemetry-gateway:
    env:
      SRC_LOG_LEVEL: debug

You can increase the frequency of exports to monitor behaviour closer to real-time by setting the following in sg.config.yaml:

env:
  TELEMETRY_GATEWAY_EXPORTER_EXPORT_INTERVAL: "10s"

In development, a gRPC interface is enabled for Telemetry Gateway as well at http://127.0.0.1:10085/debug/grpcui/.

Testing against a remote Telemetry Gateway

A test deployment is available at telemetry-gateway.sgdev.org, which publishes events to a test topic and development pipeline - currently sourcegraph-telligent-testing/event-telemetry-test. In local development, you can configure Sourcegraph to export to this test deployment by setting the following in sg.config.yaml:

env:
  TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR: "https://telemetry-gateway.sgdev.org:443"

For details about live Telemetry Gateway deployments, refer to the handbook Telemetry Gateway page.