Skip to main content

​
Observability Setup for Self-Hosted Deployments

This guide explains how to configure logging levels and enable OpenTelemetry for comprehensive observability in your self-hosted Databrain deployment.

​
Log Level Configuration

Databrain supports configurable log levels to help with troubleshooting. By default, the log level is set to info.

​
Available Log Levels

LevelDescription
errorOnly error messages
warnWarnings and errors
infoInformational messages, warnings, and errors (default)
debugAll messages including detailed debug information

​
Configuration

Add to your .env file: 
LOG_LEVEL=info
To enable debug logging for troubleshooting: 
LOG_LEVEL=debug

​
OpenTelemetry Integration

Databrain supports OpenTelemetry (OTel) for distributed tracing and metrics. This enables you to:
  • Track API latency for all endpoints
  • Monitor error rates
  • See processing time breakdowns (database queries, external API calls, etc.)
  • Correlate logs with traces for easier debugging

​
Prerequisites

To use OpenTelemetry, you need an OTLP-compatible collector or backend. Popular options include:

​
Configuration

Add these environment variables to your .env file: 
# Enable OpenTelemetry (disabled by default)
OTEL_ENABLED=true

# Your OTLP collector endpoint
OTEL_EXPORTER_OTLP_ENDPOINT=http://your-collector:4318

# Service name for traces (optional, defaults to databrain-api)
OTEL_SERVICE_NAME=databrain-api

​
Docker Compose Setup

If you want to run an OpenTelemetry Collector alongside Databrain, uncomment the collector service in your docker-compose.yml: 
otel-collector:
  image: otel/opentelemetry-collector-contrib:latest
  restart: always
  ports:
    - "4317:4317"   # OTLP gRPC receiver
    - "4318:4318"   # OTLP HTTP receiver
  volumes:
    - ./otel-config.yaml:/etc/otelcol/config.yaml
  networks:
    - datab
Then update your backend environment: 
OTEL_ENABLED=true
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318

​
Example: Collector Configuration for Jaeger

Create an otel-config.yaml file: 
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 1s
    send_batch_size: 1024

exporters:
  jaeger:
    endpoint: jaeger:14250
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger]

​
What Gets Instrumented

When OpenTelemetry is enabled, Databrain automatically instruments:
ComponentWhat’s Tracked
HTTP RequestsIncoming API requests with latency, status codes
Express RoutesRoute-level timing and errors
PostgreSQLDatabase query duration and errors
RedisCache operations timing
Outgoing HTTPExternal API calls (Hasura, Keycloak, etc.)

​
Example Trace

A typical API request trace shows: 
GET /api/v2/metric/execute (1,247ms)
├── middleware.auth (12ms)
│   └── HTTP POST keycloak/auth/verify (10ms)
├── getMetricConfig (45ms)
│   └── HTTP POST hasura/v1/graphql (43ms)
├── buildQuery (8ms)
├── executeQuery (1,150ms)
│   └── PostgreSQL Query (1,148ms)
└── formatResponse (32ms)

​
Troubleshooting

​
Logs Not Appearing at Expected Level

  1. Verify LOG_LEVEL is set correctly in your .env
  2. Restart the backend service after changing the value
  3. Log levels are case-insensitive (DEBUG, debug, Debug all work)

​
OpenTelemetry Not Working

  1. Verify OTEL_ENABLED=true (must be exactly true)
  2. Check that OTEL_EXPORTER_OTLP_ENDPOINT is reachable from the backend container
  3. Check backend logs for [Telemetry] OpenTelemetry initialized message
  4. If you see OTEL_ENABLED=true but OTEL_EXPORTER_OTLP_ENDPOINT not set, configure the endpoint

​
High Overhead Concerns

OpenTelemetry adds minimal overhead when enabled. If you experience performance issues:
  1. Ensure your collector can handle the volume
  2. Consider sampling in the collector configuration
  3. Disable OTel temporarily by setting OTEL_ENABLED=false

​
Structured Logging

Databrain uses Winston for structured, JSON-formatted logging. All logs include:
  • Timestamp: ISO format with milliseconds
  • Level: error, warn, info, or debug
  • Message: Human-readable description
  • Metadata: Structured data as key-value pairs
  • Trace Context: When OpenTelemetry is enabled, includes trace_id and span_id

​
Example Log Output

{
  "level": "info",
  "message": "API request completed",
  "timestamp": "2026-02-03 10:30:45.123",
  "method": "POST",
  "url": "/api/v2/metric/execute",
  "statusCode": 200,
  "duration": 1247,
  "userId": "123",
  "trace_id": "1a2b3c4d5e6f7g8h",
  "span_id": "9i0j1k2l3m4n"
}

​
Usage in Code

import logger from 'utils/logger';

// Good - structured logging
logger.info('User logged in', { userId: '123', method: 'oauth' });
logger.error('Database error', { error: err.message, query: sqlQuery });

// Avoid - unstructured logging
console.log('User 123 logged in'); // ❌ Don't use console.log

​
Default Behavior (No Configuration Required)

If you don’t configure any observability settings:
  • Log level defaults to info
  • OpenTelemetry is disabled (zero overhead)
  • Logs output to stdout in JSON format
  • Structured logging with Winston is always enabled
This maintains backward compatibility with existing deployments.