Integrating Databrain with SigNoz

This guide explains how to send OpenTelemetry traces, metrics, and logs from your self-hosted Databrain instance to SigNoz, an open-source observability platform.

Why SigNoz?

SigNoz is a great choice for self-hosted observability:

100% Open Source: No vendor lock-in
OpenTelemetry Native: Built specifically for OTel
All-in-One: Traces, metrics, and logs in a single platform
Cost-Effective: Self-hosted means no per-event pricing
Easy to Deploy: Docker Compose or Kubernetes deployment

Prerequisites

Databrain self-hosted version with OpenTelemetry support
Docker and Docker Compose (for SigNoz installation)
4GB+ RAM for SigNoz (8GB recommended for production)

Option 1: SigNoz Cloud (Easiest)

If you prefer a managed solution, use SigNoz Cloud:

Go to https://signoz.io/teams/
Create a free account
Note your ingestion endpoint and ingestion key

2. Configure Databrain

# Enable OpenTelemetry
OTEL_ENABLED=true

# SigNoz Cloud endpoint (adjust for your region)
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.{region}.signoz.cloud:443

# SigNoz ingestion key
OTEL_EXPORTER_OTLP_HEADERS=signoz-access-token=YOUR_INGESTION_KEY

# Service name
OTEL_SERVICE_NAME=databrain-api

# Optional: Enable debug logging
LOG_LEVEL=info

3. Docker Compose Configuration

services:
  databrainbackend:
    environment:
      OTEL_ENABLED: "true"
      OTEL_EXPORTER_OTLP_ENDPOINT: "https://ingest.{region}.signoz.cloud:443"
      OTEL_SERVICE_NAME: "databrain-api"
      OTEL_EXPORTER_OTLP_HEADERS: "signoz-access-token=${SIGNOZ_INGESTION_KEY}"
      LOG_LEVEL: "info"

Option 2: Self-Hosted SigNoz (Recommended for Production)

1. Install SigNoz

Using Docker Compose (Quick Start)

# Clone SigNoz repository
git clone -b main https://github.com/SigNoz/signoz.git
cd signoz/deploy/

# Start SigNoz
docker compose -f docker/clickhouse-setup/docker-compose.yaml up -d

This starts:

SigNoz UI: http://localhost:3301
OTLP Receiver (gRPC): localhost:4317
OTLP Receiver (HTTP): localhost:4318
ClickHouse: For data storage
Query Service: For querying data

Using Docker Compose (Custom Network)

If Databrain is on the same Docker network:

# signoz-docker-compose.yaml
version: "3.8"

networks:
  databrain:
    external: true
  signoz:
    name: signoz

services:
  clickhouse:
    image: clickhouse/clickhouse-server:23.7.3-alpine
    container_name: signoz-clickhouse
    volumes:
      - ./clickhouse-config.xml:/etc/clickhouse-server/config.d/config.xml
      - ./clickhouse-users.xml:/etc/clickhouse-server/users.d/users.xml
      - signoz-data:/var/lib/clickhouse/
    networks:
      - signoz
    restart: on-failure

  otel-collector:
    image: signoz/signoz-otel-collector:0.88.11
    container_name: signoz-otel-collector
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - "4317:4317"  # OTLP gRPC receiver
      - "4318:4318"  # OTLP HTTP receiver
    networks:
      - signoz
      - databrain
    depends_on:
      - clickhouse
    restart: on-failure

  query-service:
    image: signoz/query-service:0.38.0
    container_name: signoz-query-service
    command: ["-config=/root/config/prometheus.yml"]
    volumes:
      - ./prometheus.yml:/root/config/prometheus.yml
    environment:
      - ClickHouseUrl=tcp://clickhouse:9000
    networks:
      - signoz
    depends_on:
      - clickhouse
    restart: on-failure

  frontend:
    image: signoz/frontend:0.38.0
    container_name: signoz-frontend
    ports:
      - "3301:3301"
    environment:
      - FRONTEND_API_ENDPOINT=http://query-service:8080
    networks:
      - signoz
    depends_on:
      - query-service
    restart: on-failure

volumes:
  signoz-data:

2. Configure Databrain for Self-Hosted SigNoz

Same Docker Network

# docker-compose.yml
services:
  databrainbackend:
    environment:
      OTEL_ENABLED: "true"
      OTEL_EXPORTER_OTLP_ENDPOINT: "http://signoz-otel-collector:4318"
      OTEL_SERVICE_NAME: "databrain-api"
      LOG_LEVEL: "info"
    networks:
      - databrain
      - signoz

networks:
  databrain:
  signoz:
    external: true

Separate Hosts

If SigNoz is on a different host:

# Enable OpenTelemetry
OTEL_ENABLED=true

# SigNoz OTLP endpoint (adjust host)
OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-host:4318

# Service name
OTEL_SERVICE_NAME=databrain-api

3. Kubernetes Deployment

Deploy SigNoz on Kubernetes

# Add SigNoz Helm repo
helm repo add signoz https://charts.signoz.io
helm repo update

# Create namespace
kubectl create namespace signoz

# Install SigNoz
helm install signoz signoz/signoz -n signoz \
  --set clickhouse.persistence.size=50Gi \
  --set frontend.ingress.enabled=true \
  --set frontend.ingress.hosts[0].host=signoz.your-domain.com

Configure Databrain

apiVersion: apps/v1
kind: Deployment
metadata:
  name: databrain-backend
spec:
  template:
    spec:
      containers:
      - name: backend
        env:
          - name: OTEL_ENABLED
            value: "true"
          - name: OTEL_EXPORTER_OTLP_ENDPOINT
            value: "http://signoz-otel-collector.signoz.svc.cluster.local:4318"
          - name: OTEL_SERVICE_NAME
            value: "databrain-api"
          - name: LOG_LEVEL
            value: "info"

What Gets Sent to SigNoz

Telemetry Type	Description
Traces	Full distributed traces with all spans
Metrics	Request rates, latency histograms, error rates
Logs	Structured logs with trace correlation

All three are integrated in SigNoz’s unified interface.

Verification

1. Check SigNoz is Running

# Check containers
docker ps | grep signoz

# Check OTLP receiver is listening
curl http://localhost:4318/v1/traces
# Should return 405 Method Not Allowed (endpoint exists but expects POST)

2. Restart Databrain

docker compose restart databrainbackend

3. Generate Test Traffic

# Health check
curl https://your-databrain-instance.com/api/health

# Sample request
curl -X POST "https://your-databrain-instance.com/api/v2/metric/execute" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{"metricId": "test-123"}'

4. View in SigNoz UI

Open SigNoz: http://localhost:3301 (or your configured URL)
Go to Services → You should see databrain-api
Click on service to see metrics:
- Request rate (requests per second)
- Error rate (%)
- Latency (p50, p90, p95, p99)
- Apdex score
Go to Traces → Filter by serviceName=databrain-api
Click on any trace to see the full waterfall view

5. Check Logs

Go to Logs tab
Filter: service_name = databrain-api
Click on any log entry
See Related Traces section for correlated traces

SigNoz Features

1. Service Dashboard

The service dashboard shows:

RED metrics: Rate, Errors, Duration
Apdex score: User satisfaction metric
Operations: Individual endpoint performance
Database calls: Query performance breakdown
External calls: Downstream service latency

2. Traces Explorer

Advanced trace filtering:

serviceName = databrain-api
AND duration > 1000ms
AND statusCode >= 500

Available filters:

Service name
Operation name
HTTP status code
Duration
Error status
Custom tags (userId, metricId, etc.)

3. Service Map

Visualize service dependencies:

Go to Service Map
See databrain-api and its dependencies:
- PostgreSQL
- Redis
- Hasura
- Keycloak
- S3
Click on connections to see request rates and error rates

4. Logs Management

SigNoz provides powerful log querying: Query examples:

service_name = databrain-api AND level = error
service_name = databrain-api AND userId = 123
service_name = databrain-api AND duration > 1000

Log to trace correlation:

Click on any log → see Trace button
Jump directly to the trace that generated this log

5. Alerts

Create alerts in SigNoz:

High Error Rate Alert

Go to Alerts → New Alert
Alert Type: Metrics Alert
Metric Query:

A: error_count = count(signoz_latency_bucket{service_name="databrain-api", status_code=~"5.."})
B: total_count = count(signoz_latency_bucket{service_name="databrain-api"})
Formula: (A / B) * 100

Condition: > 5 (5% error rate)
For: 5 minutes
Add notification channel (Slack, email, webhook)

High Latency Alert

Metric Query:

histogram_quantile(0.95, 
  sum by (le)(rate(signoz_latency_bucket{service_name="databrain-api"}[5m]))
)

Condition: > 2000 (2 seconds)
For: 5 minutes

6. Dashboards

Create custom dashboards:

Go to Dashboards → New Dashboard
Add panels:

Request Rate Panel:

rate(signoz_calls_total{service_name="databrain-api"}[5m])

Error Rate Panel:

rate(signoz_calls_total{service_name="databrain-api", status_code=~"5.."}[5m])
/
rate(signoz_calls_total{service_name="databrain-api"}[5m])

Latency Distribution Panel:

histogram_quantile(0.50, sum(rate(signoz_latency_bucket[5m])) by (le))
histogram_quantile(0.95, sum(rate(signoz_latency_bucket[5m])) by (le))
histogram_quantile(0.99, sum(rate(signoz_latency_bucket[5m])) by (le))

Database Query Latency:

histogram_quantile(0.95,
  sum(rate(signoz_latency_bucket{
    service_name="databrain-api",
    db_system="postgresql"
  }[5m])) by (le)
)

Advanced Configuration

Sampling

Add sampling to the OTel collector config:

# otel-collector-config.yaml
processors:
  # Tail-based sampling
  tail_sampling:
    decision_wait: 10s
    num_traces: 100
    policies:
      # Always keep errors
      - name: error-policy
        type: status_code
        status_code:
          status_codes: [ERROR]
      
      # Keep slow requests
      - name: latency-policy
        type: latency
        latency:
          threshold_ms: 1000
      
      # Sample 10% of normal traffic
      - name: probabilistic-policy
        type: probabilistic
        probabilistic:
          sampling_percentage: 10

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [tail_sampling, batch]
      exporters: [clickhouse]

Resource Attributes

Add custom resource attributes:

processors:
  resource:
    attributes:
      - key: deployment.environment
        value: production
        action: upsert
      - key: cloud.provider
        value: aws
        action: upsert
      - key: cloud.region
        value: us-east-1
        action: upsert

Troubleshooting

Issue	Solution
No data in SigNoz	Check OTel collector logs: `docker logs signoz-otel-collector`
Cannot connect to SigNoz	Verify networks and firewall rules
Missing logs	Ensure Winston is configured with JSON format
High disk usage	Adjust ClickHouse retention settings
Slow queries	Increase ClickHouse resources or add indexes

Debug SigNoz Collector

# Check collector logs
docker logs signoz-otel-collector -f

# Check for OTLP receiver
docker logs signoz-otel-collector | grep "OTLP receiver"

# Should see:
# OTLP receiver started on 0.0.0.0:4317
# OTLP receiver started on 0.0.0.0:4318

Debug Databrain Connection

# Test connectivity from Databrain container
docker exec databrain-backend curl -v http://signoz-otel-collector:4318/v1/traces

Check ClickHouse

# Connect to ClickHouse
docker exec -it signoz-clickhouse clickhouse-client

# Check traces table
SELECT count() FROM signoz_traces.signoz_index_v2;

# Check for recent traces
SELECT timestamp, serviceName, name FROM signoz_traces.signoz_index_v2 
ORDER BY timestamp DESC LIMIT 10;

Data Retention

Configure retention in ClickHouse:

-- Connect to ClickHouse
docker exec -it signoz-clickhouse clickhouse-client

-- Set 30-day retention for traces
ALTER TABLE signoz_traces.signoz_index_v2 
MODIFY TTL toDateTime(timestamp) + INTERVAL 30 DAY;

-- Set 90-day retention for metrics
ALTER TABLE signoz_metrics.distributed_samples_v2 
MODIFY TTL toDateTime(timestamp) + INTERVAL 90 DAY;

Or configure in clickhouse-config.xml:

<clickhouse>
  <merge_tree>
    <ttl_only_drop_parts>1</ttl_only_drop_parts>
  </merge_tree>
</clickhouse>

Performance Tuning

For High Traffic (>10K requests/minute)

Increase ClickHouse resources:

clickhouse:
  resources:
    limits:
      memory: 8Gi
      cpu: 4

Batch processing:

processors:
  batch:
    timeout: 10s
    send_batch_size: 10000

Use buffer:

processors:
  memory_limiter:
    limit_mib: 512
    spike_limit_mib: 128
    check_interval: 5s

Backup and Restore

Backup ClickHouse Data

# Stop SigNoz
docker compose down

# Backup data directory
tar -czf signoz-backup-$(date +%Y%m%d).tar.gz \
  ./deploy/docker/clickhouse-setup/data/

# Restart SigNoz
docker compose up -d

Restore from Backup

# Stop SigNoz
docker compose down

# Restore data
tar -xzf signoz-backup-20260203.tar.gz -C ./deploy/docker/clickhouse-setup/

# Restart
docker compose up -d

Upgrading SigNoz

cd signoz/deploy/

# Pull latest images
docker compose -f docker/clickhouse-setup/docker-compose.yaml pull

# Restart with new images
docker compose -f docker/clickhouse-setup/docker-compose.yaml up -d

Community & Support

Documentation: https://signoz.io/docs/
Slack Community: https://signoz.io/slack
GitHub: https://github.com/SigNoz/signoz
GitHub Discussions: https://github.com/SigNoz/signoz/discussions

For Databrain configuration issues, contact your Databrain support team.

​Integrating Databrain with SigNoz

​Why SigNoz?

​Prerequisites

​Option 1: SigNoz Cloud (Easiest)

​1. Sign Up for SigNoz Cloud

​2. Configure Databrain

​3. Docker Compose Configuration

​Option 2: Self-Hosted SigNoz (Recommended for Production)

​1. Install SigNoz

​Using Docker Compose (Quick Start)

​Using Docker Compose (Custom Network)

​2. Configure Databrain for Self-Hosted SigNoz

​Same Docker Network

​Separate Hosts

​3. Kubernetes Deployment

​Deploy SigNoz on Kubernetes

​Configure Databrain

​What Gets Sent to SigNoz

​Verification

​1. Check SigNoz is Running

​2. Restart Databrain

​3. Generate Test Traffic

​4. View in SigNoz UI

​5. Check Logs

​SigNoz Features

​1. Service Dashboard

​2. Traces Explorer

​3. Service Map

​4. Logs Management

​5. Alerts

​High Error Rate Alert

​High Latency Alert

​6. Dashboards

​Advanced Configuration

​Sampling

​Resource Attributes

​Troubleshooting

​Debug SigNoz Collector

​Debug Databrain Connection

​Check ClickHouse

​Data Retention

​Performance Tuning

​For High Traffic (>10K requests/minute)

​Backup and Restore

​Backup ClickHouse Data

​Restore from Backup

​Upgrading SigNoz

​Community & Support