log_to_metric does not support aggregated_histogram metrics output by vector itself

[oivindoh]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Vector's internal metrics emits metrics with the "aggregated_histogram" type. As part of my pipeline I convert these metrics to logs to be able to ship them to a remote http_server vector source.

This might not be the optimal setup when one could use a native vector source instead, but it has uncovered that log_to_metric emits an ERROR when it encounters logs with the aggregated_histogram property, as discussed here.

Relevant log

2024-05-14T10:45:01.277458Z ERROR transform{component_kind="transform" component_id=logs_to_metrics component_type=log_to_metric}: vector::internal_events::log_to_metric: Internal log [Missing required metric details. Required one of gauge, distribution, histogram, summary, counter] has been suppressed 279 times.
2024-05-14T10:45:01.277528Z ERROR transform{component_kind="transform" component_id=logs_to_metrics component_type=log_to_metric}: vector::internal_events::log_to_metric: Missing required metric details. Required one of gauge, distribution, histogram, summary, counter error_code="missing_metric_details" error_type="parser_failed" stage="processing" internal_log_rate_limit=true
2024-05-14T10:45:01.277585Z ERROR transform{component_kind="transform" component_id=logs_to_metrics component_type=log_to_metric}: vector_common::internal_event::component_events_dropped: Internal log [Events dropped] has been suppressed 279 times.
2024-05-14T10:45:01.278201Z ERROR transform{component_kind="transform" component_id=logs_to_metrics component_type=log_to_metric}: vector_common::internal_event::component_events_dropped: Events dropped intentional=false count=1 reason="Missing required metric details. Required one of gauge, distribution, histogram, summary, counter" internal_log_rate_limit=true
2024-05-14T10:45:01.278310Z ERROR transform{component_kind="transform" component_id=logs_to_metrics component_type=log_to_metric}: vector::internal_events::log_to_metric: Internal log [Missing required metric details. Required one of gauge, distribution, histogram, summary, counter] is being suppressed to avoid flooding.
2024-05-14T10:45:01.278348Z ERROR transform{component_kind="transform" component_id=logs_to_metrics component_type=log_to_metric}: vector_common::internal_event::component_events_dropped: Internal log [Events dropped] is being suppressed to avoid flooding.

Currently I work around this problem simply by using a route transform that matchese these events, and then using log_to_metric on that route's _unmatched route.

Configuration

sources:
  logs_only_receiver:
    type: http_server
    address: 0.0.0.0:8081
    decoding:
      codec: json

  vector_metrics:
    type: internal_metrics

transforms:
  metrics_to_logs:
    type: metric_to_log
    inputs: [ vector_metrics ]

  # Drop aggregated histogram events because log_to_metric does not support it as of 0.38.1
  drop_route:
    type: route
    route:
      aggregated_histogram: .aggregated_histogram != null
    inputs:
      - logs_only_receiver

  logs_to_metrics:
    type: log_to_metric
    metrics: []
    all_metrics: true
    inputs:
      - drop_route._unmatched
  
sinks:
  send_to_http_server:
    type: http
    inputs:
      - metrics_to_logs
    uri: http://localhost:8081
    encoding:
      codec: json

  export_received_metrics:
    type: prometheus_exporter
    address: 0.0.0.0:9199
    inputs: [ logs_to_metrics ]

Version

0.38.1

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response