β

Announcing Pulsar Reporting: Near-Real-Time Metric

eBay Tech Blog 192 阅读

We are excited to announce the first open-source release of Pulsar Reporting.

Earlier this year, we announced http://gopulsar.io , an open-source project that included Pulsar Pipeline, a real-time analytics platform and stream processing framework. One of the frequently requested features for Pulsar has been integration with a metrics store for visualizing the near-real-time metrics. We’ve provided this feature with this release, which adds the Pulsar Reporting API and the Pulsar Reporting UI Framework under the same license terms. The public GitHub repository is https://github.com/pulsarIO .

What is Pulsar Reporting?

Pulsar Reporting is an extensible data visualization and reporting framework designed to provide real-time insights from Pulsar Pipeline. The framework includes a rich set of charting widgets and a visual reporting editor for users to easily create reports. It has a robust data query engine that can be extended to support many different types of data sources. With the Pulsar Reporting Framework, users can quickly create multi-dimensional and interactive reports that include drill-down and slice-and-dice capabilities.

Features

Why Pulsar Reporting?

The Pulsar Reporting Framework complements Pulsar, an open-source, real-time analytics platform and stream processing framework. Pulsar generates huge amounts of data, and visualization is the best way to provide intuitive and meaningful insights into that data. However, building dashboards and reports for big data from scratch is cumbersome and error-prone. The Pulsar Reporting Framework allows user to create reports easily and quickly without requiring complex data processing and UI logic.

Architecture

The raw events and session events from Pulsar Pipeline flow to Kafka using the Pulsar Kafka channel. The Druid cluster then ingests the raw events as well as the sessions from Kafka topics into two tables, one for sessions and one for events. Both tables are indexed in one-second granularity to enable real-time reporting. The Pulsar Reporting API provides an abstract layer to access the tables. The Reporting UI gets the data from the API to build different charts.

pulsar_reporting_architecture

Sample API requests

What’s next?

We have open-sourced the Pulsar Reporting Framework, and we plan to continue developing the code in the open. We welcome your suggestions and contributions. Here are some of the features we are thinking about.

Please visit http://gopulsar.io for source code, documentation, and more information.

The Team

Tracking-COE2

作者:eBay Tech Blog
Where e-commerce meets world-class technology
原文地址:Announcing Pulsar Reporting: Near-Real-Time Metric, 感谢原作者分享。

发表评论