VictoriaMetrics Offers Prometheus Replacement for Time Series Monitoring

Prometheus has emerged as the leading open source observability tool for cloud native infrastructure monitoring. But during its development during the past years, some say there has been a difference in needs among its user base with how Prometheus is evolving.

One user and maintainer was Roman Khavronenko, co-founder at VictoriaMetrics,  which continues to expand on its flagship open source MetricsQL for time series data monitoring solutions.

“Our query language is designed to address the issues we encountered with Prometheus. When we initially experimented with Prometheus, we were satisfied with its capabilities, but as we delved deeper, we saw problems on an architectural level. Language problems, for example, were pointed out by Prometheus community members, but were rejected by maintainers,” Khavronenko said. “VictoriaMetrics has listened to the community, and no longer dependent on Prometheus libraries, we were able to shape our query engine to fulfill the new requirements.”

Conversely, projects such as Thanos, Cortex and Mimir re-use libraries developed by Prometheus maintainers, Khavronenko noted. Doing this helps to maintain “the highest level of compatibility, as all listed solutions are basically using the same code,” said. “But once any of them wants to change something, it is a very long process to convince other parties that change is needed and satisfy all their requirements,” Khavronenko said. “VictoriaMetrics uses none of Prometheus libraries. And while it decreases compatibility level, it gives us a lot of flexibility to add features we want whenever we want.”

For the application developer, it’s crucial to maintain a fast pace and have control over the development process, Khavronenko said. Depending on external libraries can create vulnerabilities and other issues. “The main reason for developing our own query engine is that we made it more efficient and flexible. For example, MetricsQL was multithreaded from the very beginning, while PromQL remains single-threaded.,” Khavronenko said. “This language retains the capabilities of the Prometheus query language while addressing the issues we encountered.

Indeed, there is room for applications that can better meet the different needs of Prometheus users. “There currently are too many opportunities to make significant mistakes during Prometheus deployment and configuration. As the number of Kubernetes clusters increase throughout the enterprise, there are simply too many potential failure points to consistently ensure reliable and accurate monitoring across all clusters,” Torsten Volk, an analyst for Enterprise Management Associates (EMA), said. “Ideally, each new Kubernetes cluster should automatically include monitoring and alerting for all of its relevant metrics. The fact that these metrics and alerts can be different depending on the applications running on a specific cluster makes this even more challenging.”

Khavronenko said MetricsQL was specifically designed to:

• help users solve most-common metrics queries.
• be compatible with industry-standard Prometheus PromQL.
• offer HDR-like histograms for accurate analysis of extreme data ranges.

MetricsQL is designed for querying time series data. It provides a rich list of functions for various aggregations, transformations, rollups and other functionality specific for time series, and “remains simple and efficient to use on any scale,” Khavronenko said.

Applications include streaming services for video games, online music services, scientific research and other similar applications involving the distribution of streaming data. The applications can easily require the monitoring of billions of metrics that may be spread across multiple cloud deployments and physically located anywhere in the world, Khavronenko noted. This is where Prometheus typically falls short, he said.

Prometheus does not handle the function well of revealing the number of requests an application process per second, which MetricsQL is designed to offer, Khavronenko noted. “Prometheus often provides extrapolated results rather than exact ones, leading to misleading information and potential issues,” Khavronenko said. “This issue was discussed extensively in the Prometheus GitHub repository as far as 2019.”

Volk agreed: “There is simply too much knowledge needed to reliably monitor Kubernetes clusters,” Volk said. “If DevOps teams have to worry about how to optimally configure their queries so that they actually measure the correct data without causing resource issues on the cluster, effective monitoring is too difficult for a mainstream technology like Kubernetes.”

VictoriaMetrics’ revenue primarily comes from the enterprise version and the services provided to large companies. “We offer architectural support and additional features tailored to the needs of large organizations,” Khavronenko said. “Since we didn’t seek external investments and started earning money within six months of the release, we have been profitable from the beginning.”

VictoriaMetrics also recently introduced VictoriaLogs for monitoring applications for what the company says is a “more strategic ‘state of all systems’ enterprise-wide observability.” VictoriaLogs VictoriaLogs works with both structured and unstructured logs for maximum backward compatibility with large-scale infrastructure needed by users whether they are academic or commercial, working in e-commerce or video gaming teams, the company says.

While logs, metrics and traces make up the three pillars of observability, “many companies do not rely on traces at all and I’ve seen many fewer organizations not using metrics,” Khavronenko said. “But I haven’t seen a single IT company not doing logs,” Khavronenko said. “So, while VictoriaMetrics offers a scalable performance solution for metrics, VictoriaLogs now does the same for logs.”