by Quesma

Breaking the event data lock-in

Many customers we talk to are dealing with huge increases in the volume of their logs, leading to skyrocketing costs and challenges with implementing observability. Applications have become increasingly complex, consisting of more and more moving parts, and good operations of such architectures require a continuous stream of detailed, always-on observability data.

While this event influx is growing yearly, many companies are stuck with analytical backends designed for the economics of data volumes from the early 2000s. New vendors appear on the horizon with very interesting value propositions, but there is often friction and risk involved in migrating the data and well established processes.  Meanwhile, generative AI and LLMs will require even more data for training, observability, and security use cases. To handle the increasing volume of events,  new generation of data platforms is required.

Enter Hydrolix

One of the vendors that appeared recently with a completely new approach to cost-effective log management at scale is Hydrolix. With its streaming data lake, designed to ingest petabytes of data at a fraction of typical costs, they've generated a lot of interest, including partnering with Akamai, and recently raised $35M in Series B funding. Hydrolix's platform combines real-time stream processing, decoupled storage, and low-latency indexed search to create a high-performance yet cost-efficient log management system. These features make Hydrolix the ideal data platform to power log data-intensive applications. 

Removing migration friction

Still, many customers hesitate to take advantage of the obvious benefits of migrating to a cost-effective solution like Hydrolix. In addition to the friction that often comes with migrating to a new data solution, teams may prefer certain aspects of their current solution's tooling. An example is user interfaces and dashboards, such as Kibana, a visualization and analytics tool that is designed to be used only with Elastic.

One of the reasons for that are different responsibilities, goals, and motivations of teams owning infrastructure costs and teams that take advantage of data kept in this infrastructure. While cost factors are important for the first, the latter does not desire the change required to improve on these. These customers look for the holy grail - streamlining the migration process to cheaper and faster data platforms in a way that is transparent to end users.

Quesma was founded to tackle this problem - unbundle the application (UI) layer from the database so customers can use the UI they prefer with the data solution that works best for their use case - for example - use the UI they prefer with the data solution that works best for their use case. One of the first use cases we delivered is a translation from Elastic Query Language to SQL, allowing customers to migrate easily from Elasticsearch to modern columnar databases such as Hydrolix's ClickHouse-compatible streaming data lake, all without a need to change the analytics (UI) layer used by their analysts.

Hydrolix is now compatible with Kibana

And now, we’re excited to announce that Quesma’s smart translation technology is powering Hydrolix’s new Kibana compatibility feature! In addition to supporting platforms like Grafana, Redash, and (recently) Splunk, Hydrolix's customers can now connect their Kibana UI directly to the Hydrolix cloud. The best part? No migration is necessary—your existing dashboards and queries work seamlessly with Hydrolix data right out of the box.

Below is a screenshot of Kibana running on Hydrolix.

Quesma works as a bridge between the Hydrolix platform and your Kibana/OpenSearchDashboards UI. It translates Elastic Query Language queries into SQL and also returns the results in a format compatible with Kibana and OpenSearch Dashboards interfaces.

For customers still holding onto Elasticsearch, this means an easy, painless migration to Hydrolix’s powerful data lake—all without disrupting your end users. And for those already enjoying Hydrolix, it opens the door to a new world of log data analysis with Kibana’s unparalleled power.

Want to see how Quesma and Hydrolix can transform your log management?

Contact us for a demo today, and get ahead of the data curve.