May 10, 2024
OpenSearch and Elasticsearch are two of the most popular full text search engines on the market today. Both are powerful tools that can be used for a variety of purposes, including log analytics, enterprise search, and security information and event management (SIEM). However, there are also some key differences between the two platforms.
In this blog post, we will take a closer look at OpenSearch and Elasticsearch, comparing and contrasting their features, licensing models, and community support. We will also discuss some of the factors to consider when choosing between the two search engines.
History
Elasticsearch was originally developed by Elasticsearch N.V., originally under Apache open source license. In 2019, after a copyright dispute resulting from Elasticsearch as a service launch by AWS cloud, Amazon forked Elasticsearch to create the OpenSearch Project that is governed by a consortium of companies and developers.
Features
OpenSearch was developed from a relatively advanced ( 7.10.2 ) fork of Elasticsearch, so all the basic functionality of search, analytics, and dashboards in the two applications are the same. However, they first had to remove all code that was not compatible with the Apache 2.0 license and this included disabling all telemetry collection functionality and removing the entire Elastic X-Pack code. OpenSearch and Elasticsearch still share many of the same features, including:
Full-text search
Distributed search
Multi-tenant architecture
Analytics engine
Amazon fork of Elasticsearch resulted in a significant loss of functionality for the earliest versions of OpenSearch, but they are continuing to deliver new OpenSearch features via external plugins/connectors that can replace these capabilities. There’s functionality such as authentication and authorization, index management, alerting and so on that in Elasticsearch was traditionally proprietary, so OpenSearch has implemented open-source alternatives. For example, while Elasticsearch has Index Lifecycle Management, OpenSearch has Index State Management. By and large, they do the same thing, the difference is in the details. Also - instead of the original telemetry collection features from Elasticsearch, AWS customers can use the OpenTelemetry Collector plugin to collect and format telemetry data. There’s also an OpenSearch Observability plugin that can be used to analyze telemetry data from distributed applications.
Elasticsearch and Kibana are still typically used together, while Amazon has developed its own visualization tool called OpenSearch Dashboards by forking Kibana. There are also companies like Quesma that allow connecting these great UI apps to SQL columnar databases like ClickHouse.
Licensing
OpenSearch is released under the Apache 2.0 license, which is a permissive open-source license. This means that it is free to use and modify for any purpose.
Elasticsearch has a more complex licensing model. The basic features of Elasticsearch are free to use under the Server Side Public License (SSPL) and Elastic License. However, some of the more advanced features require a commercial license.
Community Support
Elasticsearch has objectively a larger and more active community than OpenSearch. This means that there are more resources available for Elasticsearch, such as documentation, tutorials, and forums. It has also more average contributions, but to be fair it also has larger code base, including X-pack, etc
This is natural as OpenSearch is a younger project. It is however growing fast and the fully Open source license may be more attractive for future contributors allowing OpenSearch to outgrow Elasticsearch in the long run.
Choosing Between OpenSearch and Elasticsearch
The decision of whether to use OpenSearch or Elasticsearch depends on your specific needs and priorities. Here are some factors to consider:
Features: If you need access to all of the features that Elasticsearch offers, then Elasticsearch is the better choice. However, if you only need the basic features of a search engine, then OpenSearch may be a good option.
Licensing: If you are on a tight budget and need a free and open-source search engine, then OpenSearch is the clear choice. However, if you are willing to pay for a commercial license, then Elasticsearch may be a better option.
Community Support: If you need access to a large and active community of users and developers, then Elasticsearch is the better choice. However, the OpenSearch community is growing rapidly and may be able to provide you with the support you need.
Conclusion
OpenSearch and Elasticsearch are both powerful search engines that can be used for a variety of purposes. The best choice for you will depend on your specific needs and priorities.
Additional Notes
Elasticsearch and OpenSearch are diverging in some notable ways, but they’re ultimately cut from the same cloth - Elasticsearch 7.10.2. And while there are some minor differences, the two solutions are pretty comparable at this point for the vast majority of mainstream use cases.
So whether you’re choosing to adopt Elasticsearch or Amazon OpenSearch to enable log analytics at scale, you’ll likely face many of the same challenges we’ve
High cost of ownership that increases exponentially as daily log ingest increases
Stability and uptime challenges as search indices grow increasingly large
Issues related to reverted index concept not being the best choice for analytical purposes
Data retention trade-offs where users start limiting data retention to reduce storage costs
As an alternative, you may consider keeping parts of Elastic/OpenSearch ecosystem like Kibana/OSDashboards and Logstash/Prepper and connect them to more efficient, faster and cheaper columnar SQL databases like ClickHouse. Quesma can help with that by providing a translation layer to connect these two worlds together.
I hope this blog post has helped you to understand the differences between OpenSearch and Elasticsearch.
