What it is and the problem it solves
Elasticsearch is a source-available, distributed full-text search engine. It solves the problem of fast, scalable search over high-volume, schema-free JSON data.
Shay Banon created its precursor, Compass, in 2004. While working on a third version of Compass he concluded that a full rewrite was needed to build a scalable, distributed search solution with JSON over HTTP as the common interface, and he released the first version of Elasticsearch in February 2010. Elastic NV was founded in 2012 to provide commercial services and products around it.
How it works
Elasticsearch distributes JSON documents across sharded, replicated indices. Each shard can have replicas on different cluster nodes. It uses Apache Lucene for indexing and search, exposed via HTTP.
Routing and rebalancing across the cluster are handled automatically. One constraint matters at design time: once an index has been created, the number of primary shards cannot be changed. A gateway module handles long-term index persistence, so an index can be recovered after a node failure. Beyond full-text queries, the engine supports percolation, a form of prospective search in which stored queries are matched against incoming documents rather than the reverse. Official client libraries exist for Java, C#, PHP, Python and Ruby, among others.
What works
Full-text search works. Faceted search works. Real-time search works. Multitenancy works. Sharding and replication across nodes work.
Real-time GET requests make Elasticsearch usable as a NoSQL datastore. It is developed alongside Logstash (data collection and log parsing), Kibana (analytics and visualisation) and Beats (lightweight data shippers); the four are designed for use together as the Elastic Stack, formerly the ELK Stack.
What does not
It does not remain open source after January 2021. Version 7.11 dropped Apache License 2.0 for dual SSPL/Elastic License. AGPL was added only in August 2024 (v8.16.0).
Elastic said the change was a response to Amazon Web Services offering Elasticsearch and Kibana as a service without what Elastic described as adequate collaboration. Critics noted that Elastic had previously promised never to alter the Apache 2.0 licensing, and predicted the move would damage the ecosystem. AWS announced a fork to continue under Apache 2.0, backed by other members of the ecosystem including Logz.io, CrateDB and Aiven, and rebranded it OpenSearch in April 2021 because of potential trademark conflicts with the Elasticsearch name. The 2024 addition of the AGPL made the software available under a free and open-source license again, alongside the two source-available options. Separately, Elasticsearch does not support distributed transactions.
What it changes
It changes how organisations index and query large volumes of log, metrics, and event data—enabling fast faceted and real-time search where relational databases falter.
In May 2019 Elastic made the core security features of the Elastic Stack available without charge, including TLS for encrypted communications, file and native realm authentication, and role-based access control for cluster APIs and indices; the source code for those features is available under the Elastic License. SIEM and machine learning capabilities are part of the commercial offerings.
Is it worth your time
Yes—if you need scalable, real-time full-text search over unstructured or semi-structured data and can manage licensing shifts post-2021.
