Skip to content

Sub-second search & analytics engine on cloud storage

License

Notifications You must be signed in to change notification settings

kikokikok/quickwit

ย 
ย 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

CI codecov Contributor Covenant License: AGPL V3 Twitter Follow Discord



Quickwit Cloud-Native Search Engine Quickwit Cloud-Native Search Engine

Cloud-native search engine for observability (logs, traces, and soon metrics!). An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.


We just released Quickwit 0.8! Read the blog post to learn about the latest powerful features!

Quickwit is the fastest search engine on cloud storage. It's the perfect fit for observability use cases

๐Ÿš€ Quickstart


gharchive-demo.mp4

๐Ÿ’ก Features

  • Full-text search and aggregation queries
  • Elasticsearch-compatible API, use Quickwit with any Elasticsearch or OpenSearch client
  • Jaeger-native
  • OTEL-native for logs and traces
  • Schemaless or strict schema indexing
  • Schemaless analytics
  • Sub-second search on cloud storage (Amazon S3, Azure Blob Storage, Google Cloud Storage, โ€ฆ)
  • Decoupled compute and storage, stateless indexers & searchers
  • Grafana data source
  • Kubernetes ready - See our helm-chart
  • RESTful API

Enterprise ready

  • Multiple data sources Kafka / Kinesis / Pulsar native
  • Multi-tenancy: indexing with many indexes and partitioning
  • Retention policies
  • Delete tasks (for GDPR use cases)
  • Distributed and highly available* engine that scales out in seconds (*HA indexing only with Kafka)

๐Ÿ“‘ Architecture overview

Quickwit Distributed TracingQuickwit Distributed Tracing

๐Ÿ“• Documentation

๐Ÿ“š Resources

๐Ÿ”ฎ Roadmap

  • Quickwit 0.9 (July 2024)

    • Indexing and search performance improvements
    • Index configuration updates (retention policy, indexing and search settings)
    • Concatenated field
  • Quickwit 0.10 (October 2024)

    • Schema (doc mapping) updates
    • Native distributed ingestion
    • Index templates

๐Ÿ™‹ FAQ

How can I switch from Elasticsearch or OpenSearch to Quickwit?

Quickwit supports a large subset of Elasticsearch/OpenSearch API.

For instance, it has an ES-compatible ingest API to make it easier to migrate your log shippers (Vector, Fluent Bit, Syslog, ...) to Quickwit.

On the search side, the most popular Elasticsearch endpoints, query DSL, and even aggregations are supported.

The list of available endpoints and queries is available here, while the list of supported aggregations is available here.

Let us know if part of the API you are using is missing!

If the client you are using is refusing to connect to Quickwit due to missing headers, you can use the extra_headers option in the node configuration to impersonate any compatible version of Elasticsearch or OpenSearch.

How is Quickwit different from traditional search engines like Elasticsearch or Solr?

The core difference and advantage of Quickwit is its architecture built from the ground to search on cloud storage. We optimized IO paths, revamped the index data structures and made search stateless and sub-second on cloud storage.

How does Quickwit compare to Elastic in terms of cost?

We estimate that Quickwit can be up to 10x cheaper on average than Elastic. To understand how, check out our blog post about searching the web on AWS S3.

What license does Quickwit use?

Quickwit is open-source under the GNU Affero General Public License Version 3 - AGPLv3. Fundamentally, this means you are free to use Quickwit for your project if you don't modify Quickwit. However, if you do and you are distributing your modified version to the public, you have to make the modifications public. We also provide a commercial license for enterprises to provide support and a voice on our roadmap.

Is it possible to set up Quickwit for a High Availability (HA)?

HA is available for search, for indexing it's available only with a Kafka source.

What is Quickwit's business model?

Our business model relies on our commercial license. There is no plan to become SaaS soon.

๐Ÿค Contribute and spread the word

We are always thrilled to receive contributions: code, documentation, issues, or feedback. Here's how you can help us build the future of log management:

โœจ After your contributions are accepted, don't forget to claim your swag by emailing us at [email protected]. Thank you for contributing!

๐Ÿ’ฌ Join Our Community

We welcome everyone to our community! Whether you're contributing code or just saying hello, we'd love to hear from you. Here's how you can connect with us:

About

Sub-second search & analytics engine on cloud storage

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Rust 96.2%
  • TypeScript 1.8%
  • Python 0.7%
  • HCL 0.4%
  • JavaScript 0.3%
  • Shell 0.2%
  • Other 0.4%