| mapped_pages | ||||||
|---|---|---|---|---|---|---|
| products |
|
|||||
| applies_to |
|
|||||
| description | The Elastic Stack is a group of open source products that work together to securely store, search, analyze, and visualize your data. |
The {{stack}} is a suite of open-source products, like {{es}}, {{kib}}, {{agent}}, {{ls}}, and more, that work together to ingest, store, search, and visualize data at scale. This page describes each component and explains how they fit together across Elastic deployments.
All Elastic deployments and projects share the same open source foundation:
- {{es}}: The distributed data store and search engine that handles indexing, querying, and analytics.
- {{kib}}: The user interface with dashboards, visualizations, and management tools.
Depending on your use case, you might need to install more products that work together with {{es}} and {{kib}} (referred to as the {{stack}} or ELK). For example:
- {{agent}}: A lightweight data shipper that collects and forwards data to {{es}}.
- {{ls}}: The data ingestion and transformation engine, often used for more complex ETL (extract, transform, load) pipelines.
$$$stack-components$$$ The {{stack}} includes products for ingesting, storing, and exploring data at scale:
Continue reading to learn how these products work together.
All deployments include {{es}}. {{es}} is the distributed search and analytics engine, scalable data store, and vector database at the heart of all Elastic deployments and solutions. You can use the {{es}} clients to access data directly by using common programming languages.
{{es}} is a data store and vector database that provides near real-time search and analytics for all types of data. Whether you have structured or unstructured text, time series (timestamped) data, vectors, or geospatial data, {{es}} can efficiently store and index it in a way that supports fast searches. It also includes multiple query languages, aggregations, and robust features for querying and filtering your data.
{{es}} is built to be a resilient and scalable distributed system. It runs as a cluster of one or more servers, called nodes. When you add data to an index, which is the fundamental unit of storage in {{es}}, it's divided into pieces called shards, which are spread across the various nodes in the cluster. This architecture allows {{es}} to handle large volumes of data and ensures that your data remains available even if a node fails. If you use {{serverless-full}}, it has a unique Search AI Lake cloud-native architecture and automates the nodes, shards, and replicas for you.
{{es}} also includes AI-powered features and built-in {{nlp}} (NLP) models that enable you to make predictions, run {{infer}}, and integrate with LLMs faster.
Nearly every aspect of {{es}} can be configured and managed programmatically through its REST APIs. This allows you to automate repetitive tasks and integrate Elastic management into your existing operational workflows. For example, you can use the APIs to manage indices, update cluster settings, run complex queries, and configure security. This API-first approach is fundamental to enabling infrastructure-as-code practices and managing deployments at scale.
Learn more about the {{es}} data store, its distributed architecture, and APIs.
The clients provide a convenient mechanism to manage API requests and responses to and from {{es}} from popular languages such as Java, Ruby, Go, and Python. Both official and community contributed clients are available.
Learn more about the {{es}} clients.
Use {{kib}} to explore, manage, and visualize the data that's stored in {{es}} and to manage components of the {{stack}}.
{{kib}} provides the user interface for all Elastic solutions and {{serverless-short}} projects. It's a powerful tool for visualizing and analyzing your data and for managing and monitoring the {{stack}}. Although you can use {{es}} without it, {{kib}} is required for most use cases and is included by default when you deploy using some deployment types, including {{serverless-full}}.
With {{kib}}, you can:
- Use Discover to interactively search and filter your raw data.
- Build custom visualizations like charts, graphs, and metrics with tools like Lens, which offers a drag-and-drop experience.
- Assemble your visualizations into interactive dashboards to get a comprehensive overview of your information.
- Perform geospatial analysis and add maps to your dashboards.
- Configure notifications for significant data events and track incidents with alerts and cases.
- Manage resources such as processors, pipelines, data streams, trained models, and more.
Each solution or project type provides access to customized features in {{kib}} such as built-in dashboards and AI assistants.
{{kib}} also has query tools such as Console, which provides an interactive way to send requests directly to the {{es}} API and view the responses. For secure, automated access, you can create and manage API keys to authenticate your scripts and applications.
Before you can search it, visualize it, and use it for insights, you must get your data into {{es}}. There are multiple methods for ingesting data. The best approach depends on the type of data and your specific use case. For example, you can collect and ship logs, metrics, and other types of data with {{agent}} or collect detailed performance information with {{product.apm}}. If you want to transform and enrich data before it's stored, you can use {{es}} ingest pipelines or {{ls}}.
Trying to decide which ingest components to use? Refer to and .
{{agent}} is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, and forward data from remote services or hardware. Each agent has a single policy to which you can add integrations for new data sources, security protections, and more. You can also use {{agent}} processors to sanitize or enrich your data.
To monitor the state of all your {{agents}}, manage agent policies, and upgrade {{agent}} binaries or integrations, refer to Central management in {{fleet}}.
{{product.apm}} is an application performance monitoring system. It allows you to monitor software services and applications in real-time by collecting detailed performance information on response time for incoming requests, database queries, calls to caches, external HTTP requests, and more. This makes it efficient to pinpoint and fix performance problems quickly.
Learn more about {{product.apm}}.
:::{include} /manage-data/_snippets/otel.md :::
With EDOT, you can use vendor-neutral instrumentation and stream native OTel data such as standardized traces, metrics, and logs without proprietary agents.
:::{include} /manage-data/_snippets/beats.md :::
Learn more about {{beats}}.
Ingest pipelines let you perform common transformations on your data before indexing them into {{es}}. You can configure one or more "processor" tasks to run sequentially, making specific changes to your documents before storing them in {{es}}.
Learn more about ingest pipelines.
{{ls}} is a data collection engine with real-time pipelining capabilities. It can dynamically unify data from disparate sources and normalize the data into destinations of your choice. {{ls}} supports a broad array of input, filter, and output plugins, with many native codecs further simplifying the ingestion process.
Learn more about {{ls}}.
serverless: unavailable
:::{include} /deploy-manage/deploy/_snippets/stack-version-compatibility.md :::
$$$installation-order$$$ :::{include} /deploy-manage/deploy/_snippets/installation-order.md :::