Optimize segment layout and location
Segments and infrastructure
We're in the process of migrating this content. Check back soon.
Expert interview
We're in the process of migrating this content. Check back soon.
Layout
Segments in Druid are generated during ingestion and stored in Deep Storage. It's critical to optimize the size and number of these files with an understanding of the impact on query efficiency and on data management.
Open JupyterLab in your learn-druid environment.
Work through the following notebooks:
With those completed, you should be able to set up your ingestion to produce different numbers and volumes of segment files, and have a sense of how those layouts impact query execution.
Learn more about tiering by following this series of notebooks. The first walks through general tiering, while the second walks through how to query data that has not been pre-fetched to Historicals.
Learn more
Before you complete this section, be sure to have looked at these docs pages to give you broader knowledge:
Take a look at these videos and articles to deepen your understanding:
- Optimizing Druid from Pinterest
- Storing Data in Druid for Performance from Imply
- Powering Observability from Confluent
Tiering in action
We're in the process of migrating this content. Check back soon.