Apache® Druid logs

Druid logs - expert interview

info

We're in the process of migrating this content. Check back soon.

Exercises

Take time to look at the log files in Druid, familiarizing yourself with their location and contents, and the configuration that applies to them.

Run through these notebooks from the learn-druid repository.:

• The notebook on Apache Druid logs, where you'll see examples of how Druid uses Log4j and the contents of those logs.
• The notebook on streaming and SQL-based ingestion logs, which focuses on task logs.

Remember to take a look at the reference material at the end of each notebook. Some questions in the exam require you to have read the related pages, or at least to know where to get to them.

Learn more

Equipped with what you have learned, you may consider it a good idea to:

• Look at and record the Log4j configuration in your running environment.
• Follow one of your own ingestion tasks through the logs to understand, in detail, what Druid is doing.
• Decide whether you will implement a solution to centralize logs for storage and analysis.
• Work out a meaningful retention policy for all your logs.
• Use a test environment to try out different configuration options.
• Determine a configuration management strategy that will allow you to centrally manage the log configurations of your Druid instances.

For extra depth and context, take a look at these resources:

• Read about setting the query Id in the query context parameters.
• Find out about masking things like passwords in log files.
• Watch the presentation “Tales from Support” by Ben Krug and Bala Thiyagarajan of Imply.
• Check out Druid Optimizations for Scaling Customer Facing Analytics by Conviva.