Design a Log Aggregation Service - Follow-up questions - Intuition First, Need more How
Here is solutions for Design a Log Aggregation Service.
Main topic:
-
Log store in Kafka
-
Query in Elastic Search
-
Fetch in S3
1. Requirements
-
Real-time log viewing for debugging (latency < a few seconds)
-
Regex-based querying on real-time logs for live troubleshooting
-
Tar-archived log export for analytics teams covering 24-48 hours
2. High-level Design

July 26, 2026