Huge updates

posts/career/monitoring-and-logging.org

@@ -2,14 +2,118 @@
 #+OPTIONS: num:nil
 #+DATE: <2026-01-18 Sun 23:00> 
 #+filetags: :learning:notes:
-#+WIP: t
+#+WIP:
 #+COMMENTS: t
 #+SLUG: monitoring-and-logging
 
-* TODO
+* Monitoring and Logging Tools Overview
 
-Talk about Kibana, Grafana, Prometheus, and others.
-    - [ ] Research popular monitoring and logging tools
-    - [ ] Write about their features and use cases
-    - [ ] Include examples of how to set them up
-    - [ ] Discuss best practices for monitoring and logging
+Modern systems, especially cloud and distributed applications, require tools to observe performance, detect issues, and analyse logs. Tools like Kibana, Grafana, and Prometheus are widely used for these purposes.
+
+* Kibana
+** Purpose
+Log visualisation and analysis.
+
+** How it works
+Kibana is part of the Elastic Stack (formerly ELK Stack) and is used to visualise data stored in Elasticsearch.
+
+** Key Features
+- Search and analyse large log datasets
+- Interactive dashboards
+- Log filtering and querying
+- Security and anomaly detection features
+
+** Common Use Cases
+- Viewing application logs
+- Debugging errors in production systems
+- Security monitoring
+
+** Example Setup
+1. Install Elasticsearch
+2. Send logs using Logstash or Filebeat
+3. Use Kibana to visualise logs
+
+* Prometheus
+** Purpose
+Metrics collection and monitoring.
+
+Prometheus is designed to collect numeric metrics over time from systems and applications.
+
+** Key Features
+- Time-series database
+- Powerful query language (PromQL)
+- Built-in alerting
+- Pull-based metrics collection
+
+** Common Use Cases
+- Monitoring servers and containers
+- Tracking CPU, memory, and request latency
+- Infrastructure monitoring in Kubernetes
+
+** Example Setup
+1. Install Prometheus
+2. Configure targets to scrape metrics
+3. Expose metrics via a /metrics endpoint
+4. Query metrics using PromQL
+
+* Grafana
+** Purpose
+Visualisation and dashboards.
+
+Grafana is commonly used with Prometheus but can connect to many different data sources.
+
+** Key Features
+- Highly customisable dashboards
+- Supports many data sources (Prometheus, Elasticsearch, databases)
+- Alerting and notifications
+- Real-time visual monitoring
+
+** Common Use Cases
+- Infrastructure monitoring dashboards
+- Business metrics visualisation
+- Combining logs, metrics, and traces
+
+** Example Setup
+1. Install Grafana
+2. Connect a data source (Prometheus, Elasticsearch, etc.)
+3. Build dashboards using panels and queries
+
+* Other Popular Tools
+
+** Logstash
+- Log processing pipeline
+- Collects, transforms, and sends logs to Elasticsearch
+
+** Filebeat
+- Lightweight log shipper
+- Sends logs from servers to Elasticsearch
+
+** Loki
+- Log aggregation system designed by Grafana Labs
+- Integrates well with Grafana dashboards
+
+* Best Practices for Monitoring and Logging
+
+** Monitor Key Metrics
+- CPU usage
+- Memory usage
+- Request latency
+- Error rates
+
+** Centralise Logs
+Send logs from all services to a single platform.
+
+** Use Alerts
+Configure alerts to notify you of abnormal behavior.
+
+** Combine Logs and Metrics
+- Metrics tell you that something is wrong
+- Logs help you understand why it is wrong
+
+** Create Meaningful Dashboards
+Focus on actionable information instead of displaying excessive data.
+
+* Simple Summary
+- Prometheus collects metrics
+- Grafana visualises metrics
+- Kibana analyses logs