Kubernetes Monitoring and Logs Explained: Metrics, Events, Alerts and Troubleshooting

Kubernetes Monitoring and Logs Explained#

Kubernetes monitoring helps you understand whether your cluster, nodes, Pods and applications are healthy. Logs help you understand what happened inside containers. Together, metrics, logs, events and alerts give DevOps and SRE teams the visibility needed to troubleshoot production issues quickly.

Animated mental model: Kubernetes observability flow#

Table of contents#

• Quick answer
• What is Kubernetes observability?
• kubectl logs
• Kubernetes events
• Metrics and kubectl top
• Prometheus and Grafana
• Alerts
• Troubleshooting workflow
• Cheat sheet
• Best practices

Quick answer#

Logs#

Logs show messages written by containers. Use them to debug application errors, crashes and startup failures.

Events#

Events show cluster-level decisions and failures such as scheduling problems, image pull errors and volume mount issues.

Metrics#

Metrics show CPU, memory, restarts, latency and resource usage over time.

Alerts#

Alerts notify teams when something crosses a risky threshold, such as high CPU or repeated Pod restarts.

1. What is Kubernetes observability?#

Kubernetes observability means understanding what is happening inside your cluster. It is not only about checking whether a Pod is running. A production team needs to know why a Pod restarted, why a deployment is slow, whether nodes are under pressure, whether traffic is reaching a Service and whether users are impacted.

• Signal — Question it answers — Example tool/command
• Logs — What did the application print? — kubectl logs
• Events — What did Kubernetes report? — kubectl get events
• Metrics — How much CPU/memory is used? — kubectl top , Prometheus
• Alerts — When should the team act? — Alertmanager, Grafana alerts

2. View Pod and container logs#

The first command most engineers use during an incident is kubectl logs . It reads the container logs from a Pod.

kubectl logs pod-name kubectl logs -f pod-name kubectl logs --tail=100 pod-name kubectl logs pod-name --since=30m

Logs for a Pod with multiple containers#

If a Pod has multiple containers, you must specify the container name.

kubectl logs pod-name -c app kubectl logs pod-name -c sidecar kubectl logs -f pod-name -c app

Logs from the previous crashed container#

When a container restarts, the current logs may not show the original crash. Use --previous .

kubectl logs pod-name --previous kubectl logs pod-name -c app --previous

3. Understand Kubernetes events#

Kubernetes events are short messages created by the cluster. Events are extremely useful when Pods are pending, images cannot be pulled, volumes cannot be mounted, or probes are failing.

kubectl get events kubectl get events -A kubectl get events --sort-by=.metadata.creationTimestamp kubectl get events -n production --sort-by=.metadata.creationTimestamp

Common events and meaning#

• Event reason — Meaning — Common fix
• FailedScheduling — Scheduler cannot place the Pod — Check resources, taints, node selectors and affinity.
• FailedMount — Volume cannot be mounted — Check PVC, StorageClass, permissions and volume status.
• BackOff — Container keeps failing — Check logs and app startup command.
• Unhealthy — Probe is failing — Check readiness/liveness probe path, port and timeout.
• ImagePullBackOff — Image cannot be pulled — Check image name, tag, registry access and imagePullSecrets.

4. Metrics and kubectl top#

Metrics show resource usage. The common beginner command is kubectl top , but it requires Metrics Server to be installed in the cluster.

kubectl top nodes kubectl top pods kubectl top pods -A kubectl top pod pod-name --containers

Install Metrics Server in a lab cluster#

For learning clusters, Metrics Server is often installed separately. Use the official manifest for your environment and verify compatibility.

kubectl get deployment metrics-server -n kube-system kubectl top nodes kubectl top pods

5. Prometheus and Grafana in Kubernetes#

For production monitoring, teams usually need more than kubectl top . Prometheus is commonly used to collect metrics, and Grafana is commonly used to visualize dashboards.

• Component — Purpose
• Prometheus — Collects and stores metrics.
• Grafana — Visualizes metrics in dashboards.
• Alertmanager — Routes alerts to email/chat/on-call systems.
• kube-state-metrics — Exposes Kubernetes object state metrics.
• Node exporter — Exposes node-level metrics.

Common metrics to monitor#

• Node CPU and memory usage.
• Pod CPU and memory usage.
• Pod restart count.
• Deployment replica availability.
• Pending Pods.
• Container OOMKilled events.
• Ingress 4xx/5xx errors.
• Request latency.
• Persistent volume usage.

6. Useful alert examples#

Alerts should be actionable. A good alert tells the team that something requires attention, not just that a metric moved slightly.

• Alert — Why it matters — First check
• Pod restart count high — App may be crashing repeatedly. — kubectl logs --previous
• Node memory pressure — Pods may be evicted. — kubectl describe node
• Deployment replicas unavailable — Service may be degraded. — kubectl rollout status
• High 5xx rate — Users may see failures. — Ingress/app logs
• PVC nearly full — Stateful app may fail writes. — Storage dashboard

7. Troubleshooting workflow#

When something fails, use a consistent workflow instead of guessing.

1. Check the namespace and context.
2. Check Pod status.
3. Describe the Pod.
4. Read logs.
5. Check previous logs if it restarted.
6. Check events.
7. Check Service and Endpoints.
8. Check node and resource pressure.
9. Check rollout status.

kubectl config current-context kubectl get pods -n app kubectl describe pod pod-name -n app kubectl logs pod-name -n app kubectl logs pod-name -n app --previous kubectl get events -n app --sort-by=.metadata.creationTimestamp kubectl get svc,endpoints -n app kubectl top pods -n app kubectl rollout status deployment/myapp -n app

8. Kubernetes monitoring and logs cheat sheet#

• Task — Command
• Get Pods — kubectl get pods -A
• View logs — kubectl logs pod-name
• Follow logs — kubectl logs -f pod-name
• Previous crash logs — kubectl logs pod-name --previous
• Describe Pod — kubectl describe pod pod-name
• View events — kubectl get events --sort-by=.metadata.creationTimestamp
• Node metrics — kubectl top nodes
• Pod metrics — kubectl top pods -A
• Rollout status — kubectl rollout status deployment/name

Common mistakes#

• Looking only at logs and ignoring events.
• Checking the wrong namespace.
• Forgetting --previous for restarted containers.
• Assuming Running means the app is healthy.
• Not monitoring restart counts.
• Creating too many noisy alerts.
• Not saving logs centrally for production workloads.

Best practices#

• Use structured logs where possible.
• Add request IDs/correlation IDs to application logs.
• Monitor Pods, nodes, deployments, Services and Ingress.
• Alert on user-impacting symptoms, not every small metric movement.
• Use dashboards for trends and alerts for action.
• Install Metrics Server for basic metrics.
• Use Prometheus/Grafana for production observability.
• Protect log data because logs can contain sensitive details.

Mini practice lab#

1. Create a test deployment.
2. View Pod logs.
3. Scale the deployment.
4. Check rollout status.
5. Break the image tag intentionally in a lab.
6. Check events and fix the image tag.

kubectl create deployment demo --image=nginx kubectl get pods kubectl logs deploy/demo kubectl scale deployment demo --replicas=3 kubectl rollout status deployment/demo kubectl delete deployment demo

Related Melochip guides#

• Kubernetes Learning Path
• kubectl Commands Explained
• Kubernetes Troubleshooting Guide
• Kubernetes Pods Explained
• Docker Logs and Troubleshooting Guide
• GitLab CI/CD Troubleshooting Guide

Official references#

• Kubernetes logging architecture
• Kubernetes resource metrics pipeline
• kubectl reference