[StackOverflow/kubernetes] Configure local filesystem storage for Loki

### ROOT CAUSE The issue arises because Loki's default configuration uses in-memory storage (Gin), which is ephemeral and unsuitable for persistent data. The user needs to explicitly configure Loki to use the local filesystem for storage persistence, including specifying the correct directory and ensuring proper permissions. ### CODE FIX 1. **Configure Loki's `loki.yaml`**: Add the filesystem storage configuration to Loki's YAML file (typically located at `/etc/loki/loki.yaml`): ```yaml storage: filesystem: directory: /var/lib/loki/data ``` Ensure the `directory` path exists and has write permissions (e.g., `sudo mkdir -p /var/lib/loki/data && sudo chown -R lofi-user:loki-data /var/lib/loki/data`). 2. **Deploy with Persistent Volume (Kubernetes)**: Use a `PersistentVolumeClaim` in the Loki deployment manifest to ensure data persistence: ```yaml spec: volumes: - name: loki-data persistentVolumeClaim: claimName: loki-data-pvc ``` Create a matching `PersistentVolume` (PVC) definition to allocate storage. 3. **Restart Loki**: After updating the configuration, restart the Loki service: ```bash sudo systemctl restart loki ``` 4. **Verify Configuration**: Check Loki's logs (`journalctl -u loki`) for errors related to storage initialization. Validate the data directory is being used with: ```bash ls -la /var/lib/loki/data ``` This ensures Loki uses the local filesystem with persistent storage, resolving issues with data loss upon pod restarts.