Using Cloudflare AI Gateway patchLog to Grow LLM Evaluation Logs
A small experiment on adding feedback, score, and metadata back to Cloudflare AI Gateway logs so LLM request logs can become the start of an evaluation loop.
Category
A small experiment on adding feedback, score, and metadata back to Cloudflare AI Gateway logs so LLM request logs can become the start of an evaluation loop.
A validation note on sending a failed Eve weather-tool eval to Langfuse, comparing the failed run with a revised run, and improving the evaluation criteria.
Making a personal blog discoverable by AI agents as Perplexity and ChatGPT replace traditional search. Implementation of AIO (AI Optimization) with llms.txt, JSON API, and MCP Server design—decision criteria for a 46-post blog.
I compare Langfuse Monitors with my custom Slack morning briefing worker to figure out which responsibilities should move to the official feature and which should stay custom.
A practical experiment that aggregates Langfuse traces in a Cloudflare Worker and sends only token, cost, and latency anomalies to Slack.
Manually checking traces every day is unsustainable. A personal GenAIOps plan to aggregate Langfuse failures, token spikes, and low-score outputs into a single Slack morning briefing.
Using Cloudflare AI Gateway as an OpenAI-compatible endpoint, I walk through its logging, payload controls, metadata, cost estimation, and OTel integration to see how far it can serve as an entry point for LLM observability.