What I am building
While working on LocalAI-RAG I found that the final answer alone does not explain pipeline failures. I created RAG Debug Toolkit, a VS Code extension for large JSON, JSONC, and JSONL traces. It lets me inspect rendered prompts, compare state across steps, follow action order, and build a flow diagram from the trace.
Visibility instead of guesswork
I retain execution traces and inspect not just action names but inputs, meaningful state changes, and outcomes. CodeLens opens decoded prompts and chat messages as readable Markdown, showing what the model actually received.
Flow, metrics, and bottlenecks
Trace Explorer shows the step sequence, identifiers, counters, and duration. I can turn events into a PlantUML activity diagram, while comparing durations of similar actions helps flag possible slowdowns. It is a small tool born from a real debugging need.
Project scope
- CodeLens for rendered_prompt and rendered_chat_messages
- Trace Explorer with step state and metrics
- Pipeline step-by-step state diff views
- Activity UML generation with PlantUML/PlantText
- Action duration diagnostics and slow-step highlighting
- VSIX packaging for VS Code