LlamaIndex

LlamaIndex AI

LlamaIndex AI

LlamaIndex AI

LlamaIndex AI

LlamaIndex AI

LlamaIndex AI

LlamaIndex AI

LlamaIndex AI

LlamaIndex AI

LlamaIndex AI

Filesystems are having a moment

The AI agent ecosystem keeps rediscovering filesystems as a persistence and interoperability layer. LlamaIndex, LangChain, Oracle, and others now advocate for file-based context over massive tool integrations — coding agents like Claude Code thrive precisely because they read and write files locally. Context windows act like erasable whiteboards, not real memory, and files offer a boring but effective fix: write things down, read them back. Yet an ETH Zürich paper found that bloated context files actually hurt agent performance, suggesting they should stay minimal. Meanwhile, fragmentation reigns — CLAUDE.md, AGENTS.md, .cursorrules all coexist — though Anthropic's SKILL.md format has gained cross-platform adoption. The deeper argument: filesystems could restore personal data ownership, acting as an open interoperability layer where your preferences, skills, and memory travel between tools without vendor lock-in. submitted by /u/fagnerbrack [link] [comments]

Most multi-hop RAG goes stale the moment your data changes, what about a training-free approach that skips the graph rebuild?

Most methods that get strong multi-hop answers (GraphRAG, HippoRAG, RAPTOR, trained retrievers) build a knowledge graph or fine-tune a retriever over the corpus. That's fine until the data changes — then you re-extract / rebuild / retrain before the new facts are usable. For a corpus that updates daily, that's a real cost. MOTHRAG does the multi-hop reasoning at query time over a plain dense index instead. An update is just embed + append (one embedding call) — no graph reconstruction, no retraining — so it stays current as the corpus changes. And dropping the graph doesn't cost accuracy. F1, Llama-3.3-70B reader, n=1000 each: System HotpotQA 2Wiki MuSiQue Avg Hardware MOTHRAG 78.1 76.3 50.5 68.3 commodity API, no GPU HippoRAG2 75.5 71.0 48.6 65.0 — GraphRAG 68.6 58.6 38.5 55.2 — RAPTOR 69.5 52.1 28.9 50.2 — Competitor rows reproduced from HippoRAG2 (ICML 2025), Table 2. MOTHRAG is within ~0.7 avg F1 of the GPU-bound research frontier (a fine-tuned, GPU-served stack — not shown). (Fair note: graph-RAG systems like GraphRAG shine on small curated / sensemaking corpora — this is multi-hop factoid QA over changing data, a different regime.) Deterministic by design: instead of a free-form agent loop it runs a small ensemble of reasoning arms (direct read, decomposition, an iterative grounding-driven arm) under a deterministic arbitrator, over a bridge retrieval substrate with multi-hop chain filtering. Every answer is proof-tree-structured, so you can audit why it answered. Measured ≈$0.018/query, ~44% cheaper at matched accuracy. Open source, ~1 week old — genuinely after feedback and failure cases: pip install mothrag Code: https://github.com/juliangeymonat-jpg/mothrag Paper: https://doi.org/10.5281/zenodo.20668567 Live demo (BYO free key): https://huggingface.co/spaces/JUBOX99/mothrag-demo submitted by /u/ObjectiveEntrance740 [link] [comments]

You asked for DeepLearning.ai-style notebooks for AgentSwarms—so we built 67 of them (TypeScript/LangChain/LangGraph/LlamaIndex/AgentsSDK/VercelAI).

Hey everyone, A few months ago, We shared the visual canvas we built for AgentSwarms. The response was incredible, but the most common piece of feedback was: "The visual canvas is great for architecture, but I need to see the actual code to really understand how to deploy this." You wanted deep-dive, code-first labs—the kind you see on DeepLearning.ai—but for multi-agent systems, faster and with more flexibility. We’ve spent the last few weeks heads-down engineering a completely new Interactive Notebooks section. As of today, we have 67 TypeScript-based notebooks live on the site (with more dropping soon). What’s in the library: We’ve covered everything from basic LangChain fundamentals to complex enterprise-level multi-agent workflows. Everything runs entirely in your browser using TypeScript—no Docker, no Python venv, no local dependencies. A personal favorite: I’m particularly excited about the "Failure Mode & Error Handling" notebook. We’ve all seen agents that work perfectly in a demo but crash in production the moment a tool times out or an LLM returns garbage. This notebook walks through: How to build deterministic validation gates between nodes. How to force an orchestrator to "catch" a worker failure and dynamically re-route or re-prompt. How to handle state recovery when a multi-agent loop gets stuck in a hallucination cycle. Why we built this: I’m tired of seeing AI "tutorials" that are just static blog posts. To master Agentic AI, you need to be able to tweak a system prompt, break the code, watch the error trace, and fix the routing logic in real-time. The entire library of 67 labs is 100% free to use. If you’re currently wrestling with how to make your agents production-grade, I’d love for you to check them out and let me know if there’s a specific "failure mode" or architecture pattern you’d like us to add to the next batch of notebooks. Try it out here: agentswarms.fyi submitted by /u/Outside-Risk-8912 [link] [comments]

You asked for DeepLearning.ai-style notebooks for AgentSwarms—so we built 67 of them (TypeScript/LangChain/LangGraph/LlamaIndex/OpenAI-AgentsSDK/VercelAI).

Hey everyone, A few months ago, We shared the visual canvas we built for AgentSwarms. The response was incredible, but the most common piece of feedback was: "The visual canvas is great for architecture, but I need to see the actual code to really understand how to deploy this." You wanted deep-dive, code-first labs—the kind you see on DeepLearning.ai—but for multi-agent systems, faster and with more flexibility. We’ve spent the last few weeks heads-down engineering a completely new Interactive Notebooks section. As of today, we have 67 TypeScript-based notebooks live on the site (with more dropping soon). What’s in the library: We’ve covered everything from basic LangChain fundamentals to complex enterprise-level multi-agent workflows. Everything runs entirely in your browser using TypeScript—no Docker, no Python venv, no local dependencies. A personal favorite: I’m particularly excited about the "Failure Mode & Error Handling" notebook. We’ve all seen agents that work perfectly in a demo but crash in production the moment a tool times out or an LLM returns garbage. This notebook walks through: How to build deterministic validation gates between nodes. How to force an orchestrator to "catch" a worker failure and dynamically re-route or re-prompt. How to handle state recovery when a multi-agent loop gets stuck in a hallucination cycle. Why we built this: I’m tired of seeing AI "tutorials" that are just static blog posts. To master Agentic AI, you need to be able to tweak a system prompt, break the code, watch the error trace, and fix the routing logic in real-time. The entire library of 67 labs is 100% free to use. If you’re currently wrestling with how to make your agents production-grade, I’d love for you to check them out and let me know if there’s a specific "failure mode" or architecture pattern you’d like us to add to the next batch of notebooks. Try it out here: agentswarms.fyi submitted by /u/Outside-Risk-8912 [link] [comments]

Learn Agentic AI with quick, easy to run hands on labs, visual canvases and notebooks for free!

If you’re a full-stack engineer or technical architect willing to learn production-grade enterprise agents, you need architecture, security, and type-safe systems. That’s why we builtAgentSwarms.fyi—the ultimate hands-on educational platform for teaching agentic AI and multi-agent workflows. 🚀 The Core AgentSwarms Ecosystem: Real-World Architectures: Skip the generic hello-world loops. Learn production-grade systems like human-in-the-loop validation, automated multi-platform content multiplexers, and secure code-sandbox environments. Deterministic Cloud Guardrails: Deep dives into multi-cloud token economics, dynamic cost-optimized routing, and model evaluation metrics. Grassroots Engineering Focus: No corporate marketing fluff. Just raw, practical code patterns designed to bridge the gap between fragile prototypes and stable cloud deployments. 💣 The New Drop: 60+ Browser-Native TypeScript Notebooks We just completely re-engineered our learning workspace. We’ve added 60+ fully interactive TypeScript Notebooks running 100% natively in your browser. No pip install dependency hell, no local Docker setup, and zero environment friction. Read the architecture, tweak the system prompts or Zod schemas, hit play, and watch the streaming terminal execute live across the five absolute best frameworks in the ecosystem: 🟢 LangChain.js (Fundamentals & Middleware Guardrails) 🔀 LangGraph.js (Cyclic Graphs & Stateful Orchestration) 💾 LlamaIndex.ts (Sentence-Window Retrieval & RAG Triad Evals) ⚡ Vercel AI SDK (Streaming UI Integration) 🤖 OpenAI Agents SDK (Lightweight, low-boilerplate loops) Stop passively scrolling through video courses. Open a canvas, break the graph nodes, and start compiling real multi-agent swarms. 👉 Dive in for free: agentswarms.fyi/learn submitted by /u/Outside-Risk-8912 [link] [comments]

Learn Agentic AI with quick, easy to run hands on labs, visual canvases and notebooks for free!

If you’re a full-stack engineer or technical architect willing to learn production-grade enterprise agents, you need architecture, security, and type-safe systems. That’s why we builtAgentSwarms.fyi—the ultimate hands-on educational platform for teaching agentic AI and multi-agent workflows. 🚀 The Core AgentSwarms Ecosystem: Real-World Architectures: Skip the generic hello-world loops. Learn production-grade systems like human-in-the-loop validation, automated multi-platform content multiplexers, and secure code-sandbox environments. Deterministic Cloud Guardrails: Deep dives into multi-cloud token economics, dynamic cost-optimized routing, and model evaluation metrics. Grassroots Engineering Focus: No corporate marketing fluff. Just raw, practical code patterns designed to bridge the gap between fragile prototypes and stable cloud deployments. 💣 The New Drop: 60+ Browser-Native TypeScript Notebooks We just completely re-engineered our learning workspace. We’ve added 60+ fully interactive TypeScript Notebooks running 100% natively in your browser. No pip install dependency hell, no local Docker setup, and zero environment friction. Read the architecture, tweak the system prompts or Zod schemas, hit play, and watch the streaming terminal execute live across the five absolute best frameworks in the ecosystem: 🟢 LangChain.js (Fundamentals & Middleware Guardrails) 🔀 LangGraph.js (Cyclic Graphs & Stateful Orchestration) 💾 LlamaIndex.ts (Sentence-Window Retrieval & RAG Triad Evals) ⚡ Vercel AI SDK (Streaming UI Integration) 🤖 OpenAI Agents SDK (Lightweight, low-boilerplate loops) Stop passively scrolling through video courses. Open a canvas, break the graph nodes, and start compiling real multi-agent swarms. 👉 Dive in for free: agentswarms.fyi/learn submitted by /u/Outside-Risk-8912 [link] [comments]

I measured my Claude Code MCP stack on two axes — byte savings AND cache-friendliness. My "best" byte-saver was defeating Anthropic's prompt cache (counter-example + open benchmark)

TL;DR — Single-axis benchmarks for MCPs, compressors, and retrieval layers can recommend a system that's strictly worse in production. The missing axis: cache-friendliness — whether the same input produces byte-identical bytes across runs, so Anthropic's prompt cache hits. In my coding-agent stack, my biggest byte-saver (retrieval MCP, 60–70% reduction) was defeating the 5-min TTL prompt cache on every call. Two runs of the same query produced different bytes because of rg --files-with-matches output order leaking through a Map insertion sequence into the final context. The fix was 2 lines: sort the rg hits before slicing, sort the Map entries by path. Byte savings unchanged, cache_friendly_score went from ~0% to 100%. https://preview.redd.it/x5foipotq93h1.png?width=1600&format=png&auto=webp&s=c0930422e882e23d1fc34ded25934c74db692a21 Article + open benchmark harness: Article: https://gregshevchenko.com/research/mcp-stack-token-economy/ Harness (stdlib-only Python, offline): https://github.com/g-shevchenko/mcp-token-savers — see methods/ for formal definitions, cluster-bootstrap CIs, Wilson CIs, preregistration, real-data Cohen's κ. What the harness measures: mean_ratio + CV across N≥5 runs per fixture → byte-saving axis unique_md5_count == 1 check → cache-friendliness axis (0–100%) 12-anti-pattern audit on tool definitions (DSA reference) What named alternatives publicly disclose: I surveyed the public docs for Cursor codebase index, Sourcegraph Cody, Aider repo-map, Microsoft LLMLingua / LLMLingua-2, Firecrawl / Jina Reader, RouteLLM / Martian (May 2026). https://preview.redd.it/ailemo1wq93h1.png?width=1600&format=png&auto=webp&s=4732f5d03f53ba95d2b5aaac0c7f21f1858a36a4 Limitations: I hypothesized that the prep layer triggers more downstream cache hits on subsequent turns. It didn't reach significance: Welch p=0.32, Cohen's d ≈ 0.18, N=137. Two-judge Cohen's κ on the corpus (cerebras-llama × groq-llama, N=25): κ = 0.5955 (moderate, below the 0.7 substantial threshold). 4 of 5 inter-judge disagreements concentrate on one task with an ambiguous acceptance criterion. Sharpening the spec would push κ to ~0.83. Disclosure: I'm the author. No commercial affiliation with the listed tools. The harness is MIT-licensed and takes any compressor as (str) -> str. Curious what cache_friendly_score looks like on others' Claude Code stacks. submitted by /u/Level_Credit1535 [link] [comments]

On "harness engineering": Are people actually building things or just giving impressive labels to "tweaking?"

I see a lot of posts and videos talking about harness engineering, or it could be context engineering, RAG, etc. The thing is, most of them talk about the concepts. And then I hear about all these people actually doing it. And my question is about this disconnect: what does it look like in practice? The way I understand it tools like Claude Code or OpenAI Codex are agents, and the logic that controls what gets fed to the model is the harness. So when people talk about "engineering the context," are they: writing actual programs CLI tools, pipelines, custom API wrappers that manage what gets sent to the model? or mostly just structuring their prompts well and calling it engineering? Same question for RAG--or any other oft-discussed topics: are people actually building retrieval pipelines from scratch, or are they standing up LlamaIndex / Mem0 and saying they're "using RAG" to infomaxx their AI agents? Not trying to be dismissive. I'm genuinely curious about what people are actually doing when they say they have applied these concepts to their agentic workflows. submitted by /u/josh_apptility [link] [comments]

LLM rankings are not a ladder: experimental results from a transitive benchmark graph [D]

https://preview.redd.it/668yjlucu80h1.png?width=2800&format=png&auto=webp&s=ca541488abb5262b06cfc13a9586efb19f24d644 I built a small website called LLM Win: https://llm-win.com It turns LLM benchmark results into a directed graph: If model A beats model B on benchmark X, add an edge A -> B. Then it searches for the shortest transitive chain between two models. The meme version is: Can LLaMA 2 7B beat Claude Opus 4.7? In an absurd transitive benchmark sense, sometimes yes. But I added a Report tab because the structure itself seems useful for model evaluation. Some experimental findings from the current Artificial Analysis data snapshot: Weak-to-strong reachability is high. I checked 126,937 pairs where the source model has lower Intelligence Index than the target model. 119,514 of them are reachable through benchmark win chains, for a reachable rate of 94.2%. Most paths are short. Among reachable weak-to-strong pairs: 2-3 hop paths account for 91.4%. So this is not mostly long-chain cherry-picking. Direct reversal triples are abundant. After treating non-positive benchmark values as missing, there are still about 119k direct weak-over-strong triples of the form: (source model, target model, benchmark), where the source has lower Intelligence Index but higher score on that benchmark. Some benchmarks create more reversals than others. Current high-reversal / useful-signal candidates include: Humanity's Last Exam, IFBench, AIME 2025, TAU2, SciCode Different benchmarks have different interpretations. For example, IFBench has roughly: reversal rate: ~17.5%, coverage: ~80.0%, correlation with Intelligence Index: r≈0.82. This suggests it may provide an independent skill signal rather than simply duplicating the overall ranking. My current interpretation: LLM rankings are better represented as a benchmark-specific capability graph than as a single ladder. Some reversals probably reflect real specialization; some reflect benchmark coverage limits, volatility, or measurement noise. The next question is whether reversal structure can help build better evaluation metrics: identify specialist models; identify volatile benchmarks; build robust generalist scores; select complementary benchmark sets; decompose models into capability fingerprints. Curious what people think: Is benchmark reversal structure a useful evaluation signal, or mostly an artifact of noisy benchmarks? submitted by /u/Spico197 [link] [comments]

I built a benchmark for AI “memory” in coding agents. looking for others to beat it.

Most AI memory benchmarks test semantic recall. But coding agents don't really fail like that. They don't just "forget", they break their own earlier decisions while they're still in the code. So I built a benchmark for that. It checks if an agent can actually stay consistent with project rules WHILE it's working, not just after the fact. It looks at things like: whether edits actually respect earlier architectural decisions if behavior stays consistent across multiple sessions (even when you throw noise at it) whether retrieval kicks in at the right moment — not just "yeah it's in memory somewhere" Repo (full harness + dataset + scoring): https://github.com/Alienfader/continuity-benchmarks Early numbers vs baseline + the usual RAG-style memory setups: ~3× better action alignment way stronger multi-session consistency retrieval timing matters way more than retrieval just being there I'm not saying this is the final word on agent memory. But it's exposing a failure mode most benchmarks aren't even looking at. So heres the challenge If you're building an agent memory system, RAG for code, long-context coding agents, persistent state / memory layers, run it on this benchmark. Drop your results, your setup, your comparisons. I really wanna see how tools like LangChain, LlamaIndex, and custom RAG stacks hold up in mutation-heavy workflows. We need memory systems we can actually compare, not just ones that sound good on paper. https://preview.redd.it/dkm2ulxsyzzg1.png?width=2624&format=png&auto=webp&s=67f0299395708818aa3d7346ddae2ad0c5c4a6ba submitted by /u/Alienfader [link] [comments]

I built persistent memory for Claude — local stack, MCP integration, 39ms retrieval. Sharing the architecture.

If you use Claude heavily, you've felt this: every session starts from zero. You re-explain context, Claude helps, the window closes, and the next session has no idea what you decided yesterday. The standard workaround is a markdown wiki Claude reads — but as the wiki grows, every "what did we decide about X" question burns thousands of tokens grepping and re-reading whole pages. I spent the last few weeks building a persistent memory layer to fix both problems. It runs entirely on my own machine, integrates via MCP, and lives between Claude and my existing wiki. Sharing the architecture and what I learned in case anyone wants to build their own. What it does Semantic retrieval over my wiki. Instead of Claude grepping pages, my MCP server returns the most relevant chunks for any query in ~50ms. 82% mean token reduction on a 10-query eval set vs the grep+Read baseline. F1 retrieval quality is also better — cheaper and more accurate. Session crystallization. End-of-session, conversations get compressed into a structured "L4 node" with summary + decisions + open threads, indexed alongside wiki content. Tomorrow I can ask "what did we decide about X" and Claude pulls last session's decision verbatim. Lazy-spawned local models. Embedder + chat model run as subprocesses that the supervisor spawns on first use and reaps after 1 hour idle. Boot cost is zero — nothing loaded until needed. The architecture (four layers) Inspired by Andrej Karpathy's writing on LLM-native wikis, then formalized into a build spec: L0 — append-only event log (SQLite). Every input/output, content-hashed. L1 — structured facts with confidence + decay (deferred to next phase) L2/L3 — derived prose + cross-cutting summaries (the hand-edited wiki plays this role for now) L4 — crystallized session nodes. Summary, decisions, open threads. Indexed in the same vector store as wiki chunks so retrieval finds both naturally. The stack Qdrant in Docker for vector search llama.cpp running Qwen3-Embedding-4B (GPU) and Qwen3.5-2B-Q4_K_M (CPU) FastMCP server exposing 7 tools (retrieve, crystallize_session, list_sessions, get_l4_node, index_status, reindex, shutdown_models) Cowork plugin for Claude Desktop integration; also works with Claude Code via standard MCP config No cloud, no API keys, $0 marginal cost per query. Numbers Token reduction: 82.7% mean, 86.2% median vs grep+Read baseline Retrieval F1: 0.50 vs 0.20 baseline Embed cold-start: ~4s. Hot-path p95: 39ms (was 2241ms before fixing one specific bug — see below) L4 session retrieval eval: 0.920 mean score (gate 0.6) 738 chunks currently indexed across 104 markdown files The most useful thing I learned Hot-path retrieve was inexplicably stuck at 2241ms p95 even though the embedding model was fully GPU-resident on a 4070 Ti Super. Spent hours blaming GPU offload, prompt cache, KV pre-allocation. The actual cause: every httpx.post() was opening a fresh TCP connection, and Windows localhost handshakes take ~2 seconds. A 5-line change — switching to a persistent httpx.Client with keep-alive — dropped p95 to 39ms. 57× speedup. Lesson: latency that's suspiciously consistent (2240, 2237, 2241, 2227, 2239 ms) is a fixed cost, not a compute cost. If your local-MCP integration feels slow on Windows, check connection reuse before you blame the model. A few other things that surprised me Qwen3 thinking mode silently consumes the generation budget. Crystallization was returning empty content. Logs showed exactly 2000 tokens generated (the cap). Turned out Qwen3 emits ... blocks the chat handler strips before populating message.content. With JSON grammar enforced, the model spent all 2000 tokens "thinking" and never emitted JSON. Fix: pass chat_template_kwargs: {enable_thinking: false} via extra_body (requires --jinja on llama-server). The MCP plugin needed to register against the right config file. Cowork (Claude Desktop's agentic mode) doesn't read ~/.claude.json like Claude Code does. The first attempt at MCP registration silently went to the wrong file. The fix was packaging the LKS service as a proper Cowork plugin (.plugin bundle) — Cowork has a plugin system distinct from raw MCP server registration. If you're trying to wire a custom MCP server into Cowork, this is the path. What it doesn't do (yet) No automatic conversation capture — L0 ingestion is manual or via end-of-session crystallization No L1 fact extraction yet (next phase) — retrieval is over markdown chunks + L4 nodes today Wiki is still source-of-truth; no automatic conflict resolution Solo deployment only; no federation or multi-user Tested on Windows; Linux/Mac would need a small tweak to the supervisor (it uses subprocess.CREATE_NEW_PROCESS_GROUP for clean Windows termination) Full write-up Architecture, phased build narrative, all five lessons-learned bug stories, the setup walkthrough, and the roadmap: https://gist.github.com/tyoung515-svg/5fd5279f46d935f517cda89146c94685

eTPS Site Plan – Simple Leaderboard + What You’ll Actually See

Building on the last post, here’s what the first version of effectiveTPS will look like. **Core display (v1):** - Clean table comparing popular local models - Raw TPS (the marketing number everyone shows) - eTPS (the new metric that actually measures useful output in real conversations) - Time to First Token (how long you wait before it starts replying) - Effectiveness Index = (eTPS ÷ Raw TPS) × 100 — higher is better **Example leaderboard (early test data):** | Model | Raw TPS | eTPS | Time to First Token | Effectiveness Index | |--------------------|---------|--------|---------------------|---------------------| | Llama 3.1 70B | 45.2 | 38.7 | 1.4s | **86** | | Qwen2.5-32B | 68.4 | 52.1 | 0.8s | **76** | | Gemma 2 27B | 71.3 | 44.6 | 0.6s | **63** | I’ve been running these tests through a structured multi-turn analysis framework I built to evaluate complex workflows. That’s how eTPS was stress-tested — not just single-turn benchmarks, but real back-and-forth sessions. Advanced mode (toggle) will add latency percentiles, cost-per-quality, and consistency scoring later. For v1 the goal is to keep it dead simple and immediately useful, even if you’re not deep into AI. The whole point is to cut through the noise and show which models actually deliver useful work, not just raw speed. What do you think should be added (or removed) for the first version? Any metrics you’d want to see front-and-center? **TL;DR:** Simple leaderboard with Raw TPS, eTPS, Time to First Token, and a clear Effectiveness Index. Advanced stuff stays hidden until you want it. Feedback welcome. submitted by /u/axendo [link] [comments]

A Hackable ML Compiler Stack in 5,000 Lines of Python [P]

Hey r/MachineLearning, The modern ML (LLM) compiler stack is brutal. TVM is 500K+ lines of C++. PyTorch piles Dynamo, Inductor, and Triton on top of each other. Then there's XLA, MLIR, Halide, Mojo. There is no tutorial that covers the high-level design of an ML compiler without dropping you straight into the guts of one of these frameworks. I built a reference compiler from scratch in ~5K lines of pure Python that emits raw CUDA. It takes a small model (TinyLlama, Qwen2.5-7B) and lowers it to a sequence of CUDA kernels through six IRs. The goal isn't to beat Triton; it is to build a hackable, easy-to-follow compiler. Full article: A Principled ML Compiler Stack in 5,000 Lines of Python Repo: deplodock The pipeline consists of six IRs, each closer to the hardware than the last. Walking the following PyTorch code through every stage (real reference compiler output with names shortened for brevity and comments added): torch.relu(torch.matmul(x + bias, w)) # x: (16, 64), bias: (64,), w: (64, 16) Torch IR. Captured FX graph, 1:1 mirror of PyTorch ops: bias_bc = bias[j] -> (16, 64) float32 add = add(x, bias_bc) -> (16, 64) float32 matmul = matmul(add, w, has_bias=False) -> (16, 16) float32 relu = relu(matmul) -> (16, 16) float32 Tensor IR. Every op is decomposed into Elementwise / Reduction / IndexMap. Minimal unified op surface, so future frontends (ONNX, JAX) plug in without touching downstream passes: bias_bc = bias[j] -> (16, 64) float32 w_bc = w[j, k] -> (16, 64, 16) float32 add = add(x, bias_bc) -> (16, 64) float32 add_bc = add[i, j] -> (16, 64, 16) float32 prod = multiply(add_bc, w_bc) -> (16, 64, 16) float32 red = sum(prod, axis=-2) -> (16, 1, 16) float32 matmul = red[i, na, j] -> (16, 16) float32 relu = relu(matmul) -> (16, 16) float32 The (16, 64, 16) intermediate looks ruinous, but it's never materialized; the next stage fuses it out. Loop IR. Each kernel has a loop nest fused with adjacent kernels. Prologue, broadcasted multiply, reduction, output layout, and epilogue all collapse into a single loop nest with no intermediate buffers. === merged_relu -> relu === for a0 in 0..16: # free (M) for a1 in 0..16: # free (N) for a2 in 0..64: # reduce (K) in0 = load bias[a2] in1 = load x[a0, a2] in2 = load w[a2, a1] v0 = add(in1, in0) # prologue (inside reduce) v1 = multiply(v0, in2) acc0 <- add(acc0, v1) v2 = relu(acc0) # epilogue (outside reduce) merged_relu[a0, a1] = v2 Tile IR. The first GPU-aware IR. Loop axes get scheduled onto threads/blocks, Stage hoists shared inputs into shared memory, and a 2×2 register tile lets each thread accumulate four outputs at once. The K-axis is tiled into two outer iterations of 32-wide reduce. Three-stage annotations below carry the heaviest optimizations: buffers=2@a2 — double-buffer the smem allocation along the a2 K-tile loop, so loads for iteration a2+1 overlap compute for a2. async — emit cp.async.ca.shared.global so the warp doesn't block on global→smem transfers; pairs with commit_group/wait_group fences in Kernel IR. pad=(0, 1, 0) — add 1 element of padding to the middle smem dim so warp-wide loads don't all hit the same bank.kernel k_relu_reduce Tile(axes=(a0:8=THREAD, a1:8=THREAD)): for a2 in 0..2: # K-tile # meta: double-buffered, sync (small, no async needed) bias_smem = Stage(bias, origin=((a2 * 32)), slab=(a3:32@0)) buffers=2@a2 ​ kernel k_relu_reduce Tile(axes=(a0:8=THREAD, a1:8=THREAD)): for a2 in 0..2: # K-tile bias_smem = Stage(bias, origin=((a2 * 32)), slab=(a3:32@0)) buffers=2@a2 x_smem = Stage(x, origin=(0, (a2 * 32)), slab=(a0:8@0, a3:32@1, cell:2@0)) pad=(0, 1, 0) buffers=2@a2 async w_smem = Stage(w, origin=((a2 * 32), 0), slab=(a3:32@0, a1:8@1, cell:2@1)) buffers=2@a2 async # reduce for a3 in 0..32: in0 = load bias_smem[a2, a3] in1 = load x_smem[a2, a0, a3, 0]; in2 = load x_smem[a2, a0, a3, 1] in3 = load w_smem[a2, a3, a1, 0]; in4 = load w_smem[a2, a3, a1, 1] # prologue, reused 2× across N v0 = add(in1, in0); v1 = add(in2, in0) # 2×2 register tile acc0 <- add(acc0, multiply(v0, in3)) acc1 <- add(acc1, multiply(v0, in4)) acc2 <- add(acc2, multiply(v1, in3)) acc3 <- add(acc3, multiply(v1, in4)) # epilogue relu[a0*2, a1*2 ] = relu(acc0) relu[a0*2, a1*2 + 1] = relu(acc1) relu[a0*2 + 1, a1*2 ] = relu(acc2) relu[a0*2 + 1, a1*2 + 1] = relu(acc3) Kernel IR. Schedule materialized into hardware primitives. THREAD/BLOCK become threadIdx/blockIdx, async Stage becomes Smem + cp.async fill with commit/wait fences, sync Stage becomes a strided fill loop. Framework-agnostic: same IR could lower to Metal or HIP: kernel k_relu_reduce Tile(axes=(a0:8=THREAD, a1:8=THREAD)): Init(acc0..acc3, op=add) for a2 in 0..2: # K-tile Smem bias_smem[2, 32] (float) StridedLoop(flat = a0*8 + a1; < 32; += 64): bias_smem[a2, flat] = load bias[a2*32 + flat] Sync # pad row to 33 to kill bank conflicts Smem x_smem[2, 8, 33, 2] (float) StridedLoop(flat = a0*8 + a1; < 512; += 64): cp.async x_smem[a2, flat/64, (flat/2)%32, flat%2] <- x[flat/64*2 + flat%2, a2*3

How would you build an automated commentary engine for daily trade attribution at scale? [R]

Hey everyone, I'm currently working through a problem in the market risk reporting space and would love to hear how you all would architect this. The Use Case: &gt; I have thousands of trades coming in at varying frequencies (daily, monthly). I need to build a system that automatically analyzes this time-series data and generates a precise, human-readable commentary detailing exactly what changed and why. For example, the output needs to be a judgment like: "The portfolio variance today was +$50k, driven primarily by a shift in the Equities asset class, with the largest single contributor being Trade XYZ." The Dilemma: * The Math: Absolute precision is non-negotiable. I know I can't just dump raw data into an LLM and ask it to calculate attribution, because it will hallucinate the math. I usually rely on Python and Polars for the high-performance deterministic crunching. * The Rigidity: If I hardcode every single attribution scenario (by asset class, by region, by specific trade) into a static ETL pipeline before feeding it to an LLM for summarization, the system becomes too rigid to handle new business scenarios automatically. My Question: How would you strike the balance between deterministic mathematical precision and dynamic natural language generation? Are you using Agentic workflows (e.g., having an LLM dynamically write and execute Polars/pandas code in a sandbox)? Or are you sticking to pre-calculated cubes and heavily structured context prompts? Any specific frameworks (LangChain, LlamaIndex, PandasAI, etc.) or design patterns you've had success within financial reporting? Appreciate any insights!

Lessons learned building a no-hallucination RAG for Islamic finance similarity gates beat prompt engineering

Lessons learned building a no-hallucination RAG for Islamic finance similarity gates beat prompt engineering I kept getting blocked trying to share this so I'll cut straight to the technical meat. The problem: Islamic finance rulings vary by jurisdiction and a wrong answer has real consequences. Telling an LLM "refuse if unsure" in a system prompt is not enough. It still speculates. The fix that actually worked: kill the LLM call entirely at retrieval time. If top-k chunks score below 0.7 cosine similarity, the function returns a hardcoded refusal string. The LLM never sees the query. No amount of clever prompting is as reliable as just not calling the model. Other things worth knowing: FAISS on HuggingFace Spaces free tier is ephemeral. Every cold start wipes it. Solution: push the index to a private HF Dataset, pull it on startup via FastAPI lifespan event. PyPDF2 on scanned PDFs returns nothing. AAOIFI documents are scanned images. trafilatura on clean HTML beats OCR every time if a web version exists. Jurisdiction metadata on every chunk is not optional. source_name + source_url + jurisdiction in every chunk. A Malaysian SC ruling and a Gulf fatwa can say opposite things on the same question. Stack: FastAPI + LlamaIndex + FAISS + sentence-transformers + Mistral-Small-3.1-24B via HF Inference API. Netlify Function as proxy so credentials never touch the browser. What threshold do you use for retrieval refusal in high-stakes domains? submitted by /u/Particular-Plate7051 [link] [comments]