DSPy

DSPy AI

DSPy AI

DSPy AI

DSPy AI

DSPy AI

DSPy AI

DSPy AI

DSPy AI

DSPy AI

DSPy AI

CANTANTE: Optimizing Agentic Systems via Contrastive Credit Attribution [R]

LLM-based multi-agent systems have demonstrated strong performance across complex real-world tasks, such as software engineering, predictive modeling, and retrieval-augmented generation. Yet, automating their configuration remains a structural challenge. Researchers are often forced into manual, trial-and-error prompt tuning, where a change to a single agent shifts the global output in ways that are difficult to trace. The core bottleneck is credit assignment: while the parameters governing agent behavior are local, performance scores are only available at the global system level. This makes optimization fundamentally difficult because we do not inherently know which agents contributed positively or negatively to the outcome. CANTANTE is an attempt to take a different path: treating agent prompts as parameters learned from task rewards rather than tuned by hand. By solving the credit assignment problem, we can move from brittle, hand-crafted agent demos to trustworthy systems that are actually autonomous and useful in practice. CANTANTE's algorithm in short (see second image): Let local optimizers suggest configurations (e.g., prompts). Evaluate different configurations on the same queries, capturing reasoning traces and system scores. Let an attributer compare these rollouts and assign each agent a credit, thereby decomposing the global reward into per-agent update signals. Feed those credits to any local optimizer; for the experiments, we use CAPO, our prompt optimizer from prior work at AutoML 2025. Evaluated against the DSPy-solutions GEPA and MIPROv2 on MBPP (Programming Benchmark), GSM8K (Mathematical Reasoning Benchmark), and HotpotQA (Retrieval Benchmark), CANTANTE: • Achieves the best average rank, • beats the strongest baseline by +18.9 points on MBPP and +12.5 on GSM8K, and • maintains inference time cost compared to unoptimized prompts. 🔗 Link to the paper: https://arxiv.org/abs/2605.13295 💻 Link to the repo: https://github.com/finitearth/cantante If you're researching multi-agent architectures or automated prompt engineering, I'd love to hear what's working (and breaking) for you right now. submitted by /u/finitearth [link] [comments]

[D] Litellm supply chain attack and what it means for api key management

If you missed it, litellm versions 1.82.7 and 1.82.8 on pypi got compromised. malicious .pth file that runs on every python process start, no import needed. it scrapes ssh keys, aws/gcp creds, k8s secrets, crypto wallets, env vars (aka all your api keys). karpathy posted about it. the attacker got in through trivy (a vuln scanner ironically) and stole litellm's publish token. 2000+ packages depend on litellm downstream including dspy and mlflow. the only reason anyone caught it was because the malicious code had a fork bomb bug that crashed machines. This made me rethink how i manage model api keys. having keys for openai, anthropic, google, deepseek all sitting in .env files across projects is a massive attack surface. switched to running everything through zenmux a while back so theres only one api key to rotate if something goes wrong. not a perfect solution but at least i dont have 6 different provider keys scattered everywhere. Run pip show litellm right now. if youre on anything above 1.82.6 treat it as full compromise. submitted by /u/Zestyclose_Ring1123 [link] [comments]

If DSPy is so great, why isn't anyone using it?