PayloopPayloop
CommunityVoicesToolsDiscoverLeaderboardReportsBlog
Save Up to 65% on AI
Powered by Payloop — LLM Cost Intelligence
Free Tool · No signup

Free LLM Stack Analyzer

Paste a public GitHub repo URL. We’ll detect every AI provider, model, framework, and anti-pattern — and tell you what it would cost to run in production.

Try:LangChainLlamaIndexAnthropic SDKOpenAI Python

How it works

Step 1
Paste the repo URL

Any public GitHub repository — owner/name or a full URL. Monorepos, private mirrors, forks all welcome.

Step 2
We scan source + config

The scanner reads source code, lockfiles, env files, CI config, and the GitHub API to build a complete picture.

Step 3
Read the audit

27 graded categories — costs, architecture, security, production readiness, deprecation risk, and more.

What we detect

Each audit lands in a sub-page (one per dimension). Open the overview and drill into Costs, Optimization, Security, RAG, Per-Request, Production, Team — 25 graded views with repo-specific narrative.

Architecture

Providers, models, frameworks, agent loops, vector stores, embeddings, observability — stack composition end-to-end.

Costs

Per-model pricing, per-request token floor, system-prompt overhead, tool-definition cost, agent-loop multipliers.

Optimization

30+ anti-patterns: missing prompt caching, no rate limiting, no fallback model, unbounded loops, wrong model tier.

Security

Guardrails, content filtering, rate limiting, error handling, OWASP LLM Top 10 coverage, exposed API keys.

RAG pipeline

Vector stores, embedding models, retrievers, rerankers, chunking strategy, and overall retrieval cost.

Production readiness

CI maturity, releases, Docker / Kubernetes setup, README quality, contributor velocity, deployment cadence.

Frequently asked questions

What does the free LLM audit detect?

The audit scans the repository's source code, lockfiles, environment files, and CI configuration to detect AI providers (OpenAI, Anthropic, Google, Bedrock, etc.), specific models in use, framework choice (LangChain, LlamaIndex, Vercel AI, CrewAI, etc.), vector stores, embedding models, agent loops, guardrails, observability tooling, and ~30 well-known anti-patterns (missing prompt caching, no rate limiting, no fallback model, unbounded loops, etc.).

Is the analysis private?

The repository must be public. We don't store source code or secrets — we read public files via the GitHub API, compute an analysis, and persist only the derived metrics (providers, models, scores, optimization opportunities). The audit URL is shareable and indexable so others can see the analysis too.

How accurate are the cost estimates?

Cost estimates are derived from detected call sites and the per-million-token pricing of each model in use, applied to a typical request shape (system prompt + tools + agent multiplier). Without production usage telemetry the absolute numbers are directional; the relative cost between models and the savings opportunities are typically within ±15% of measured spend.

What's the difference between this and `/audit/{owner}/{repo}`?

There's no difference — the free audit page is a thin entry point that runs the same analysis pipeline. Submitting a repo here routes you to /audit/{owner}/{repo}, the persistent audit URL that you can share or come back to. Repeated visits use the cached analysis.

Does it work for monorepos?

Yes. The scanner walks the entire tree and aggregates findings across every package or subdirectory. The resulting audit shows file-level call sites so you can attribute usage to specific apps in the monorepo.

Do I need an API key or to sign up?

No signup, no API key. The free audit is fully public and runs on a public GitHub URL. If you want to track audits over time, integrate observability, or rerun on a schedule, sign in — but it's not required to get an analysis.

Want something deeper? Generate a custom AI cost report or run the API key exposure scanner. Browse live audits or compare LangChain stacks across the ecosystem.