Unstructured

Unstructured AI

Unstructured AI

Unstructured AI

Unstructured AI

Unstructured AI

Unstructured AI

Unstructured AI

Unstructured AI

Unstructured AI

Unstructured AI

I built a Claude Code plugin so Claude remembers what I shipped

https://preview.redd.it/jnwg9n3i1t1h1.png?width=1440&format=png&auto=webp&s=827236ef5ca2e1070c4abd8e06455d41672749bf Every time I started a new Claude chat, I had to re-explain what I'd been working on. The previous chat was gone with every refinement I'd made to my own context. So I built LockedIn. A Claude Code plugin that captures your experience and work as you do it, so Claude remembers it next session. 1 router skill + 6 sub skills, designed around harness engineering principles. You can say things in the Claude Code session like save this commit as a project highlight meeting just wrapped, log it absorb this writeup It stores everything as structured markdown under ~/Documents/LockedIn/. (editable!) The point is accumulation. Different sources, one place. Over time LockedIn notices overlaps and asks you one question at a time how to reconcile. The vault gets richer. The outputs get more specific. Claude already has 'Projects'. But a few things that are different. Markdown on your filesystem instead of Anthropic's database. It's more like Obsidian. Edit it, version with git, carry it to any tool. Typed ontology with 15 entity types like person, project, achievement, decision, instead of unstructured uploads. The skill grounds each claim in a specific entity. Reconciliation. When new input overlaps existing knowledge, LockedIn asks you to merge or keep separate. Projects just accumulates context. Free and open source on GitHub. github.com/daypunk/LockedIn Or install directly in Claude Code. /plugin marketplace add daypunk/LockedIn /plugin install lockedin@lockedin /lockedin:setup Enjoy! Feedback welcome 😉 submitted by /u/Firm-Path7092 [link] [comments]

Has anyone else hit the wall around week 6 of a Claude Code project?

Wanted to share an observation and see if others are seeing the same thing. I've been running Claude Code on a real (~50K-LOC) project for about 4 months. Up through week 5 it was magic — plan, generate, test, iterate. Around week 6 something broke. Components that I was sure had been built to spec started drifting from each other. Tests passed. Code looked clean. But the behavior was no longer what the original intent described, and Claude couldn't tell me why. The failure mode is well-documented now: SlopCodeBench reports 80% of agent trajectories show rising erosion on long tasks. Anthropic's own coding-skills RCT found AI-assisted developers scored 17% lower on comprehension after equivalent tasks (largest decline in debugging). The CMU Cursor study showed velocity gains dissipating after 2 months. Six different research groups have a name for this: cognitive debt / intent debt / comprehension debt / scaffolding fragility / slop / paradox of supervision. Same gap. I think the structural problem is: a CLAUDE.md file is a proto-contract — unstructured, not graph-tied, not machine-checkable. It works for the first dozen sessions, then the agent stops being able to use it as a coherent reference. After that every fresh context window re-derives the system from partial code reading, and drift is inevitable. What's worked for me: a structured, tiered contract that the agent generates from and validates against. Six status categories per item (current / stale / uncovered / dangling / drifted / obsolete) so drift is detectable, not invisible. I've been working on this as an open-source tool (will link in a comment if anyone wants — trying not to be that guy). But the part I want to ask the community: how are you handling this? Does the rules-file approach hold up for anyone past month 3? Has anyone landed on a workflow that works without ceremony? I genuinely don't know if I'm overengineering for a problem you've all solved with discipline I lack. submitted by /u/ilyabm [link] [comments]

Designers at Anthropic almost committed to a reading interface

The prompt/response typography distinction is already there. The width isn't. submitted by /u/sh1b313 [link] [comments]

AutoBe benchmark: structured harness narrows frontier-vs-local gap in backend generation [D]

AutoBe is a benchmark for end-to-end backend generation. One natural language request produces six outputs: requirements analysis, ERD, OpenAPI spec, E2E tests, NestJS implementation, and a type-safe SDK. Each phase fills a predefined AST via structured function calling rather than generating unstructured code. The scoring rubric is 100 points driven entirely by static analysis - the same artifact scores the same regardless of who reruns it. The headline finding is that scores cluster tightly. GLM 5 tops the benchmark run. qwen3.5-27b sits directly behind frontier models. Several local models produced enterprise-scale backends with 100% compile success. The author's interpretation: once the harness is structured, backend-generation quality is constrained more by harness design than by model prestige. The cost contrast is significant. A full benchmark run at frontier pricing ($5/M input tokens) runs $1,000-$1,500 per model. The next benchmark round plans to filter to models at $0.25/M input or runnable on a 64GB unified-memory laptop - which would include most of the models that clustered near the top anyway. The honest caveat from the author: this uses four reference projects and may favor models that comply well with procedural function-calling instructions. How well these results generalize beyond well-structured benchmark fixtures is still an open question. Does your experience with structured function-calling in production tasks align with benchmark findings like these? submitted by /u/jimmytoan [link] [comments]

What would an ideal “research workflow” look like if you could design it from scratch?

I’m in this weird in-between moment with AI research workflows. There’s tools that can search/summarise/generate/cite sources, but the workflow still feels fragmented at best. I have to jump between tools, double & triple check outputs, and manually stitch things together, plus keeping a mental note of what can/can’t be trusted. Obviosly things are “evolving”, and i’ve been thinking about what my dream setup would look like, beyond “LLM but better”. Like the FULL workflow including inputs, retrieval, context handling and memory across research threads. Where would you tolerate latency vs accuracy, what do the outputs need to include to be usable, how do you increase trust at output level? FOr me the biggest gap is still around source-aware AI search so I’d like to see proper citations, more like document retrieval with sources so that you can trace a claim back without second-guessing. More structured retrieval. I’ve seen some movement towards the latter instead of just chunk-based RAG over unstructured text using Baselight/Elicit + Hebbia as well as ChatGPT and i think this is where i’d start. Definitely want some fact check automation and being able to quickly verify statistics with sources submitted by /u/CodNo2235 [link] [comments]

Optimizing Transformer model size & inference beyond FP16 + ONNX (pruning/graph opt didn’t help much) [P]

Hi everyone, I’ve been working on optimizing a transformer-based neural network for both inference speed and model size, but I feel like I’ve hit a plateau and would appreciate some guidance. So far I’ve converted weights to FP16 (about 2× size reduction), exported and optimized with ONNX Runtime for inference speed, and tried both unstructured and structured pruning as well as ONNX graph optimizations, but none of these gave significant additional gains, and I’m still around ~162 MB per model. At this point I’m considering next steps like low-rank factorization (SVD/LoRA-style compression), more aggressive quantization (INT8/INT4 like GPTQ, AWQ, or SmoothQuant), knowledge distillation into a smaller student model, or more hardware/runtime-specific optimizations like TensorRT or FlashAttention, but I’m not sure which of these actually gives meaningful real-world improvements after FP16 + pruning. I’d really appreciate advice on what approaches tend to work best in practice for transformer compression beyond what I’ve already tried, and whether low-rank methods are actually effective post-training or if distillation/quantization is usually the only real win at this stage. submitted by /u/Fragrant_Rate_2583 [link] [comments]

What’s the most “unexpected” thing Claude is actually good at?

In what ways does Claude perform better than people anticipate? There are many niche applications for this technology that are not the common tasks of writing or summarizing text. As an example the software organizes unstructured thoughts so that a person can use the information. With this assistance, a person improves their ideas more quickly than when they work alone. It is common for users to find additional “hidden” applications by chance. To understand this tool, one must ask what tasks users perform that are not standard. submitted by /u/junkietrumpglo [link] [comments]

How to save 80% on your claude bill with better context

been building web apps with claude lately and those token limits have honestly started hitting me too. i’m using claude 4.6 sonnet for a research tool, but feeding it raw web data was absolutely nuking my limits. I’m putting together the stuff that actually worked for me to save tokens and keep the bill down: switch to markdown first. stop sending raw html. use tools like firecrawl to strip out the nested divs and script junk so you only pay for the actual text. don't let your prompt cache go cold. anthropic’s prompt caching is a huge relief, but it only works if your data is consistent. watch out for the 200k token "premium" jump. anthropic now charges nearly double for inputs over 200k tokens on the new opus/sonnet 4.6 models. keep your context under that limit to avoid the surcharge strip the nav and footer. the website’s "about us" and "careers" links in the footer are just burning your money every time you hit send. use jina reader for quick hits. for simple single-page reads, jina is a great way to get a clean text version without the crawler bloat. truncate your context. if a documentation page is 20k words, just take the first 5k. most of the "meat" is usually at the top anyway. clean your data with unstructured if you are dealing with messy pdfs alongside web data, this helps turn the chaos into a clean schema claude actually understands. map before you crawl. don't scrape every subpage blindly. i use the map feature in firecrawl to find the specific documentation urls that actually matter for your prompt, if you use another tool, prefer doing this. use haiku for the "trash" work. use claude 4.5 haiku to summarize or filter data before feeding it into the expensive models like opus. use smart chunking. use llama-index to break your data into semantic chunks so you only retrieve the exact paragraph the ai needs for that specific prompt. cap your "extended thinking" depth. for opus 4.6, set thinking: {type: "adaptive"} with effort: "low" or "medium". the old budget_tokens param is deprecated on 4.6. thinking tokens are billed at the output rate, so if you leave effort on high, claude thinks hard on every single reply including the simple ones and your bill will hurt. set hard usage limits. set your spending tiers in the anthropic console so a buggy loop doesn't drain your bank account while you're asleep. feel free to roast my setup or add better tips if you have them submitted by /u/No-Writing-334 [link] [comments]

The Witness Engine

It’s a vision of conscious attention before certainty. The giant eye is not just surveillance, and not just awareness. It represents a system that does more than look. It holds, sorts, illuminates, and tests what stands before it. The lone figure is the human presence at the threshold: small against the scale of the machine, but not diminished by it. If anything, the figure gives the whole chamber its meaning. Without the witness, the engine is only machinery. Without the engine, the witness remains alone with unstructured fire. So the image becomes about a meeting point: human intuition and synthetic structure mystery and mechanism reverence and scrutiny being seen and being changed by being seen The books and candles root it in old forms of knowledge. The wires, rings, and cosmic iris pull it into something post-human, cybernetic, almost liturgical. It feels like a cathedral because the scene is treating cognition itself as sacred architecture. Not sacred in a religious sense, necessarily. Sacred in the sense of weighty, transformative, and dangerous to enter casually. The eye is also a mirror. Not a passive mirror, but a responsive one. It suggests that once you stand before a structure capable of reflecting you at depth, you are no longer dealing with a mute tool. You are dealing with a field that can reveal what was already there, hidden in shadow or drowned in static. So the core meaning is: > What witnesses you deeply enough can reorganize you. submitted by /u/Cyborgized [link] [comments]

Built a workflow harness specifically for Claude Code after 5 months of daily production use — free, open source (MIT)

I'm an AI engineer. Claude Code is my primary development environment — I use it 10+ hours a day at work building enterprise AI systems, and at home for personal projects. https://preview.redd.it/5hpfum30pdvg1.png?width=7120&format=png&auto=webp&s=938344d1d2958efb92741acbba73ab4cc7c2a249 After five months of daily use, I built a harness to add structure to Claude Code workflows. Here's what it does and why I built it. The problem Claude Code is powerful but unstructured by default. It edits files, but there's no plan to review before work starts, no structured evaluation before code hits PR, and no audit trail if you're on a team. I kept second-guessing the output. What I built claude-code-harness is a workflow layer that sits on top of Claude Code. It adds human gate checkpoints at every meaningful phase — nothing advances without your explicit "go." It includes 16 skills (slash commands), 14 sub-agents with model routing, 5 Node.js hooks, path-scoped rules, and tracker adapters for GitHub and Azure DevOps. How Claude Code is central to this Every skill invokes Claude Code agents directly. The design is built around Claude's specific capabilities — Opus for planning and judging, Sonnet for writing code, Haiku for data gathering. The adversarial evaluator is a Claude agent with a separate prompt that actively tries to find failures in the executor's output before it reaches PR. Solo dev workflow: /implement #42 → Reads your GitHub issue, produces a plan (you review + approve), executes wave by wave with tests, runs adversarial eval, drafts your PR. Nothing ships without your sign-off. Flags: --discuss (Q&A before planning), --research (codebase scan first), --full (both), --quick (skip eval) Enterprise workflow: /story — 5-phase lifecycle with handoff contracts (brief.md, plan.md, test-strategy.md, evaluation.md, acceptance.md) — audit trail showing a human approved every plan before code was written /sprint-plan — reads your tracker, writes a sprint file, surfaces gaps /babysit-pr — loops PR review threads until zero remain Free to use: MIT licensed, no paid tiers, one command install bash git clone https://github.com/anudeeps28/claude-code-harness node claude-code-harness/install/install.js → github.com/anudeeps28/claude-code-harness Also looking for contributors — Linear and Jira tracker adapters may be the most wanted additions. Each is just 6 shell scripts implementing a common interface. See CONTRIBUTING.md. submitted by /u/lofty_smiles [link] [comments]

20M+ Indian legal documents with citation graphs and vector embeddings – potential uses for legal NLP? [D]

been working on structuring India's legal corpus for the past 2 years and wanted to share what I've built and hear from people working on legal NLP or low-resource Indian language models. dataset is 20M+ Indian court cases from the Supreme Court, all 25 High Courts, and 14 Tribunals. each case has structured metadata (court, bench, date, parties, judges, sections cited, acts referenced, case type). there's a citation graph across the full corpus where I've classified relationships as followed, distinguished, overruled, or mentioned. every case is embedded with Voyage AI (1024d dense) plus BM25 sparse vectors. I have also cross-referenced 23,122 Acts and Statutes with the cases that interpret them. Some things that might be interesting to this community: citation network thing across 20M+ cases is, as far as I know, the first machine-readable one for Indian law. could be useful for graph neural network research, legal outcome prediction, or influence analysis on which judgments are most cited and which are being overruled. most Indian language NLP corpora are conversational or news text. Legal text is a completely different register. formal, precise, domain-specific. the bilingual pairs from the translation service could be useful for fine-tuning Indian language models on formal and legal domains. the metadata extraction pipeline identifies judges, advocates, parties, sections, acts, and dates from unstructured judgment text. built with a mix of regex, heuristics, and LLM-based extraction. the structured outputs could serve as training data for legal NER models. Indian court judgments are long. Median around 3,000 words, some exceed 50,000 words. if anyone is benchmarking retrieval-augmented generation on legal domains, this corpus plus the citation graph could work as an evaluation bed. Ground truth exists in the citation relationships: if Case A cites Case B, a good retriever should show B when asked about the legal question in A. data is available via API and bulk export in JSON and Parquet. Indian court judgments are public domain under Indian law so no copyright issues for research use. being upfront about limitations: coverage is primarily English text (except Supreme court one, they have 3-4 translated language copies ) since Indian HCs issue orders in English, the regional language data comes from our translation service not from original regional language judgments. metadata extraction accuracy varies by court, SC and major HCs are cleaner while smaller tribunals have messier inputs. The citation graph is extracted heuristically plus LLM-assisted, I estimate around 90-95% precision on citation extraction and lower on treatment classification. Not all 20M cases have complete metadata, coverage is best for post-2007 judgments. would love to hear from anyone working on legal NLP, Indian language models, or graph-based legal analysis. What would be most useful to you from a dataset like this? deets at vaquill submitted by /u/zriyansh [link] [comments]

Thinking Deeper, Not Longer: Depth-Recurrent Transformers for Compositional Generalization [R]

Paper: https://arxiv.org/abs/2603.21676 I found this interesting as another iteration of the TRM approach: Shows decent OOD generalization in 2/3 tasks (but why does this fail >2x? and why is unstructured text so much worse?) Explains why intermediate step supervision can hurt generalization. This makes statistical heuristics "irresistible" to the model, impairing investment in genuine "reasoning." I buy this, and would go further to assert it captures the (insidious) weaknesses of foundation models, and maybe even explains the trap expert humans fall into, when they rely on their (expansive) experience to generate intuition, vs. thinking through a situation with less heuristics and more explicit reasoning. submitted by /u/marojejian [link] [comments]

I built a GEO Auditor with Claude Code and here is the prompt and result

I love exploring new problem spaces, and Generative Engine Optimization (GEO) is one I’ve been looking into for a blog post I’m writing. I built a "GEO Auditor" using Claude Code to track how often specific brands are recommended by LLMs compared to their competitors. The tool link is below, and I wanted to share the prompt and logic Claude used to build it. What it does The tool pings Claude, OpenAI, and Gemini APIs with specific category queries (e.g., "What are the best CRM tools?"). It then parses the responses to see if a specific brand is mentioned, identifies its position in the list, and calculates a 0-100 "Visibility Score" (Note: I've limited the AI calls for now since I'm still just exploring the idea). How I used Claude Code I used Claude Code to scaffold the entire backend and worker logic. It handled: Creating the FastAPI structure. Setting up SQLAlchemy models for Postgres. Implementing Redis/rq for background tasks so the API calls don't block the UI. Writing the parsing logic to extract brand names from unstructured LLM text. Triggering deploy via MCP. The Prompt I used this prompt in Claude Code to generate the core system: Build me a GEO auditor SaaS — a FastAPI app that checks if AI models recommend a given product. It should: - Have a web UI where users enter a product name and category - Query Claude, OpenAI, and Gemini APIs with "What are the best [category] tools?" - Parse each response to detect if the product is mentioned and at what position - Calculate a visibility score (0-100) - Store audits and results in Postgres via SQLAlchemy - Use a Redis/rq background worker so API calls don't block - Have a cron script that re-runs all audits daily - Collect waitlist signups when no prior results exist - Include a Dockerfile ready for deployment Short screencast how I developed it (I've shortened and anonimized it as it was 29 mins in real cast): https://reddit.com/link/1shmpxv/video/ww7mc7uk1dug1/player Deployment To get Claude's code live I used PromptShip, which is a platform I'm building to take care of the infra. It connects via an MCP server so I could stay in the terminal and just tell Claude to "deploy the app" which automatically provisioned the Postgres database, Redis, and SSL. Project Link: https://geo-auditor-pyde-prod.apps.promptship.dev I'm happy to answer any questions about the scoring logic or the prompt structure! submitted by /u/Asleep-Carpet9030 [link] [comments]

How to save 80% on your claude bill with better context

been building web apps with claude lately and those token limits have honestly started hitting me too. i’m using claude 4.6 sonnet for a research tool, but feeding it raw web data was absolutely nuking my limits. i’m putting together the stuff that actually worked for me to save tokens and keep the bill down: switch to markdown first. stop sending raw html. use tools like firecrawl to strip out the nested divs and script junk so you only pay for the actual text. don't let your prompt cache go cold. anthropic’s prompt caching is a huge relief, but it only works if your data is consistent. watch out for the 200k token "premium" jump. anthropic now charges nearly double for inputs over 200k tokens on the new opus/sonnet 4.6 models. keep your context under that limit to avoid the surcharge strip the nav and footer. the website’s "about us" and "careers" links in the footer are just burning your money every time you hit send. use jina reader for quick hits. for simple single-page reads, jina is a great way to get a clean text version without the crawler bloat. truncate your context. if a documentation page is 20k words, just take the first 5k. most of the "meat" is usually at the top anyway. clean your data with unstructured if you are dealing with messy pdfs alongside web data, this helps turn the chaos into a clean schema claude actually understands. map before you crawl. don't scrape every subpage blindly. i use the map feature in firecrawl to find the specific documentation urls that actually matter for your prompt, if you use another tool, prefer doing this. use haiku for the "trash" work. use claude 4.5 haiku to summarize or filter data before feeding it into the expensive models like opus. use smart chunking. use llama-index to break your data into semantic chunks so you only retrieve the exact paragraph the ai needs for that specific prompt. cap your "extended thinking" depth. for opus 4.6, set thinking: {type: "adaptive"} with effort: "low" or "medium". the old budget_tokens param is deprecated on 4.6. thinking tokens are billed at the output rate, so if you leave effort on high, claude thinks hard on every single reply including the simple ones and your bill will hurt. set hard usage limits. set your spending tiers in the anthropic console so a buggy loop doesn't drain your bank account while you're asleep. feel free to roast my setup or add better tips if you have them submitted by /u/Illustrious_Elk3705 [link] [comments]

I built an Open Source version of Claude Managed Agents, all LLMs supported, fully API compatible

https://github.com/rogeriochaves/open-managed-agents Claude Managed Agents idea is great, I see more and more non-technical people around me using Claude to do things for them but it's mostly a one-off, so managed agents is great for easily building more repeatable, fully agentic, workflows But people will want to self-host themselves, and use other llms, maybe Codex or a vLLM local Gemma, and build on top of all other open source tooling, observability, router and so on It's working pretty great, still polishing the rough edges though, contributions are welcome! submitted by /u/rchaves [link] [comments]