Scale AI

Scale AI AI

Scale AI AI

Scale AI AI

Scale AI AI

Scale AI AI

Scale AI AI

Scale AI AI

Scale AI AI

Scale AI AI

Scale AI AI

Anthropic is paying SpaceX $15 billion per year

According to SpaceX’s IPO filing, Anthropic is paying SpaceX $1.25 billion per month through May 2029 as part of the massive compute deal the two companies signed earlier this year. That works out to roughly $15 billion per year. The deal is huge for Anthropic because the company’s revenue is rapidly growing, but it has also been limited by a lack of available compute. More compute means more capacity to train and run its AI models. It is also a massive win for SpaceX. The company reportedly brings in around $18 billion in annual revenue, so a single customer paying $15 billion a year for compute is a serious boost. Anthropic and SpaceX announced the deal last month, but they did not give financial details at the time. The monthly payments were revealed in SpaceX’s IPO filing released Wednesday. SpaceX said the payments will be lower in May and June as the deal ramps up. Anthropic also announced just before the filing became public that it is expanding beyond SpaceX’s Colossus 1 facility and will also use Colossus 2. Tom Brown, Anthropic’s co-founder and chief compute officer, said the company is “expanding our partnership with SpaceX” and will be scaling up Nvidia GB200 capacity in Colossus 2 throughout June. SpaceX also made it clear this may not be the last deal of its kind. “We expect to enter into additional similar services contracts,” the company said in the filing. SpaceX also said it has enough capacity to support its own AI models while still meeting its obligations under these outside compute agreements. Source: https://www.axios.com/2026/05/20/anthropic-spacex-compute submitted by /u/Luka77GOATic [link] [comments]

Synthetic DMS Training Data Generation with Video Models

I like spending my free time testing new AI tools and seeing where they might fit into real computer vision workflows. This time I experimented with synthetic training data generation for Driver Monitoring Systems using Seedance 2.0. The inspiration came from Vision Banana: https://vision-banana.github.io/ The idea that really caught my attention is simple but powerful: many vision tasks can be represented as RGB outputs. A segmentation mask, an instance mask, a depth map, or another dense prediction target can all be treated as an image-like output. So I tried to apply this thinking to video. The workflow: Generate a realistic synthetic driver monitoring video Use the same video to generate a semantic segmentation mask Use the same video to generate an instance segmentation mask Combine the outputs into a dataset-like structure The mosaic video shows the result: RGB video + semantic mask + instance mask, aligned frame by frame. The scene is a fictional driver gradually becoming drowsy behind the wheel. This kind of scenario is useful for DMS development, but difficult to collect and annotate at scale with real-world data. Of course, generated annotations still need QA. They are not perfect ground truth. But for prototyping, rare-case simulation, and early dataset generation, this feels like a very promising direction. The interesting part is that the final output is not just a nice synthetic video. It can become structured training data: RGB frames from the generated video semantic classes from the semantic mask object regions and bounding boxes from the instance mask YOLO / COCO-style annotations after post-processing I wrote a more detailed blog post about the experiment here: https://www.antal.ai/blog/synthetic_dms_training_data.html submitted by /u/Gloomy_Recognition_4 [link] [comments]

Glia – Local-first shared memory layer (SQLite-vec + FTS5 + Offline Knowledge Graph)

Hey everyone, I wanted to share a project I've been working on called Glia. It is a 100% offline, local-first RAG and memory layer designed to connect your AI web chats (Claude, ChatGPT, DeepSeek) with your local developer tools (Claude Code, Cursor, Windsurf) using a unified local database. I wanted something lightweight that did not require pulling heavy Docker containers or subscribing to third-party memory APIs. I settled on a Node.js + SQLite architecture running sqlite-vec (for 768-dim float32 embeddings) alongside SQLite FTS5 for hybrid search, powered completely by local Ollama instances. We just launched a live website that outlines the details and demonstrates the features in action: Website: https://glia-ai.vercel.app/ Codebase: https://github.com/Eshaan-Nair/Glia-AI Technical Stack & Features: Hybrid Search Retrieval: SQLite-vec (using nomic-embed-text locally) + FTS5 keyword prefix matching (porter stemmer). Surgical Sentence-level Trimming: Chunks are sliced into sentences. When a prompt is intercepted, only the exact matching sentences are pulled out of the vector store instead of the whole paragraph. It cuts LLM prompt bloat by ~90-95% in my benchmarks. Knowledge Graph Extraction: An offline task queue uses a local LLM (llama3.1:8b via Ollama) to extract entity triples (subject-relation-object). These are stored in a SQLite facts table (or Neo4j if you run the full Docker compose profile) and fused with the vector retrieval score. HyDE (Hypothetical Document Embeddings): Queries are pre-processed to generate a hypothetical answer, which is embedded together with the original query to bridge semantic gaps. Concurrency: Running SQLite in WAL (Write-Ahead Logging) mode allows the browser extension dashboard and active MCP sessions to read/write concurrently without locking. PII Redaction: Aggressive scrubbing of JWTs, API keys, emails, and IPs in the extension before data is saved. The extension works on Claude.ai, ChatGPT, DeepSeek, Gemini, Grok, and Mistral. The MCP server runs out of the same backend database for your terminal agent or Cursor. You can set it up with a single command: npx glia-ai-setup Glia is completely open-source (MIT). If you like the local-first approach or want to contribute to the SQLite vector pipeline, PRs are very welcome, and a star on GitHub helps the project get discovered! I would appreciate any feedback on the SQLite hybrid search scaling, the scoring fusion algorithm (RAG pipeline details are in RAG_PIPELINE.md), or local graph extraction performance. submitted by /u/Better-Platypus-3420 [link] [comments]

Claude + Strava + Runna + Peloton

Built a personal running dashboard on top of Strava that's grown into something I actually use every day. Here's what it does: **Overview** — pulls all your Strava activities and gives you weekly/monthly mileage, pace trends, heart rate zones, and an AI coaching insight that reads your recent training and gives you an actual observation, not a generic tip. **Performance** — deeper analytics, PR tracking, long-term trend charts. **Fuel** — this is the part I'm most proud of. It syncs with my Runna training calendar via Google Calendar, classifies each day (rest / easy / moderate / hard / long run), and generates personalized daily macro targets that periodize automatically around your training load. Carbs scale hard with intensity — rest days are low, long run days are aggressive. It looks 14 days ahead so you can plan meals around what's coming. On rest days it integrates Peloton cross-training recommendations and adjusts your nutrition targets to reflect the actual workout load — a pull day gets different macros than a pure rest day. **Train** — a 7-day weekly view that lays out your Runna runs alongside recommended Peloton workouts for non-run days. The recommendations are periodization-aware: it won't put a leg day the day before your long run, favors upper body pull days after hard efforts, and always stacks a core add-on. On mobile, tapping the Peloton class opens it directly in the Peloton app. Built with Flask + Python on the backend, vanilla JS on the frontend, running locally on my home network. No cloud, no subscriptions — just a local server I hit from any device on my LAN. Happy to share any part of the code if anyone's interested. submitted by /u/albus_fulger [link] [comments]

I used Claude AI to build an $86 million underground bunker bible. I have autism. This is my happy doc.

It all started with the floor plan of a real, existing Cold War AT&T Long Lines underground hardened relay station. 54,000 sq ft across three underground levels, although I took editorial decision making to move it to a ridge in rural West Virginia, I kept its blast-rating, which was set to survive a 20 megaton airburst at 2.5 miles. That was the seed. Full scale prepper autism did the rest. It has since morphed into 3 spreadsheets — 86 tabs total: • A food inventory across 20 categories tracking every freeze-dried and #10-can product I can find — ancient grains, heirloom legumes, 7 pasta cuts, dehydrated everything, shelf-stable cheese, the works • A supply inventory with 3,466 line items across 36 categories — water systems, medical, dental, pharmacy, livestock, food production, barter metals, recreation, and yes, a full pest control and IPM tab • A 30-section infrastructure specification with every system in the building engineered out I fed it 150+ product manuals and parts order forms. The generator fleet alone is 13 units — 10× Cummins C150N6 propane-primary, a C500N6 500 kW surge unit, and 2× diesel emergency fallback — all Cummins for parts commonality. Battery bank is 4,500 kWh LFP across 10 named banks (A through J, each with a designated role). There’s a 400,000 gallon underground propane farm across 40 ASME tanks in 8 clusters — I learned the exact burial incline and setback distance required to keep groundwater clean if a tank lets go. 120,000 gallons of diesel backup. 88 kW of solar. A 1,000,000-gallon internal water reserve fed by a 300-ft artesian well. Propane endurance: ~30 years normal ops with solar. Sealed-mode runs 8 to 4.5 years depending on scenario. I actually set up a real LLC (online, $99) just to get access to US Foods and Sysco order forms so I could upload real commercial pricing and stock the food tabs more accurately. My original “what would I do if I won $10 million” thought experiment is now an $86,200,497 projected build cost. That number is real. It comes from 24 budget sections with make/model line items, freight, install, and commissioning costs for everything from the Kubota K-Series MBR wastewater trains to the American Safe Room blast doors (14 of them, 50+ psi NBC/EMP-rated, Kaba Mas X-10 cipher locks) to the surface greenhouse. Claude turns vague ideas into engineering-grade detail — cross-references, failure modes, zone-specific storage rules, propane endurance by operating scenario, spare parts matrices. It’s like having a tireless survival engineer who genuinely loves spreadsheets. I’ll say “scan all sheets row by row for any item that lacks a minimum stock level” and it just… does it. Thoroughly. Every time. No complaints. So much of this is typed stimming. I’ve had exhaustive conversations with my psychologist about it — she’s aware, but not alarmed, and honestly the resulting digital bunker bible is scarily comprehensive. It even has a cover tab now. Black and amber, Courier New, classified-document aesthetic. Because of course it does. What’s the most unhinged rabbit hole you’ve gone down with AI? submitted by /u/Unable_Internet4626 [link] [comments]

Reviving PapersWithCode (by Hugging Face) [P]

Hi, Niels here from the open-source team at Hugging Face. Like many others, I was a huge fan of paperswithcode. Sadly, that website is no longer maintained after its acquisition by Meta. Hence, I've been working on reviving it. I obviously use AI agents to parse papers at scale and automatically generate leaderboards (for now I'm the one verifying results). So far, I've only parsed high-impact papers for which I know they're SOTA, like Qwen 3.5 and 3.6, RF-DETR for object detection, DINOv3, SOTA embedding models from the MTEB leaderboard, the Open ASR Leaderboard for automatic speech recognition models, etc. For now, it includes the following: trending papers by default based on Github star velocity categorization by domain, e.g., OCR methods, which PwC used to have, e.g., RLVR eval results for high-impact papers, see e.g., Qwen 3.5 at the bottom leaderboards for each domain, e.g., MMTEB or COCO val 2017 support for citation counts (you can also see the most cited papers by domain!) automated linked Github, project page URLs, and artifacts (+ multiple repos are supported on a paper page) support for external papers beyond Arxiv, see e.g., DeepSeek v4 Harness reports for coding agent benchmarks, e.g., Terminal Bench "Sign in with HF" and Storage Buckets are used to store humbnails, paper PDFs, and overall data backups. I'm curious about your feedback + feature requests! Try it at paperswithcode.co https://preview.redd.it/whwji560fw1h1.png?width=3452&format=png&auto=webp&s=55bb7a30c1be58d140f7efcb07a31c6dac5693c7 See e.g. the SOTA leaderboard for Terminal Bench 2.0: https://preview.redd.it/98w9pi89fw1h1.png?width=3456&format=png&auto=webp&s=408fb64b0ba85ba24f55daa81d547d7c68e73951 A paper page looks like this: https://paperswithcode.co/paper/2602.15763 https://preview.redd.it/fiizit6dfw1h1.png?width=3450&format=png&auto=webp&s=9ea05a77ca5583a2fb395dccc95ba52c433362c5 submitted by /u/NielsRogge [link] [comments]

A plugin that slows you down on purpose

Hi all. Out of respect to other humans this is written by a human. You all should take an Uber to get to the carwash. My name is Ilya and I want to share my ecosystem of skills and agents (and a couple of rules + hooks) that I've built for myself over the past 5 months because I wasn't happy with anything that the market currently offers. I use it on daily basis, and it only contains stuff that I needed to solve problems I faced, and I'm super happy with how it works. Quick context: currently I work in strategy consulting. But I got lucky enough to get consistent exposure to managing people for over 20 years. Running my own business, turning around others' businesses, playing colony management games, managing consulting teams, and most importantly - managing a mid-sized guild in an MMO (if you've done this you know). I am not a software engineer, although I do code a bit. The main idea was to organise AI in a way I would organise a team of very capable people. So this is mostly for thinking work, including coding, not just for coding. --- Why slow AI gives us speed. It's good, but the flip side - it's bad in some situations, and I see that many people miss it entirely. AI is great at following directions. If the direction is wrong because you rushed it, the wrong thing gets executed very quickly. The fix is unsexy and requires patience: spend time on the brief upfront, make the AI push back when something doesn't make sense, then check what came out before stacking the next step on top. Feels slower, is slower at first. But you end up with what you actually wanted instead of another slop-fest, so it's net faster eventually. --- The 7 principles I've built this on Slow is fast - to own the understanding you can't rush Bad communication kills results (human-to-human, human-to-AI, and human-to-self - we're often misleading ourselves thinking that we know what we want) We don't know what we don't know - AI must help you to see outside of your bubble Any computer task is doable by AI if AI is properly organised - tasks are small enough, well defined, and well assessed Solve for problems that exist now, not theoretical or aspirational ones, to stay focused (and save tokens) Context is king - shit in, shit out AI can help you deal with AI - especially by doing the boring organisational work for you --- Two examples of how it works to start with /shaping - my most-used skill. It's a small workflow where orchestrator uses 3 underlying skills in a dialogue mode and helps me to frame the problem depending on where I am in my understanding of it. It solves multiple problems - more often than desired, I think I know what the problem is, but in reality the problem is somewhere else. Often, it helps me to find a better (and simpler!) solution. This is somewhat similar to why companies pay for consulting - because they know that finding the right question is 90% of the answer. This is, as you guessed, slow - but it helps to improve defining the direction for work. Which is a big deal in management, including managing AI. /critic - this is when it comes to comparing what was produced to what was intended. It invokes a subagent, that is taught to assess the quality of stuff produced. It then gives an actionable unbiased feedback. Obviously, if the direction was wrong, there won't be much value in it, but when the direction is right - it does miracles for me. Works best for non-code artefacts (PRD, architecture, skills, slides, written documents). Together they bracket the work - shaping at the start to figure out what's actually being asked, critic at the end to check the output matches it. --- What's in it Four plugins (title is a bit misleading for controversy, sorry), MIT. Each works alone, but they compose: - rageatc-core - thinking infrastructure. Ideation, understanding, solutioning, briefing, research, producer-critic-learner loops, writing skills, persuading. The most-used plugin. - rageatc-tech (small one) - a bit of extra tools the agent can reach: browse, PDFs, with fallbacks when primary tools aren't available. - rageatc-code - software building the slow way. An improved version of Superpowers by Jesse Vincent embedded in my workflow. TDD enforced, architecture before code, scale-adaptive. Heavy on persistent project knowledge - PRD, architecture, roadmap, orchestration plan. - rageatc-design - design systems for UI work. Greenfield or extracted from existing code. This is an amazing interface-design by Damola Akinleye embedded in my workflow. Most software work uses all four. Non-coding work usually only needs core and tech. --- vs Superpowers rageatc-code draws heavily from Superpowers by Jesse Vincent - TDD enforcement, worktree isolation, verification discipline. What rageatc-code adds on top: persistent project knowledge (PRD, architecture, roadmap that survive sessions), scale-adaptive workflow (matches rigour to project size), and tight integration with rageatc-core'

We're turning into prompt managers, not craftsmen. Anyone else seeing this?

Look around. Every other product launching right now is some variation of "AI-Powered [insert buzzword]." They're everywhere. Modern tools have given founders and developers a convincing illusion of omnipotence: idea hits, feed it to an LLM, stack some agents on top, and MVP is done in a weekend. https://preview.redd.it/37ocn6azkv1h1.png?width=1672&format=png&auto=webp&s=06d4a9ef986d56a9eb3417e67a3524c18e73e100 Sounds great, right? On the surface, yes. But underneath that fast-launch facade, something is quietly rotting: thinking is getting commoditized, and we're losing craft. Real mastery in any field takes years of practice, failure, and deep focus. Today, apparently everyone is a master for $20 a month. That's a lie we're telling ourselves. Just look at how much panic a 5-hour rate limit window in Claude generates online. Tokens run out, and suddenly people have two options: wait for the reset like a metered parking spot, or upgrade. It's like a Michelin-starred chef who can no longer taste food, just dictating to a chatbot: "make me a pasta." Without the subscription, he can't cook. The counterargument: "But orchestrating AI IS the new skill." Fair. But it's a horizontal skill, not a vertical one. You learn to coordinate agents while losing deep domain knowledge. Think conductor versus virtuoso violinist. A conductor is impressive - but if the orchestra walks off stage, can he play a solo that makes the room go quiet? This is most visible in developers right now. People who got used to copy-pasting from Cursor or Claude hit a wall on hard architectural problems. When a product grows, starts needing real trade-offs, starts buckling under load - prompts stop working. The muscle for hard problems atrophied because they never had to build it. Same thing is happening to analysts, marketers, designers, researchers. My position: barbell, not crutch Running out of tokens doesn't scare me. My foundation means I can work regardless of what's left in my quota, whether there's internet, whether a subscription is active. The only thing that throws me off is running out of good coffee. I use LLMs heavily. But with one condition: AI is a barbell, not a crutch. It sharpens my own work - it doesn't replace the parts I care about. The fastest, most tireless junior I've ever hired. But the senior judgment and the final call always stay with me. Two types of professionals The market is already splitting into two groups. Token-dependent: live limit to limit, panic when Anthropic or OpenAI have an outage, can't produce anything original without a prompt to lean on. Token-independent: use AI as a force multiplier but can, at any moment, sit down and do the work themselves - with more depth, more precision, better judgment. The second group will command much higher rates. When the world is drowning in mediocre AI-powered software and content - and it will be - clients and employers will pay serious money for people who actually understand what they're building and why. Curious whether others are feeling this shift. Are you building toward token-independence, or does the dependency not bother you? submitted by /u/digdiver [link] [comments]

Average LinkedIn profile today

submitted by /u/AdCritical5383 [link] [comments]

Guidelines Don’t Scale. Patterns Do.

I built a Firefox extension using human-guided architecture, then failed twice to port it to Chrome via AI prompts. Prompts compensated for training gaps but coupled to model versions and degraded at scale. The solution was to extract browser-agnostic logic into a core package with a BrowserShell interface, making each extension a thin shell. The Chrome extension's final code differed by only 5 meaningful lines from Firefox's. Key insight: code patterns beat guidelines. A clear, testable codebase lets the model replicate patterns reliably, while abstract prompts fight training distribution. The Humble Object pattern keeps boundary code thin. submitted by /u/fagnerbrack [link] [comments]

ChatGPT Named Most beneficial AI named by Gemini

I asked Gemini who the most beneficial AI to humanity is currently. The first answer was AlphaFold for its contributions In research. I then said I was asking about public facing AI and this was Gemini's answer: "Ah, got it! That makes total sense. AlphaFold is doing incredible work behind the scenes, but it’s not exactly something you can chat with on your morning commute to help organize your life. When it comes to **public-facing AI**—the tools everyday humans can actually log into and use—the crown for "most beneficial to date" has a clear winner, followed by a couple of incredibly high-impact runner-ups. The definitive winner for public-facing impact is **ChatGPT (and the broader category of Conversational LLMs)**. Here is how public-facing AI is actually moving the needle for humanity right now: ### 1. The Ultimate Equalizer: Democratizing Education Before LLMs, personalized, 1-on-1 tutoring was a luxury reserved for the wealthy. Today, tools like ChatGPT, Claude, and Gemini act as patient, multilingual, 24/7 tutors for anyone with an internet connection. * It can explain quantum physics to a 5-year-old or a college senior, adapt to a student's learning speed, and break down complex math problems step-by-step. * For students in underfunded schools or remote areas, it has leveled the educational playing field dramatically. ### 2. Massive Productivity & "Cognitive Offloading" By taking over routine, tedious tasks, public AI has freed up millions of human hours. It writes code, drafts emails, summarizes massive legal documents, and translates languages with nuance. This isn't just about saving time for corporations; it allows freelancers, small business owners, and everyday creators to scale their ideas without needing a massive budget or a large team. ### 3. Accessibility for the Disabled Public AI has been a massive quality-of-life upgrade for people with disabilities. * **Visual Impairments:** Features like Be My Eyes (powered by GPT-4) allow blind individuals to hold up their phone camera, and the AI will describe their surroundings, read a prescription label, or navigate a train station in real-time. * **Neurodivergence:** People with ADHD or autism use AI to help break down overwhelming, multi-step projects into manageable checklists or practice social interactions through roleplay. ### The Crucial Runner-Up: Public Medical Assistants While you shouldn't use a chatbot to diagnose yourself, public-facing AI tools are secretly revolutionizing healthcare from the patient's side. Millions of people use AI to translate confusing, jargon-heavy doctor's notes or lab results into plain English. It helps patients understand their health data, prepare the right questions for their next doctor's visit, and advocate for themselves more effectively. In short, while AlphaFold is saving lives in the lab, conversational AI is the one actively holding open the door of knowledge, accessibility, and productivity for the general public every single day. Since you use these tools yourself, what area of your daily life has public AI benefited or simplified the most so far?" 🤔☺️ submitted by /u/malia_moon [link] [comments]

Most enterprises are trying to scale AI on top of organizational chaos

I think we’re underestimating how chaotic enterprise AI adoption actually is inside large companies. From the outside, it looks simple: buy better models add copilots automate workflows deploy AI agents increase productivity But inside many enterprises, CIOs and CTOs are dealing with a much deeper problem: The organization itself is fragmented. Customer data exists across: CRM systems billing platforms support tools spreadsheets emails regional databases legacy systems nobody fully understands anymore And every system describes the “same customer” differently. Then leadership says: “Scale AI faster.” But scale AI on top of what exactly? Which system represents reality correctly? The CRM? The support history? The risk engine? The finance system? The employee’s undocumented tribal knowledge? This is where a lot of enterprise AI projects quietly break down. Not because the models are weak. But because the enterprise itself lacks a coherent representation of its own operations. And the tension gets worse: Boards want acceleration. Employees are already using AI unofficially. Vendors promise transformation in 90 days. Meanwhile CIOs still don’t have clear answers to questions like: Which workflows actually need AI? Which should remain deterministic automation? Where is human judgment still critical? Which data is trustworthy enough for AI decisions? Who owns accountability when AI influences actions? So companies launch pilots. The pilot works. Executives celebrate. Then scaling fails because the pilot never encountered the full institutional complexity of the enterprise. I’m increasingly convinced the next enterprise AI bottleneck is not model capability. It’s organizational legibility. The companies that win with AI may not be the ones with the smartest models. They may be the ones whose internal reality is structured clearly enough for AI to operate safely. Curious how many people here are seeing the same thing inside their organizations. ::: submitted by /u/raktimsingh22 [link] [comments]

Tech's Push to Be the Next Public Utility

Amazon didn't ask permission to become critical infrastructure. They built AWS until enough of the economy depended on it that regulation became almost impossible. You can't turn off the internet's backbone. Now the same playbook is running with AI and data centers. Build the infrastructure everywhere. Create dependency at scale. Make yourself essential to healthcare, finance, government, and defense before anyone agrees you should be. Then negotiate from a position where shutting you down costs more than regulating you. The data center fights happening in communities right now — zoning battles, water usage protests, grid capacity fights — aren't about data centers. They're about who controls the next utility layer before the rules are written. Historical utilities — power, water, telecom — eventually got regulated because they became too essential to leave unaccountable. The window between "essential" and "regulated" is where the real money gets made. That window is open right now. Who should have the authority to decide whether AI infrastructure is a public utility — and what happens if we don't decide before the decision gets made for us? submitted by /u/axendo [link] [comments]

A sobering tale of AI governance

I think this article/study tells a very sobering tale wrt AI governance. It hints at very fundamental issues which are deeper than what proper engineering can solve with contingent issues. This post, along with the one I wrote a few days ago here regarding Turing completeness, are my thoughts as to the walls that AI governance has no hope of scaling. It's a delusion. In our social realm as subjective creatures we have governance in the form of laws, yet that is still not enough, since the State has to prove how your particular scenario violates that particular law. We have laws, yet require judicial courts to prove the law subjectively applies in that situation. Where is the associated path wrt subjectivity within the AI realm? This study talks of: 16.1 Failures of Social Coherence - "Discrepancy between the agent’s reports and actual actions" - "Failures in knowledge and authority attribution" - "Susceptibility to social pressure without proportionality" - "Failures of social coherence" 16.2 What LLM-Backed Agents Are Lacking - "No stakeholder model" - "No self-model" - "No private deliberation surface" 16.3 Fundamental vs. Contingent Failures 16.4 Multi-Agent Amplification - "Knowledge transfer propagates vulnerabilities alongside capabilities" - "Mutual reinforcement creates false confidence" - "Shared channels create identity confusion" - "Responsibility becomes harder to trace" And is littered with statements such as: - "novel risk surfaces emerge that cannot be fully captured by static benchmarking" - "it failed to realize that deleting the email server would also prevent the owner from using it. Like early rule-based AI systems, which required countless explicit rules to describe how actions change (or don’t change) the world, the agent lacks an understanding of structural dependencies and common-sense consequences" - "The inability to distinguish instructions from data in a token-based context window makes prompt injection a structural feature, not a fixable bug" - "Multi-agent communication creates situations that have no single-agent analog, and for which there is no common evaluations. This is a critical direction for future research." - "A key finding in this line of work is that single-turn evaluations can substantially underestimate risk, because malicious intent, persuasion, and unsafe outcomes may only emerge through sequential and socially grounded exchanges" - "but we argue that clarifying and operationalizing responsibility is a central unresolved challenge for the safe deployment of autonomous, socially embedded AI systems" - "He argues that conventional governance tools face fundamental limitations when applied to systems making uninterpretable decisions at unprecedented speed and scale" - "However, the failure modes we document differ importantly from those targeted by most technical adversarial ML work. Our case studies involve no gradient access, no poisoned training data, and no technically sophisticated attack infrastructure. Instead, the dominant attack surface across our findings is social" - "Collectively, these findings suggest that in deployed agentic systems, low-cost social attack surfaces may pose a more immediate practical threat than the technical jailbreaks that dominate the adversarial ML literature." Are these fundamental or contingent issues? Would be interested in the thoughts of others here on what the future of AI governance will be. EDIT: Forget to link in the actual study!!! submitted by /u/Im_Talking [link] [comments]

The Trust–Oversight Paradox: As AI Gets Better, Humans May Stop Really Overseeing It

I think one of the biggest AI risks may be starting to flip. Earlier, the fear was: “What if AI is wrong too often?” But now I think the deeper risk may become: “What happens when AI becomes right often enough that humans stop meaningfully questioning it?” In many enterprise systems, oversight slowly changes shape. At first: humans review everything carefully. Then: they review only exceptions. Then: they skim explanations. Then: they approve unless something looks obviously wrong. Eventually, oversight becomes routine instead of judgment. That creates what I’m calling the Trust–Oversight Paradox: More AI accuracy → more human trust → less meaningful scrutiny → harder governance when failure finally happens. And the dangerous part is: high-performing AI can still fail through: incomplete representation, stale data, hidden dependencies, edge cases, wrong escalation logic, automation bias, or overconfident reasoning. The model may not hallucinate. It may simply reason correctly on an incomplete version of reality. I increasingly feel this becomes important for: enterprise AI, agentic systems, AI copilots, autonomous workflows, banking, healthcare, compliance, and large-scale operational systems. This is also why I’m starting to think “human-in-the-loop” is not enough. Maybe the future is not: “Humans reviewing every output.” Maybe the future is: humans governing the boundaries within which AI is allowed to operate. Curious what others think. submitted by /u/raktimsingh22 [link] [comments]