ComfyUI

ComfyUI AI

ComfyUI AI

ComfyUI AI

ComfyUI AI

ComfyUI AI

ComfyUI AI

ComfyUI AI

ComfyUI AI

ComfyUI AI

ComfyUI AI

Scenema Audio: Zero-shot expressive voice cloning and speech generation [N]

We've been building Scenema Audio as part of our video production platform at scenema.ai, and we're releasing the model weights and inference code. The core idea: emotional performance and voice identity are independent. You describe how the speech should be performed (rage, grief, excitement, a child's wonder), and optionally provide reference audio for voice identity. The reference provides the "who." The prompt provides the "how." Any voice can perform any emotion, even if that voice has never been recorded in that emotional state. Limitations (and why we still use it) This is a diffusion model, not a traditional TTS pipeline. Common issues include repetition and gibberish on some seeds. Different seeds give different results, and you will not get a perfect output with 0% error rate. This model is meant for a post-editing workflow: generate, pick the best take, trim if needed. Same way you'd work with any generative model. That said, we keep coming back to Scenema Audio over even Gemini 3.1 Flash TTS, which is already more controllable than most TTS systems out there. The reason is simple: the output just sounds more natural and less robotic. There's a quality to diffusion-generated speech that autoregressive TTS doesn't quite match, especially for emotional delivery. Audio-first video generation As this video points out, generating audio first and then using it to drive video generation is a powerful workflow. That's actually how we've used Scenema Audio in some cases. Generate the voice performance, then feed it into an A2V pipeline (LTX 2.3, Wan 2.6, Seedance 2.0, etc.) to generate video that matches the speech. Here's an example of that workflow in action. On distillation and speed A few people have asked this. Our bottleneck is not denoising steps. The diffusion pass is a small fraction of total generation time. The real costs are elsewhere in the pipeline. We're already at 8 steps (down from 50 in the base model), and that's the sweet spot where quality holds. Prompting matters This model is sensitive to prompting, the same way LTX 2.3 is for video. A generic voice description gives you generic output. A specific, theatrical description with action tags gives you a performance. There's also a pace parameter that controls how much time the model gets per word. Takes some experimentation to find what works for your use case, but once you do, you can generate hours of audio with minimal quality loss. Complex words and proper nouns benefit from phonetic spelling. Unlike traditional TTS, it doesn't have a phoneme-to-audio pipeline or a pronunciation dictionary. If it garbles "Tchaikovsky," you would spell it "Chai-koff-skee" or whatever makes sense to you. Docker REST API with automatic VRAM management We ship this as a Docker container with a REST API. Same setup we use in production on scenema.ai. The service auto-detects your GPU and picks the right configuration: VRAM Audio Model Gemma Notes 16 GB INT8 (4.9 GB) CPU streaming Needs 32 GB system RAM 24 GB INT8 (4.9 GB) NF4 on GPU Default config 48 GB bf16 (9.8 GB) bf16 on GPU Best quality We went with Docker because that's how we serve it. No dependency hell, no conda environments. Pull, set your HF token for Gemma access, then docker compose up. ComfyUI Native ComfyUI node support is planned. We're hoping to release it in the coming weeks, unless someone from the community beats us to it. In the meantime, the REST API is straightforward to call from a custom node since it's just a local HTTP service. Links All demos + article: scenema.ai/audio Model weights: huggingface.co/ScenemaAI/scenema-audio Code + setup: github.com/ScenemaAI/scenema-audio YouTube demo: youtu.be/VnEQ_ImOaAc This is fully open source. The model weights derive from the LTX-2 Community License but all inference and pipeline code is MIT. submitted by /u/a__side_of_fries [link] [comments]

How much RAM does Cowork actually use on macOS

I'm thinking about picking up a Mac Mini M4 24GB primarily to run Cowork + Claude Code alongside some local video generation (ComfyUI/Wan 2.2). On my Windows PC I can see vmmem absolutely hammering RAM whenever Cowork is running, which I assume is the Hyper-V sandbox it spins up. Curious how that translates on macOS though — is the VM footprint similarly heavy, or does macOS handle it more gracefully? Basically trying to figure out how much headroom I realistically have for other workloads running alongside Cowork. Any real world numbers from Mac users would be super helpful. submitted by /u/Obvious-Outside3434 [link] [comments]

How do I preserve my AI character as Sora is shutting down

With Sora shutting down, I’m trying to figure out how to keep my character alive across other AI video platforms, bcz I don't wanna start from scratch again. So I put together a reference package that may help ppl like me. Structure of my saved prompts like this: [Appearance] Hair: color, style, length Eyes: color, shape, distinguishing features Build, height, skin tone Marks: scars, tattoos, birthmarks [Motion] Gait: bouncy, heavy, military Gestures: hand talker, still, deliberate [Style] Color palette Rendering: realistic, anime, stylized Common settings or environments File naming: char_front_happy_natural_light.mp4, it's convenient if you're searching for something specific. If static shots are needed, just screenshot images from your vids For the voice, I prompt my character inside a soundproof booth, and then have him deliver lines in various emotional states. So you have some of the best voice samples you can get from Sora. There are many AI voice-cloning tools that can recreate your original voice, as long as you have enough high-quality material. It isn’t perfect, but it's a reliable backup for the toolbox. Where to Rebuild: Platform Character Fidelity Notes Kling AI Very good Strong consistency Runway Gen-3 Good Reference image support Hailuo Good Budget-friendly Pika Moderate Short clips work better ComfyUI + AnimateDiff Best control Needs local GPU I'm using kling 3.0 on AtlasCloud.ai, just test two or three now, don't wait until you're locked out. I don’t think there’s an AI that has an extension that actually works re-create the things you want, but for now all we can do is save as many vids of your character as possible, maybe in the future there is a model powerful enough to allow you continue using your character submitted by /u/Fresh-Resolution182 [link] [comments]

Open-source model alternatives of sora

Since someone asked in the comments of my last post about open-source alternatives to Sora, I spent some time going through opensource video models. Not all of it is production-ready, but a few models have gotten good enough to consider for real work. Wan 2.2 Results are solid, motion is smooth, scene coherence holds up better than most at this tier. If you want something with strong prompts following, less censorship and cost-efficient, this is the one to try. Best for: nsfw, general-purpose video, complex motion scenes, fast iteration cycles. Available on AtlasCloud.ai LTX 2.3 The newest in the open-source space, runs notably faster than most open alternatives and handles motion consistency better than expected. Best for: short clips, product visuals, stylized content. Available on ltx.io CogVideoX Handles multi-object scenes well. Trained on Chinese data, so it has a different aesthetic register than Western models, worth testing if you're doing anything with Asian aesthetics or characters. Best for: narrative scenes, multi-character sequences, consistent character work. AnimateDiff AnimateDiff adds motion to SD-style images and has a massive LoRA ecosystem behind it. It requires a decent GPU and some technical setup. If you're comfortable with ComfyUI and have the hardware, this integrates cleanly. Best for: style transfer, LoRA-driven character animation, motion graphics. SVD Quality is solid on short clips; longer sequences tend to drift, still one of the most reliable open options. Local deployment via ComfyUI or diffusers. Best for: product shots, converting illustrations to motion, predictable camera moves. Tbh none of these are Sora. But for a lot of use cases, they cover enough ground. Anyway, worth building familiarity with two or three of them before Sora locks you down. submitted by /u/Which-Jello9157 [link] [comments]

open-workshop - A tool I made for managing your own personal "workshop" or "studio"

https://preview.redd.it/s4hj7gxks1og1.png?width=728&format=png&auto=webp&s=41953a383ed5d6bec0f8ddd1cd364a8b07ffd9f3 open-workshop is something I built because I really struggle with managing all the different projects that I work on at any given time and tend to start new things over finishing "old" things it's a plugin for claude code / cli based harnesses, so it lives in all your claude code sessions and tracks your projects as you register them. It ships with an "R&D" department which it uses to build other "departments" for your workshop to create specialized subagent workers that your project may need for instance, my personal open-workshop department list consists of: R&D (perform web research on topics, a "meta" department that mutates your personal version of open-workshop to get your other departments the tools they need for their "job") Art/Design (ComfyUI asset generation, working with vision models to critique generated assets, figma mcp, etc.) Game Dev (Enabling agents working directly with game engines since its a little different than just writing code) Engineering (More traditional software planning and coding) Marketing (Market research) My first iteration on it was just called "dream-factory" because I wanted to be able to work on multi-disciplinary projects from a single location and track the scattered progress I was making on different things in the brief windows I get for my personal projects. I hope it helps someone else! Here's the more formal marketing page: https://look-itsaxiom.github.io/open-workshop/ And the repo directly: https://github.com/look-itsaxiom/open-workshop edit: fixed img submitted by /u/look-itsaxiom [link] [comments]

I built the first AI image generation plugin with Claude Code — 1,300+ curated prompts, local ComfyUI support, and it actually works

I've been using Claude Code daily for months and kept running into the same gap: every time I needed a quick mockup, product shot, or design asset, I had to leave Claude and switch to a completely different tool. So I built an MCP plugin to fix that. It's already been listed in Awesome MCP Servers (82k+ stars). What it does https://reddit.com/link/1rnh5vh/video/wohzdskf0ong1/player The Plugin gives Claude Code full image generation capabilities — not just "call an API and return a URL", but actual creative workflow orchestration: Describe a vague idea → Claude enhances your prompt with proper lighting, composition, and style details → generates the image Ask for "5 logo concepts" → it writes 5 genuinely distinct prompts and generates them in parallel Pick the one you like → "now put this on a mug and a t-shirt" → it uses your logo as a reference and generates mockups All without leaving your terminal The free stuff (no API key needed) This was important to me — you shouldn't have to sign up for anything just to try a plugin: 1,300+ curated prompt library — search by keyword, browse by category, copy full prompts. These are hand-picked high-quality prompts, not scraped garbage Prompt enhancement — give it "a cat" and it'll expand that into a detailed prompt with camera lens, lighting direction, material textures Model listing — see what's available across all your configured providers Three backends, your choice Provider Cost Privacy Cloud server Token-based credits Cloud Any OpenAI-compatible API Your provider's pricing Cloud Local ComfyUI Free 100% local, nothing leaves your machine The ComfyUI support is what I'm most proud of. You can import your existing workflows, and the plugin auto-detects the key nodes (KSampler, CLIPTextEncode, etc.) and fills in prompt/seed/dimensions at runtime. If you're already running ComfyUI locally, you literally just point it at localhost:8188 and you're done. How to install GitHub: https://github.com/jau123/MeiGen-AI-Design-MCP Would love feedback, especially from anyone running ComfyUI locally. What workflows would you want to see supported? submitted by /u/Deep-Huckleberry-752 [link] [comments]

Is it better Text2vid or img2vid? AnimateDiff or Wan?

Animating this? Or something like this? Trying to do something like this imgs, I already have AnimateDiff + comfyui, using Gemini for guidance but it's not that helpful and most of the tutorials are from 3y ago submitted by /u/Mastah-Blastah [link] [comments]