> We found that the model's modulation weights (~40% of the total parameters) could be pruned and replaced with a functionally equivalent lookup table, dramatically shrinking the memory footprint with no loss in output quality.
Is this a common approach to reducing weights with "no loss in output quality", assuming this is true? Seems almost too simple to work. If this is doable, would this be applicable to LLMs as well?
Neat with native frame-to-frame generation, but wonder how easy it is to "link" together clips at the intersection, typically the models kind of lose the "momentum" across these stiches, being able to merge things with frame-to-frame between clips might help with this it feels like.
It is a well-known trick, given that the timestep is between 0 to 1, you can slicing them at any resolution (1000, or 10000, give or take), and then keep a look-up table for modulation scale / bias etc for each. It is quite different from quantization and it is indeed lossless.
It is also only applicable to diffusion models as only these operates at per-timestep.
They ship a complete checkpoint for easily management (inference & training) in their own infrastructure. Moving to a LUT would make training on these layers impossible. BTW, these are not useful for lightweight fine-tuning, but might still be useful if you do serious post-training work.
Of course, these are also not an issue for things like FLUX.2 which adopts DiT-Air arch, that doesn't have this wasted space issue.
It may or may not be true. The people who made this modification and the other commenters didn't do anything rigorous to verify what they did. They just eyeball it. They could very well make some other error - this has happened frequently - that developing on prod, not knowing what they are doing, has and hopefully will again solve.
"Modulation weights" here refers to weights used to adjust layer normalization depending on the task (adaLN). General-purpose LLMs generally don't have those in the first place.
One thing similar would be projecting both the head.weight and the final LLM activations into a smaller vector space, since that is basically just cosine similarity ranking step (so that would reduce the head.weight size). But again, it must be tried many times and just not working as well. LLM space is pretty saturated with tricks.
Also begs the question whether this is applicable for high-throughput applications on FPGAs, which are to my novice mind basically LUTs, right?
I remember a paper which was posted on HN a few weeks ago where somebody implemented KAN networks in FPGAs, since those can readily be approximated as LUTs.
Whoah, could this mean we can treat layers like a jpg, where we come up with a formula that estimates the weight values of a layer instead of storing all of the weights?
This isn't the same with lookup tables, but in Explaining Attention with Program Synthesis they were able to replace some attention heads with Python programs:
FWIW on a 6000 Pro it takes 68 seconds for 10 seconds 480p video, (cold) same demo workflow as you used. Set "megapixels" to 2.0 (1920x1088) and same video seems to take 5+ minutes, not sure if everything is right/correct at the moment.
As far as I can tell, the current ComfyUI nodes don't even do compilation, and I haven't looked into what attention mechanism they're using, but I'm sure with time these durations will come down even more.
People on reddit have definitely pointed out that sageattention will speed up the renders.
And it's literally the first day. Someone will make a distilled 4-8 step LoRA and we're off to the races.
Edit: did a couple of 10 second long 864/480 i2v videos on my RTX Pro 6000: sageattention bumps them up 33%, that is to say, 140.89 seconds without sageattention becomes 105.69 with sageattention on (if using the KJ Sageattention node, "allow_compile" doesn't seem to affect it, just "sage_attention" set to "auto" works fine).
EasyCache also appears to work, but does affect quality, at least with the default threshold or even down to 0.10. Still, at 0.10 threshold the same render above, with sageattention, is down to 71.33 seconds, so depending on your use case the quality hit might be worth it. Also it seems that with EasyCache the video still matches the un-EasyCached video (with the same seed), so you could use it to do seed hunting.
I'm on an RTX 5090. I told it to make a 5 second 864x480 video, it's been running for over 30 minutes and is only 35% done in the SamplerCustomAdvanced step.
EDIT: Oh, I'm an idiot. Forgot I had a llama.cpp webserver running with a model loaded. Killed it and it finished very fast.
I am particularly curious how multimodal models will work with types of knowledge that are inherently non-text. For example, SOTA LLMs really suck at electronics, especially analog electronics.
Is MiniMax H3 capable of logical / technical reasoning, or is it purely art oriented?
Suck at what aspect of (analog) electronics specifically? Not contradicting the claim, just want to understand it.
I have not tested yet, but I suspect that LLMs with a harness that can execute code can do SPICE simulations rather ok these days? I have seen MCPs for measurement equipment also, maybe they can even close the physical loop?
> I have seen MCPs for measurement equipment also, maybe they can even close the physical loop?
I'm working on this but for various reasons can't have my physical lab up at the moment. But yes there are many options for connected test equipment that could rather trivially be interacted with via LLM or pretty easy to write libraries.
spice is a very bad simulation. it's not like a unit test or a VM. it works very differently to the real thing, only approximating it in some highly controlled situations.
If you have accurate device models, and you know how to use simulations, you can predict quite well the behavior of the analog circuits that you design.
Simulations that work like the real thing are an absolute necessity in the industry. It is frequent to design the analog parts of some integrated circuit, when you might have to wait months to get prototype samples that you can measure in the laboratory.
If the prototypes do not work exactly as expected, you will probably have the chance to make once a correction of the schematics and/or layout, which will be incorporated in the final product.
But when not even the second try does not work as designed, that is likely to be a failure, because any other redesign would add months and many millions of $ until the product can reach the market.
So "spice" simulations that can be trusted are a necessity. When designing integrated circuits you cannot use empirical methods that work when you make manually a single-use board, where you can replace components or adjust resistors, capacitors or inductors until you get what you want. For integrated circuits, you must know in advance how the circuit will work, because you cannot tweak it.
It is true however that it is easy to configure simulations that will give non-sense results, due to lack of convergence or other numerical errors, so to avoid this, one typically needs both experience with laboratory measurements and experience in using the simulator.
I doubt that an LLM would successfully run SPICE simulations, because there are few examples of written instructions of how to do this in the right way (which varies depending on the domain of applications, and each SPICE-like simulator may have its own quirks).
> work with types of knowledge that are inherently non-text
What are those things exactly? AFAIK, everything we can "know" can be written down, one way or another, even analog circuits.
Also, what SOTA LLMs are you referring to? GPTs been handling analog circuits fine for quite some time, I want to say for at least one year? I've been "pair programming" a bunch of working circuits with GPT models since probably GPT 5 or so.
> AFAIK, everything we can "know" can be written down, one way or another, even analog circuits.
writing down circuit diagrams is like cooking about music.
> what SOTA LLMs are you referring to?
I have done a survey among analog electronics designers just a couple weeks ago and they all said that their forays into LLMs were great for digital electronics, code, and firmware, but for analog they were pretty terrible, with a variety of LLMs, according to everyone.
I think that's different though, that's "doing" rather than just "knowing". You can ride a bike without knowing how it works, and obviously vice-versa too. I don't see circuit diagrams as "doing" though, but the soldering part of building circuits definitely is that way though, you can't just read about it and excel first time you pick up an iron, you have to practice and understand it with your body, like bicycling.
I dug up a few old parody ideas I’d had back in high school and threw them at MiniMax M3 on my RTX. There’s definitely still a lot of jank once you move away from fairly normal scenarios. The moment you start to veer into weirder concepts, things tend to break down a bit especially in the game show where someone is strapped to a wheel and being spun.
Still tho, I was actually shocked by how well the text-to-video turned out overall, and how fast it ran. A 10-second, half-megapixel video gen took only a few minutes which is kinda crazy especially thinking back early WAN days.
Oh yeah definitely agree. I think one of the older tests for LTX and WAN text-to-video models used to involve having a person do a cartwheel or really anything that required putting them in a non‑upright orientation.
The mouse render is surprisingly good. Several of those clips stood out to be a pretty big leap in terms of current SOTA models.
The only one that looks "off" is the beverage ad video during the can opening clip, it still has that "AI smoothening" effect. Good thing this can be done pretty well using traditional rendering.
I feel like for a good while now we'll transition into a process that uses traditional "close-up" rendering/shots + AI generated wide-shots or quick cuts.
Exciting, but also troubling. This being open-weights is a massive win for the community though.
This is still about a year and a half behind Seedance 2.0/ Seedance 2.5
But it represents a coming price pressure that will face the leading foundation models. Open source will prevent runaway costs.
Moreover, it prevents the hair-trigger platform safety checkers from shutting down creative work. Video models are notoriously bad at shutting down a huge number of requests.
Creatives will prefer to work on cloud or private GPU clusters. Waiting 10 minutes for a few seconds of 480p is unacceptable. Hobbyists will have fun, but most actual production work is happening in the cloud.
Artist's time is worth money, and they like to spin up dozens of concurrent generations at a time to more quickly explore the generation state space and make progress on completing work.
I haven't seen any user preference comparisons between Seedance 2.5 and MiniMax H3. As an upper bound, H3 cannot be more than 6 months behind Seedance 2.5 since H3 is already ahead of where Seedance was 6 months ago.
> A year and a half behind Seedance 2.0? That is a bold claim that needs evidence.
SOTA a year ago was Kling 2.5, and H3 does not look or perform at that level.
The Artificial Analysis rankings are whack. They rank Omni first, which is incredulously wrong. Google's models broadly suck, and there they all are - right at the top.
Artificial Analysis has notoriously ranked models such as Grok Imagine highly and continues to rank Happy Horse as a good model despite the model being absolute garbage.
Could it be because they are subject to broad based statistical attacks? It's easy to encode information about the origin of media in either its metadata or output frames. Or maybe there's simply no overlap between creatives and people who click on ELO scores.
I've spent thousands upon thousands of dollars generating video. I will stand by the claim that nothing touches Seedance 2.0 / 2.5.
It's good that we're getting better open weights. It puts price pressure on the foundation model companies. But these open models are not a substitute for Kling or Seedance yet. Not even close.
right before that, there's the part with the person hiking up the dish and 'breathing', and the clouds of water vapor coming out of their mouth don't line up with their breaths
> The result gives a total memory footprint reduced by 66%, from 123.6 GB in full precision to 42.5 GB with the smallest models variants. Combining this with our dynamic VRAM offloading enables a next-generation 2K video model to run locally on a GPU like the RTX 3060.
Pretty cool.
But assuming you have a 16GB 3060, how long would it take to generate a 15 second clip?
I remember reading a report where people running AI-model instagram account were using insanely long and detailed prompts about the setting, lighting, makeup, pose, disposition, clothing, etc. about their models. Presumably with some reference image of the face / body to remain consistent across images.
It‘s not clear to me whether a sufficiently detailed prompt can generate actually interesting video with a natural ”texture” (for lack of a better word).
That would be the prompt. With the right assistance from Qwen3.5/Ornith I was able to achieve some amazing results. Unfortunately due to their licensing I'm not allowed to use it in the USA, so I had to halt testing.
I mean, even the demo prompts on the ComfyUI page aren't adhered to by the model. From the first prompt, one of the four lines:
> TRANSITION: a violent WHIP PAN off the rooftop that SMEARS the floating words away with it, motion-streaked —
And the video just didn't do any of that transition at all, it just replaced it with a cut. If you look at the rest of the prompts, you'll find similar lines that are just totally ignored. Except maybe the mouse one, I didn't see anything wrong with that off the bat.
"Regions such as the EU, UK, South Korea, and the US are currently developing or enforcing AI-related regulations that may have specific implications for generative video models"
Reference-to-video mode seems like all that was missing to enable completely independent cinematography as right now one couldn't stitch different scenes together properly without altering substantial portions of the scene.
I've said it before and I'll say it again, human directors are still valuable, as they use AI video editing tools to generate the shots they want and put them together in a cohesive way. Previously they might've used film and actors but if they can just prompt the AI (or create workflows as seen with ComfyUI) then they arrange them together just like how an EDM producer doesn't actually play the instruments but instead the creativity is in the arrangement.
I suspect it'll be quite a while until AI gets a good enough aesthetic sense to do this, as even with static HTML websites humans can easily see that it's AI slop.
case in point, the entire SaaS multimedia space has pivoted this year to agentic workflows, as in, no more generating AI for that sensitive audience, but instead automating the human work of editing and compositing of real media
if you so happen to supply generative media it will form a cohesive edit of that too
also website slop is distinct from the AI generated sites that blend in. you only notice the ones that don’t.
I'm familiar with the space, my point is it's still not as good, in the same way that AI generated webpages are still not as good. Cohesion is the main issue, usually AI generates too many disparate elements.
or option 2 we'll keep making productions with film and actors. I get it's easy to feel that it's all over with how good these video models are getting but thinking we'll all be slaves to the slop machine once it gets good enough is pretty pessimistic depressing and IMO unlikely.
(I do think it will get a foothold in the "crap people are ashamed to admit they watch" sector though, which it basically already has)
Just like there are still plays even though movies overtook them as the main form of entertainment with acting, so too will AI generation overtake movies with real people. It's simply cheaper and more freeform creatively to use AI so expect that to be the future.
The fundamental difference between AI and all the previous advances like that is that people actually liked and wanted films, synchronized sound, color, CGI, etc. and queued down the block for them. Same thing with electronic music, people enjoy listening to it. Not really the same thing with AI, most of the praise is about the impact it will have on the production process, cost-effectiveness, etc., what actually comes out tends to take the back seat. From what we've seen it's pretty difficult getting people to like the output as well.
You should read up on the history of each of those, they all had major protesters until everyone acclimated. So too for AI media in a decade or two. We already have people talking to their AI girl and boyfriends, it's not a big leap to think it'll be normalized.
of course they had critics, that's kind of what my point is addressing. but we didn't have to wait years for society to potentially acclimate to those previous advancements, they had an overwhelming proportion of fans on day one. and not sure what ai boy/girlfriends have to do with anything
We did in fact have to wait years for them to be accepted, such as photography in the art world, as I said I suggest you read up more heavily on their history. I brought up AI girlfriends as an example of such normalization of AI already occuring. As well, even for AI slop there are fans, such as https://en.wikipedia.org/wiki/Fruit_Love_Island. Just because you or I aren't fans does not mean they do not exist or are unknown.
The example video just looks like the highly produced art (TV, commercials, games) other people have created. I find it impossible to believe this wasn't trained on other people's work, and there is no protection for it. Terribly sad. A lack of original thinking is coming.
It is coming up with Seinfeld episodes even with voice overs. Unless it is trained to do so, don't think that would happen. This model is just around the corner from getting banned due to this copyright issue. (which i hope not)
As with the arts, 99.9% of people can't use these models to express vision, get attention, or achieve distribution.
The game is the same as it has always been. You still need hard work, taste, something important to say, the ability to articulate it, good timing, and luck.
Nothing has changed. We can just build faster.
What this does enable is for more to be created that caters to a wider variety of interests. It disrupts existing structures of capital allocation, production, and distribution and gives new players a chance to reshape the game.
The bar will rise and people will still be running at the same pace on the treadmill. There will be more to see, but less time to see it.
Eh, the technical examples are impressive. But normal people really don't like AI. And I don't think that's going to improve if we really start to see job displacement.
I see a future here for digital advertising, for design mock-ups, for political propaganda, for robotics model training/world simulation, for fan movies, etc.
But the film industry is struggling to get eyeballs and butts in seats already; using a tool that most people inside-and-outside the industry loathe isn't the answer to fixing that.
At most, I could see it used for pre-viz/pre-production work in Hollywood.
> We found that the model's modulation weights (~40% of the total parameters) could be pruned and replaced with a functionally equivalent lookup table, dramatically shrinking the memory footprint with no loss in output quality.
Is this a common approach to reducing weights with "no loss in output quality", assuming this is true? Seems almost too simple to work. If this is doable, would this be applicable to LLMs as well?
Neat with native frame-to-frame generation, but wonder how easy it is to "link" together clips at the intersection, typically the models kind of lose the "momentum" across these stiches, being able to merge things with frame-to-frame between clips might help with this it feels like.
It is a well-known trick, given that the timestep is between 0 to 1, you can slicing them at any resolution (1000, or 10000, give or take), and then keep a look-up table for modulation scale / bias etc for each. It is quite different from quantization and it is indeed lossless.
It is also only applicable to diffusion models as only these operates at per-timestep.
So... why didn't the model ship this way to begin with? They just wanted to waste VRAM for fun?
They ship a complete checkpoint for easily management (inference & training) in their own infrastructure. Moving to a LUT would make training on these layers impossible. BTW, these are not useful for lightweight fine-tuning, but might still be useful if you do serious post-training work.
Of course, these are also not an issue for things like FLUX.2 which adopts DiT-Air arch, that doesn't have this wasted space issue.
It may or may not be true. The people who made this modification and the other commenters didn't do anything rigorous to verify what they did. They just eyeball it. They could very well make some other error - this has happened frequently - that developing on prod, not knowing what they are doing, has and hopefully will again solve.
"Modulation weights" here refers to weights used to adjust layer normalization depending on the task (adaLN). General-purpose LLMs generally don't have those in the first place.
One thing similar would be projecting both the head.weight and the final LLM activations into a smaller vector space, since that is basically just cosine similarity ranking step (so that would reduce the head.weight size). But again, it must be tried many times and just not working as well. LLM space is pretty saturated with tricks.
Also begs the question whether this is applicable for high-throughput applications on FPGAs, which are to my novice mind basically LUTs, right?
I remember a paper which was posted on HN a few weeks ago where somebody implemented KAN networks in FPGAs, since those can readily be approximated as LUTs.
Whoah, could this mean we can treat layers like a jpg, where we come up with a formula that estimates the weight values of a layer instead of storing all of the weights?
This is more or less what those lovely 1.5 bit quantisations are already doing
This isn't the same with lookup tables, but in Explaining Attention with Program Synthesis they were able to replace some attention heads with Python programs:
https://arxiv.org/abs/2606.19317
Im running this on my 4070ti super (16 gb vram), and it takes 10 minutes for a 10-seconds 480p video. but the results are spectacular.
FWIW on a 5080 16GB it takes 3 minutes for 10 seconds 480p video (the mouse video workflow with length changed from 5 seconds to 10 seconds)
FWIW on a 6000 Pro it takes 68 seconds for 10 seconds 480p video, (cold) same demo workflow as you used. Set "megapixels" to 2.0 (1920x1088) and same video seems to take 5+ minutes, not sure if everything is right/correct at the moment.
As far as I can tell, the current ComfyUI nodes don't even do compilation, and I haven't looked into what attention mechanism they're using, but I'm sure with time these durations will come down even more.
People on reddit have definitely pointed out that sageattention will speed up the renders.
And it's literally the first day. Someone will make a distilled 4-8 step LoRA and we're off to the races.
Edit: did a couple of 10 second long 864/480 i2v videos on my RTX Pro 6000: sageattention bumps them up 33%, that is to say, 140.89 seconds without sageattention becomes 105.69 with sageattention on (if using the KJ Sageattention node, "allow_compile" doesn't seem to affect it, just "sage_attention" set to "auto" works fine).
EasyCache also appears to work, but does affect quality, at least with the default threshold or even down to 0.10. Still, at 0.10 threshold the same render above, with sageattention, is down to 71.33 seconds, so depending on your use case the quality hit might be worth it. Also it seems that with EasyCache the video still matches the un-EasyCached video (with the same seed), so you could use it to do seed hunting.
Knowing r/StableDiffusion this will run on a Raspberry Pico's PIO core in real-time by end of this week.
Waiting for the “FWIW on a B200” reply
I was kind of hoping to induce that too! But sadly, no one has come to our rescue yet.
Runpod waiting for you
What’s the vram usage? I’ve got a 5090 so as long as I can fit it in memory the gen times should be roughly equivalent ~10%
Depends on what weights, I landed on:
Ends up at ~83GB, but they also shipped bunch of pruned + quantized versions of the diffusion model, might fit with a 5090: https://huggingface.co/Comfy-Org/MiniMax-H3I must be doing something incredibly wrong.
I'm on an RTX 5090. I told it to make a 5 second 864x480 video, it's been running for over 30 minutes and is only 35% done in the SamplerCustomAdvanced step.
EDIT: Oh, I'm an idiot. Forgot I had a llama.cpp webserver running with a model loaded. Killed it and it finished very fast.
I am particularly curious how multimodal models will work with types of knowledge that are inherently non-text. For example, SOTA LLMs really suck at electronics, especially analog electronics.
Is MiniMax H3 capable of logical / technical reasoning, or is it purely art oriented?
Suck at what aspect of (analog) electronics specifically? Not contradicting the claim, just want to understand it.
I have not tested yet, but I suspect that LLMs with a harness that can execute code can do SPICE simulations rather ok these days? I have seen MCPs for measurement equipment also, maybe they can even close the physical loop?
> I have seen MCPs for measurement equipment also, maybe they can even close the physical loop?
I'm working on this but for various reasons can't have my physical lab up at the moment. But yes there are many options for connected test equipment that could rather trivially be interacted with via LLM or pretty easy to write libraries.
spice is a very bad simulation. it's not like a unit test or a VM. it works very differently to the real thing, only approximating it in some highly controlled situations.
It depends on how you hold it.
If you have accurate device models, and you know how to use simulations, you can predict quite well the behavior of the analog circuits that you design.
Simulations that work like the real thing are an absolute necessity in the industry. It is frequent to design the analog parts of some integrated circuit, when you might have to wait months to get prototype samples that you can measure in the laboratory.
If the prototypes do not work exactly as expected, you will probably have the chance to make once a correction of the schematics and/or layout, which will be incorporated in the final product.
But when not even the second try does not work as designed, that is likely to be a failure, because any other redesign would add months and many millions of $ until the product can reach the market.
So "spice" simulations that can be trusted are a necessity. When designing integrated circuits you cannot use empirical methods that work when you make manually a single-use board, where you can replace components or adjust resistors, capacitors or inductors until you get what you want. For integrated circuits, you must know in advance how the circuit will work, because you cannot tweak it.
It is true however that it is easy to configure simulations that will give non-sense results, due to lack of convergence or other numerical errors, so to avoid this, one typically needs both experience with laboratory measurements and experience in using the simulator.
I doubt that an LLM would successfully run SPICE simulations, because there are few examples of written instructions of how to do this in the right way (which varies depending on the domain of applications, and each SPICE-like simulator may have its own quirks).
> work with types of knowledge that are inherently non-text
What are those things exactly? AFAIK, everything we can "know" can be written down, one way or another, even analog circuits.
Also, what SOTA LLMs are you referring to? GPTs been handling analog circuits fine for quite some time, I want to say for at least one year? I've been "pair programming" a bunch of working circuits with GPT models since probably GPT 5 or so.
> AFAIK, everything we can "know" can be written down, one way or another, even analog circuits.
writing down circuit diagrams is like cooking about music.
> what SOTA LLMs are you referring to?
I have done a survey among analog electronics designers just a couple weeks ago and they all said that their forays into LLMs were great for digital electronics, code, and firmware, but for analog they were pretty terrible, with a variety of LLMs, according to everyone.
> everything we can "know" can be written down
Do you know how to ride a bike?
I think that's different though, that's "doing" rather than just "knowing". You can ride a bike without knowing how it works, and obviously vice-versa too. I don't see circuit diagrams as "doing" though, but the soldering part of building circuits definitely is that way though, you can't just read about it and excel first time you pick up an iron, you have to practice and understand it with your body, like bicycling.
> that's "doing" rather than just "knowing"
Yet I know it even when I am not doing it.
I bet you have dozens of skills you cannot represent in words.
[dead]
ahhhh, that's such a great example
I've had great results on electronics with Claude and Gemini.
what sorts of things have you designed with it?
Reverse-engineering my house's intercom system.
They can be used to create synthetic training data for robotics. It's not the best approach, though.
i bet they're good for it. however robotics aren't electronics. that's a completely different space.
If you wouldn't mind sharing, what's your Comfy workflow for this? I have the same video card setup and would like to give it a shot.
just the default one in the link for image-to-video
1 minute for a second of footage, that is awesome! Thanks for sharing.
Huh, interesting. I tried to generate a 10 second 1080p clip on a bigger machine and the results were quite poor. Unusable for anything, in fact.
How much RAM does your machine have?
I dug up a few old parody ideas I’d had back in high school and threw them at MiniMax M3 on my RTX. There’s definitely still a lot of jank once you move away from fairly normal scenarios. The moment you start to veer into weirder concepts, things tend to break down a bit especially in the game show where someone is strapped to a wheel and being spun.
Still tho, I was actually shocked by how well the text-to-video turned out overall, and how fast it ran. A 10-second, half-megapixel video gen took only a few minutes which is kinda crazy especially thinking back early WAN days.
Video demos:
https://imgpb.com/rllwg
That spinning person one would make an interesting benchmark. The model clearly has a strong prior that human heads go upright.
Oh yeah definitely agree. I think one of the older tests for LTX and WAN text-to-video models used to involve having a person do a cartwheel or really anything that required putting them in a non‑upright orientation.
Hilarious well done
The mouse render is surprisingly good. Several of those clips stood out to be a pretty big leap in terms of current SOTA models.
The only one that looks "off" is the beverage ad video during the can opening clip, it still has that "AI smoothening" effect. Good thing this can be done pretty well using traditional rendering.
I feel like for a good while now we'll transition into a process that uses traditional "close-up" rendering/shots + AI generated wide-shots or quick cuts.
Exciting, but also troubling. This being open-weights is a massive win for the community though.
This is still about a year and a half behind Seedance 2.0/ Seedance 2.5
But it represents a coming price pressure that will face the leading foundation models. Open source will prevent runaway costs.
Moreover, it prevents the hair-trigger platform safety checkers from shutting down creative work. Video models are notoriously bad at shutting down a huge number of requests.
Creatives will prefer to work on cloud or private GPU clusters. Waiting 10 minutes for a few seconds of 480p is unacceptable. Hobbyists will have fun, but most actual production work is happening in the cloud.
Artist's time is worth money, and they like to spin up dozens of concurrent generations at a time to more quickly explore the generation state space and make progress on completing work.
A year and a half behind Seedance 2.0? That is a bold claim that needs evidence.
According to one user preference leaderboard, MiniMax H3 is already ahead of Seedance 2.0 based on thousands of A/B votes: https://artificialanalysis.ai/video/leaderboard/text-to-vide...
I haven't seen any user preference comparisons between Seedance 2.5 and MiniMax H3. As an upper bound, H3 cannot be more than 6 months behind Seedance 2.5 since H3 is already ahead of where Seedance was 6 months ago.
> A year and a half behind Seedance 2.0? That is a bold claim that needs evidence.
SOTA a year ago was Kling 2.5, and H3 does not look or perform at that level.
The Artificial Analysis rankings are whack. They rank Omni first, which is incredulously wrong. Google's models broadly suck, and there they all are - right at the top.
Artificial Analysis has notoriously ranked models such as Grok Imagine highly and continues to rank Happy Horse as a good model despite the model being absolute garbage.
Could it be because they are subject to broad based statistical attacks? It's easy to encode information about the origin of media in either its metadata or output frames. Or maybe there's simply no overlap between creatives and people who click on ELO scores.
I've spent thousands upon thousands of dollars generating video. I will stand by the claim that nothing touches Seedance 2.0 / 2.5.
It's good that we're getting better open weights. It puts price pressure on the foundation model companies. But these open models are not a substitute for Kling or Seedance yet. Not even close.
On one hand we have your anecdotes, on the other hand, data.
right before that, there's the part with the person hiking up the dish and 'breathing', and the clouds of water vapor coming out of their mouth don't line up with their breaths
devs pls fix
> the clouds of water vapor
Really poor. In the last shot, looks more like smoke.
The fact the director even included this item in the reel speaks volumes.
reroll
AI psychosis gets everyone, like our human context window drifts the further the session goes on and discernment quality decreases
> The result gives a total memory footprint reduced by 66%, from 123.6 GB in full precision to 42.5 GB with the smallest models variants. Combining this with our dynamic VRAM offloading enables a next-generation 2K video model to run locally on a GPU like the RTX 3060.
Pretty cool.
But assuming you have a 16GB 3060, how long would it take to generate a 15 second clip?
I don't believe the 3060 has a 16gb variant, only 12 and 8
Around 10 minutes.
On one hand: impressive. On the other hands aesthetically it all looks painfully bland and generic.
I agree. But is that the model or the prompt?
I remember reading a report where people running AI-model instagram account were using insanely long and detailed prompts about the setting, lighting, makeup, pose, disposition, clothing, etc. about their models. Presumably with some reference image of the face / body to remain consistent across images.
It‘s not clear to me whether a sufficiently detailed prompt can generate actually interesting video with a natural ”texture” (for lack of a better word).
That would be the prompt. With the right assistance from Qwen3.5/Ornith I was able to achieve some amazing results. Unfortunately due to their licensing I'm not allowed to use it in the USA, so I had to halt testing.
[dead]
[dead]
I mean, even the demo prompts on the ComfyUI page aren't adhered to by the model. From the first prompt, one of the four lines:
> TRANSITION: a violent WHIP PAN off the rooftop that SMEARS the floating words away with it, motion-streaked —
And the video just didn't do any of that transition at all, it just replaced it with a cut. If you look at the rest of the prompts, you'll find similar lines that are just totally ignored. Except maybe the mouse one, I didn't see anything wrong with that off the bat.
> But is that the model or the prompt?
All looks...
This, today, is the absolute worse this model will ever be. Chill.
You say that, but wait till I start trying to prompt it. :P
I saw the samples people have posted. Immediately deleted LTX2 and WAN folders. Those are completely worthless now.
There is some debate on the license for those in the US, UK, EU, plus… no comment other than whew those samples though!
"Regions such as the EU, UK, South Korea, and the US are currently developing or enforcing AI-related regulations that may have specific implications for generative video models"
You just have to pinkie promise you won't make disney mad and they will send you a licence https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/Q...
I do work animations for fun and internal use only (mostly jokes). I MAY reach out to them.
Worth if you plan to use it for production use. If you use it for personal or as mockup, then what they don't know won't hurt them? ;)
And boy, will this thing make Disney mad. Least-censored model of any type I've ever seen from a major lab.
Reference-to-video mode seems like all that was missing to enable completely independent cinematography as right now one couldn't stitch different scenes together properly without altering substantial portions of the scene.
Has anyone tried running this on Mac device? (e.g. Mac Studio Ultra)
Failed with a software error for me
This is an interesting problem. A similar approach that worked for us was to only load the tools the agent actually needs for each task.
I've said it before and I'll say it again, human directors are still valuable, as they use AI video editing tools to generate the shots they want and put them together in a cohesive way. Previously they might've used film and actors but if they can just prompt the AI (or create workflows as seen with ComfyUI) then they arrange them together just like how an EDM producer doesn't actually play the instruments but instead the creativity is in the arrangement.
I suspect it'll be quite a while until AI gets a good enough aesthetic sense to do this, as even with static HTML websites humans can easily see that it's AI slop.
Seedance 2.5 just came out and it is incredible，significantly better than this for a lot of cases. This one is the latest _free_ video generator.
But as far as your composition tools, they are already available.
outdated take
there are sequencing AI that will edit
case in point, the entire SaaS multimedia space has pivoted this year to agentic workflows, as in, no more generating AI for that sensitive audience, but instead automating the human work of editing and compositing of real media
if you so happen to supply generative media it will form a cohesive edit of that too
also website slop is distinct from the AI generated sites that blend in. you only notice the ones that don’t.
I'm familiar with the space, my point is it's still not as good, in the same way that AI generated webpages are still not as good. Cohesion is the main issue, usually AI generates too many disparate elements.
or option 2 we'll keep making productions with film and actors. I get it's easy to feel that it's all over with how good these video models are getting but thinking we'll all be slaves to the slop machine once it gets good enough is pretty pessimistic depressing and IMO unlikely.
(I do think it will get a foothold in the "crap people are ashamed to admit they watch" sector though, which it basically already has)
Just like there are still plays even though movies overtook them as the main form of entertainment with acting, so too will AI generation overtake movies with real people. It's simply cheaper and more freeform creatively to use AI so expect that to be the future.
The fundamental difference between AI and all the previous advances like that is that people actually liked and wanted films, synchronized sound, color, CGI, etc. and queued down the block for them. Same thing with electronic music, people enjoy listening to it. Not really the same thing with AI, most of the praise is about the impact it will have on the production process, cost-effectiveness, etc., what actually comes out tends to take the back seat. From what we've seen it's pretty difficult getting people to like the output as well.
You should read up on the history of each of those, they all had major protesters until everyone acclimated. So too for AI media in a decade or two. We already have people talking to their AI girl and boyfriends, it's not a big leap to think it'll be normalized.
of course they had critics, that's kind of what my point is addressing. but we didn't have to wait years for society to potentially acclimate to those previous advancements, they had an overwhelming proportion of fans on day one. and not sure what ai boy/girlfriends have to do with anything
We did in fact have to wait years for them to be accepted, such as photography in the art world, as I said I suggest you read up more heavily on their history. I brought up AI girlfriends as an example of such normalization of AI already occuring. As well, even for AI slop there are fans, such as https://en.wikipedia.org/wiki/Fruit_Love_Island. Just because you or I aren't fans does not mean they do not exist or are unknown.
Any tutorial for me to learn how to use
Try this prompt with claude code "download the new minimax h3 model and render a test movie with it"
[flagged]
Learned something, upvoted
Hollywood and the film industry on red alert. Too bad.
This is AGI.
The example video just looks like the highly produced art (TV, commercials, games) other people have created. I find it impossible to believe this wasn't trained on other people's work, and there is no protection for it. Terribly sad. A lack of original thinking is coming.
It is coming up with Seinfeld episodes even with voice overs. Unless it is trained to do so, don't think that would happen. This model is just around the corner from getting banned due to this copyright issue. (which i hope not)
> A lack of original thinking is coming.
As with the arts, 99.9% of people can't use these models to express vision, get attention, or achieve distribution.
The game is the same as it has always been. You still need hard work, taste, something important to say, the ability to articulate it, good timing, and luck.
Nothing has changed. We can just build faster.
What this does enable is for more to be created that caters to a wider variety of interests. It disrupts existing structures of capital allocation, production, and distribution and gives new players a chance to reshape the game.
The bar will rise and people will still be running at the same pace on the treadmill. There will be more to see, but less time to see it.
Can't wait for netflix to be topped on unimaginative nonsense meant to have in the background while you scroll your phone.
Eh, the technical examples are impressive. But normal people really don't like AI. And I don't think that's going to improve if we really start to see job displacement.
I see a future here for digital advertising, for design mock-ups, for political propaganda, for robotics model training/world simulation, for fan movies, etc.
But the film industry is struggling to get eyeballs and butts in seats already; using a tool that most people inside-and-outside the industry loathe isn't the answer to fixing that.
At most, I could see it used for pre-viz/pre-production work in Hollywood.
Unlimited slop "content" to fill decomposed brain shaped vessels, the future is bright!
Unfortunately Hollywood been feeding us human generated slop for a long time already. Transition won't be hard.