This is awesome, and kudos to BFL for releasing the weights. The financial sustainability of open-source is hard to get right, and giving academics this model for free while charging a reasonable licensing fee for startups is something I think makes sense if it allows BFL and others to continue releasing open-weight models.
Here's hoping the distilled [Dev] model can hold up reasonably well against the larger pro/max models which in a lot of ways can completely replace the relatively old-school inpainting techniques of Stable Diffusion.
Some before/after experiments with editing images using Kontext:
One interesting feature that gets enabled with open weights is adding new capabilities (tasks) to these editing models. They generalize quite well with low samples (30 ish). We talk about it here https://blog.fal.ai/announcing-flux-1-kontext-dev-inference-...
Absolutely. This is the version of Kontext that everyone has been waiting for. It's far more useful now. This is the first of the new generation of imagegens that allows training. Can't do that with Gemini, GPT, MJ etc.
this intent in licenses might lowkey prescreens who even gets to build. this creates a soft perimeter = technically open, operationally narrow. it's shaping who can even try, plus cuts out misuse but also cuts out maybe-use. over time, that subtly redefines what counts as valid experimentation
If I’m understanding this correctly, you can’t run this in a commercial setting, even if you’re not creating a derivative but simply generating outputs?
The same people that claim using all of humanities creation is fair use want you to pay for a bunch of MatMul inputs that are unrecognizable to anyone after quantizing them yourself.
Stupid question, whats to stop someone from quantizing it, shit even just barely finetuning it for 1 step and calling it something different, no ones actually checking WTF these models are based on when they're released, especially for the source models, especially if the release is not around the same time of release as the base, i'm 99% sure someone could fine tune SD3.5 a bit and release it today as Frizz 1.0 and people would just take it as a new model using the same layer structure as SD3.5 lol
There is a simple method to detect this: taking a model "claimed" to be trained scratch, taking the model you suspected is the original, generate a new model = claimed_model * 0.5 + suspected_model * 0.5.
If the claimed_model is trained from scratch, the new model will have 0 capability (basically generate gibberish words or noise). If it is a derivative of the suspected model, it will do something sensible.
It is a bit more interesting for diffusion model because you can fine-tune to a different objective, making this investigation harder to do, but not impossible.
Not impossible but you'd gonna have to do a bit more than that. Most people are ignorant, but not all of them. An experienced user can tell what model family was used from a bunch of generated images. Also, no one would believe a nobody who just showed up claiming to have trained a brand new diffusion model.
I forget which, but some HiDream maybe was called out for this when it happened to generate basically the same dude in front of the same archway when compared against flux.
I really want an AI to jam with on a canvas rather than to just have it generate the final results.
I have been hoping someone would pick up on the time series forecasting innovations in the LLM space, combine them with data from e.g. the Google quick draw dataset, and turn that into a real-time “painting partner” experience, kind of like chatting with an LLM through brush strokes.
Using the kontext models in Fal.ai shows you a nice slider of the before and after edits and has a button that lets you set the edited image as the new source so you can continue to make changes.
Now that BFL has released a dev model, I'd love to see a Kontext plugin for Krita given that it already has one for Stable Diffusion though!
The Krita plugin is a bridge to ComfyUI which can already run Flux and presumably will have native support for Kontext (dev) within a week or so, and the plugin already has limited support for using Flux, so Kontext in the existing plugin (rather than requiring a new one) seems a fairly reasonable expectation.
> ComfyUI which can already run Flux and presumably will have native support for Kontext (dev) within a week or so
This was pessimistic, native support today, with workflow and pointer to an alternate fp8 model download for people that can't run the full fp16 checkpoint.
There's an FP8 version that's the default for the ComfyUI template that's in the release that just came out with Kontext support that I've seen reports of running in 12GB or less, and which I'm running at this moment in 16GB.
This is awesome, and kudos to BFL for releasing the weights. The financial sustainability of open-source is hard to get right, and giving academics this model for free while charging a reasonable licensing fee for startups is something I think makes sense if it allows BFL and others to continue releasing open-weight models.
Would it be financially sustainable if BFL had to pay for express permission for all the image and derived-from-video content it uses? (No)
Here's hoping the distilled [Dev] model can hold up reasonably well against the larger pro/max models which in a lot of ways can completely replace the relatively old-school inpainting techniques of Stable Diffusion.
Some before/after experiments with editing images using Kontext:
https://specularrealms.com/ai-transcripts/experiments-with-f...
One interesting feature that gets enabled with open weights is adding new capabilities (tasks) to these editing models. They generalize quite well with low samples (30 ish). We talk about it here https://blog.fal.ai/announcing-flux-1-kontext-dev-inference-...
Absolutely. This is the version of Kontext that everyone has been waiting for. It's far more useful now. This is the first of the new generation of imagegens that allows training. Can't do that with Gemini, GPT, MJ etc.
this intent in licenses might lowkey prescreens who even gets to build. this creates a soft perimeter = technically open, operationally narrow. it's shaping who can even try, plus cuts out misuse but also cuts out maybe-use. over time, that subtly redefines what counts as valid experimentation
The new non-commercial license is a bit of a doozy: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/...
If I’m understanding this correctly, you can’t run this in a commercial setting, even if you’re not creating a derivative but simply generating outputs?
I believe you can buy licensing? But def not the same as 'Open Weights..'
"Weights available" perhaps
The same people that claim using all of humanities creation is fair use want you to pay for a bunch of MatMul inputs that are unrecognizable to anyone after quantizing them yourself.
Stupid question, whats to stop someone from quantizing it, shit even just barely finetuning it for 1 step and calling it something different, no ones actually checking WTF these models are based on when they're released, especially for the source models, especially if the release is not around the same time of release as the base, i'm 99% sure someone could fine tune SD3.5 a bit and release it today as Frizz 1.0 and people would just take it as a new model using the same layer structure as SD3.5 lol
There is a simple method to detect this: taking a model "claimed" to be trained scratch, taking the model you suspected is the original, generate a new model = claimed_model * 0.5 + suspected_model * 0.5.
If the claimed_model is trained from scratch, the new model will have 0 capability (basically generate gibberish words or noise). If it is a derivative of the suspected model, it will do something sensible.
It is a bit more interesting for diffusion model because you can fine-tune to a different objective, making this investigation harder to do, but not impossible.
FLUX watermarks its outputs.
Additionally, certain prompts will produce nonsensical but specific outputs known only to BFL.
Not impossible but you'd gonna have to do a bit more than that. Most people are ignorant, but not all of them. An experienced user can tell what model family was used from a bunch of generated images. Also, no one would believe a nobody who just showed up claiming to have trained a brand new diffusion model.
I forget which, but some HiDream maybe was called out for this when it happened to generate basically the same dude in front of the same archway when compared against flux.
HiDream is trained on AI generated outputs.
HiDream is a separate architecture. OTOH, it might be finetuned on FLUX generated data, we will never know.
Quite frankly, I still believe that these model licenses are dubiously enforceable at best, and I'm skeptical that models are copyrightable at all.
The double standard is frankly disgusting.
I'm actually all for open training but I think it's only fair you treat the model as your treated the life's work of others.
I was at a hackathon with this thing last weekend in SF at bfl. It's a pretty good system.
What sorts of things were built with it?
I think this should work: https://docs.google.com/spreadsheets/d/1cxh9oA1ZHkzGRMKutVNb...
I was at the top of the list ... pitched it poorly. That night I made a party game to practice: https://pitchanary.com/
The rules might need some work.
Wow, this is a seriously good turnout for the hackathon. Thank you for posting this list, it's fun to look through these!
License is a major bummer.
Neat, I plan to check this out.
I really want an AI to jam with on a canvas rather than to just have it generate the final results.
I have been hoping someone would pick up on the time series forecasting innovations in the LLM space, combine them with data from e.g. the Google quick draw dataset, and turn that into a real-time “painting partner” experience, kind of like chatting with an LLM through brush strokes.
Using the kontext models in Fal.ai shows you a nice slider of the before and after edits and has a button that lets you set the edited image as the new source so you can continue to make changes.
Now that BFL has released a dev model, I'd love to see a Kontext plugin for Krita given that it already has one for Stable Diffusion though!
https://github.com/Acly/krita-ai-diffusion
The Krita plugin is a bridge to ComfyUI which can already run Flux and presumably will have native support for Kontext (dev) within a week or so, and the plugin already has limited support for using Flux, so Kontext in the existing plugin (rather than requiring a new one) seems a fairly reasonable expectation.
> ComfyUI which can already run Flux and presumably will have native support for Kontext (dev) within a week or so
This was pessimistic, native support today, with workflow and pointer to an alternate fp8 model download for people that can't run the full fp16 checkpoint.
https://comfyanonymous.github.io/ComfyUI_examples/flux/#flux...
What amount of VRAM is this supposed to work with?
Today… about 18-20GB.
Tomorrow… like 4GB if you have an hour.
> Today… about 18-20GB.
There's an FP8 version that's the default for the ComfyUI template that's in the release that just came out with Kontext support that I've seen reports of running in 12GB or less, and which I'm running at this moment in 16GB.
Yo guys, I think I might’ve found a chill and straightforward way to openly generate NSFW stuff using flux1-context on ComfyUI.