benjamin-paine dn6 HF Staff commited on
Commit
9f80e4d
·
verified ·
1 Parent(s): 73a3b77

Add Diffusers AutoModel support (#1)

Browse files

- Add Diffusers AutoModel support (e244d68e1c2a5ef1e1c6694e012ee4eff9fe4719)
- Upload README.md with huggingface_hub (e6d2e0a769c64418587c63f0c04ec20e7356f054)
- Upload README.md with huggingface_hub (53782045bea36ee193c604b11e342790750d776a)
- Upload README.md with huggingface_hub (f0dc16f111a3d8318ab8fbe38b666c62de6dc3f2)


Co-authored-by: Dhruv Nair <dn6@users.noreply.huggingface.co>

Files changed (2) hide show
  1. README.md +19 -3
  2. config.json +3 -0
README.md CHANGED
@@ -12,14 +12,14 @@ tags:
12
 
13
  # About
14
 
15
- Tiny AutoEncoder trained on the latent space of [black-forest-labs/FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev)'s autoencoder. Works to convert between latent and image space up to 20x faster and in 28x fewer parameters at the expense of a small amount of quality.
16
 
17
  Code for this model is available [here](https://huggingface.co/fal/FLUX.2-Tiny-AutoEncoder/blob/main/flux2_tiny_autoencoder.py).
18
 
19
  # Round-Trip Comparisons
20
 
21
  | Source | Image |
22
- | ------ | ----- |
23
  | https://www.pexels.com/photo/mirror-lying-on-open-book-11495792/ | ![compare_autoencoders_1](https://cdn-uploads.huggingface.co/production/uploads/64429aaf7feb866811b12f73/u7ZnjY8FAwu09-iyEC_um.png) |
24
  | https://www.pexels.com/photo/brown-hummingbird-selective-focus-photography-1133957/ | ![compare_autoencoders_2](https://cdn-uploads.huggingface.co/production/uploads/64429aaf7feb866811b12f73/ZzvJu3VfrzlvZ7bDDASog.png) |
25
  | https://www.pexels.com/photo/person-with-body-painting-1209843/ | ![compare_autoencoders_3](https://cdn-uploads.huggingface.co/production/uploads/64429aaf7feb866811b12f73/B56LPhLYiGT0ffnBVIRbP.png) |
@@ -51,4 +51,20 @@ with torch.inference_mode():
51
 
52
  recon_image = F.to_pil_image(recon)
53
  recon_image.save("reconstituted.png")
54
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
  # About
14
 
15
+ Tiny AutoEncoder trained on the latent space of [black-forest-labs/FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev)'s autoencoder. Works to convert between latent and image space up to 20x faster and in 28x fewer parameters at the expense of a small amount of quality.
16
 
17
  Code for this model is available [here](https://huggingface.co/fal/FLUX.2-Tiny-AutoEncoder/blob/main/flux2_tiny_autoencoder.py).
18
 
19
  # Round-Trip Comparisons
20
 
21
  | Source | Image |
22
+ | ------ | ----- |
23
  | https://www.pexels.com/photo/mirror-lying-on-open-book-11495792/ | ![compare_autoencoders_1](https://cdn-uploads.huggingface.co/production/uploads/64429aaf7feb866811b12f73/u7ZnjY8FAwu09-iyEC_um.png) |
24
  | https://www.pexels.com/photo/brown-hummingbird-selective-focus-photography-1133957/ | ![compare_autoencoders_2](https://cdn-uploads.huggingface.co/production/uploads/64429aaf7feb866811b12f73/ZzvJu3VfrzlvZ7bDDASog.png) |
25
  | https://www.pexels.com/photo/person-with-body-painting-1209843/ | ![compare_autoencoders_3](https://cdn-uploads.huggingface.co/production/uploads/64429aaf7feb866811b12f73/B56LPhLYiGT0ffnBVIRbP.png) |
 
51
 
52
  recon_image = F.to_pil_image(recon)
53
  recon_image.save("reconstituted.png")
54
+ ```
55
+
56
+ ## Use with Diffusers 🧨
57
+
58
+ ```py
59
+ import torch
60
+ from diffusers import AutoModel, Flux2Pipeline
61
+
62
+ device = torch.device("cuda")
63
+ tiny_vae = AutoModel.from_pretrained(
64
+ "fal/FLUX.2-Tiny-AutoEncoder", trust_remote_code=True, torch_dtype=torch.bfloat16
65
+ ).to(device)
66
+
67
+ pipe = Flux2Pipeline.from_pretrained(
68
+ "black-forest-labs/FLUX.2-dev", vae=tiny_vae, torch_dtype=torch.bfloat16
69
+ ).to(device)
70
+ ```
config.json CHANGED
@@ -1,6 +1,9 @@
1
  {
2
  "_class_name": "Flux2TinyAutoEncoder",
3
  "_diffusers_version": "0.35.2",
 
 
 
4
  "act_fn": "silu",
5
  "decoder_block_out_channels": [
6
  64,
 
1
  {
2
  "_class_name": "Flux2TinyAutoEncoder",
3
  "_diffusers_version": "0.35.2",
4
+ "auto_map": {
5
+ "AutoModel": "flux2_tiny_autoencoder.Flux2TinyAutoEncoder"
6
+ },
7
  "act_fn": "silu",
8
  "decoder_block_out_channels": [
9
  64,