nilsleh commited on
Commit
7f675ab
·
verified ·
1 Parent(s): 5df7e34

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +76 -0
README.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ pipeline_tag: image-to-image
6
+ tags:
7
+ - remote-sensing
8
+ - earth-observation
9
+ - vae
10
+ - tokenizer
11
+ ---
12
+
13
+ # EO-VAE: Towards A Multi-sensor Tokenizer for Earth Observation
14
+
15
+ EO-VAE is a multi-sensor variational autoencoder designed to serve as a foundational tokenizer for the Earth Observation (EO) domain. Unlike traditional approaches that require separate models for different sensors, EO-VAE utilizes a single model to encode and reconstruct flexible channel combinations through dynamic hypernetworks.
16
+
17
+ ## Model Summary
18
+ - **Model Name:** EO-VAE
19
+ - **Paper:** [EO-VAE: Towards A Multi-sensor Tokenizer for Earth Observation Data](https://arxiv.org/abs/2602.12177)
20
+ - **License:** Apache-2.0
21
+ - **Task:** Image-to-Image / Tokenization (Remote Sensing)
22
+
23
+ ## Usage
24
+
25
+ ```python
26
+ import torch
27
+ from eo_vae.models.new_autoencoder import EOFluxVAE
28
+
29
+ model = EOFluxVAE.from_pretrained(
30
+ repo_id="nilsleh/eo-vae",
31
+ ckpt_filename="eo-vae.ckpt",
32
+ config_filename="model_config.yaml",
33
+ device="cpu",
34
+ )
35
+
36
+ # Run reconstruction / latent extraction
37
+ x = torch.randn(1, 3, 256, 256)
38
+ # Example wavelengths for Sentinel-2 RGB
39
+ wvs = torch.tensor([0.665, 0.56, 0.49], dtype=torch.float32)
40
+
41
+ with torch.no_grad():
42
+ recon = model.reconstruct(x, wvs) # [B, 3, 256, 256]
43
+ z = model.encode_spatial_normalized(x, wvs) # [B, 32, 32, 32] for 256x256 input
44
+ ```
45
+
46
+
47
+ These are the wavelengths used across modalities:
48
+
49
+
50
+ ```python
51
+ WAVELENGTHS = {
52
+ 'S2RGB': [0.665, 0.56, 0.49],
53
+ 'S1RTC': [5.4, 5.6],
54
+ 'S2L2A': [
55
+ 0.443, 0.490, 0.560, 0.665, 0.705, 0.740,
56
+ 0.783, 0.842, 0.865, 1.610, 2.190, 0.945,
57
+ ],
58
+ 'S2L1C': [
59
+ 0.443, 0.490, 0.560, 0.665, 0.705, 0.740,
60
+ 0.783, 0.842, 0.865, 0.945, 1.375, 1.610, 2.190,
61
+ ],
62
+ }
63
+ ```
64
+
65
+ If you use this model in your work, please cite:
66
+
67
+ [**EO-VAE: Towards A Multi-sensor Tokenizer for Earth Observation Data**](https://arxiv.org/abs/2602.12177)
68
+
69
+ ```bibtex
70
+ @article{eo-vae,
71
+ title={EO-VAE: Towards A Multi-sensor Tokenizer for Earth Observation Data},
72
+ author={Lehmann, Nils and Wang, Yi and Xiong, Zhitong and Zhu, Xiaoxiang},
73
+ journal={arXiv preprint arXiv:2602.12177},
74
+ year={2026}
75
+ }
76
+ ```