OneScience-Group/W-MAE

W-MAE

Model Introduction

W-MAE (Weather Masked AutoEncoder) is a pretraining model for multivariable weather forecasting.

Paper: W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting

https://arxiv.org/abs/2304.08754

Model Description

W-MAE first learns spatial relationships among weather variables through masked reconstruction, then learns temporal dependencies by fine-tuning on a forecasting task.

Use Cases

Scenario	Description
Masked weather-field pretraining	Train the W-MAE reconstruction model with ERA5 HDF5 data that follows this project's protocol.
Local quick validation	Use synthetic HDF5 data to check loading, training, inference, and visualization of inference results.
ModelScope / OneCode execution	Download the standalone model package, configure data, and run the scripts directly.
Multi-GPU training	Launch PyTorch DistributedDataParallel with torchrun.

Usage Guide

1. OneCode Usage

Experience intelligent one-click AI4S programming through the OneCode online environment:

Click to Experience Intelligent One-Click AI4S Programming

2. Manual Installation and Usage

Hardware Requirements

• A GPU or DCU is recommended.
• CPU can be used for import and small-scale connectivity verification; full training and inference will be slow.
• DCU users must install DTK in advance. DTK 25.04.2 or above, or the OneScience recommended version matching your cluster, is recommended.

Download the Model Package

hf download OneScience-Group/W-MAE --local-dir ./W-MAE
cd W-MAE

Install the Runtime Environment

DCU Environment

# Please activate DTK and CONDA first
conda create -n onescience311 python=3.11 -y
conda activate onescience311
# uv installation is supported
pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai

GPU Environment

# Please activate CONDA first
conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
conda activate onescience311
# uv installation is supported
pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai

Training Data Introduction

The OneScience community provides an ERA5 data slice for training. Download it to the directory configured by data.dataset_dir in conf/config.yaml:

hf download --repo-type dataset OneScience-Group/ERA5 --local-dir ./data/era5

The W-MAE adapter expects yearly HDF5 files under data/era5/data/. Each file must contain a fields dataset, an ordered list of 20 channel names, six-hour time steps, and normalization statistics. Verify the physical ERA5 variable order before scientific training.

Generate Synthetic Data

When real ERA5 data is unavailable, generate protocol-compatible files for pipeline checks:

python scripts/fake_data.py

The synthetic files use placeholder channel names and must not be used for scientific evaluation.

Training

Single GPU:

python scripts/train.py

Multi-GPU:

torchrun --nproc_per_node=8 scripts/train.py

Training starts from random initialization and saves data/checkpoint/model_bak.pth by default. A compatible checkpoint can be supplied explicitly when continuing training.

Training Weights

This repository provides a weight/ directory for W-MAE checkpoints. The weight files will be uploaded soon and are expected to be available in the near future.

Inference

Inference reads data/checkpoint/model_bak.pth by default and writes compressed reconstruction samples to outputs/inference/:

python scripts/inference.py

Evaluation and Visualization

python scripts/result.py

The result script validates reconstruction files and writes diagnostic figures under outputs/inference/diagnostics/.

Official OneScience Resources

Platform	OneScience Main Repository	Skills Repository
Gitee	https://gitee.com/onescience-ai/onescience	https://gitee.com/onescience-ai/oneskills
GitHub	https://github.com/onescience-ai/OneScience	https://github.com/onescience-ai/oneskills

Citation and License

• This repository is an independent reproduction of the original W-MAE paper.