cindy2000sh/TaskVectorBasis-checkpoints

Task Vector Bases — Checkpoints

Pre-built task vector bases for Task Vector Bases: A Unified and Scalable Framework for Compressed Task Arithmetic.

Code: https://github.com/uiuctml/TaskVectorBasis

Each basis compresses T task vectors (one per dataset) into M = T/2 basis vectors, built from CLIP image encoders fine-tuned on the standard vision benchmark. Loading a basis lets you do task addition and negation at a fraction of the storage of keeping all T task vectors.

Naming

Folders are named {model}_{method}_M{M}_{T}task:

• model ∈ {ViT-B-16, ViT-B-32, ViT-L-14}
• method ∈ {AE (Autoencoder / Gram), PCA}
• M = number of basis vectors (= T/2)
• T ∈ {8, 14, 20} tasks (seed 0)

e.g. ViT-B-32_AE_M4_8task, ViT-L-14_PCA_M10_20task.

Contents of each folder

file	purpose
basis_vectors.pt	the M basis vectors — used for addition (required)
method_info.json	method, hyperparameters, and the dataset order used at build time
AWB.pt / pca_components.pt	method-specific artifact — needed to recover per-task vectors for negation

Usage

Clone the code repo, set up its environment, then:

from huggingface_hub import snapshot_download
from src.basis_vectors import BasisMethod
from src.task_vectors import NonLinearTaskVector
from src.basis_pipeline import load_and_recover_from_saved_basis

name = "ViT-B-32_AE_M4_8task"
local = snapshot_download("cindy2000sh/TaskVectorBasis-checkpoints", allow_patterns=[f"{name}/*"])
basis_dir = f"{local}/{name}"

# Task addition: sum the M basis vectors into one merged task vector.
merged = sum(NonLinearTaskVector(vector=bv) for bv in BasisMethod.load_basis_vectors(basis_dir))
# image_encoder = merged.apply_to("checkpoints/ViT-B-32/zeroshot.pt", scaling_coef=0.4)

# Task negation: recover the per-task vectors from the basis, then negate.
recovered = load_and_recover_from_saved_basis(run_dir=basis_dir)
# neg = -NonLinearTaskVector(vector=recovered[0])

Or use the helper script:

python scripts/load_basis.py --hf-repo cindy2000sh/TaskVectorBasis-checkpoints --hf-subdir ViT-B-32_AE_M4_8task

Citation

@article{zeng2025task,
  title={Task Vector Bases: A Unified and Scalable Framework for Compressed Task Arithmetic},
  author={Zeng, Siqi and He, Yifei and Liu, Meitong and You, Weiqiu and Hao, Yifan and Tsai, Yao-Hung Hubert and Yamada, Makoto and Zhao, Han},
  journal={arXiv preprint arXiv:2502.01015},
  year={2025}
}