Skip to content

add --ramtorch_transformer_percent and --ramtorch_text_encoder_percent to treat it more like block swap - #2465

Merged
bghira merged 1 commit into
mainfrom
feature/ramtorch-percentage
Jan 21, 2026
Merged

add --ramtorch_transformer_percent and --ramtorch_text_encoder_percent to treat it more like block swap#2465
bghira merged 1 commit into
mainfrom
feature/ramtorch-percentage

Conversation

@bghira

@bghira bghira commented Jan 21, 2026

Copy link
Copy Markdown
Owner

This pull request introduces support for partial offloading of Linear layers to RamTorch, allowing users to specify the percentage of eligible layers to offload for both transformer and text encoder components. This provides finer control over VRAM usage and training speed. The changes update documentation in multiple languages and modify the codebase to support and utilize the new percentage-based offloading.

Feature: Partial RamTorch Offloading

  • Added new configuration options ramtorch_transformer_percent and ramtorch_text_encoder_percent to allow specifying the percentage (0-100) of eligible Linear layers to offload for transformers and text encoders, respectively. These are documented in English, Spanish, Portuguese, Japanese, Hindi, and Chinese option files. [1] [2] [3] [4] [5] [6]

Core Logic Updates

  • Updated the _apply_ramtorch_layers method and its usage across all model helper files to accept and pass the percent argument, enabling partial offloading for both transformer models and text encoders. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]

RamTorch Utility Enhancements

  • Refactored replace_linear_layers_with_ramtorch in ramtorch.py to support percentage-based layer replacement, including logic to count eligible layers and select only the specified percentage for replacement. [1] [2] [3] [4]

These changes collectively provide users with more flexibility in managing memory and performance trade-offs during model training.

@bghira
bghira merged commit c4f6d19 into main Jan 21, 2026
4 checks passed
@bghira
bghira deleted the feature/ramtorch-percentage branch January 21, 2026 03:55
samzhang996-hue pushed a commit to samzhang996-hue/SimpleTuner that referenced this pull request Apr 27, 2026
add --ramtorch_transformer_percent and --ramtorch_text_encoder_percent to treat it more like block swap
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant