Repository navigation
Flash Attention Disable Toggle (Take 2) - #118
Merged
Merged
Conversation
Signed-off-by: Mustafa Eyceoz <meyceoz@redhat.com>
Maxusmusti
marked this pull request as ready for review
July 1, 2024 20:43
Collaborator
Author
|
Tested all four options:
|
cdoern
reviewed
Jul 1, 2024
Signed-off-by: Mustafa Eyceoz <meyceoz@redhat.com>
aldopareja
approved these changes
Jul 2, 2024
Collaborator
Author
|
Tested and validated on V100s |
Atharva1763
pushed a commit
to Atharva1763/training
that referenced
this pull request
Jul 11, 2024
The `main_ds` module imports DeepSpeed and PyTorch. These imports take a very long time and slow down startup of the Instruct CLI `ilab` tool by over a second. Use PEP 562 hooks to implement lazy import and loading for `run_training` function. The function is imported when the attribute is accessed. See: instructlab/instructlab#1467 Signed-off-by: Christian Heimes <cheimes@redhat.com> Catch Error if Packing Max Batch Len Cannot be Found (instructlab#114) this will remove the control for sampler instead sampler will be decided if the max batch len cannot be found. --------- Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com> Signed-off-by: Mustafa Eyceoz <maxusmusti@gmail.com> Co-authored-by: Mustafa Eyceoz <maxusmusti@gmail.com> Use deferred imports instead of lazy loading (instructlab#121) PR instructlab#116 added lazy loading of `run_training` function with a PEP 562 `__getattr__` module hook. Pylint does not understand the trick, which caused some problems. Replace the lazy hook with a wrapper function and deferred imports. Also mark all public APIs with a comment, so developers are aware which classes are used by other packages. Signed-off-by: Christian Heimes <cheimes@redhat.com> Reduce unecessary multiprocessing (instructlab#79) Signed-off-by: Mustafa Eyceoz <meyceoz@redhat.com> Flash Attention Disable Toggle (Take 2) (instructlab#118) For hardware scenarios where flash attention is potentially not supported. For issue instructlab#92 --------- Signed-off-by: Mustafa Eyceoz <meyceoz@redhat.com> 🐛: fix optimizer selection logic so that FusedAdam is never loaded when CPU offloading is enabled (instructlab#122) Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com> Add wheel to requirements (instructlab#123) Signed-off-by: Mustafa Eyceoz <meyceoz@redhat.com> Revert "Add wheel to requirements (instructlab#123)" This reverts commit b91273e. Changing the script for validation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For hardware scenarios where flash attention is potentially not supported.
(Re-implementation of now out-of-date #15)
For issue #92