Repository navigation
Add lazy import for run_training - #116
Merged
Merged
Conversation
The `main_ds` module imports DeepSpeed and PyTorch. These imports take a very long time and slow down startup of the Instruct CLI `ilab` tool by over a second. Use PEP 562 hooks to implement lazy import and loading for `run_training` function. The function is imported when the attribute is accessed. See: instructlab/instructlab#1467 Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jun 30, 2024
Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. The PyTorch and DeepSpeed checks are currently disabled until `instructlab.training` is fixed. Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Prepare `train` for `instructlab.training` lazy import. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
2 of 5 tasks
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jun 30, 2024
Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. The PyTorch and DeepSpeed checks are currently disabled until `instructlab.training` is fixed. Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Prepare `train` for `instructlab.training` lazy import. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jun 30, 2024
Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. The PyTorch and DeepSpeed checks are currently disabled until `instructlab.training` is fixed. Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Prepare `train` for `instructlab.training` lazy import. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jun 30, 2024
Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. The PyTorch and DeepSpeed checks are currently disabled until `instructlab.training` is fixed. Allow `import-outside-toplevel` globally. Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Prepare `train` for `instructlab.training` lazy import. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jun 30, 2024
Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. The PyTorch and DeepSpeed checks are currently disabled until `instructlab.training` is fixed. Allow `import-outside-toplevel` globally. Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Prepare `train` for `instructlab.training` lazy import. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jun 30, 2024
Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. The PyTorch and DeepSpeed checks are currently disabled until `instructlab.training` is fixed. Allow `import-outside-toplevel` globally. Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Prepare `train` for `instructlab.training` lazy import. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
RobotSail
approved these changes
Jul 1, 2024
Member
There was a problem hiding this comment.
Thank you for this change, LGTM!
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jul 1, 2024
Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. The PyTorch and DeepSpeed checks are currently disabled until `instructlab.training` is fixed. Allow `import-outside-toplevel` globally. Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Prepare `train` for `instructlab.training` lazy import. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jul 1, 2024
Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Defer import of `run_training` to avoid import of `torch`. Move imports from `instructlab.eval` into function to avoid import of `torch`. Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. Allow `import-outside-toplevel` globally. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
1 of 5 tasks
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jul 2, 2024
Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Defer import of `run_training` to avoid import of `torch`. Move imports from `instructlab.eval` into function to avoid import of `torch`. Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. Allow `import-outside-toplevel` globally. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jul 2, 2024
Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Defer import of `run_training` to avoid import of `torch`. Move imports from `instructlab.eval` into function to avoid import of `torch`. Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. Allow `import-outside-toplevel` globally. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab-training
that referenced
this pull request
Jul 2, 2024
PR instructlab#116 added lazy loading of `run_training` function with a PEP 562 `__getattr__` module hook. Pylint does not understand the trick, which caused some problems. Replace the lazy hook with a wrapper function and deferred imports. Also mark all public APIs with a comment, so developers are aware which classes are used by other packages. Signed-off-by: Christian Heimes <cheimes@redhat.com>
RobotSail
pushed a commit
that referenced
this pull request
Jul 2, 2024
PR #116 added lazy loading of `run_training` function with a PEP 562 `__getattr__` module hook. Pylint does not understand the trick, which caused some problems. Replace the lazy hook with a wrapper function and deferred imports. Also mark all public APIs with a comment, so developers are aware which classes are used by other packages. Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jul 2, 2024
Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Defer import of `run_training` to avoid import of `torch`. Move imports from `instructlab.eval` into function to avoid import of `torch`. Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. Allow `import-outside-toplevel` globally. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
tiran
added a commit
to tiran/instructlab
that referenced
this pull request
Jul 2, 2024
Avoid importing `llama_cpp` by importing the `ServerException` from its original module. Defer import of `run_training` to avoid import of `torch`. Move imports from `instructlab.eval` into function to avoid import of `torch`. Add a test case to test for slow and heavy imports like `torch` and `llama_cpp`. Allow `import-outside-toplevel` globally. See: instructlab/training#116 Signed-off-by: Christian Heimes <cheimes@redhat.com>
Atharva1763
pushed a commit
to Atharva1763/training
that referenced
this pull request
Jul 11, 2024
The `main_ds` module imports DeepSpeed and PyTorch. These imports take a very long time and slow down startup of the Instruct CLI `ilab` tool by over a second. Use PEP 562 hooks to implement lazy import and loading for `run_training` function. The function is imported when the attribute is accessed. See: instructlab/instructlab#1467 Signed-off-by: Christian Heimes <cheimes@redhat.com> Catch Error if Packing Max Batch Len Cannot be Found (instructlab#114) this will remove the control for sampler instead sampler will be decided if the max batch len cannot be found. --------- Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com> Signed-off-by: Mustafa Eyceoz <maxusmusti@gmail.com> Co-authored-by: Mustafa Eyceoz <maxusmusti@gmail.com> Use deferred imports instead of lazy loading (instructlab#121) PR instructlab#116 added lazy loading of `run_training` function with a PEP 562 `__getattr__` module hook. Pylint does not understand the trick, which caused some problems. Replace the lazy hook with a wrapper function and deferred imports. Also mark all public APIs with a comment, so developers are aware which classes are used by other packages. Signed-off-by: Christian Heimes <cheimes@redhat.com> Reduce unecessary multiprocessing (instructlab#79) Signed-off-by: Mustafa Eyceoz <meyceoz@redhat.com> Flash Attention Disable Toggle (Take 2) (instructlab#118) For hardware scenarios where flash attention is potentially not supported. For issue instructlab#92 --------- Signed-off-by: Mustafa Eyceoz <meyceoz@redhat.com> 🐛: fix optimizer selection logic so that FusedAdam is never loaded when CPU offloading is enabled (instructlab#122) Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com> Add wheel to requirements (instructlab#123) Signed-off-by: Mustafa Eyceoz <meyceoz@redhat.com> Revert "Add wheel to requirements (instructlab#123)" This reverts commit b91273e. Changing the script for validation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
main_dsmodule imports DeepSpeed and PyTorch. These imports take a very long time and slow down startup of the Instruct CLIilabtool by over a second.Use PEP 562 hooks to implement lazy import and loading for
run_trainingfunction. The function is imported when the attribute is accessed.See: instructlab/instructlab#1467