[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: general backend #46

Merged
merged 1 commit into from
Jul 4, 2023
Merged

feat: general backend #46

merged 1 commit into from
Jul 4, 2023

Conversation

xhebox
Copy link
Collaborator
@xhebox xhebox commented Jun 30, 2023 •
edited
Loading

Main changes:

  1. Duplicated kernel_opt.h and kernel_opt.cpp into every platform (x86/arm)
  2. Use cmake to control whether compile src/kern/optimized/${arch} or src/kern/gpu
  3. Remove platform macros like ifdef INFER_XXX in src/kern/optimized/${arch}

This allows more flexiblity in platform implementation. I mean that every arch can implement different kernels. You don't need to write every kernel in kernel_opt.h. You can also write some kernels if you like.

For example, for rvv arch, you don't need to write vectorized cpp code for compilers. That does not quite work on rvv.

@xhebox
feat: general backend
3bfeb79 
Signed-off-by: xhe <xw897002528@gmail.com>
@chenqy4933
Copy link
Collaborator

👍

@xhebox
Copy link
Collaborator Author
xhebox commented Jul 3, 2023

Thanks for the invitation, so, does this PR LGTM to you? @chenqy4933

@chenqy4933
Copy link
Collaborator

make local there are error:

/tmp/cclkOYZK.s:39948: Error: symbol `_ZZN8inferllm3opt24vec_vec_dot_q40_with_q80EiPKvS2_E19__PRETTY_FUNCTION__' is already defined
/tmp/cclkOYZK.s:39953: Error: symbol `_ZZN8inferllm3opt24vec_vec_dot_q40_with_q80EiPKvS2_E19__PRETTY_FUNCTION__' is already defined
/tmp/cclkOYZK.s:39983: Error: symbol `_ZZN8inferllm3opt19dequantize_row_q4_0EPKvPfiE19__PRETTY_FUNCTION__' is already defined

@xhebox
Copy link
Collaborator Author
xhebox commented Jul 3, 2023

make local there are error:

/tmp/cclkOYZK.s:39948: Error: symbol `_ZZN8inferllm3opt24vec_vec_dot_q40_with_q80EiPKvS2_E19__PRETTY_FUNCTION__' is already defined
/tmp/cclkOYZK.s:39953: Error: symbol `_ZZN8inferllm3opt24vec_vec_dot_q40_with_q80EiPKvS2_E19__PRETTY_FUNCTION__' is already defined
/tmp/cclkOYZK.s:39983: Error: symbol `_ZZN8inferllm3opt19dequantize_row_q4_0EPKvPfiE19__PRETTY_FUNCTION__' is already defined

I'll check later.

@xhebox
Copy link
Collaborator Author
xhebox commented Jul 3, 2023

make local there are error:

/tmp/cclkOYZK.s:39948: Error: symbol `_ZZN8inferllm3opt24vec_vec_dot_q40_with_q80EiPKvS2_E19__PRETTY_FUNCTION__' is already defined
/tmp/cclkOYZK.s:39953: Error: symbol `_ZZN8inferllm3opt24vec_vec_dot_q40_with_q80EiPKvS2_E19__PRETTY_FUNCTION__' is already defined
/tmp/cclkOYZK.s:39983: Error: symbol `_ZZN8inferllm3opt19dequantize_row_q4_0EPKvPfiE19__PRETTY_FUNCTION__' is already defined

@chenqy4933 Can you provide me some more info? For example, the output of cmake, with the following lines inserted at L61

message(STATUS "${INFER_ARCH}")
message(STATUS "${SRC}")
message(STATUS "${PLTSRC}")

@chenqy4933
Copy link
Collaborator

it is right，I test on other x86 machine。the log is:

-- INFER_ARCH is : x86
-- SRC list: /home/cqy/project/InferLLM/src/core/device.cpp;/home/cqy/project/InferLLM/src/core/device.h;/home/cqy/project/InferLLM/src/core/graph.cpp;/home/cqy/project/InferLLM/src/core/graph.h;/home/cqy/project/InferLLM/src/core/kvstorage.h;/home/cqy/project/InferLLM/src/core/kvstotage.cpp;/home/cqy/project/InferLLM/src/core/model.cpp;/home/cqy/project/InferLLM/src/core/model_imp.cpp;/home/cqy/project/InferLLM/src/core/model_imp.h;/home/cqy/project/InferLLM/src/core/op.cpp;/home/cqy/project/InferLLM/src/core/op.h;/home/cqy/project/InferLLM/src/core/tensor.cpp;/home/cqy/project/InferLLM/src/core/tensor.h;/home/cqy/project/InferLLM/src/core/thread_pool.cpp;/home/cqy/project/InferLLM/src/core/thread_pool.h;/home/cqy/project/InferLLM/src/file.cpp;/home/cqy/project/InferLLM/src/file.h;/home/cqy/project/InferLLM/src/graph/baichuan.cpp;/home/cqy/project/InferLLM/src/graph/baichuan.h;/home/cqy/project/InferLLM/src/graph/chatGLM.cpp;/home/cqy/project/InferLLM/src/graph/chatGLM.h;/home/cqy/project/InferLLM/src/graph/graph_imp.cpp;/home/cqy/project/InferLLM/src/graph/llama.cpp;/home/cqy/project/InferLLM/src/graph/llama.h;/home/cqy/project/InferLLM/src/kern/kernel.h;/home/cqy/project/InferLLM/src/kern/kernel_define.h;/home/cqy/project/InferLLM/src/kern/naive/naive.cpp;/home/cqy/project/InferLLM/src/kern/naive/naive.h;/home/cqy/project/InferLLM/src/kern/naive/quantize.h;/home/cqy/project/InferLLM/src/utils.cpp;/home/cqy/project/InferLLM/src/utils.h
-- PLTSRC list: /home/cqy/project/InferLLM/src/kern/optimized/x86/common.h;/home/cqy/project/InferLLM/src/kern/optimized/x86/kernel.cpp;/home/cqy/project/InferLLM/src/kern/optimized/x86/kernel.h;/home/cqy/project/InferLLM/src/kern/optimized/x86/optimized.h;/home/cqy/project/InferLLM/src/kern/optimized/x86/quantize.h

by the way, there are error when enable GPU， although the GPU is not complete

@xhebox
Copy link
Collaborator Author
xhebox commented Jul 3, 2023 •
edited
Loading

it is right，I test on other x86 machine。the log is:

Hmm, this is strange. Can you paste your full log with rm -rf build; cmake -Bbuild .; cmake --build build -v? Just paste it to https://pastebin.com/ or somewhere else. I can't reproduce.

@chenqy4933
Copy link
Collaborator

I will debug it in my machine， LGTM

@xhebox xhebox merged commit ce1f576 into MegEngine:main Jul 4, 2023
@xhebox xhebox deleted the dd branch July 4, 2023 03:05
@xhebox xhebox mentioned this pull request Jul 21, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@xhebox @chenqy4933