Draft: [onert] Support RoPE operation #14090
Draft
+607
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ys44kim
added
the
DRAFT
ys44kim
force-pushed
the
draft/onert/rope
branch
2 times, most recently
from
f3f19e3
to
c43dfa6
Compare
This commit supports RoPE for onerte ONE-DCO-1.0-Signed-off-by: youngsik kim <[email protected]>
- check data type of sin/cos input ONE-DCO-1.0-Signed-off-by: youngsik kim <[email protected]>
This commit adds cker test for RoPE ONE-DCO-1.0-Signed-off-by: youngsik kim <[email protected]>
ys44kim
force-pushed
the
draft/onert/rope
branch
from
ad223c2
to
12a419d
Compare
This commit change assert to throw in cker rope ONE-DCO-1.0-Signed-off-by: youngsik kim <[email protected]>
ys44kim
changed the title
ys44kim
commented
Oct 2, 2024
| kGptNeox = 0, | ||
| kGptJ = 1, | ||
| }; | ||
|
|
There was a problem hiding this comment.
It was added to support two modes (GPT-NEOX, GPT-J) of RoPE.
Currently, only GPT-NEOX is supported.
| throw std::runtime_error("the dimension(3) of input and sin_table do not match"); | ||
|
|
||
| if (input_shape.Dims(3) != cos_table_shape.Dims(3)) | ||
| throw std::runtime_error("the dimension(3) of input and cos_table do not match"); |
There was a problem hiding this comment.
if the sin/cos table and input dim(3) are different, an error occurs in the following operation
and it is processed as throw.
| auto input_tensor = _tensor_reg->getPortableTensor(input_index); | ||
| auto sin_tensor = _tensor_reg->getPortableTensor(sin_table); | ||
| auto cos_tensor = _tensor_reg->getPortableTensor(cos_table); | ||
| auto output_tensor = _tensor_reg->getPortableTensor(output_index); |
There was a problem hiding this comment.
The current implementation has three inputs(input, sin_table, and cos_table)
so, input the sin/cos table at the upper layer (ex.app).
This commit adds cker test for RoPE ONE-DCO-1.0-Signed-off-by: youngsik kim <[email protected]>
ys44kim
force-pushed
the
draft/onert/rope
branch
from
dda779e
to
f829437
Compare
ys44kim
commented
Oct 2, 2024
| } | ||
| } | ||
| } | ||
| else |
There was a problem hiding this comment.
BUILD_TYPE = Release
before fuse
$ ./Product/x86_64-linux.release/out/bin/onert_run ~/model/rope/Net_RoPE_000.circle
===================================
MODEL_LOAD takes 0.251 ms
PREPARE takes 3.557 ms
EXECUTE takes 0.036 ms
- MEAN : 0.036 ms
- MAX : 0.036 ms
- MIN : 0.036 ms
- GEOMEAN : 0.036 ms
===================================
after fuse
$ ./Product/x86_64-linux.release/out/bin/onert_run ~/model/rope/RoPE_000.circle
===================================
MODEL_LOAD takes 0.169 ms
PREPARE takes 2.384 ms
EXECUTE takes 0.020 ms
- MEAN : 0.020 ms
- MAX : 0.020 ms
- MIN : 0.020 ms
- GEOMEAN : 0.020 ms
===================================
|
@glistening @hseok-oh @ragmani @zetwhite |
This commit restore the wrong code applied ONE-DCO-1.0-Signed-off-by: youngsik kim <[email protected]>
This commit adds data type for RoPE ONE-DCO-1.0-Signed-off-by: youngsik kim <[email protected]>
|
@nnfw-bot test-all |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit supports RoPE Operation for onert
ONE-DCO-1.0-Signed-off-by: youngsik kim [email protected]