Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Autodiff Upstreaming - enzyme frontend #129458
base: master
Are you sure you want to change the base?
Autodiff Upstreaming - enzyme frontend #129458
Changes from all commits
a6fec80
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question: what's the significance of offset being
-1
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's the bane of my existence. :D Enzyme uses a type layout representation which is suboptimal, it should move over to type trie's, but another student and I both gave up on our refactor PRs and the Enzyme core Lead dev also isn't actively working this. Luckily there are some AD users which seem to have the resources to eventually rewrite the whole typetree infrastructure, so I consider it just an implementation detail.
In less dramatic,
-1
in Enzyme speech means everywhere.that is
{0:-1: Float}
means at index 0 you have a ptr, if you dereference it it will be floats everywhere. Thus* f32
.If you have
{-1:int}
it means int's everywhere, e.g.[i32; N]
.{0:-1:-1 float}
then means one pointer at offset 0, if you dereference it there will be only pointers, if you dereference these new pointers they will point to array of floats.Generally, it allows byte-specific descriptions.
This design has no way of handling recursive datastructures, it skips things that have more than 5 indirections and there are some hacks to make it handle gaps in layouts, as well as other issues with it. If Enzyme is slow at compile time than this is usually the culprit. Also it should be extended at some point to make use of const/mut knowledge, right now that get's lost once we have more than one indirection. To be fair I find it pretty cool that Enzyme is already so extremely fast while leaving some information like here still unused.
The middle-end PR will include tests for typetrees, since that's where we construct them.
I'll add this also to the docs, so I can link to them next time.