Add ApplyBucketsWithInterpolation TFTransform #31291
Conversation
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
|
assign set of reviewers |
|
looks like the python 3.11 ML tests have some sort of setup issue (#31287) |
|
Assigning reviewers. If you would like to opt out of this review, comment R: @shunping for label python. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
for now you can consider this a new suite that never worked. |
|
our ml tests are in a bad shape, but they were not running extensively previously. |
|
Reminder, please take a look at this pr: @shunping |
| [0, 1]. | ||
|
|
||
| Input values are bucketized based on the provided boundaries such that the | ||
| input is mapped to a positive index i for which bucket_boundaries[i-1] <= |
There was a problem hiding this comment.
do we need some escaping for code-snippets with backticks or something similar to make it look nicer in pydoc?
There was a problem hiding this comment.
We can try that out, added some backticks
| input is mapped to a positive index i for which bucket_boundaries[i-1] <= | ||
| element < bucket_boundaries[i], if it exists. The values are then | ||
| normalized to the range [0,1] within the bucket, with NaN values being | ||
| mapped to 0.5. |
There was a problem hiding this comment.
should we link to TFT docs for more info as in some other ML Ops?
|
|
||
| Args: | ||
| columns: A list of column names to apply the transformation on. | ||
| bucket_boundaries: A rank 2 Tensor or list representing the bucket |
There was a problem hiding this comment.
Is the typehint set correctly for bucket_boundaries? Is Rank 2 Tensor a 2d matrix?
There was a problem hiding this comment.
it's consistent with what we're accepting as valid input in ApplyBuckets, both co-opt the language from TFT. Updating those to be more accurate to our function signature seems reasonable, changed for both functions
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @tvalentyn for label python. Available commands:
|
|
waiting on author |
Co-authored-by: tvalentyn <tvalentyn@users.noreply.github.com>
Co-authored-by: tvalentyn <tvalentyn@users.noreply.github.com>
Implements the
apply_buckets_with_interpolation()Tensorflow Transform into MLTransform.Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.