Bug fix for case where aspect ratio is a float

Summary:
- Fix the calculation of the non square NDC range when the H and W are not integer multiples.
- Add test for this case

Reviewed By: gkioxari

Differential Revision: D26613213

fbshipit-source-id: df6763cac602e9f1d516b41b432c4d2cfbaa356d
This commit is contained in:
Nikhila Ravi
2021-02-24 10:05:07 -08:00
committed by Facebook GitHub Bot
parent 0345f860d4
commit 13429640d3
4 changed files with 15 additions and 18 deletions

View File

@@ -10,7 +10,9 @@
__device__ inline float NonSquareNdcRange(int S1, int S2) {
float range = 2.0f;
if (S1 > S2) {
range = ((S1 / S2) * range);
// First multiply S1 by float range so that division results
// in a float value.
range = (S1 * range) / S2;
}
return range;
}

View File

@@ -10,7 +10,7 @@
inline float NonSquareNdcRange(int S1, int S2) {
float range = 2.0f;
if (S1 > S2) {
range = ((S1 / S2) * range);
range = (S1 * range) / S2;
}
return range;
}