Skip to content

Commit

Permalink
Fix childName to consider ReaderOption fileColumnNamesReadAsLowerCase (
Browse files Browse the repository at this point in the history
…#10687)

Summary:
Converts `childName` to lowercase if hive config `isFileColumnNamesReadAsLowerCase`
is enabled.

cc: yingsu00 aditi-pandit

Pull Request resolved: #10687

Reviewed By: Yuhta

Differential Revision: D61032300

Pulled By: xiaoxmeng

fbshipit-source-id: 38520b69cf575f0a7acc2184963573b432b76a85
  • Loading branch information
pramodsatya authored and facebook-github-bot committed Aug 9, 2024
1 parent c0f0a31 commit 6d156e4
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 1 deletion.
5 changes: 4 additions & 1 deletion velox/dwio/parquet/reader/ParquetReader.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,10 @@ std::unique_ptr<ParquetTypeWithId> ReaderBase::getParquetColumnInfo(
auto curSchemaIdx = schemaIdx;
for (int32_t i = 0; i < schemaElement.num_children; i++) {
++schemaIdx;
auto& childName = schema[schemaIdx].name;
auto childName = schema[schemaIdx].name;
if (isFileColumnNamesReadAsLowerCase()) {
folly::toLowerAscii(childName);
}
auto childRequestedType =
requestedType ? requestedType->asRow().findChild(childName) : nullptr;
auto child = getParquetColumnInfo(
Expand Down
2 changes: 2 additions & 0 deletions velox/dwio/parquet/tests/reader/ParquetReaderTest.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,8 @@ TEST_F(ParquetReaderTest, parseReadAsLowerCase) {
const std::string upper(getExampleFilePath("upper.parquet"));

dwio::common::ReaderOptions readerOptions{leafPool_.get()};
auto outputRowType = ROW({"a", "b"}, {BIGINT(), BIGINT()});
readerOptions.setFileSchema(outputRowType);
readerOptions.setFileColumnNamesReadAsLowerCase(true);
auto reader = createReader(upper, readerOptions);
EXPECT_EQ(reader->numberOfRows(), 2ULL);
Expand Down

0 comments on commit 6d156e4

Please sign in to comment.