org.apache.parquet

Class CorruptStatistics

java.lang.Object

org.apache.parquet.CorruptStatistics

public class CorruptStatistics
extends Object

There was a bug (PARQUET-251) that caused the statistics metadata for binary columns to be corrupted in the write path.

This class is used to detect whether a file was written with this bug, and thus it's statistics should be ignored / not trusted.

Constructor Summary

Constructors

Constructor and Description
CorruptStatistics()

Method Summary

Modifier and Type	Method and Description
static boolean	shouldIgnoreStatistics(String createdBy, PrimitiveType.PrimitiveTypeName columnType) Decides if the statistics from a file created by createdBy (the created_by field from parquet format) should be ignored because they are potentially corrupt.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CorruptStatistics

public CorruptStatistics()

Method Detail

shouldIgnoreStatistics

public static boolean shouldIgnoreStatistics(String createdBy,
                                             PrimitiveType.PrimitiveTypeName columnType)

Decides if the statistics from a file created by createdBy (the created_by field from parquet format) should be ignored because they are potentially corrupt.

Parameters:

createdBy - the created-by string from a file footer

columnType - the type of the column that this is checking

Returns:

true if the statistics may be invalid and should be ignored, false otherwise