org.kitesdk.data.spi

Class SchemaUtil

java.lang.Object

org.kitesdk.data.spi.SchemaUtil

public class SchemaUtil
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SchemaUtil.SchemaVisitor<T>

Constructor Summary

Constructors

Constructor and Description
SchemaUtil()

Method Summary

Methods

Modifier and Type	Method and Description
static void	checkPartitionedBy(DatasetDescriptor descriptor, String fieldName)
static void	checkTypeConsistency(Schema schema, PartitionStrategy strategy, String fieldName, Object... values) Checks that the type of each of values is consistent with the type of field fieldName declared in the Avro schema (from descriptor).
static void	checkTypeConsistency(Schema schema, String fieldName, Object... values) Checks that the type of each of values is consistent with the type of field fieldName declared in the Avro schema (from descriptor).
static Schema.Field	copy(Schema.Field field) Creates a new field with the same name, schema, doc, and default value as the incoming schema.
static Schema	fieldSchema(Schema schema, PartitionStrategy strategy, String name) Returns a Schema for the given field name, which could be either a schema field or a partition field.
static Schema	fieldSchema(Schema schema, String name) Returns the nested Schema for the given field name.
static <T> T	fromString(String value, Schema schema)
static Class<?>	getClassForType(Schema.Type type)
static <S,T> Class<? extends T>	getPartitionType(FieldPartitioner<S,T> fp, Schema schema)
static <S> Class<? extends S>	getSourceType(FieldPartitioner<S,?> fp, Schema schema)
static boolean	isConsistentWithExpectedType(Schema.Type type, Class<?> expectedClass) Checks that a schema type should produce an object of the expected class.
static boolean	isConsistentWithMappingType(Schema.Type type, FieldMapping.MappingType mappingType)
static boolean	isField(Schema schema, PartitionStrategy strategy, String name) Return whether there is a field named name in the schema or partition strategy.
static Schema	keySchema(Schema schema, PartitionStrategy strategy) Creates a Schema for the keys of a PartitionStrategy based on an entity Schema.
static Schema	merge(Iterable<Schema> schemas) Merges Schema instances if they are compatible.
static Schema	merge(Schema left, Schema right) Merges two Schema instances if they are compatible.
static Schema	mergeOrUnion(Iterable<Schema> schemas) Merges Schema instances and creates a union of schemas if any are incompatible.
static boolean	nullOk(Schema schema) Returns whether null is allowed by the schema.
static Schema	partitionFieldSchema(FieldPartitioner<?,?> fp, Schema schema) Builds a Schema for the FieldPartitioner using the given Schema to determine types not fixed by the FieldPartitioner.
static String	toString(Object value, Schema schema)
static <T> T	visit(Schema schema, SchemaUtil.SchemaVisitor<T> visitor)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SchemaUtil

public SchemaUtil()

Method Detail

getClassForType

public static Class<?> getClassForType(Schema.Type type)

getSourceType

public static <S> Class<? extends S> getSourceType(FieldPartitioner<S,?> fp,
                                   Schema schema)

getPartitionType

public static <S,T> Class<? extends T> getPartitionType(FieldPartitioner<S,T> fp,
                                        Schema schema)

isConsistentWithExpectedType

public static boolean isConsistentWithExpectedType(Schema.Type type,
                                   Class<?> expectedClass)

Checks that a schema type should produce an object of the expected class. This check uses Class.isAssignableFrom(Class).

Parameters:

type - an avro Schema.Type

expectedClass - a java class

Returns:

true if the type produces objects assignable to the class, false if the class is not assignable from the avro type.

isConsistentWithMappingType

public static boolean isConsistentWithMappingType(Schema.Type type,
                                  FieldMapping.MappingType mappingType)

checkTypeConsistency

public static void checkTypeConsistency(Schema schema,
                        String fieldName,
                        Object... values)

Checks that the type of each of values is consistent with the type of field fieldName declared in the Avro schema (from descriptor).

checkTypeConsistency

public static void checkTypeConsistency(Schema schema,
                        PartitionStrategy strategy,
                        String fieldName,
                        Object... values)

Checks that the type of each of values is consistent with the type of field fieldName declared in the Avro schema (from descriptor).

partitionFieldSchema

public static Schema partitionFieldSchema(FieldPartitioner<?,?> fp,
                          Schema schema)

Builds a Schema for the FieldPartitioner using the given Schema to determine types not fixed by the FieldPartitioner.

Parameters:

fp - a FieldPartitioner

schema - an entity Schema that will be partitioned

Returns:

a Schema for the field partitioner

isField

public static boolean isField(Schema schema,
              PartitionStrategy strategy,
              String name)

Return whether there is a field named name in the schema or partition strategy.

Parameters:

schema - a Schema

strategy - a PartitionStrategy

name - a String field name

Returns:

true if name identifies a field

fieldSchema

public static Schema fieldSchema(Schema schema,
                 PartitionStrategy strategy,
                 String name)

Returns a Schema for the given field name, which could be either a schema field or a partition field.

Parameters:

schema - an entity Schema that will be partitioned

strategy - a PartitionStrategy used to partition entities

name - a schema or partition field name

Returns:

a Schema for the partition or schema field

fieldSchema

public static Schema fieldSchema(Schema schema,
                 String name)

Returns the nested Schema for the given field name.

Parameters:

schema - a record Schema

name - a String field name

Returns:

the nested Schema for the field

keySchema

public static Schema keySchema(Schema schema,
               PartitionStrategy strategy)

Creates a Schema for the keys of a PartitionStrategy based on an entity Schema.

The partition strategy and schema are assumed to be compatible and this will result in NullPointerExceptions if they are not.

Parameters:

schema - an entity schema Schema

strategy - a PartitionStrategy

Returns:

a Schema for the storage keys of the partition strategy

checkPartitionedBy

public static void checkPartitionedBy(DatasetDescriptor descriptor,
                      String fieldName)

visit

public static <T> T visit(Schema schema,
          SchemaUtil.SchemaVisitor<T> visitor)

toString

public static String toString(Object value,
              Schema schema)

fromString

public static <T> T fromString(String value,
               Schema schema)

merge

public static Schema merge(Iterable<Schema> schemas)

Merges Schema instances if they are compatible.

Schemas are incompatible if:

• The Schema.Type does not match.
• For record schemas, the record name does not match
• For enum schemas, the enum name does not match

Map value, array element, and record field types types will use unions if necessary, and union schemas are merged recursively.

Parameters:

schemas - a set of Schema instances to merge

Returns:

a merged Schema

Throws:

IncompatibleSchemaException - if the schemas are not compatible

mergeOrUnion

public static Schema mergeOrUnion(Iterable<Schema> schemas)

Merges Schema instances and creates a union of schemas if any are incompatible.

Schemas are incompatible if:

• The Schema.Type does not match.
• For record schemas, the record name does not match
• For enum schemas, the enum name does not match

Map value, array element, and record field types types will use unions if necessary, and union schemas are merged recursively.

Parameters:

schemas - a set of Schema instances to merge

Returns:

a combined Schema

merge

public static Schema merge(Schema left,
           Schema right)

Merges two Schema instances if they are compatible.

Two schemas are incompatible if:

• The Schema.Type does not match.
• For record schemas, the record name does not match
• For enum schemas, the enum name does not match

Map value and array element types will use unions if necessary, and union schemas are merged recursively.

Parameters:

left - a Schema

right - a Schema

Returns:

a merged Schema

Throws:

IncompatibleSchemaException - if the schemas are not compatible

copy

public static Schema.Field copy(Schema.Field field)

Creates a new field with the same name, schema, doc, and default value as the incoming schema.

Fields cannot be used in more than one record (not Immutable?).

nullOk

public static boolean nullOk(Schema schema)

Returns whether null is allowed by the schema.

Parameters:

schema - a Schema

Returns:

true if schema allows the value to be null