com.datastax.spark.connector.rdd.partitioner

TokenRangeClusterer

class TokenRangeClusterer[V, T <: Token[V]] extends AnyRef

Divides a set of token ranges into groups containing not more than maxRowCountPerGroup rows and not more than maxGroupSize token ranges. Each group will form a single CassandraRDDPartition.

The algorithm is as follows: 1. Sort token ranges by endpoints lexicographically. 2. Take the highest possible number of token ranges from the beginning of the list, such that their sum of rowCounts does not exceed maxRowCountPerGroup and they all contain at least one common endpoint. If it is not possible, take at least one item. Those token ranges will make a group. 3. Repeat the previous step until no more token ranges left.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. TokenRangeClusterer
  2. AnyRef
  3. Any
Implicitly
  1. by any2stringadd
  2. by any2stringfmt
  3. by any2ArrowAssoc
  4. by any2Ensuring
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TokenRangeClusterer(maxRowCountPerGroup: Long, maxGroupSize: Int = scala.Int.MaxValue)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. def +(other: String): String

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to StringAdd performed by method any2stringadd in scala.Predef.
    Definition Classes
    StringAdd

  5. def ->[B](y: B): (TokenRangeClusterer[V, T], B)

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to ArrowAssoc[TokenRangeClusterer[V, T]] performed by method any2ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()

  6. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  7. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  8. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  9. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  10. def ensuring(cond: (TokenRangeClusterer[V, T]) ⇒ Boolean, msg: ⇒ Any): TokenRangeClusterer[V, T]

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to Ensuring[TokenRangeClusterer[V, T]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring

  11. def ensuring(cond: (TokenRangeClusterer[V, T]) ⇒ Boolean): TokenRangeClusterer[V, T]

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to Ensuring[TokenRangeClusterer[V, T]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring

  12. def ensuring(cond: Boolean, msg: ⇒ Any): TokenRangeClusterer[V, T]

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to Ensuring[TokenRangeClusterer[V, T]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring

  13. def ensuring(cond: Boolean): TokenRangeClusterer[V, T]

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to Ensuring[TokenRangeClusterer[V, T]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring

  14. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  15. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  16. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  17. def formatted(fmtstr: String): String

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to StringFormat performed by method any2stringfmt in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @inline()

  18. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  19. def group(tokenRanges: Seq[TokenRange[V, T]]): Iterable[Seq[TokenRange[V, T]]]

    Groups small token ranges on the same server(s) in order to reduce task scheduling overhead.

    Groups small token ranges on the same server(s) in order to reduce task scheduling overhead. Useful mostly with virtual nodes, which may create lots of small token range splits. Each group will make a single Spark task.

  20. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  21. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  22. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  23. final def notify(): Unit

    Definition Classes
    AnyRef

  24. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  25. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  26. def toString(): String

    Definition Classes
    AnyRef → Any

  27. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  28. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  29. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  30. def →[B](y: B): (TokenRangeClusterer[V, T], B)

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to ArrowAssoc[TokenRangeClusterer[V, T]] performed by method any2ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc

Shadowed Implicit Value Members

  1. val self: Any

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to StringAdd performed by method any2stringadd in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (tokenRangeClusterer: StringAdd).self
    Definition Classes
    StringAdd

  2. val self: Any

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to StringFormat performed by method any2stringfmt in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (tokenRangeClusterer: StringFormat).self
    Definition Classes
    StringFormat

Deprecated Value Members

  1. def x: TokenRangeClusterer[V, T]

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to ArrowAssoc[TokenRangeClusterer[V, T]] performed by method any2ArrowAssoc in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (tokenRangeClusterer: ArrowAssoc[TokenRangeClusterer[V, T]]).x
    Definition Classes
    ArrowAssoc
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) Use leftOfArrow instead

  2. def x: TokenRangeClusterer[V, T]

    Implicit information
    This member is added by an implicit conversion from TokenRangeClusterer[V, T] to Ensuring[TokenRangeClusterer[V, T]] performed by method any2Ensuring in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (tokenRangeClusterer: Ensuring[TokenRangeClusterer[V, T]]).x
    Definition Classes
    Ensuring
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) Use resultOfEnsuring instead

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion any2stringadd from TokenRangeClusterer[V, T] to StringAdd

Inherited by implicit conversion any2stringfmt from TokenRangeClusterer[V, T] to StringFormat

Inherited by implicit conversion any2ArrowAssoc from TokenRangeClusterer[V, T] to ArrowAssoc[TokenRangeClusterer[V, T]]

Inherited by implicit conversion any2Ensuring from TokenRangeClusterer[V, T] to Ensuring[TokenRangeClusterer[V, T]]

Ungrouped