Packages

c

ch.ninecode.cim

CIMSubsetter

class CIMSubsetter[A <: Product] extends Serializable

Subclass extractor.

Extracts the given type of object from the full Element Resilient Distributes Dataset (RDD), to create another RDD of just those elements, and creates a DataFrame of that RDD, and registers it as a temporary table for access via SQL (e.g. JDBC and SparkR::sql()).

Note: This must be serializable and can't depend on the companion objects for the CIM case classes.

Linear Supertypes
Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CIMSubsetter
  2. Serializable
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CIMSubsetter()(implicit arg0: ClassTag[A], arg1: scala.reflect.api.JavaUniverse.TypeTag[A])

Type Members

  1. type basetype = A
  2. type rddtype = RDD[A]

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def asThisClass(element: Element): Option[A]

    Return the provided Element as this class if possible.

    Return the provided Element as this class if possible.

    element

    the element to convert

    returns

    Some(A) or None if the Element is not this Subsetter class type.

  6. val classname: String
  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  8. val cls: String
  9. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  11. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  12. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  13. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  14. def like(name: String): ((Int, RDD[_])) ⇒ Boolean

    Match names with pattern "name|xxx".

    Match names with pattern "name|xxx".

    name

    the name to match

    returns

    true if the RDD name matches the pattern

  15. def make(context: SQLContext, rdd: RDD[Element], storage: StorageLevel): Unit

    Create the Dataframe for Typeclass A.

    Create the Dataframe for Typeclass A.

    context

    The SQL context for creating the views.

    rdd

    The raw Element RDD to subset.

    storage

    The storage level to persist the subset RDD with.

  16. def modify_schema(rtc: Class[_], schema: StructType): StructType

    Alter the schema so sup has the correct superclass name.

    Alter the schema so sup has the correct superclass name.

    rtc

    The runtime class for Typeclass A.

    schema

    The SQL schema for Typeclass A, e.g. org.apache.spark.sql.types.StructType = StructType(StructField(sup,StructType(StructField(sup,StructType(StructField(sup,...

  17. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  18. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  19. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  20. val runtime_class: Class[_]
  21. def save(context: SQLContext, rdd: rddtype, storage: StorageLevel): Unit

    Create the Dataframe for Typeclass A.

    Create the Dataframe for Typeclass A.

    context

    The SQL context for creating the views.

    rdd

    The raw Element RDD to subset.

    storage

    The storage level to persist the subset RDD with.

  22. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  23. val tag: scala.reflect.api.JavaUniverse.TypeTag[A]
  24. def toString(): String
    Definition Classes
    AnyRef → Any
  25. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  27. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated @deprecated
    Deprecated

    (Since version ) see corresponding Javadoc for more information.

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped