Packages

c

ch.ninecode.cim

CIMJoin

class CIMJoin extends CIMRDD with Serializable

Join CIM files from NIS Strom and SAP ISU.

Resolve ServiceLocation objects based on the Name user attribute.

Linear Supertypes
Serializable, Serializable, CIMRDD, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CIMJoin
  2. Serializable
  3. Serializable
  4. CIMRDD
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CIMJoin(spark: SparkSession, storage: StorageLevel)

    spark

    The session with CIM RDD defined, for which the topology should be calculated

    storage

    The storage level for new and replaced CIM RDD.

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def asDataSet[T <: Product](implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], spark: SparkSession): Dataset[T]

    Get a typed DataSet for the given class.

    Get a typed DataSet for the given class.

    T

    The type of RDD.

    spark

    The Spark session.

    returns

    A typed dataset of the RDD.

    Definition Classes
    CIMRDD
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def biggest(name: String): (Int, (Int, RDD[_])) ⇒ Int

    Find the largest integer after the name| pattern in a foldLeft operation.

    Find the largest integer after the name| pattern in a foldLeft operation.

    name

    the name to check

    Definition Classes
    CIMRDD
  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  8. def delete_name(a: (Name, Option[(ServiceLocation, ServiceLocation)])): Boolean
  9. def delete_service_location(a: (ServiceLocation, Option[(ServiceLocation, ServiceLocation)])): Boolean
  10. def do_join(): RDD[Element]

    Join NIS CIM file with ISU CIM file.

    Join NIS CIM file with ISU CIM file.

    The join uses the Name objects that contain both the SAP ISU id and the NIS number. The tasks are:

    1. Create new (or edit) the SAP ServiceLocation objects with:
      • mRID (rdf:ID) is the SAP ISU number (unchanged)
      • IdentifiedObject.name is the NIS number
      • IdentifiedObject.aliasName is the NIS internal id and class name
      • IdentifiedObject.description is the ISU description (aliasName)
      • Location.mainAddress is the ISU address data (unchanged)
    2. Change the location attribute of the PositionPoint object for the NIS ServiceLocation to point to the ISU ServiceLocation (i.e. replace MST# with ISU#)
    3. Change the UserAttribute objects that link the EnergyConsumer to ServiceLocation to point to the new (or edited) SAP ServiceLocation (i.e. replace MST# with ISU#)
    4. Optionally delete the NIS ServiceLocation
    5. Optionally delete the old Name object referencing the NIS ServiceLocation (this should clean out the Name RDD I think)
    6. Create a new Name object with the reverse orientation (Name.name = NIS MST# and Name.IdentifiedObject = SAP ISU#) [Not required if NIS ServiceLocation is deleted]

    So, in summary, edit these RDD:

    • ServiceLocation (merge & delete)
    • PositionPoint (edit)
    • UserAttribute (edit)
    • Name (delete)
    returns

    The updated Elements RDD.

  11. def edit_position_point(a: (PositionPoint, Option[(ServiceLocation, ServiceLocation)])): PositionPoint
  12. def edit_service_location(a: (ServiceLocation, Option[(String, (ServiceLocation, ServiceLocation))])): ServiceLocation
  13. def edit_user_attribute(a: (UserAttribute, Option[(ServiceLocation, ServiceLocation)])): UserAttribute
  14. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  16. def get[T](implicit arg0: ClassTag[T], spark: SparkSession, log: Logger): RDD[T]

    Get the typed RDD.

    Get the typed RDD.

    Convenience method where the name of the RDD is the same as the contained class type (the usual case).

    T

    The type of the RDD, e.g. RDD[T].

    spark

    The Spark session which persisted the typed RDD.

    log

    A logger for error messages.

    returns

    The RDD with the given type of objects, e.g. RDD[ACLineSegment].

    Definition Classes
    CIMRDD
  17. def get[T](name: String)(implicit arg0: ClassTag[T], spark: SparkSession, log: Logger): RDD[T]

    Get the named RDD.

    Get the named RDD.

    T

    The type of objects contained in the named RDD.

    name

    The name of the RDD, usually the same as the CIM class.

    spark

    The Spark session which persisted the named RDD.

    log

    A logger for error messages.

    returns

    The typed RDD, e.g. RDD[T].

    Definition Classes
    CIMRDD
    Example:

    1. The RDD of all elements is somewhat special, currently it is named Elements (plural), so this method must be used:

      val elements: RDD[Element] = get[Element]("Elements")
  18. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  19. def getOrElse[T](implicit arg0: ClassTag[T], spark: SparkSession): RDD[T]

    Get the typed RDD or an empty RDD if none was registered.

    Get the typed RDD or an empty RDD if none was registered.

    Convenience method where the name of the RDD is the same as the contained class type (the usual case).

    T

    The type of the RDD, e.g. RDD[T].

    spark

    The Spark session which persisted the typed RDD.

    returns

    The RDD with the given type of objects, e.g. RDD[ACLineSegment], or an empty RDD of the requested type.

    Definition Classes
    CIMRDD
  20. def getOrElse[T](name: String)(implicit arg0: ClassTag[T], spark: SparkSession): RDD[T]

    Get the named RDD or else an empty RDD of the requested type.

    Get the named RDD or else an empty RDD of the requested type.

    T

    The type of objects contained in the named RDD.

    name

    The name of the RDD, usually the same as the CIM class.

    spark

    The Spark session which persisted the named RDD.

    returns

    The typed RDD, e.g. RDD[T], as either the persisted RDD or an empty one if none was found.

    Definition Classes
    CIMRDD
  21. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  22. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  23. implicit val level: StorageLevel
  24. def like(name: String): ((Int, RDD[_])) ⇒ Boolean

    Match names with pattern "name|xxx".

    Match names with pattern "name|xxx".

    name

    the name to match

    returns

    true if the RDD name matches the pattern

    Definition Classes
    CIMRDD
  25. implicit val log: Logger
  26. def nameOf[T](implicit arg0: ClassTag[T]): String

    Get the base name of the CIM class.

    Get the base name of the CIM class.

    T

    The type of the class.

    returns

    The base name of the class.

    Definition Classes
    CIMRDD
  27. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  28. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  29. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  30. def put[T <: Product](rdd: RDD[T], keep: Boolean = false)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], spark: SparkSession, storage: StorageLevel): Unit

    Persist the typed RDD using the class name, checkpoint it if that is enabled, and create the SQL view for it.

    Persist the typed RDD using the class name, checkpoint it if that is enabled, and create the SQL view for it.

    T

    The type of RDD.

    rdd

    The RDD to persist

    keep

    If true then don't unpersist any existing RDD with the given name.

    spark

    The Spark session.

    storage

    The storage level for persistence.

    Definition Classes
    CIMRDD
  31. def put[T <: Product](rdd: RDD[T], name: String, keep: Boolean)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], spark: SparkSession, storage: StorageLevel): Unit

    Persist the typed RDD using the given name, checkpoint it if that is enabled, and create the SQL view for it.

    Persist the typed RDD using the given name, checkpoint it if that is enabled, and create the SQL view for it.

    Since the use of RDD persistence can be problematic for memory reuse, the keep parameter has special processing. When keep is true any existing RDD of the same name is renamed to name|n where n is sequentially increasing to the next available integer. When keep is false all existing RDD of the above form are unpersisted.

    This facilitates the use-case where the new RDD depends on the existing one. For example, the Elements RDD is rewritten by about, dedup, join, normalize and topological processing, where the new RDD is derived from the original. But when a completely new CIM RDF file is read in, keep=false will unpersist all the derived RDD.

    T

    The type of RDD.

    rdd

    The RDD to persist

    name

    The name under which to persist it.

    keep

    If true then don't unpersist the existing RDD with the given name.

    spark

    The Spark session.

    storage

    The storage level for persistence.

    returns

    The named, viewed and possibly checkpointed original RDD.

    Definition Classes
    CIMRDD
  32. implicit val session: SparkSession
  33. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  34. def test[T](implicit arg0: ClassTag[T], spark: SparkSession): Boolean

    Check for the typed RDD.

    Check for the typed RDD.

    Convenience method where the name of the RDD is the same as the contained class type (the usual case).

    T

    The type of the RDD, e.g. RDD[T].

    spark

    The Spark session which persisted the typed RDD.

    returns

    true if the named RDD exists, false otherwise.

    Definition Classes
    CIMRDD
  35. def test[T](name: String)(implicit arg0: ClassTag[T], spark: SparkSession): Boolean

    Check for the named RDD.

    Check for the named RDD.

    T

    The type of objects contained in the named RDD.

    name

    The name of the RDD, usually the same as the CIM class.

    spark

    The Spark session which persisted the named RDD.

    returns

    true if the named RDD exists, false otherwise.

    Definition Classes
    CIMRDD
  36. def toInt(s: String): Option[Int]
    Definition Classes
    CIMRDD
  37. def toString(): String
    Definition Classes
    AnyRef → Any
  38. def unbundle(a: ((Name, ServiceLocation), (Name, ServiceLocation))): (String, (ServiceLocation, ServiceLocation))
  39. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  40. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  41. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated @deprecated
    Deprecated

    (Since version ) see corresponding Javadoc for more information.

Inherited from Serializable

Inherited from Serializable

Inherited from CIMRDD

Inherited from AnyRef

Inherited from Any

Ungrouped