java - 为 Spark Partitioner 将 Scala 代码翻译成 Java

Question

所以我尝试使用 Spark 和 Java 来实现自定义分区器，我找到了一个很好的在线示例，但它使用的是 Scala，我终究无法弄清楚如何将它正确地转换为 Java，因此我可以尝试实现它。谁能帮忙？这是我在 Scala 中找到的示例代码:

class DomainNamePartitioner(numParts: Int) extends Partitioner {
  override def numPartitions: Int = numParts
  override def getPartition(key: Any): Int = {
    val domain = new Java.net.URL(key.toString).getHost()
    val code = (domain.hashCode % numPartitions)
    if (code < 0) {
      code + numPartitions  // Make it non-negative
    } else {
      code
    }
  } 
  // Java equals method to let Spark compare our Partitioner objects
  override def equals(other: Any): Boolean = other match {
    case dnp: DomainNamePartitioner =>
      dnp.numPartitions == numPartitions
    case _ =>
      false
  }
}

Answer 1

首先，Scala是编写Spark的首选。

下面是对应的Java代码(不是唯一版本):

查看更多信息:https://spark.apache.org/docs/latest/api/java/index.html

class DomainNamePartitioner extends Partitioner{

private int numParts;

public Partitioner()
{

}
public Partitioner(int numParts)
{
  this.numParts = numParts;
}

@Override
public int numPartitions()
{
    return numParts;
}
@Override
public int getPartition(Object key){

   String domain = new Java.net.URL(key.toString).getHost();
   int code = domain.hashCode % numPartitions;
   if (code < 0){
    return code + this.numPartitions();
   }else{
    return code;
   }
}

@Override
public boolean equals(Object obj){

    if (obj instanceof DomainNamePartitioner){
        DomainNamePartitioner pObj = (DomainNamePartitioner)obj;
        return pObj.numPartitions() == this.numPartitions;
    }else{
        return false;
    }
}
}