hadoop - 如何让HDFS的replica block的位置自己设置？

Question

我的目标是让HDFS的replica block的位置节点自己定。查看HDFS的代码后，我在这里找到了调用轨迹(hadoop 2.2版本)

BlockManager.getDatanodeDescriptors(List<String>) line: 1400    
BlockManager.chooseTarget(String, int, DatanodeDescriptor, HashMap<Node,Node>, long, List<String>) line: 1378   
FSNamesystem.getAdditionalBlock(String, long, String, ExtendedBlock, HashMap<Node,Node>, List<String>) line: 2477   
NameNodeRpcServer.addBlock(String, String, ExtendedBlock, DatanodeInfo[], long, String[]) line: 555 
ClientNamenodeProtocolServerSideTranslatorPB.addBlock(RpcController, ClientNamenodeProtocolProtos$AddBlockRequestProto) line: 387   
ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(Descriptors$MethodDescriptor, RpcController, Message) line: 59582  
ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(RPC$Server, String, Writable, long) line: 585  
ProtobufRpcEngine$Server(RPC$Server).call(RPC$RpcKind, String, Writable, long) line: 928    

我发现 BlockManager.java 有一个 chooseTarget方法:

 public DatanodeDescriptor[] chooseTarget(final String src,
  final int numOfReplicas, final DatanodeDescriptor client,
  final HashMap<Node, Node> excludedNodes,
  final long blocksize, List<String> favoredNodes) throws IOException {

我想如果我可以设置 List<String> favoredNodes的值(value)，那么我的目标就可以实现。

Answer 1

我从 DistributedFileSystem.java(hadoop 2.2 版本)找到 API

  /**
   * Same as  
   * {@link #create(Path, FsPermission, boolean, int, short, long, 
   * Progressable)} with the addition of favoredNodes that is a hint to 
   * where the namenode should place the file blocks.
   * The favored nodes hint is not persisted in HDFS. Hence it may be honored
   * at the creation time only. HDFS could move the blocks during balancing or
   * replication, to move the blocks from favored nodes. A value of null means
   * no favored nodes for this create
   */
  public HdfsDataOutputStream create(final Path f,
      final FsPermission permission, final boolean overwrite,
      final int bufferSize, final short replication, final long blockSize,
      final Progressable progress, final InetSocketAddress[] favoredNodes)
          throws IOException {