java - 尼菲 :can't import getFileSystem in customProcessor?

Question

我想使用 getFileSytem() 来提取任何比我们拥有的时间戳更新但我无法导入它的文件(我的意思是 getFileSytem()) 我有这段代码:

final FileSystem hdfs = getFileSystem();
final boolean recursive = context.getProperty(RECURSE_SUBDIRS).asBoolean();

final Set<FileStatus> statuses;
try {
    final Path rootPath = new Path(directory);
    statuses = getStatuses(rootPath, recursive, hdfs,
            createPathFilter(context));
    getLogger().debug("Found a total of {} files in HDFS", new Object[]
            {statuses.size()});
} catch (final IOException | IllegalArgumentException e) {
    getLogger().error("Failed to perform listing of HDFS due to {}", new
            Object[]{e});
    return;
}

这是我的 pom 文件依赖项:

 <dependencies>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-api</artifactId>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-utils</artifactId>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-mock</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-simple</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
        <version>3.5</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/log4j/log4j -->
    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>1.2.17</version>
    </dependency>

    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-hdfs-processors</artifactId>
        <version>0.0.2-incubating</version>
    </dependency>

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-common</artifactId>
        <version>3.0.0-beta1</version>
    </dependency>

    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-properties</artifactId>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-distributed-cache-client-service-api</artifactId>
    </dependency>

      <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-standard-nar</artifactId>
            <type>nar</type>
        </dependency>

</dependencies>

另外我想知道是否可以制作正则表达式来帮助我计算以相同字符串开头的文件数量？

Answer 1

您不会希望将 nifi-hdfs-processors 作为依赖项包含在您的自定义处理器中。相反，您可能希望将 nifi-hadoop-libraries-nar 设置为自定义处理器 NAR 的父 NAR。检查POM for the nifi-hadoop-nar举个例子。