java - 使用 Java 从大文件中删除注释

Question

我有 .sh、.txt、.sql、.pkb 等文件，文件大小大于 10 MB，这意味着超过 100k 行。

我想从这些文件中删除注释，然后进一步使用未注释的内容。我为其编写了以下代码。

/**
 * Removes all the commented part from the file content as well as returns a
 * file structure which have just lines with declaration syntax for eg.
 * Create Package packageName <- Stores all decalartion lines as separate
 * string in an array
 * 
 * @param file
 * @return file content
 * @throws IOException
 */
private static String[] filterContent(File file) throws IOException {

    String withoutComment = "";
    String declare = "";
    String[] content;
    List<String> readLines = FileUtils.readLines(file);

    int size = readLines.size();
    System.out.println(file.getName() + " Files number of lines "+ size + " at "+new Date());
    String[] declareLines = new String[size];
    int startComment = 0;
    int endComment = 0;
    Boolean check = false;
    int j = 0;
    int i=0;
    // Reading content line by line
    for (String line:readLines) {
        // If line contains */ that means comment is ending in this line,
        // making a note of the line number
        if (line.toString().contains("*/")) {
            endComment = i;
            // Removing the content before */ from the line
            int indexOf = line.indexOf("*/");
            line = line.replace(line.substring(0, indexOf + 2), "");
        }

        // If startComment is assigned fresh value and end comment hasn't,
        // that means the current line is part of the comment
        // Ignoring the line in this case and moving on to the next one
        if ((startComment > 0 && endComment == 0) || (endComment < startComment) || check)
            continue;

        // If line contains /* that means comment is starting in this line,
        // making a note of the line number
        if (line.contains("/*")) {
            startComment = i;
            // Removing the content after /* from the line
            int indexOf = line.indexOf("/*");
            line = line.replace(line.substring(indexOf), "");
            if (i == 0)
                check = true; // means comment in the very first line
        }

        // If line contains -- that means single line comment is present in
        // this line,
        // removing the content after --
        if (line.contains("--")) {
            int indexOf = line.indexOf("--");
            line = line.replace(line.substring(indexOf), "");
        }
        // If line contains -- that means single line comment is present in
        // this line,
        // removing the content after --
        if (line.contains("#")) {
            int indexOf = line.indexOf("#");
            line = line.replace(line.substring(indexOf), "");
        }

        // At this point, all commented part is removed from the line, hence
        // appending it to the final content
        if (!line.isEmpty())
            withoutComment = withoutComment + line + " \n";
        // If line contains CREATE its a declaration line, holding it
        // separately in the array
        if (line.toUpperCase().contains(("CREATE"))) {
            // If next line does not contains Create and the current line is
            // the not the last line,
            // then considering two consecutive lines as declaration line,
            if (i < size - 1 && !readLines.get(i + 1).toString().toUpperCase().contains(("CREATE"))) {
                declare = line + " " + readLines.get(i + 1).toString() + "\n";
            } else if (i < size) {// If the line is last line, including
                                    // that line alone.
                declare = line + "\n";
            }

            declareLines[j] = declare.toUpperCase();
            j++;
        }
        i++;
    }
    System.out.println("Read lines "+ new Date());
    List<String> list = new ArrayList<String>(Arrays.asList(declareLines));
    list.removeAll(Collections.singleton(null));

    content = list.toArray(new String[list.size() + 1]);

    withoutComment = withoutComment.toUpperCase();
    content[j] = withoutComment;
    System.out.println("Retruning uncommented content "+ new Date());
    return content;
}


 public static void main(String[] args) {
        String[] content = filterContent(new File("abc.txt"));
}

此代码的问题是，如果文件很大，则速度太慢。对于 10 MB 的文件，删除评论需要 6 个多小时。 (代码在 SSH 服务器上运行)。

我还可以拥有最大 100 MB 的文件，其中删除评论需要几天时间。如何更快地删除评论？

更新:这个问题不是重复的，因为我的问题不仅仅是通过改变读取行的方式来解决。它的字符串 Activity 使进程变慢，我需要一种方法来使评论删除 Activity 更快。

Answer 1

您可以创建多个线程来完成工作(需要正确分割线路)