java - 在大型迭代中划分循环

Question

我的问题的目标是通过将我的循环迭代范围拆分到一个大型数组列表来提高我的算法的性能。
例如:我有一个数组列表，其大小约为 100 亿个长值条目，我要实现的目标是从 0 到 1 亿个条目开始循环，输出任何计算的 1 亿个条目的结果在循环内；然后开始，1 亿到 2 亿做前面的事情并输出结果，然后是 300-4 亿，400-5 亿等等。在我得到所有 1000 亿/1 亿个结果之后，我可以在循环外对它们进行求和，从并行的循环输出中收集结果。

我曾尝试使用一个范围，该范围可能能够通过尝试使用动态范围偏移方法实现类似的效果，但我似乎无法像我希望的那样完全实现逻辑。

public static void tt4() {
    long essir2 = 0;
    long essir3 = 0;

    List cc = new ArrayList<>();  
    List<Long> range = new ArrayList<>();  

    // break point is a method that returns list values, it was converted to 
    // string because of some concatenations and would be converted back to long here
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }  

    // the size of the List is huge about 1 trillion entries at the minimum
    long hy = cc.size() - 1;

    for (long k = 0; k < hy; k++) { 
        long t1 = (long) cc.get((int) k);
        long t2 = (long) cc.get((int) (k + 1)); 

        // My main question: I am trying to iterate the entire list in a dynamic way 
        // which would exclude repeated endpoints on each iteration. 

        range = LongStream.rangeClosed(t1 + 1, t2)
                          .boxed()
                          .collect(Collectors.toList());  

        for (long i : range) {
            // Hard is another method call on the iteration
            // complexcalc is a method as well

            essir2 = complexcalc((int) i, (int) Hard(i)); 
            essir3 += essir2;
        } 
    }

    System.out.println("\n" + essir3);  
}

我没有任何错误，我只是在寻找一种提高性能和时间的方法。我可以直接在一秒钟内完成一百万个条目，但是当我输入我需要的大小时，它会永远运行。我给出的大小是用于说明大小大小的摘要，我不希望像 1000 亿这样的意见并不多，如果我能在一秒钟内完成一百万，我说的是巨大的数字，我需要反复做复杂的任务和调用，如果可以的话，我只需要在我试图实现的逻辑方面得到帮助。

Answer 1

我会立即建议的一件事是将您的 Breakpoint 返回值存储在一个简单的数组中，而不是使用 List。这应该会显着缩短您的执行时间:

    List<Long> cc = new ArrayList<>();
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }
    Long[] ccArray = cc.toArray(new Long[0]);

我相信您正在寻找的是跨多个线程拆分您的任务。你可以用 ExecutorService 来做到这一点“这简化了异步模式下任务的执行”。

请注意，我对整个概念并不过分熟悉，但最近对它进行了一些试验，并为您提供了一个关于如何实现它的快速草稿。

我欢迎那些在多线程方面更有经验的人更正这篇文章或在评论中提供更多信息以帮助改进这个答案。

可运行任务类

public class CompartmentalizationTask implements Runnable {

    private final ArrayList<Long> cc;
    private final long index;

    public CompartmentalizationTask(ArrayList<Long> list, long index) {

        this.cc = list;
        this.index = index;
    }

    @Override
    public void run() {
        Main.compartmentalize(cc, index);
    }
}

主类

private static ExecutorService exeService = Executors.newCachedThreadPool();
private static List<Future> futureTasks = new ArrayList<>();

public static void tt4() throws ExecutionException, InterruptedException 
{
    long essir2 = 0;
    long essir3 = 0;

    ArrayList<Long> cc = new ArrayList<>();
    List<Long> range = new ArrayList<>();

    // break point is a method that returns list values, it was converted to
    // string because of some concatenations and would be converted back to long here
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }

    // the size of the List is huge about 1 trillion entries at the minimum
    long hy = cc.size() - 1;

    for (long k = 0; k < hy; k++) {
        futureTasks.add(Main.exeService.submit(new CompartmentalizationTask(cc, k)));
    }
    for (int i = 0; i < futureTasks.size(); i++) {
        futureTasks.get(i).get();
    }
    exeService.shutdown();
}

public static void compartmentalize(ArrayList<Long> cc, long index)
{
    long t1 = (long) cc.get((int) index);
    long t2 = (long) cc.get((int) (index + 1));

    // My main question: I am trying to iterate the entire list in a dynamic way
    // which would exclude repeated endpoints on each iteration.

    range = LongStream.rangeClosed(t1 + 1, t2)
            .boxed()
            .collect(Collectors.toList());

    for (long i : range) {
        // Hard is another method call on the iteration
        // complexcalc is a method as well

        essir2 = complexcalc((int) i, (int) Hard(i));
        essir3 += essir2;
    }
}