java - Java 中的目录扫描器

Question

为一组文件名过滤器连续扫描一组目录。对于到达的每个文件名过滤器，处理文件并重复所有步骤

在 jdk 1.5 中推荐的设计是什么，可能使用 java.concurrent.Executor 和 Future

Answer 1

我用网络爬虫完成了类似的任务。只需要做一些改变......它是一个并发实现，新发现的目录被执行器框架中的线程池扫描。它使用并发集合队列和列表来保存索引文件。索引器从队列中取出文件并对它们进行处理。这是 FileFilter 实现

    public class ImageFileFilter implements FileFilter
    {
      private final String[] okFileExtensions = 
        new String[] {"jpg", "png", "gif"};

      public boolean accept(File file)
      {
        for (String extension : okFileExtensions)
        {
          if (file.getName().toLowerCase().endsWith(extension))
          {
            return true;
          }
        }
        return false;
      }
    }

这是带有主要方法的类...

     public class FileFilterTest {
        public static void main(String[] args) {
            File dir = new File("D:\\dev\\css-templates\\cms-admin");
            BlockingQueue blockingQueue = new ArrayBlockingQueue(5);
            FileCrawler fileCrawler = new FileCrawler(blockingQueue,
                    new ImageFileFilter(), dir);
            new Thread(fileCrawler).start();

            FileIndexer indexer = new FileIndexer(blockingQueue);
            new Thread(indexer).start();
        }
    }

这里是文件爬虫线程

     public class FileCrawler implements Runnable {
            private final BlockingQueue fileQueue;
            private ConcurrentSkipListSet indexedFiles = new ConcurrentSkipListSet();
            private final FileFilter fileFilter;
            private final File root;
            private final ExecutorService exec = Executors.newCachedThreadPool();

            public FileCrawler(BlockingQueue fileQueue,
                               final FileFilter fileFilter,
                               File root) {
                this.fileQueue = fileQueue;
                this.root = root;
                this.fileFilter = new FileFilter() {
                    public boolean accept(File f) {
                        return f.isDirectory() || fileFilter.accept(f);
                    }
                };
            }

            public void run() {

                    submitCrawlTask(root);

            }

            private void submitCrawlTask(File f) {
                CrawlTask crawlTask = new CrawlTask(f);
                exec.execute(crawlTask);
            }

            private class CrawlTask implements Runnable {
                private final File file;

                CrawlTask(File file ) {

                    this.file= file;
                }

             public void run() {        
                 if(Thread.currentThread().isInterrupted())
                return;

                    File[] entries = file.listFiles(fileFilter);

                    if (entries != null) {
                        for (File entry : entries)
                            if (entry.isDirectory())
                                submitCrawlTask(entry);
                            else if (entry !=null && !indexedFiles.contains(entry)){
                                indexedFiles.add(entry);
                                try {
                                    fileQueue.put(entry);
                                } catch (InterruptedException e) {
                                        Thread.currentThread().interrupt();
                                }
                            }
                    }
                }
        }
       }

这是文件索引线程

        public class FileIndexer implements Runnable {
        private final BlockingQueue queue;

        public FileIndexer(BlockingQueue queue) {
            this.queue = queue;
        }

        public void run() { 
            try {
                while (true) {
                    indexFile(queue.take());
                }
            } catch (InterruptedException e) {
                System.out.println("Indexer Interrupted");
                Thread.currentThread().interrupt();
            }
        }

        public void indexFile(File file) {
            // do something with the file...
            System.out.println("Indexing File : " + file.getAbsolutePath() + " " + file.getName());
        };
    }