java - 无法在Grails 2.4.4项目中实例化OpenCV-4.0.1 CascadeClassifier

Question

我想从其上传文件中裁剪1x1员工图片，以将其设置为我的Grails应用程序上的头像。我听说OpenCV做得很好，所以我在ImageService中使用了它。问题在于，它似乎无法找到(或读取)所需的CascadeClassifier XML文件:

class ImageService {
    final String FRONTAL_FACE_XML = "D:\\Devtools\\opencv\\build\\etc\\lbpcascades\\lbpcascade_frontalface_improved.xml"
    final String ORIGINAL_PICTURE = "D:\\Projects\\opencv\\grails-app\\assets\\4fc30smaegvq0z3mvgm9yhf6vtv9kv8bgryi9x08wuada8jxu3.jpg"
    final String CROPPED_PICTURE = "D:\\Projects\\opencv\\grails-app\\assets\\4fc30smaegvq0z3mvgm9yhf6vtv9kv8bgryi9x08wuada8jxu3_100.jpg"

    void opencvtest() {
        // Before I placed the OpenCV dll in the environment path, this line causes an error.
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME)

        // UnsatisfiedLinkError here
        CascadeClassifier faceDetector = new CascadeClassifier(this.getClass().getResource("lbpcascade_frontalface_improved.xml").getPath());

        // Same error as well.
        // File cascadeFile = new File(FRONTAL_FACE_XML);
        // CascadeClassifier faceDetector = new CascadeClassifier(cascadeFile.getAbsolutePath());

        // And also here.
        // CascadeClassifier faceDetector = new CascadeClassifier(FRONTAL_FACE_XML);

        Mat image = HighGui.imread(ORIGINAL_PICTURE)
        faceDetector.detectMultiScale(image, face_Detections)

        Rect rect_Crop = null

        for (Rect rect : face_Detections.toArray()) {
            Core.rectangle(image, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(0, 255, 0))
            rectCrop = new Rect(rect.x, rect.y, rect.width, rect.height)
        }

        Mat image_roi = new Mat(image, rectCrop)
        HighGui.imwrite(CROPPED_PICTURE, image_roi)

        return
    }
}

并导致以下错误:

java.lang.UnsatisfiedLinkError org.opencv.objdetect.CascadeClassifier.CascadeClassifier_0(Ljava/lang/String;)J

环境: Windows 7，Java 1.8，Grails 2.4.4

我所做的事情:

我已经从他们的网站https://opencv.org/releases.html安装了OpenCV 4.0.1，并在D:\opencv\4.0.1上解压缩了它们的发行版。

我已将此包含在我的环境路径中:D:\opencv\4.0.1\build\java\x64

由于找不到正式的Maven依赖关系链接，我将在jar上找到的D:\opencv\4.0.1\build\java复制到了grails lib目录中。

我已经尝试将所有dll和xml复制到C:\Windows\system32。

考虑到发行版中的jar可能是“错误的”，我通过compile "org.bytedeco.javacpp-presets:opencv:4.0.1-1.4.4"将此包替换为BuildConfig.groovy上的包，但仍然会在同一行上引起错误。

我验证了FRONTAL_FACE_XML和ORIGINAL_PICTURE是否都是正确的文件路径，并且它们都是正确的。

Answer 1

在此示例中，我们将在Windows上使用在VC15上使用OpenCV。我还没有学习如何在Linux上移植它。

传说:

APPLICATION_PATH = D:\application
JAVA_DLL_PATH = D:\opencv\4.0.1\build\java\x64
VC_DLL_PATH = D:\opencv\4.0.1\build\x64\vc15\bin

可选的:

进入%VC_DLL_PATH%目录。

创建一个名为debug(或您喜欢的名称)的目录，然后将所有*d.dll文件移到该目录中。这些以*d.dll结尾的文件在运行它们时可能会导致错误，因为它正在查找不属于基本Visual C++ 2015的vc15调试dll。

组态:

在%APPLICATION_PATH%\src目录中创建一个名为files的目录(或任何您喜欢的目录)。它应该与此类似。

%APPLICATION_PATH%
├── bin\
├── grails-app\
├── lib\
├── ...
├── src\
│ ├── groovy\
│ ├── java\
│ └── files\ --new directory
└── ...

将*.dll和%JAVA_DLL_PATH%目录中的所有%VC_DLL_PATH%复制到新创建的目录%APPLICATION_PATH%\src\files中。

将新创建的目录包括到.classpath文件中。

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
<classpathentry excluding="spring/" kind="src" path="grails-app/conf"/>
...
<classpathentry kind="src" path="src/files"/>
...
</classpath>

在您的%APPLICATION_PATH%中创建另一个名为files的目录(或您喜欢的任何一个)。

%APPLICATION_PATH%
├── bin\
├── grails-app\
├── files\ --new directory
└── ...

在此处复制您将使用的CascadeClassifiers xml文件。它们位于\opencv\4.0.1\build\etc目录中。

即使将dll复制到%APPLICATION_PATH%中，我们仍然需要将它们包括在系统路径中。编辑您的环境变量，并将%JAVA_DLL_PATH%和%VC_DLL_PATH%目录包括到PATH中。

同样，即使我们复制了dll并将库目录包含在系统路径中，我们仍然需要将其包含在Tomcat JVM库路径中。打开BuildConfig.groovy并添加以下内容:

grails.tomcat.jvmArgs = ["-Djava.library.path=D:\opencv\4.0.1\build\x64\vc15\bin;D:\opencv\4.0.1\build\java\x64"]

将目录包括到Config.groovy中。

openCV {
cascadeClassifiers = "D:\\application\\files\\opencv"
home = "D:\\opencv\\4.0.1\\build\\x64\\vc15\\bin"
java = "D:\\opencv\\4.0.1\\build\\java\\x64"
}

如何使用:

您需要在 *.java中创建一个 *.groovy文件而不是 %APPLICATION_PATH%\src\java文件。我还没有检查为什么在 .groovy文件上不起作用。

import grails.util.Holders;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.UUID;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.FileUtils;
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.MatOfRect;
import org.opencv.core.Rect;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.objdetect.CascadeClassifier;

public class ImageService {
    static final String[] OPENCV_SUPPORTED_EXTENSION = new String[]{"bmp", "dib", "jp2", "jpe", "jpeg", "jpg", "pbm", "pgm", "png", "ppm", "ras", "sr", "tif", "tiff"};

    public static File cropImage(final File originalFile) {
        loadLibrary();
        return cropImage(originalFile, originalFile.getParentFile(), getCascadeClassifiers());
    }

    public static File cropImage(final File originalFile, final ArrayList<CascadeClassifier> cascadeClassifiers) {
        return cropImage(originalFile, originalFile.getParentFile(), cascadeClassifiers);
    }

    public static File cropImage(final File originalFile, final File targetDirectory) {
        loadLibrary();
        return cropImage(originalFile, originalFile.getParentFile(), getCascadeClassifiers());
    }

    public static File cropImage(final File originalFile, final File targetDirectory, final ArrayList<CascadeClassifier> cascadeClassifiers) {
        ArrayList<File> siblingFiles = getFaces(originalFile, cascadeClassifiers);
        File maxFile = null;
        int maxWidth = 0;

        for(int x = 0; x < siblingFiles.size(); x++) {
            Mat image = Imgcodecs.imread(siblingFiles.get(x).getAbsolutePath(), Imgcodecs.IMREAD_UNCHANGED);

            if(image.width() > maxWidth) {
                maxFile = siblingFiles.get(x);
                maxWidth = image.width();
            }
        }

        File croppedFile = null;
        if(maxFile != null) {
            croppedFile = new File(targetDirectory.getAbsolutePath() +
                File.separator +
                originalFile.getName());

            try {
                FileUtils.copyFile(maxFile, croppedFile);
            }
            catch(IOException e) {}
        }

        for(int y = 0; y < siblingFiles.size(); y++) {
            siblingFiles.get(y).delete();
        }

        System.gc();
        System.runFinalization();

        return croppedFile;
    }

    public static ArrayList<CascadeClassifier> getCascadeClassifiers() {
        ArrayList<CascadeClassifier> classifiers = new ArrayList<CascadeClassifier>();

        final String[] cascadeSupportedExtensions = new String[]{"xml"};
        final String cascadeClassifierPath = Holders.getFlatConfig().get("openCV.cascadeClassifiers").toString();
        File cascadeClassifierDirectory = new File(cascadeClassifierPath);
        ArrayList<File> detectors = new ArrayList<File>(FileUtils.listFiles(cascadeClassifierDirectory, cascadeSupportedExtensions, false));

        for(int y = 0; y < detectors.size(); y++) {
            CascadeClassifier faceDetector = new CascadeClassifier();
            faceDetector.load(detectors.get(y).getAbsolutePath());

            classifiers.add(faceDetector);
        }

        return classifiers;
    }

    public static ArrayList<File> getFaces(final File originalFile) {
        loadLibrary();
        return getFaces(originalFile, getCascadeClassifiers());
    }

    public static ArrayList<File> getFaces(final File originalFile, final ArrayList<CascadeClassifier> cascadeClassifiers) {
        File temporaryFile = new File(originalFile.getParentFile().getAbsolutePath() +
            File.separator +
            UUID.randomUUID().toString() +
            "." + FilenameUtils.getExtension(originalFile.getName()));

        try {
            FileUtils.copyFile(originalFile, temporaryFile);
        }
        catch(IOException e) {}

        final int frame = 9;
        final int offset = 8;
        int rotateCounter = 0;

        Integer marginX, marginY, marginWidth, marginHeight;
        Integer pxPerOffset, excess;

        ArrayList<File> siblingFiles = new ArrayList<File>();

        while(rotateCounter < 4) {
            if(rotateCounter > 0) {
                Mat image = Imgcodecs.imread(temporaryFile.getAbsolutePath(), Imgcodecs.IMREAD_UNCHANGED);

                Core.transpose(image, image);
                Core.flip(image, image, 1);

                Imgcodecs.imwrite(temporaryFile.getAbsolutePath(), image);

                image.release();
            }

            for(int y = 0; y < cascadeClassifiers.size(); y++) {
                CascadeClassifier faceDetector = cascadeClassifiers.get(y);
                Mat image = Imgcodecs.imread(temporaryFile.getAbsolutePath(), Imgcodecs.IMREAD_UNCHANGED);

                MatOfRect faceDetections = new MatOfRect();
                faceDetector.detectMultiScale(image, faceDetections);

                Rect defaultRect = null;
                Rect marginRect = null;
                Rect[] facesRect = faceDetections.toArray();

                for(int z = 0; z < facesRect.length; z++) {
                    defaultRect = facesRect[z];
                    pxPerOffset = defaultRect.width / frame;

                    marginX = defaultRect.x - (pxPerOffset * offset);
                    marginY = defaultRect.y - (pxPerOffset * offset);

                    marginWidth = defaultRect.width + (offset * pxPerOffset * 2);
                    marginHeight = defaultRect.height + (offset * pxPerOffset * 2);

                    excess = Math.max(
                        0 - marginX,
                            Math.max(0 - marginY,
                                Math.max(marginX + marginWidth - image.width(),
                                    Math.max(marginY + marginHeight - image.height(), 0)))
                    );

                    if(excess > 0) {
                        marginX += excess;
                        marginY += excess;

                        marginWidth -= excess * 2;
                        marginHeight -= excess * 2;
                    }

                    marginRect = new Rect(marginX, marginY, marginWidth, marginHeight);
                    Mat imageWithMargin = new Mat(image, marginRect);

                    String croppedFileName = temporaryFile.getParentFile().getAbsolutePath() +
                        File.separator +
                        UUID.randomUUID().toString() + "_" +
                        y + "_" +
                        rotateCounter + "_" +
                        z + "." +
                        FilenameUtils.getExtension(temporaryFile.getName());

                    Imgcodecs.imwrite(croppedFileName, imageWithMargin);

                    siblingFiles.add(new File(croppedFileName));

                    imageWithMargin.release();
                }

                image.release();
            }

            rotateCounter++;
        }

        temporaryFile.delete();

        return siblingFiles;
    }

    public static void loadLibrary() {
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

        final String[] urls = new String[]{
            Holders.getFlatConfig().get("openCV.java").toString(),
            Holders.getFlatConfig().get("openCV.home").toString()
        };

        final String[] validExtensions = new String[]{"dll"};
        ArrayList<File> dlls = new ArrayList<File>();
        for(int y = 0; y < urls.length; y++) {
            dlls.addAll(FileUtils.listFiles(new File(urls[y]), validExtensions, false));
        }

        for(int y = 0; y < dlls.size(); y++) {
            for(int z = 0; z < validExtensions.length; z++) {
                System.loadLibrary(dlls.get(y).getName().replace("." + validExtensions[z], ""));
                System.load(dlls.get(y).getAbsolutePath());
            }
        }
    }
}

然后，将其用于常规服务类。

class TestService {
    def openCVTest() {
        File picture = new File("D:\\original.jpg");
        File savingDirectory = new File("D:\\");

        ImageService.cropImage(picture, savingDirectory);

        return;
    }
}