java - 用 Java 构建了部分上传系统，但视频损坏

Question

因此，我构建了一个图片和视频部分上传系统，该系统似乎工作正常，直到我们尝试压缩视频或用户尝试播放视频。我发现编写视频的字节并不像附加字节那么简单...我现在(稍微)熟悉原子(free、mov、mdat、uuid 等)等概念，并且想知道为什么以及如何video 只是附加到新文件的字节集合，无法识别即使所有字节都已写入，原子仍然存在。

无论如何，这里有一些源代码:

服务器端我们有一个部分上传对象:

public class PartialUpload {
    private Integer partialUploadId;
    private String urlSuffix;
    private Date dateAdded;
    private Long expiresIn;
    private Long currentBytesMin;
    private Long currentBytesMax;
    private Long totalBytes;
    private Boolean complete;
    private String filename;

    //there are getters and setters which bind to a database entity below but these are the fields
}

在服务器端，我们还有以下方法来检查部分上传是否已开始、正在继续或已达到其总字节数。

@RequestMapping(value = "/video_partial", method = RequestMethod.POST)
    public ResponseEntity<?> postVideoPartial(@RequestParam("name") String name, @RequestParam(name = "totalBytes") Long totalBytes) throws IOException {
        PartialUpload upload = new PartialUpload();
        upload.setUrlSuffix(upload.hashUrlSuffix()); // creates a url friendly hash
        Calendar expiry = Calendar.getInstance();
        upload.setDateAdded(expiry.getTime());
        expiry.add(Calendar.DAY_OF_MONTH, PARTIAL_UPLOAD_EXPIRY_TIME); // constant which is a day
        upload.setExpiresIn(expiry.getTimeInMillis() - Calendar.getInstance().getTimeInMillis());
        upload.setComplete(false);
        upload.setFilename(name);
        upload.setTotalBytes(totalBytes);
        partialUploadRepo.save(upload); //save this object to the database
        PartialUploadInitialDTO dto = new PartialUploadInitialDTO(); //initial DTO which contains the url suffix and the time it expires in
        dto.expiresIn = upload.getExpiresIn();
        dto.urlSuffix = upload.getUrlSuffix();
        return new ResponseEntity<>(dto, HttpStatus.ACCEPTED);
    }

//continuing the partial upload until complete
@RequestMapping(value = "/video_partial/{urlSuffix}", method = RequestMethod.POST)
    public ResponseEntity<?> postVideoPartial(@RequestBody byte[] partialBytes, @PathVariable("urlSuffix") String urlSuffix) throws IOException {
        try {
            PartialUpload upload = partialUploadRepo.getUploadByUrlSuffix(urlSuffix);
            if (!upload.isExpired()) {
                if (upload.getComplete() != null && upload.getComplete()) {
                    return new ResponseEntity<>("Upload already complete", HttpStatus.BAD_REQUEST);
                } else {

                    if (upload.getCurrentBytesMin() == null) { //tests if the very first chunk has been sent. If null, the chunk has yet to be sent
                        upload.setCurrentBytesMin(0L); //Tells the client to start at a 0 offset
                        upload.setCurrentBytesMax(START_BYTES_MAX); //Tells the client to upload bytes to the maximum... if there are fewer bytes than the maximum only the applicable bytes will be written
                        if (MediaHelper.initialFileWrite(partialBytes, upload, MediaType.VIDEO)) { //instantiates the file and writes bytes given the file does not yet exist
                            if (upload.getCurrentBytesMax() >= upload.getTotalBytes()) { // if the total bytes have been written
                                if (MediaHelper.moveFileToTempFolder(upload, MediaType.VIDEO)) { //moves the file for compression
                                    PartialUploadCompleteDTO completeDTO = new PartialUploadCompleteDTO();
                                    completeDTO.success = true;
                                    upload.setCurrentBytesMin(upload.getTotalBytes());
                                    upload.setCurrentBytesMax(upload.getTotalBytes());
                                    upload.setComplete(true);
                                    partialUploadRepo.save(upload); //saves that the upload has been completed
                                    return new ResponseEntity<>(completeDTO, HttpStatus.OK); //return success
                                } else {
                                    return new ResponseEntity<>("Couldn't Move File To Temp Folder", HttpStatus.INTERNAL_SERVER_ERROR);
                                }
                            } else {
                                //***************************************
                                PartialUploadInProgressDTO dto = new PartialUploadInProgressDTO(); //case where there are more chunks to upload and where I am receiving error
                                Calendar expiry = Calendar.getInstance();
                                expiry.setTime(upload.getDateAdded());
                                expiry.setTimeInMillis(expiry.getTimeInMillis() + upload.getExpiresIn());
                                dto.expirationDateTime = expiry.getTime();
                                dto.nextExpectedMin = upload.getCurrentBytesMax() + 1; //offset the next bytes by the last byte written + 1 MAY BE THE CAUSE OF THE ERRORS
                                dto.nextExpectedMax = upload.getCurrentBytesMax() + (upload.getCurrentBytesMax() - upload.getCurrentBytesMin()); // offset the next max by the current max (which is the minimum) plus the interval number of bytes
                                if (dto.nextExpectedMax >= upload.getTotalBytes()) { //if the max overshoots the total bytes make the max the new total bytes
                                    dto.nextExpectedMax = upload.getTotalBytes();
                                }
                                upload.setCurrentBytesMin(dto.nextExpectedMin);
                                upload.setCurrentBytesMax(dto.nextExpectedMax);
                                partialUploadRepo.save(upload); // saves the next expected chunk
                               return new ResponseEntity<>(dto, HttpStatus.ACCEPTED);
                            }
                        } else {
                            return new ResponseEntity<>("Retry", HttpStatus.BAD_REQUEST);
                        }
                    } else {
                        //appends bytes to the already existing file
                        if (MediaHelper.appendBytesToFile(upload, partialBytes, MediaType.VIDEO)) {
                            if (upload.getCurrentBytesMax() >= upload.getTotalBytes()) { //test if complete (total bytes achieved)
                                if (MediaHelper.moveFileToTempFolder(upload, MediaType.VIDEO)) {
                                    PartialUploadCompleteDTO completeDTO = new PartialUploadCompleteDTO();
                                    completeDTO.success = true;
                                    upload.setCurrentBytesMin(upload.getTotalBytes());
                                    upload.setCurrentBytesMax(upload.getTotalBytes());
                                    upload.setComplete(true);
                                    partialUploadRepo.save(upload); //see above
                                    return new ResponseEntity<>(completeDTO, HttpStatus.OK);
                                } else {
                                    return new ResponseEntity<>("Couldn't Move File To Temp Folder", HttpStatus.INTERNAL_SERVER_ERROR);
                                }
                            } else {
                                PartialUploadInProgressDTO dto = new PartialUploadInProgressDTO();
                                Calendar expiry = Calendar.getInstance();
                                expiry.setTime(upload.getDateAdded());
                                expiry.setTimeInMillis(expiry.getTimeInMillis() + upload.getExpiresIn());
                                dto.expirationDateTime = expiry.getTime();
                                dto.nextExpectedMin = upload.getCurrentBytesMax() + 1;
                                dto.nextExpectedMax = upload.getCurrentBytesMax() + (upload.getCurrentBytesMax() - upload.getCurrentBytesMin());
                                if (dto.nextExpectedMax >= upload.getTotalBytes()) {
                                    dto.nextExpectedMax = upload.getTotalBytes();
                                }
                                upload.setCurrentBytesMin(dto.nextExpectedMin);
                                upload.setCurrentBytesMax(dto.nextExpectedMax);
                                partialUploadRepo.save(upload);
                               return new ResponseEntity<>(dto, HttpStatus.ACCEPTED);
                            }
                        } else {
                            return new ResponseEntity<>("Retry", HttpStatus.BAD_REQUEST);
                        }
                    }
                }
            } else {
                return new ResponseEntity<>("Upload has expired", HttpStatus.BAD_REQUEST);
            }
        } catch(Exception e) {
            return new ResponseEntity<>("No File Exists At URL", HttpStatus.BAD_REQUEST);
        }
    }

这些是媒体帮助器方法:

    public boolean initialFileWrite(byte[] partialBytes, PartialUpload upload, MediaType type) {
        File blobUploadDirectory = null;
        try {
            if (type == MediaType.IMAGE) {
                blobUploadDirectory = getBlobImageUploadDir();
            } else {
                blobUploadDirectory = getBlobVideoUploadDir();
            }
            if (!blobUploadDirectory.exists()) {
                blobUploadDirectory.mkdirs();
            }
            File file = new File(blobUploadDirectory.getAbsolutePath() + "/" + upload.getFilename());
            if (!file.exists()) {
                file.createNewFile();
            }
            FileUtils.writeByteArrayToFile(file, partialBytes, false);
            return true;
        } catch(Exception e) {
            //unappend appended bytes
            e.printStackTrace();
            return false;
        }
    }

    public boolean appendBytesToFile(PartialUpload upload, byte[] partialBytes, MediaType type) {
        File blobUploadDirectory = null;
        try {
            if (type == MediaType.IMAGE) {
                blobUploadDirectory = getBlobImageUploadDir();
            } else {
                blobUploadDirectory = getBlobVideoUploadDir();
            }
            if (!blobUploadDirectory.exists()) {
                blobUploadDirectory.mkdirs();
            }
            File file = new File(blobUploadDirectory.getAbsolutePath() + "/" + upload.getFilename());
            if (!file.exists()) {
                file.createNewFile();
            }
            FileUtils.writeByteArrayToFile(file, partialBytes, true);
            return true;
        } catch(Exception e) {
            //unappend appended bytes
            e.printStackTrace();
            return false;
        }
    }

    public boolean moveFileToTempFolder(PartialUpload upload, MediaType type) {
        File blobUploadDirectory = null;
        try {
            if (type == MediaType.IMAGE) {
                blobUploadDirectory = getBlobImageUploadDir();
            } else {
                blobUploadDirectory = getBlobVideoUploadDir();
            }
            if (!blobUploadDirectory.exists()) {
                blobUploadDirectory.mkdirs();
                return false;
            }
            File file = new File(blobUploadDirectory.getAbsolutePath() + "/" + upload.getFilename());
            if (!file.exists()) {
                return false;
            }
            File outFile = type == MediaType.IMAGE ? new File(getLocalImageUploadDir(), upload.getFilename()) : new File(getLocalVideoUploadDir(), upload.getFilename());
            return file.renameTo(outFile);
        } catch(Exception e) {
            //unappend appended bytes
            return false;
        }
    }

在移动应用程序中，我使用 QTFastStart 的修改版本将 moov 原子移动到结构的前面(不是必需的，但这是一个临时解决方案)。否则 moov 原子就会被损坏。任何关于为什么原子在文件传输过程中被损坏的帮助将不胜感激。

这里是 Android 应用程序的更多源代码

if (currentMedia.getType() == VIDEO) {
                    try {
                        String newFileName = currentMedia.getPath(); //qualified media path
                        newFileName = newFileName.replaceAll("\\d+\\.mp4", "newFile.mp4"); //makes a temp mp4 for the current media. I tested with AtomicParsley and QTFastStart (PY) to make sure this wasn't the issue

                        File inFile = new File(currentMedia.getPath());
                        File outFile = new File(newFileName);
                        if (!outFile.exists()) {
                            outFile.createNewFile();
                        }
                        QtFastStart.fastStart(inFile, outFile); //moves the moov atom indices
                        outFile.renameTo(new File(currentMedia.getPath())); //makes the original file the moved file
                        file = new File(currentMedia.getPath());

                    } catch (IOException e) {
                        e.printStackTrace();
                    } catch (QtFastStart.MalformedFileException e) {
                        e.printStackTrace();
                    } catch (QtFastStart.UnsupportedFileException e) {
                        e.printStackTrace();
                    }
                } else {
                    file = new File(currentMedia.getPath());
                }
                //reads a total number of bytes from a file
                if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.O) {
                    fileContent = Files.readAllBytes(file.toPath());
                } else {
                    fileContent = readFile(file);
                }

                if (currentMedia.getType() == IMAGE) {
                    mediaApiFactory.makeRequestStartImagePartialUpload(file.getName(), fileContent.length).start(null, initialResultListener);
                    pictureProgressText.setText("Uploading " + currentMedia.getName() + ".jpg");

                } else {
                    mediaApiFactory.makeRequestStartVideoPartialUpload(file.getName(), fileContent.length).start(null, initialResultListener);
                    pictureProgressText.setText("Uploading " + currentMedia.getName() + ".mp4");

                }

然后我使用retrofit将视频/图像发送到服务器

然后我处理响应

if (result != null && result.isSuccess()) {
                PartialUploadInitialDTO dto = result.getData();
                currentUrlSuffix = dto.urlSuffix;
                currentMinBytes = 0;
                currentMaxBytes = START_BYTES_MAX;
                chunks = new ArrayList<>();
                chunks.add(Arrays.copyOfRange(fileContent, (int)currentMinBytes, (int)currentMaxBytes));

                if (currentMedia.getType() == IMAGE) {
                    mediaApiFactory.makeRequestContinueImagePartialUpload(currentUrlSuffix, chunks.get(chunks.size() - 1)).start(null, intermediateResultListener);
                } else {
                    mediaApiFactory.makeRequestContinueVideoPartialUpload(currentUrlSuffix, chunks.get(chunks.size() - 1)).start(null, intermediateResultListener);
                }
            }

然后继续上传，直到读取完总字节数(成功状态)

if (result != null && result.isSuccess()) {
                if (result.getData() instanceof PartialUploadInProgressDTO) { //contains the success status of a complete dto as well (me being lazy)
                    PartialUploadInProgressDTO dto = (PartialUploadInProgressDTO)result.getData();
                    if (dto.success != null && dto.success) { //denotes that the upload was successful
                        currentFile++;
                        if (currentMedia.isDeleted()) {
                            if (currentMedia.getPath() != null) {
                                MediaUtils.deleteFile(currentMedia.getPath());
                            }
                        }
                        if (mediaInvocationListener != null) {
                            mediaInvocationListener.onUpdateProgress(currentFile, count);
                        }
                        pictureProgressBar.setProgress(100);
                        next();
                    } else { //the upload is still in progress
                        currentMinBytes = dto.nextExpectedMin;
                        currentMaxBytes = dto.nextExpectedMax;
                        chunks.add(Arrays.copyOfRange(fileContent, (int) currentMinBytes, (int) currentMaxBytes)); //adds the next chunk of bytes
                        int progress = (int)((float)currentMinBytes / fileContent.length * 100);
                        pictureProgressBar.setProgress(progress);

                        if (currentMedia.getType() == IMAGE) {
                            mediaApiFactory.makeRequestContinueImagePartialUpload(currentUrlSuffix, chunks.get(chunks.size() - 1)).start(null, intermediateResultListener); //recursive to this method
                        } else {
                            mediaApiFactory.makeRequestContinueVideoPartialUpload(currentUrlSuffix, chunks.get(chunks.size() - 1)).start(null, intermediateResultListener); //recursive to this method
                        }
                    }
                }
            }

这是到达服务器之前和之后的平均原子树之前(但在 QTQuick 运行之后):

ftyp (24 bytes)
moov (15372 bytes)
mdat (67290713 bytes)

之后( block 成功上传)

REM This is available upon request looks something like
ftyp (bytes)
moov (bytes)
NOTMDAT(bytes)

无论如何，对于冗长的帖子感到抱歉。如果需要其他任何内容，请询问，我会对此进行编辑。

Answer 1

所以我发现问题是代码中的错误。当字节具有其起始值和终止值时，您无需考虑写入的最后一个字节，只需继续使用相同的数字即可。所以以下是正确的:

dto.nextExpectedMin = upload.getCurrentBytesMax();
dto.nextExpectedMax = upload.getCurrentBytesMax() + (upload.getCurrentBytesMax() - upload.getCurrentBytesMin());

无论我在哪里

dto.nextExpectedMin = upload.getCurrentBytesMax() + 1;
dto.nextExpectedMax = upload.getCurrentBytesMax() + (upload.getCurrentBytesMax() - upload.getCurrentBytesMin());

之前