c# - 使用 FTP 协议(protocol)列出包含大约 2,000,000 个文件的目录

Question

我在 FTP 服务器上有一个包含大约 2,000,000 个文件的目录。此 FTP 服务器是基于 Linux 的。我想列出这个目录下的文件(文件名加上最后修改日期)，但是Filezilla和Core FTP Pro都做不到，列出操作会失败。我曾尝试使用 FTPWebRequest 类在 C# 中编写自己的应用程序并运行 ListDirectory 方法，但它也无法列出目录中的文件并且 ftpwebrequest 超时。
有什么方法可以使用任何协议(protocol)(例如 FTP、SMB 等)甚至通过执行 bash 脚本来列出目录中这么多文件的名称？

在c#中列出目录的函数是:

public static List<string> ListFtpDirectory(string address, string userName, string password)
    {
        List<string> filesName = new List<string>();
        try
        {
            FtpWebRequest request = (FtpWebRequest) WebRequest.Create(address);
            request.Method = WebRequestMethods.Ftp.ListDirectory;
            request.Credentials = new NetworkCredential(userName, password);
            request.UsePassive = true;
            using (FtpWebResponse response = (FtpWebResponse) request.GetResponse())
            using (StreamReader reader = new StreamReader(response.GetResponseStream()))
            {
                filesName =
                    reader.ReadToEnd().Split(new string[] {"\r\n"}, StringSplitOptions.RemoveEmptyEntries).ToList();
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
        }
        return filesName;
    }

感谢任何帮助。

Answer 1

我的 C# 不是很流利，但是您没有包含您收到的错误的确切文本。

以下是一个应该在您的环境中工作的 Powershell 脚本:

$Server = "ftp://ftp.example.com/"
$User = "anonymous@example.com"
$Pass = "anonymous@anonymous.com"

Function Get-FtpDirectory($Directory) {

    # Credentials
    $FTPRequest = [System.Net.FtpWebRequest]::Create("$($Server)$($Directory)")
    $FTPRequest.Credentials = New-Object System.Net.NetworkCredential($User,$Pass)
    $FTPRequest.Method = [System.Net.WebRequestMethods+FTP]::ListDirectoryDetails

    # Don't want Binary, Keep Alive unecessary.
    $FTPRequest.UseBinary = $False
    $FTPRequest.KeepAlive = $False

    $FTPResponse = $FTPRequest.GetResponse()
    $ResponseStream = $FTPResponse.GetResponseStream()

    # Create a nice Array of the detailed directory listing
    $StreamReader = New-Object System.IO.Streamreader $ResponseStream
    $DirListing = (($StreamReader.ReadToEnd()) -split [Environment]::NewLine)
    $StreamReader.Close()

    # Remove first two elements ( . and .. ) and last element (\n)
    # and take into account empty directories
    If ($DirListing.Length -gt 3) {
        $DirListing = $DirListing[2..($DirListing.Length-2)]
    }
    else {
        $DirListing = @{}    $DirListing = $DirListing[2..($DirListing.Length-2)] 
    }

    # Close the FTP connection so only one is open at a time
    $FTPResponse.Close()

    # This array will hold the final result
    $FileTree = @()

    # Loop through the listings
    foreach ($CurLine in $DirListing) {

        # Split line into space separated array
        $LineTok = ($CurLine -split '\ +')

        # Get the filename (can even contain spaces)
        $CurFile = $LineTok[8..($LineTok.Length-1)]

        # Figure out if it's a directory. Super hax.
        $DirBool = $LineTok[0].StartsWith("d")

        # Determine what to do next (file or dir?)
        If ($DirBool) {
            # Recursively traverse sub-directories
            $FileTree += ,(Get-FtpDirectory "$($Directory)$($CurFile)/")
        } Else {
            # Add the output to the file tree
            $FileTree += ,"$($Directory)$($CurFile)"
        }
    }

    Return $FileTree

}