python - 谷歌云存储从字节上传失败

Question

我正在尝试在 python 中动态创建一个文件，然后将其上传到谷歌云存储，但我无法从字符串或字节对象上传。

脚本的基本设置是

from google.cloud import storage

storage_client = storage.Client()

storage_client = storage.Client()
bucket = storage_client.bucket("bucket_name")
blob = bucket.blob("destination_blob_name")

如果我然后获取一个文件并运行upload_from_filename，它会上传正常

blob.upload_from_filename("source_file_name.pdf")

如果我获取相同的文件，但尝试使用 upload_from_string 上传

with open('source_file_name.pdf', 'rb') as f:
    file_string = f.read()
    blob.upload_from_string(file_string, content_type='application/pdf')

我收到错误:

TypeError: None could not be converted to bytes

我已经检查过，file_string 是一个有效的字节对象

同样，如果我尝试upload_from_file

with open('source_file_name.pdf', 'rb') as f:
    file_string = f.read()
    blob.upload_from_file(file_string, content_type='application/pdf')

我收到错误:

AttributeError: 'bytes' object has no attribute 'tell'

在我的实际用例中，我使用 BytesIO 创建一个字节对象，如下所示

pdf = BytesIO()
pisa.CreatePDF(html, pdf)
resp = pdf.getvalue()
pdf.close()

创建一个带有值的字节对象

b'%PDF-1.4\n%\x93\x8c\x8b\x9e ReportLab Generated PDF document http://www.reportlab.com\n1 0 obj\n<<\n/F1 2 0 R /F2 3 0 R\n>>\nendobj\n2 0 obj\n<<\n/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font\n>>\nendobj\n3 0 obj\n<<\n/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font\n>>\nendobj\n4 0 obj\n<<\n/Contents 11 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 10 0 R /Resources <<\n/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]\n>> /Rotate 0 /Trans <<\n\n>> \n  /Type /Page\n>>\nendobj\n5 0 obj\n<<\n/Outlines 7 0 R /PageMode /UseNone /Pages 10 0 R /Type /Catalog\n>>\nendobj\n6 0 obj\n<<\n/Author () /CreationDate (D:20210927153854+00\'00\') /Creator (\\(unspecified\\)) /Keywords () /ModDate (D:20210927153854+00\'00\') /Producer (xhtml2pdf <https://github.com/xhtml2pdf/xhtml2pdf/>) \n  /Subject () /Title () /Trapped /False\n>>\nendobj\n7 0 obj\n<<\n/Count 2 /First 8 0 R /Last 8 0 R /Type /Outlines\n>>\nendobj\n8 0 obj\n<<\n/Count -1 /Dest [ 4 0 R /Fit ] /First 9 0 R /Last 9 0 R /Parent 7 0 R /Title (This is a document)\n>>\nendobj\n9 0 obj\n<<\n/Dest [ 4 0 R /Fit ] /Parent 8 0 R /Title (second thing)\n>>\nendobj\n10 0 obj\n<<\n/Count 1 /Kids [ 4 0 R ] /Type /Pages\n>>\nendobj\n11 0 obj\n<<\n/Filter [ /ASCII85Decode /FlateDecode ] /Length 705\n>>\nstream\nGb"/c9i\'Ou&;KZN/*9AQC>q4Ff;`fLJs?67f9opAMXB=4i9,<s$iN=ZiW-G9S!E#jB:aD?g(GmLjRM,0J?KVk>Y1#79RDk\\BOJE+b@u10]AZsX]j2.82-nZs5IP%fC<:8k$;:0b1aC"H%J9]34&.>Vf92D\\[Ajp7(1\\rL6.Tm%FE(<IQl>h0e?=m635F3h9+grs@J2$PFQjH45@OCnfPZ^c2m3cNUDR_e]LLOTW`4KCn)tj.`FN8$A<u3m=@+ZI^g?ng(2/]\\1@F-(6qX+(VK4HJ75P<L%\\RH<.?Yiu!*k>T&SGDN83\'j<6UJEUC3#&@R^5YK\'.itbk6F\\Yg^$q&S-H[R?.K(k-U:1fPCo\\e\\E?A!cf$KTHku+nq3XL?!?X$58ii)YOI&C?j%gClCr^Lki\\<<mr%ar-%P4s^;^Yg*$2Bh0Vo]6r6=urje)VY0)XnNuN.D`%M0m"uQ-1=\\EQ<Sa+rFLLB;rDG%/YDpSc6D?0N`JUaW(Fg4rRZ>k+XAAif2?!F97Y-RPe^_a5*i5e\'(9i<Jtm&guN%s70i>Z4j2,__MFmRX^;Eq/qO\\Y&ta9k<kR+!<@?LL1YQHJ%lRTM8C<Ag]kXpJCYP_9BrY%[EZF/^<_<eFnW5o)\\K"Q@$ZiS"]OPa4Z>8<UVK`WmC=r6q[Kf!<&%&)EFgi6*1YP><XfY"!SS4M?hUQ;DqZUOY2LY~>endstream\nendobj\nxref\n0 12\n0000000000 65535 f \n0000000073 00000 n \n0000000114 00000 n \n0000000221 00000 n \n0000000333 00000 n \n0000000538 00000 n \n0000000623 00000 n \n0000000875 00000 n \n0000000946 00000 n \n0000001065 00000 n \n0000001143 00000 n \n0000001203 00000 n \ntrailer\n<<\n/ID \n[<23dcacffb65069110eef5eedf4b20aaa><23dcacffb65069110eef5eedf4b20aaa>]\n% ReportLab generated PDF document -- digest (http://www.reportlab.com)\n\n/Info 6 0 R\n/Root 5 0 R\n/Size 12\n>>\nstartxref\n1999\n%%EOF\n'

但是尝试 upload_from_string 或 upload_from_file 会得到与上面相同的结果

Answer 1

我找到了一种方法来做到这一点

from google.cloud import storage
import io

storage_client = storage.Client()

storage_client = storage.Client()
bucket = storage_client.bucket("bucket_name")
blob = bucket.blob("destination_blob_name")

# Produce file bytes however needed
# either by reading file contents or on the fly
bytes = your_function_to_create_file_bytes()

# Now we have a bytes string, convert that into a stream
stream = io.BytesIO(pdf_bytes)
blob.upload_from_file(stream)