I am trying to invoke a Cloud Run service using Cloud Tasks as described in the docs here.
我正在尝试使用云任务调用云运行服务,如这里的文档中所述。
I have a running Cloud Run service. If I make the service publicly accessible, it behaves as expected.
我有一项正在运行的云运行服务。如果我使服务可公开访问,它就会按预期运行。
I have created a cloud queue and I schedule the cloud task with a local script. This one is using my own account. The script looks like this
我已经创建了一个云队列,并使用本地脚本调度云任务。这个人用的是我自己的账户。脚本如下所示
from google.cloud import tasks_v2
client = tasks_v2.CloudTasksClient()
project = 'my-project'
queue = 'my-queue'
location = 'europe-west1'
url = 'https://url_to_my_service'
parent = client.queue_path(project, location, queue)
task = {
'http_request': {
'http_method': 'GET',
'url': url,
'oidc_token': {
'service_account_email': '[email protected]'
}
}
}
response = client.create_task(parent, task)
print('Created task {}'.format(response.name))
I see the task appear in the queue, but it fails and retries immediately. The reason for this (by checking the logs) is that the Cloud Run service returns a 401 response.
我看到任务出现在队列中,但它失败并立即重试。这样做的原因(通过检查日志)是因为Cloud Run服务返回401响应。
My own user has the roles "Service Account Token Creator" and "Service Account User". It doesn't have the "Cloud Tasks Enqueuer" explicitly, but since I am able to create the task in the queue, I guess I have inherited the required permissions.
The service account "[email protected]" (which I use in the task to get the OIDC token) has - amongst others - the following roles:
我自己的用户具有角色“Service Account Token Creator”和“Service Account User”。它没有显式的“云任务入队程序”,但是因为我能够在队列中创建任务,所以我想我已经继承了所需的权限。服务帐户“[电子邮件受保护]”(我在任务中使用它来获取OIDC令牌)除其他角色外,还具有以下角色:
- Cloud Tasks Enqueuer (Although I don't think it needs this one as I'm creating the task with my own account)
- Cloud Tasks Task Runner
- Cloud Tasks Viewer
- Service Account Token Creator (I'm not sure whether this should be added to my own account - the one who schedules the task - or to the service account that should perform the call to Cloud Run)
- Service Account User (same here)
- Cloud Run Invoker
So I did a dirty trick: I created a key file for the service account, downloaded it locally and impersonated locally by adding an account to my gcloud config with the key file. Next, I run
因此,我做了一个卑鄙的把戏:我为服务帐户创建了一个密钥文件,将其下载到本地,并通过使用该密钥文件将一个帐户添加到我的gCloud配置中进行本地模拟。接下来,我会跑
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" https://url_to_my_service
That works! (By the way, it also works when I switch back to my own account)
这很管用!(顺便说一句,当我切换回我自己的账户时,它也可以使用)
Final tests: if I remove the oidc_token
from the task when creating the task, I get a 403 response from Cloud Run! Not a 401...
If I remove the "Cloud Run Invoker" role from the service account and try again locally with curl, I also get a 403 instead of a 401.
最终测试:如果我在创建任务时从任务中删除oidc_Token,则会收到Cloud Run的403响应!不是401...如果我从服务帐户中删除“Cloud Run Invoker”角色,并使用curl在本地重试,我也会得到403而不是401。
If I finally make the Cloud Run service publicly accessible, everything works.
如果我最终使Cloud Run服务可以公开访问,一切都会正常工作。
So, it seems that the Cloud Task fails to generate a token for the service account to authenticate properly at the Cloud Run service.
因此,云任务似乎无法为服务帐户生成令牌,以便在云运行服务中正确进行身份验证。
What am I missing?
我错过了什么?
更多回答
Me too.. followed docs to the letter: cloud.google.com/tasks/docs/creating-http-target-tasks but am getting 401 responses from the target service. The service account enqueuing the task should only need the permissions 1. Cloud Tasks Enqueuer
2. Service Account User
3. Cloud Run Invoker
(Or invoker for whichever google service you're targeting). The enqueueing service account email is added to the task before it is enqueued so that the Cloud Tasks Queue can use it to generate a token.... I'm going to see if this issue resolves in 24 hours like yours did. This is super frustrating
我也是。按照文档的要求执行:cloud.google.com/tasks/docs/creating-http-target-tasks,但从目标服务收到了401个响应。将任务排入队列的服务帐户应该只需要以下权限:1.云任务入队程序2.服务帐户用户3.云运行调用者(或目标Google服务的调用者)。正在入队的服务帐户电子邮件会在任务入队之前添加到任务中,以便云任务队列可以使用它来生成令牌...我要看看这个问题是否会像你的那样在24小时内解决。这真是太令人沮丧了
Maybe also worth noting: I recently also got 401 responses when trying to trigger Cloud Run from Cloud Scheduler. I was also using the OIDC token and it turned out that I set the wrong URL in the audience. For Cloud Tasks, it seems like getting the OIDC token happens behind the scenes but I now have the feeling that there is something going wrong there.
也许还值得注意的是:我最近在尝试从Cloud Scheduler触发Cloud Run时也收到了401个响应。我也在使用OIDC令牌,结果发现我在观众中设置了错误的URL。对于云任务,获取OIDC令牌似乎是在幕后进行的,但我现在感觉那里出了问题。
I figured it out. And frustrating thing for me is I've had to solve this issue before... If you don't explicitly populate the audience
field for the oidc_token
then the target url from the task is used, in your example above: https://url_to_my_service
. The problem here is that if you're using Cloud Run with custom domains (instead of the cloud run generated domain), then you'll get an error because OIDC audience
doesn't support custom domains. My fix was to explicitly populate the audience with the Cloud Run generated URL, then it worked.
我想通了。令我沮丧的是,我以前必须解决这个问题……如果没有显式填充oidc_内标识的受众字段,则使用任务中的目标URL,在上面的示例中为:https://url_to_my_service.这里的问题是,如果您将Cloud Run与自定义域(而不是Cloud Run生成的域)一起使用,那么您将收到一个错误,因为OIDC受众不支持自定义域。我的解决办法是用Cloud Run生成的URL显式地填充受众,然后它就奏效了。
Thank you @teaMonkeyFruit , this solve the issue for me! I didn't find any documentation about it
谢谢@teaMonkey Fruit,这为我解决了问题!我没有找到任何关于它的文件
I had the same issue here was my fix:
我也有同样的问题,这是我的解决办法:
Diagnosis: Generating OIDC tokens currently does not support custom domains in the audience
parameter. I was using a custom domain for my cloud run service (https://my-service.my-domain.com) instead of the cloud run generated url (found in the cloud run service dashboard) that looks like this: https://XXXXXX.run.app
诊断:生成OIDC令牌目前不支持audience参数中的自定义域。我为我的云运行服务使用了自定义域(my-service.my-domain.com),而不是云运行生成的URL(在云运行服务仪表板中找到),如下所示:https://XXXXXX.run.app
Masking behavior: In the task being enqueued to Cloud Tasks, If the audience
field for the oidc_token is not explicitly set then the target url from the task is used to set the audience
in the request for the OIDC token.
屏蔽行为:在排队到云任务的任务中,如果没有显式设置oidc_令牌的受众字段,则使用任务中的目标URL来设置OIDC令牌请求中的受众。
In my case this meant that enqueueing a task to be sent to the target https://my-service.my-domain.com/resource
the audience for the generating the OIDC token was set to my custom domain https://my-service.my-domain.com/resource
. Since custom domains are not supported when generating OIDC tokens, I was receiving 401 not authorized
responses from the target service.
在我的例子中,这意味着将一个要发送到目标https://my-service.my-domain.com/resource的任务排队,生成OIDC令牌的受众被设置为我的自定义域https://my-service.my-domain.com/resource.由于在生成OIDC令牌时不支持自定义域,因此我收到了来自目标服务的401个未经授权的响应。
My fix: Explicitly populate the audience with the Cloud Run generated URL, so that a valid token is issued. In my client I was able to globally set the audience for all tasks targeting a given service with the base url: 'audience' : 'https://XXXXXX.run.app'
. This generated a valid token. I did not need to change the url of the target resource itself. The resource stayed the same: 'url' : 'https://my-service.my-domain.com/resource'
我的解决办法:使用Cloud Run生成的URL显式填充受众,以便颁发有效的令牌。在我的客户端中,我能够使用基本URL‘Audience’:‘https://XXXXXX.run.app’.‘全局设置针对给定服务的所有任务的受众这生成了一个有效的令牌。我不需要更改目标资源本身的URL。资源保持不变:‘url’:‘https://my-service.my-domain.com/resource’
More Reading:
I've run into this problem before when setting up service-to-service authentication: Google Cloud Run Authentication Service-to-Service
更多阅读:我以前在设置服务到服务身份验证时遇到过这个问题:谷歌云运行身份验证服务到服务
1.I created a private cloud run service using this code:
1.我使用以下代码创建了一个私有云运行服务:
import os
from flask import Flask
from flask import request
app = Flask(__name__)
@app.route('/index', methods=['GET', 'POST'])
def hello_world():
target = os.environ.get('TARGET', 'World')
print(target)
return str(request.data)
if __name__ == "__main__":
app.run(debug=True,host='0.0.0.0',port=int(os.environ.get('PORT', 8080)))
2.I created a service account with --role=roles/run.invoker
that I will associate with the cloud task
2.我创建了一个服务帐户,并将其与云任务相关联
gcloud iam service-accounts create SERVICE-ACCOUNT_NAME \
--display-name "DISPLAYED-SERVICE-ACCOUNT_NAME"
gcloud iam service-accounts list
gcloud run services add-iam-policy-binding SERVICE \
--member=serviceAccount:[email protected] \
--role=roles/run.invoker
3.I created a queue
3.我创建了一个队列
gcloud tasks queues create my-queue
4.I create a test.py
4.我创建了一个test.py
from google.cloud import tasks_v2
from google.protobuf import timestamp_pb2
import datetime
# Create a client.
client = tasks_v2.CloudTasksClient()
# TODO(developer): Uncomment these lines and replace with your values.
project = 'your-project'
queue = 'your-queue'
location = 'europe-west2' # app engine locations
url = 'https://helloworld/index'
payload = 'Hello from the Cloud Task'
# Construct the fully qualified queue name.
parent = client.queue_path(project, location, queue)
# Construct the request body.
task = {
'http_request': { # Specify the type of request.
'http_method': 'POST',
'url': url, # The full url path that the task will be sent to.
'oidc_token': {
'service_account_email': "your-service-account"
},
'headers' : {
'Content-Type': 'application/json',
}
}
}
# Convert "seconds from now" into an rfc3339 datetime string.
d = datetime.datetime.utcnow() + datetime.timedelta(seconds=60)
# Create Timestamp protobuf.
timestamp = timestamp_pb2.Timestamp()
timestamp.FromDatetime(d)
# Add the timestamp to the tasks.
task['schedule_time'] = timestamp
task['name'] = 'projects/your-project/locations/app-engine-loacation/queues/your-queue/tasks/your-task'
converted_payload = payload.encode()
# Add the payload to the request.
task['http_request']['body'] = converted_payload
# Use the client to build and send the task.
response = client.create_task(parent, task)
print('Created task {}'.format(response.name))
#return response
5.I run the code in Google Cloud Shell with my user account which has Owner role.
5.我使用具有所有者角色的用户帐户在Google Cloud Shell中运行代码。
6.The response received has the form:
6.收到的答复的形式如下:
Created task projects/your-project/locations/app-engine-loacation/queues/your-queue/tasks/your-task
7.Check the logs, success
7.查看日志,成功
For those like me, struggling through documentation and stackoverflow when having continuous UNAUTHORIZED
responses on Cloud Tasks HTTP requests:
对于像我这样的人来说,在云任务HTTP请求上有连续的未经授权的响应时,努力处理文档和堆栈溢出:
As was written in thread, you better provide audience
for oidcToken
you send to CloudTasks. Ensure your requested url exactly equals to your resource.
正如在线程中编写的那样,您最好为您发送到CloudTasks的oidcToken提供受众。确保您请求的url与您的资源完全相同。
For instance, if you have Cloud Function named my-awesome-cloud-function
and your task request url is https://REGION-PROJECT-ID.cloudfunctions.net/my-awesome-cloud-function/api/v1/hello
, you need to ensure, that you set function url itself.
例如,如果您有名为my-awawed-Cloud-Function的Cloud函数,并且您的任务请求URL是https://REGION-PROJECT-ID.cloudfunctions.net/my-awesome-cloud-function/api/v1/hello,,则需要确保设置函数URL本身。
{
serviceAccountEmail: [email protected],
audience: https://REGION-PROJECT-ID.cloudfunctions.net/my-awesome-cloud-function
}
Otherwise seems full url is used and leads to an error.
否则,似乎使用了完整的URL并导致错误。
The next day I am no longer able to reproduce this issue. I can reproduce the 403 responses by removing the Cloud Run Invoker role, but I no longer get 401 responses with exactly the same code as yesterday.
I guess this was a temporary issue on Google's side?
第二天,我再也不能复制这个问题了。我可以通过删除Cloud Run调用者角色来复制403个响应,但我不再使用与昨天完全相同的代码获得401个响应。我想这是谷歌方面的一个暂时问题吧?
Also, I noticed that it takes some time before updated policies are actually in place (1 to 2 minutes).
此外,我注意到更新的策略需要一段时间才能真正到位(1到2分钟)。
Not directly related to the OP issue but I was struggling to call Cloud Workflows
from Cloud Tasks
. The issue is that I was using OIDC instead of OAuth.
It's mentioned in https://cloud.google.com/tasks/docs/reference/rpc/google.cloud.tasks.v2#google.cloud.tasks.v2.CreateTaskRequest
与运营问题没有直接关系,但我很难从云任务中调用云工作流。问题是我使用的是OIDC而不是OAuth。它在https://cloud.google.com/tasks/docs/reference/rpc/google.cloud.tasks.v2#google.cloud.tasks.v2.CreateTaskRequest中提到过
OAuthToken
If specified, an OAuth token will be generated and attached as an
Authorization header in the HTTP request.
This type of authorization should generally only be used when calling
Google APIs hosted on *.googleapis.com.
The url for workflows executions is starting with workflows.googleapis.com
, that's why OAuth should be used instead of OIDC.
工作流执行的URL以workflows.googleapis.com开头,这就是为什么应该使用OAuth而不是OIDC。
更多回答
Thanks for this breakdown, it helped me realise that my issue was audience related. Specifically, using a cloud function as the handler but with a path suffix, I needed to explicitly set the audience sans path suffix for it to work.
感谢你的细分,它帮助我意识到我的问题是与观众相关的。具体地说,使用云函数作为处理程序,但带有路径后缀,我需要显式设置受众sans路径后缀才能使其工作。
How did you set your custom URL? Was it using an external static IP load balancer? That's what I have set up and this doesn't seem to fix it.
您是如何设置您的自定义URL的?它是否使用外部静态IP负载均衡器?这就是我设置的,但这似乎不能解决它。
The custom-url for the service is created in Cloud Run. I set the audience field for the OIDC token with the cloud-run-generated-url, in the request object.
服务的自定义URL是在Cloud Run中创建的。我在请求对象中使用Cloud-Run生成的url为OIDC令牌设置了Audience字段。
+1 I was struggling for a few hours getting cloud scheduler to talk with Cloud Run Jobs. Turns out OIDC does not work with run.googleapis.com/apis/run.googleapis.com/v1/namespaces. Only OAuth works !
+1我花了几个小时让Cloud Scheduler与Cloud Run Jobs交谈。事实证明,OIDC不能与run.googleapis.com/apis/run.googleapis.com/v1/namespaces.一起工作只有OAuth有效!
@teaMonkeyFruit HTTP Target Cloud Tasks can be sent to any server (even outside of Google). Doesn't that contraindicate with not supporting custom domains in "audience"? Cloud Task might send request to a non-google machine, so why "run.app" audience would be magic here?
@teaMonkeyFruit HTTP目标云任务可以发送到任何服务器(即使在Google之外)。这与不支持“受众”中的自定义域不是矛盾的吗?云任务可能会将请求发送到非Google机器,那么为什么“run.app”用户会在这里变得神奇呢?
Thanks for sharing that. The only difference I see with what I'm doing is the way you bind the role to the service account. I used the console UI while you use a gcloud command. Would that make a difference?
谢谢你分享这个。我看到的唯一不同之处是您将角色绑定到服务帐户的方式。我使用的是控制台用户界面,而您使用的是gCloud命令。这会有区别吗?
I am one of those struggling with this audience-related. I just did as you suggest and I still get the same message: Failed to validate auth token. FirebaseAuthError: Firebase ID token has incorrect "aud" (audience) claim. Expected "XXX" but got "https://europe-west3-XXX.cloudfunctions.net/YYY". Make sure the ID token comes from the same Firebase project as the service account used to authenticate this SDK. See https://firebase.google.com/docs/auth/admin/verify-id-tokens for details on how to retrieve an ID token.
Does this ring a bell? It was worth the try :) Thanks!
我是那些与观众相关的人中的一员。我只是按照您的建议执行了操作,仍然收到相同的消息:验证auth令牌失败。Firebase AuthError:FireBase ID令牌的“AUD”(受众)声明不正确。应为“xxx”,但得到的是“https://europe-west3-XXX.cloudfunctions.net/YYY”.确保ID令牌来自与用于对此SDK进行身份验证的服务帐户相同的Firebase项目。有关如何检索ID令牌的详细信息,请参阅https://firebase.google.com/docs/auth/admin/verify-id-tokens。这有什么印象吗?值得一试:)谢谢!
Something I just discovered, which is buried in the docs, is that whichever service account you are using to enqueue tasks must have BOTH 'roles/cloudtasks.enqueuer' and 'roles/iam.serviceAccountUser'. The latter which grants permission to tell the Cloud Tasks service to actAs a service account and generate a token. Very convoluted setup compared to PubSub in my opinion. (see step #8)[cloud.google.com/tasks/docs/creating-http-target-tasks#sa]
我刚刚发现的一件事隐藏在文档中,那就是无论您使用哪个服务帐户来将任务入队,都必须同时具有‘Roles/Cloudtasks.enqueer’和‘Roles/iam.serviceAccount tUser’。后者向ACTAS授予告诉云任务服务一个服务帐户并生成令牌的权限。在我看来,与PubSub相比,它的设置非常复杂。(请参阅步骤#8)[cloud.google.com/tasks/docs/creating-http-target-tasks#sa]
@jbool24 you are a savior. This solved my problem.
@jbool24你是救世主。这解决了我的问题。
我是一名优秀的程序员,十分优秀!