docker - 如何通过 Pulumi 将环境变量传递给 Dockerfile？

Question

我制作了一个简单的 flask 应用程序来练习 Pulumi。
它通过 Dockerfile 设置环境变量，我打算将它托管在 AWS Fargate 上，并将 RDS Postgres 作为数据库。
这是 Flask 应用程序:

import os

from flask import Flask, request
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate

app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = "postgresql://{}".format(
    os.environ.get("DATABASE_URL")
)
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db = SQLAlchemy(app)
migrate = Migrate(app, db)


class CarsModel(db.Model):
    __tablename__ = "cars"

    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String())
    model = db.Column(db.String())
    doors = db.Column(db.Integer())

    def __init__(self, name, model, doors):
        self.name = name
        self.model = model
        self.doors = doors

    def __repr__(self):
        return f"<Car {self.name}>"


@app.route("/")
def hello():
    return {"hello": "world"}


@app.route("/cars", methods=["POST", "GET"])
def handle_cars():
    if request.method == "POST":
        if request.is_json:
            data = request.get_json()
            new_car = CarsModel(
                name=data["name"], model=data["model"], doors=data["doors"]
            )

            db.session.add(new_car)
            db.session.commit()

            return {"message": f"car {new_car.name} has been created successfully."}
        else:
            return {"error": "The request payload is not in JSON format"}

    elif request.method == "GET":
        cars = CarsModel.query.all()
        results = [
            {"name": car.name, "model": car.model, "doors": car.doors} for car in cars
        ]

        return {"count": len(results), "cars": results, "message": "success"}


@app.route("/cars/<car_id>", methods=["GET", "PUT", "DELETE"])
def handle_car(car_id):
    car = CarsModel.query.get_or_404(car_id)

    if request.method == "GET":
        response = {"name": car.name, "model": car.model, "doors": car.doors}
        return {"message": "success", "car": response}

    elif request.method == "PUT":
        data = request.get_json()
        car.name = data["name"]
        car.model = data["model"]
        car.doors = data["doors"]

        db.session.add(car)
        db.session.commit()

        return {"message": f"car {car.name} successfully updated"}

    elif request.method == "DELETE":
        db.session.delete(car)
        db.session.commit()

        return {"message": f"Car {car.name} successfully deleted."}


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8000)

这是 Dockerfile:

# Use an official Python runtime as a parent image
FROM python:3.8

# Set the working directory to /app
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY . /app

# Install any needed packages specified in requirements.txt
RUN pip install --trusted-host pypi.python.org -r requirements.txt

ENV FLASK_APP main.py
ENV DATABASE_URL localhost
RUN flask db init
RUN flask db migrate
RUN flask db upgrade
# Make port 80 available to the world outside this container
EXPOSE 8000

# Run app.py when the container launches
CMD ["python", "main.py"]

这是 Pulumi 的 index.ts 文件:

import * as awsx from "@pulumi/awsx";
import * as aws from "@pulumi/aws";
import * as pulumi from "@pulumi/pulumi";

const vpc = new awsx.ec2.Vpc("custom");
// Step 1: Create an ECS Fargate cluster.
const cluster = new awsx.ecs.Cluster("first_cluster", { vpc });

const securityGroupIds = cluster.securityGroups.map(g => g.id);

const dbSubnets = new aws.rds.SubnetGroup("dbsubnets", {
    subnetIds: vpc.publicSubnetIds,
});

const db = new aws.rds.Instance("postgresdb", {
    engine: "postgres",

    instanceClass: "db.t2.micro",
    allocatedStorage: 20,

    dbSubnetGroupName: dbSubnets.id,
    vpcSecurityGroupIds: securityGroupIds,

    name: "dummy",
    username: "dummy",
    password: "123456789",
    publiclyAccessible: true,
    skipFinalSnapshot: true,
});
const hosts = pulumi.all([db.endpoint.apply(e => e)]);
const environment = hosts.apply(([postgresHost]) => [
    { name: "DATABASE_URL", value: postgresHost },
]);

// Step 2: Define the Networking for our service.
const alb = new awsx.elasticloadbalancingv2.ApplicationLoadBalancer(
    "net-lb", { external: true, securityGroups: cluster.securityGroups, vpc });
const atg = alb.createTargetGroup(
    "app-tg", { port: 8000, deregistrationDelay: 0 });
const web = atg.createListener("web", { port: 80, external: true });

// Step 3: Build and publish a Docker image to a private ECR registry.
const img = awsx.ecs.Image.fromPath("app-img", "./app");

// Step 4: Create a Fargate service task that can scale out.
const appService = new awsx.ecs.FargateService("app-svc", {
    cluster,
    taskDefinitionArgs: {
        container: {
            image: img,
            cpu: 102 /*10% of 1024*/,
            memory: 50 /*MB*/,
            portMappings: [web],
            environment: environment,
        },
    },
    desiredCount: 5,
}, { dependsOn: [db] });

// Step 5: Export the Internet address for the service.
export const url = web.endpoint.hostname;

现在，当我执行 pulumi up 时，我得到了这个:

 sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
        Is the server running on host "localhost" (127.0.0.1) and accepting
        TCP/IP connections on port 5432?
    could not connect to server: Cannot assign requested address
        Is the server running on host "localhost" (::1) and accepting
        TCP/IP connections on port 5432?

    (Background on this error at: http://sqlalche.me/e/e3q8)

        at /Users/myuser/projects/practice/pulumi/simple_flask_app/node_modules/@pulumi/docker.ts:546:15
        at Generator.next (<anonymous>)
        at fulfilled (/Users/myuser/projects/practice/pulumi/simple_flask_app/node_modules/@pulumi/docker/docker.js:18:58)
        at processTicksAndRejections (internal/process/task_queues.js:97:5)

    error: The command '/bin/sh -c flask db migrate' returned a non-zero code: 1

现在，我知道这是因为它尝试连接到本地主机，因为这是默认设置，但是如何传入 db 资源的主机名？

谢谢

更新 1:尝试删除 ENV DATABASE_URL localhost

删除 ENV DATABASE_URL localhost 后:

File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 652, in __connect
    connection = pool._invoke_creator(self)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 490, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not translate host name "None" to address: Name or service not known
(Background on this error at: http://sqlalche.me/e/e3q8)

    at /Users/myuser/projects/practice/pulumi/simple_flask_app/node_modules/@pulumi/docker.ts:546:15
    at Generator.next (<anonymous>)
    at fulfilled (/Users/myuser/projects/practice/pulumi/simple_flask_app/node_modules/@pulumi/docker/docker.js:18:58)

Answer 1

我认为这种不好的做法是在 docker 构建期间运行迁移。如果之后构建失败会发生什么？您如何控制哪些更改应用于哪个环境？我认为这个问题有更好的解决方案。
当容器在 fargate 中启动时，也可以应用这些迁移，例如将这些命令放入 entrypoint脚本或在进程启动中执行迁移(基本上在您的 main.py 中)，如下所述:https://flask-migrate.readthedocs.io/en/latest/#command-reference
在 pulumi up 期间不这样做的另一个原因是，这还需要一个防火墙规则，允许您的本地计算机访问数据库(尽管您的 publiclyAccessible 设置可能已经“解决”了)。
如果您仍想在构建中保留此操作，则需要以不同的方式将数据库 url 提供给第 3 步。 env 仅在第 4 步(设置 fargate)期间使用。对于第 3 步，您可以利用构建参数 ( https://docs.docker.com/engine/reference/builder/#arg ) 并通过 pulumi 传递它们，就像 https://www.pulumi.com/docs/reference/pkg/docker/image/#dockerbuild
请记住，这会增加一些安全问题，因为您将数据库向公众开放，否则这将是不必要的。所以我肯定会采用上述不同的方法。