node.js - 在 AWS Ubuntu 实例中为 NodeJS 部署设置 GitLab CI

Question

我有一个配置了 GitLab CE 的 AWS Ubuntu 实例。现在我想配置 GitLab CI 以在每次提交后部署我的 NodeJS 应用程序。我没有任何适当的分步解决方案。

我的 NodeJS 应用程序在 /var/www/mean/my-app 中运行在 http://myapp.mydomain.com并且托管由 Apache 代理处理，

<VirtualHost *:80>
    ServerAdmin anshad@mydomain.com
    ServerName gitlab.mydomain.com
    ServerAlias www.gitlab.mydomain.com

    ServerSignature Off

    ProxyPreserveHost On

    AllowEncodedSlashes NoDecode

    <Location />
       Require all granted
       ProxyPassReverse http://localhost:8080
       ProxyPassReverse http://gitlab.mydomain.com/
    </Location>

    RewriteEngine on

    RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f [OR]
    RewriteCond %{REQUEST_URI} ^/uploads/.*
    RewriteRule .* http://127.0.0.1:8080%{REQUEST_URI} [P,QSA,NE]

    DocumentRoot /home/git/gitlab/public

    LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
    ErrorLog /var/log/apache2/gitlab_error.log
    CustomLog /var/log/apache2/gitlab_forwarded.log common_forwarded
    CustomLog /var/log/apache2/gitlab_access.log combined env=!dontlog
    CustomLog /var/log/apache2/gitlab.log combined
 </VirtualHost>

该应用程序是使用永久模块引导的

forever start app.js

gitlab 配置检查 sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production给，

Checking GitLab Shell ...

GitLab Shell version >= 4.0.0 ? ... OK (4.0.0)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ... 
dev / my-app ... ok
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /home/git/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Reply by email ...

Reply by email is disabled in config/gitlab.yml

Checking Reply by email ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... yes
Init script exists? ... yes
Init script up-to-date? ... yes
projects have namespace: ... 
dev / my-app ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.3.1)
Your git bin path is "/usr/bin/git"
Git version >= 2.7.3 ? ... yes (2.7.4)
Active users: 1

Checking GitLab ... Finished

我曾经使用 SSH 从我的系统登录到实例，

ssh -i API-Key.pem ubuntu@ec2-XX-XX-XXX-XXX.ap-south-1.compute.amazonaws.com

使用命令创建 key

ssh-keygen -t rsa

运行器配置 /etc/gitlab-runner/config.toml

concurrent = 1
check_interval = 0

[[runners]]
  name = "Production Runner"
  url = "http://gitlab.mydomain.com/ci"
  token = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
  executor = "ssh"
  [runners.ssh]
    user = "ubuntu"
    host = "ip-XXX-XX-XX-XXX"
    identity_file = "/home/ubuntu/.ssh/id_rsa"
  [runners.cache]

代码在 .gitlab-ci.yml

test_async:   
 script:    
  - npm install

由于我的错误配置，运行者给出了错误，

Running with gitlab-ci-multi-runner 1.7.1 (f896af7)
Using SSH executor...
ERROR: Preparation failed: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Will be retried in 3s ...

我的困惑是:
.gitlab-ci.yml的内容应该是什么文件将提交的代码部署到应用程序位置？

以及如何为此配置运行者？如果我必须使用 ssh runner，那边应该是什么配置？

更新:

提供后 .pem文件为 identity_file ，得到以下错误

Running with gitlab-ci-multi-runner 1.7.1 (f896af7)
Using SSH executor...
Running on ip-xxx-xx-xx-xxx via ip-xxx-xx-xx-xxx...
Cloning repository...
Cloning into 'builds/a92f1b91/0/dev/my-app'...
fatal: unable to access 'http://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab.mydomain.com/dev/my-app.git/': The requested URL returned error: 500
ERROR: Build failed: Process exited with: 1. Reason was:  ()

现在有一个问题，git clone 与 http 不工作，但 ssh 克隆工作。

注: gitlab 和构建环境都是同一个主机(同一个 aws 实例)

Bug GitLab 中也有报告(http 克隆问题)。

Answer 1

在你的/etc/gitlab-runner/config.toml

concurrent = 1
check_interval = 0

[[runners]]
  name = "Production Runner"
  url = "http://gitlab.mydomain.com/ci"
  token = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
  executor = "ssh"
  [runners.ssh]
    user = "ubuntu"
    host = "ip-XXX-XX-XX-XXX"
    identity_file = "/home/ubuntu/.ssh/id_rsa"
  [runners.cache]

你定义

主持人

用户

和身份文件

主机应该是您的构建主机 IP(换句话说，您将在何处执行构建)
user 应该是您在 上的用户搭建主机 . 不在 gitlab 主机上 .
您可以通过以下方式测试您的密码 less ssh 是如何工作的

以 root 身份登录到 gitlab 主机

ssh -i/home/ubuntu/.ssh/id_rsa ubuntu@ip-XXX-XX-XX-XXX

如果这有效并且不要求您输入密码 - 一切都很好。
如果这中断 - 意味着您没有正确设置密码 less auth 。
设置基于无密码公钥的身份验证的最简单方法是使用名为的命令

ssh-copy-id

例如，我想在我的 gitlab 和我的构建主机之间设置无密码的 ssh 身份验证。
我的构建主机 ip 是 192.168.0.42，主机名是 build.home
我已经在 gitlab 主机上的/home/ubuntu/.ssh 下生成了我的 id_rsa 和 id_rsa.pub 。
现在让我们将公钥从 gitlab 主机推送到我们的构建主机。第一次它会要求你输入密码。

[root@gitlab ~]# ssh-copy-id -i /home/ubuntu/.ssh/id_rsa.pub ubuntu@build.home
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
ubuntu@build.home's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'ubuntu@build.home'"
and check to make sure that only the key(s) you wanted were added.  

请注意，在上面的示例中，我将公钥推送到远程主机。
但是当我现在连接到这个远程主机时 - 我将指定我的私钥。

[root@gitlab ~]# ssh -i /home/ubuntu/.ssh/id_rsa ubuntu@build.home
[ubuntu@build ~]$ hostname
build.home

尝试在 gitlab 主机和远程主机之间测试您的公钥身份验证并更新您的问题。
资源:

https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/executors/ssh.md
P.S:我会在今天晚些时候回家时发布我的工作环境配置。

编辑 1:
这是我的配置。
我的 gitlab 主机名为 gitlab.home 192.168.0.41
我有另一个名为 sshbuild.home 192.168.0.43 的虚拟机
下面是我如何添加 ssh runner
步骤 1. 在我的 gitlab.home 上安装 yum install gitlab-ci-multi-runner并将我的远程 sshbuild.home VM 注册为 ssh runner


我还需要确保在我的 gitlab.home 和 sshbuild.home 之间使用无密码身份验证，所以

[root@gitlab gitlab-runner]# ssh-copy-id 192.168.0.43
The authenticity of host '192.168.0.43 (192.168.0.43)' can't be established.
ECDSA key fingerprint is b4:6a:1b:72:d1:7d:1f:34:f7:bb:ef:ad:69:42:11:13.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.0.43's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '192.168.0.43'"
and check to make sure that only the key(s) you wanted were added.

[root@gitlab gitlab-runner]# ssh 192.168.0.43
Last login: Fri Nov 18 17:05:06 2016 from 192.168.0.101
[root@sshbuild ~]# exit

然后我禁用了我的另一个运行器(shell 运行器)并创建了特定的新 SSH 运行器项目，以确保当我提交时 - 它会在 ssh 运行器上执行

我确实提交了，瞧 - 我们在 sshbuild.home 主机上运行了成功的测试

这里有几个链接可能有助于更好地理解这个主题

https://about.gitlab.com/2016/07/29/the-basics-of-gitlab-ci/

https://docs.gitlab.com/ce/ci/runners/README.html

http://docs.gitlab.com/runner/commands/README.html

https://docs.gitlab.com/ce/ci/yaml/README.html

P.S:这是我的/etc/gitlab-runner/config.toml 文件

[root@gitlab gitlab-runner]# cat /etc/gitlab-runner/config.toml 
concurrent = 1
check_interval = 0

[[runners]]
  name = "sshbuild"
  url = "http://gitlab.home/"
  token = "2bc1825d8fbde09fd632637c60e9e7"
  executor = "ssh"
  [runners.ssh]
    user = "root"
    host = "192.168.0.43"
    port = "22"
    identity_file = "/root/.ssh/id_rsa"
  [runners.cache]

P.S:如果我在 Web 界面的“设置”下为我的存储库禁用 HTTP，我会遇到与您类似的错误。但是错误不是 500，而是 403。

编辑 2:
现在我将基于一个简单的 HelloWorld 项目介绍 .gitlab-ci.yml
在我的 HelloWorld 中，我有一个名为 server.js 的文件，当它从 Node 运行时 - 将简单地创建在端口 3000 上运行的 Web 服务器，并将在 GET 请求上回复“Hello World”。

 1  const http = require('http');
 2  
 3  const hostname = '0.0.0.0';
 4  const port = 3000;
 5  
 6  const server = http.createServer((req, res) => {
 7    res.statusCode = 200;
 8    res.setHeader('Content-Type', 'text/plain');
 9    res.end('Hello World!\n');
10  });
11  
12  server.listen(port, hostname, () => {
13    console.log(`Server running at http://${hostname}:${port}/`);
14  });

我的目标是能够针对它运行测试用例。在这种情况下，我将运行简单

curl localhost:3000 | grep "Hello World" 

但是我需要将它放入一个单独的脚本中，该脚本成功时退出状态为 0，失败时退出状态为非零

cat -n simpletest.sh 
     1  #!/bin/bash
     2  
     3  cleanup () 
     4  {
     5   count=`netstat -anp|grep ":3000"|grep LISTEN|awk '{print $NF}'|cut -d\/ -f1|wc -l`
     6   if [ $count -ne 0 ]
     7    then 
     8     pid=`netstat -anp|grep ":3000"|grep LISTEN|awk '{print $NF}'|cut -d\/ -f1`; 
     9     echo "Need to kill PID $pid"; 
    10     kill $pid
    11   fi
    12  }
    13  
    14  echo "Running simple test"
    15  curl localhost:3000|grep "Hello World"
    16  if [ $? -eq 0 ]
    17   then
    18    echo "Test was successfull"
    19    echo "Clean up node.js process"
    20    cleanup
    21    exit 0
    22   else
    23    echo "Test failed"
    24    echo "Clean up node.js process"
    25    cleanup
    26    exit 1
    27  fi

现在让我们介绍我的 .gitlab-ci.yml

cat -n .gitlab-ci.yml
     1  test:
     2  
     3    before_script:
     4    - echo "Before script"
     5    - hostname
     6    - /bin/bash cleanup.sh
     7  
     8    script: 
     9    - echo "Main Script"
    10    - node server.js &
    11    - sleep 3
    12    - /bin/bash simpletest.sh

我有一个名为 test.In before_script 的工作，它运行 cleanup.sh 脚本，如果发现这种情况，它会简单地终止监听端口 3000 的 PID。

cat -n cleanup.sh 
     1  #!/bin/bash
     2  count=`netstat -anp|grep ":3000"|grep LISTEN|awk '{print $NF}'|cut -d\/ -f1|wc -l`
     3  if [ $count -ne 0 ]
     4   then 
     5    pid=`netstat -anp|grep ":3000"|grep LISTEN|awk '{print $NF}'|cut -d\/ -f1`; 
     6    echo "Need to kill PID $pid"; 
     7    kill $pid
     8  fi
     9  exit 0

在脚本下:它使用我的 server.js 运行 Node ，给它 3 秒的时间来启动，然后针对它运行测试。该测试还将在测试完成后处理终止 Node PID。
所以让我们提交并检查构建的状态

现在让我们将 server.js 更改为输出的不是“Hello World”而是“HelloWorld”，因此中间没有空格。我希望我的测试用例会失败，因为它期望字面意思是“Hello World”。它确实失败了。

这是我能想到的最简单的 CI 用例。
现在，如果基于测试用例的状态，您想将代码部署到另一个环境 - 您必须开始使用

阶段
和

环境

所以你的 .gitlab-ci.yml 会变成这样(真实的工作示例)

cat -n .gitlab-ci.yml
     1  stages:
     2  - test
     3  - deploy
     4  
     5  run_test_case:
     6   stage: test
     7   before_script:
     8   - echo "Before script"
     9   - hostname
    10   - /bin/bash cleanup.sh
    11  
    12   script: 
    13   - echo "Main Script"
    14   - node server.js &
    15   - sleep 3
    16   - /bin/bash simpletest.sh
    17  
    18  deploy_to_production:
    19   stage: deploy
    20   script: 
    21   - echo "Run code here to do production deployment"
    22   environment:
    23    name: production

git push 会成功。在第 21 行，我只是运行了 echo，但这可以替换为一个脚本，该脚本将推送到您的远程登台或生产环境。