ruby - 从Rails应用程序调用Chef更新命令时出现Encoding::UndefinedConversionError

Question

我正在构建一个使用Chef-DK的Rails应用。

我想要实现的是执行与chef update <path/to/policyfile>相同的代码。

这是代码中有趣的部分:

require 'chef-dk/command/update'

chef_update = ChefDK::Command::Update.new
update_options = [policyfile_path]
update_options << '--debug' if Rails.env.development?
update = chef_update.run(update_options)

运行此命令时，它将开始更新策略文件，但失败，并显示以下输出:

Attributes already up to date
Building policy k8sworker
Expanded run list: recipe[fail2ban], recipe[os-hardening], recipe[rkhunter], recipe[ssh-hardening], recipe[ssh-keys], recipe[tincvpn]
Caching Cookbooks...
Installing tincvpn ~> 0.1.10 from github
Installing fail2ban 6.0.0
Error: Failed to generate Policyfile.lock
Reason: (Encoding::UndefinedConversionError) "\x8B" from ASCII-8BIT to UTF-8
/usr/local/lib/ruby/2.6.0/delegate.rb:349:in `write'
/usr/local/lib/ruby/2.6.0/delegate.rb:349:in `block in delegating_block'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:528:in `block in stream_to_tempfile'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:497:in `call_block'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:488:in `<<'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:163:in `read'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:293:in `block in read_body_0'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:253:in `inflater'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:283:in `read_body_0'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:204:in `read_body'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:527:in `stream_to_tempfile'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:230:in `block in streaming_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http/basic_client.rb:92:in `block in request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1518:in `block in transport_request'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:165:in `reading_body'
/usr/local/lib/ruby/2.6.0/net/http.rb:1517:in `transport_request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1479:in `request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1472:in `block in request'
/usr/local/lib/ruby/2.6.0/net/http.rb:920:in `start'
/usr/local/lib/ruby/2.6.0/net/http.rb:1470:in `request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http/basic_client.rb:69:in `request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:370:in `block in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:411:in `block in retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `loop'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:365:in `send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:388:in `block (2 levels) in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:485:in `follow_redirect'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:383:in `block in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:411:in `block in retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `loop'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:365:in `send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:228:in `streaming_request'
/usr/local/bundle/gems/cookbook-omnifetch-0.8.1/lib/cookbook-omnifetch/artifactserver.rb:42:in `install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile/cookbook_location_specification.rb:81:in `ensure_cached'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:181:in `block in install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:176:in `each'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:176:in `install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_services/install.rb:101:in `generate_lock_and_install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_services/install.rb:63:in `run'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/command/update.rb:93:in `run'
/application/app/interactors/node_actions/run_chef_update.rb:24:in `call'
...

这是带有 trace日志级别的Chef输出:

worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Initiating GET to https://supermarket.chef.io/universe
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: HTTP 1.1 200 OK
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: cache-control: max-age=0, private, must-revalidate
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: content-encoding: gzip
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: content-type: application/json; charset=utf-8
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:19 GMT
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: etag: W/"461b5134d1467f6950f82af1510078a0"
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: server: nginx
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-content-type-options: nosniff
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-frame-options: SAMEORIGIN
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-request-id: 0bc2cc4a-0399-4953-93db-699176e11e36
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-runtime: 0.301233
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-xss-protection: 1; mode=block
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: transfer-encoding: chunked
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: connection: Close
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads.
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Decompressing gzip response
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Initiating GET to https://supermarket.chef.io/api/v1/cookbooks/fail2ban/versions/6.0.0/download
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: HTTP 1.1 302 Found
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: cache-control: no-cache
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-type: text/html; charset=utf-8
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:25 GMT
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: location: https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: server: nginx
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-content-type-options: nosniff
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-frame-options: SAMEORIGIN
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-request-id: 17a0d07d-cf03-45b3-956f-8eb7fcabd2a7
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-runtime: 0.044161
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-xss-protection: 1; mode=block
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-length: 209
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: connection: Close
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Response Body ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: <html><body>You are being <a href="https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785">redirected</a>.</body></html>
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Response Body -----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_stream_complete
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: No content-length information collected for the streamed download, cannot identify streamed download.
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_stream_complete
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_stream_complete
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Following redirect 1/10
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Initiating GET to https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: HTTP 1.1 200 OK
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-amz-id-2: a2iOnve5iX8paa/pT2s+A21Bm8IvSqbjzYeP8mc2I2kNGSwgj9Wen8IyQlpiQ955grAF7xZKaQE=
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-amz-request-id: E74D7A5F59B4B3B0
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:26 GMT
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: last-modified: Wed, 08 May 2019 21:59:47 GMT
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: etag: "5c15105980cf9bfb2467ed00b789be0a"
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: accept-ranges: bytes
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-type: application/gzip
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-length: 9020
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: server: AmazonS3
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: connection: close
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Streaming download from https://supermarket.chef.io/api/v1/cookbooks/fail2ban/versions/6.0.0/download to tempfile /tmp/chef-rest20190703-1-cbdl7x
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content_encoding = ''               initializing noop stream deflator.
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::ValidateContentLength::ContentLengthCounter#handle_chunk
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::Decompressor::NoopInflater#handle_chunk
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::ValidateContentLength::ContentLengthCounter#handle_chunk
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::Decompressor::NoopInflater#handle_chunk

因此，就像从Amazon S3清除fail2ban归档文件一样，由于某些编码问题，它失败了。

从这一点来看:

当我从终端运行chef update命令时，它运行正常，没有错误

当我再次运行我的代码时，它也可以正常工作而没有任何错误(我想是由于厨师缓存了食谱的版本，所以不再经历失败的代码了)

因此，对我来说，为了使该部分正常工作，我的代码中缺少某些内容，但我不知道这可能是什么。

我该如何解决这个问题？

附言:该应用程序在具有Linux Alpine的Docker容器中运行。

Answer 1

Chef库使用Tempfile以便将下载的数据写入某处，因此我对它的write方法进行了别名化，以便在传递的参数上调用force_encoding('UTF-8')，然后再将其返回给原始的write方法。

这是我的替代(在config/initializers/tempfile_override.rb中:

# frozen_string_literal: true

require 'tempfile'

#
# Overrides the Tempfile write method in order to always use `force_encode` with
# UTF-8 in order to solve the issue where `chef update` fails to write
# downloaded cookbooks data and write it to the disk.
# See https://stackoverflow.com/questions/56866746/encodingundefinedconversionerror-when-calling-chef-update-command-from-a-rails
#
class Tempfile
  alias old_write write
  def write(data)
    old_write(data.force_encoding('UTF-8'))
  end
end

我有点被迫这样做，因为我需要更改 Chef::UI对象才能在我的应用程序中获取Chef日志。

那不是很干净，但是我没有更好的方法来解决这个问题，而不必修补厨师本身。

除非找到更好的答案，否则此答案将为公认的答案。

如果您知道更好的方法，请帮助我 here :)