Spring Feign 不压缩响应

Question

我正在使用 spring feign 压缩请求和响应

服务器端:

server:
  servlet:
    context-path: /api/v1/
  compression:
    enabled: true
    min-response-size: 1024

当我从 chrome 访问 api 时，我看到它添加了 'Accept-Encoding': "gzip, deflate, br"
客户端:

    server:
      port: 8192
      servlet:
        context-path: /api/demo



feign.compression.response.enabled: true

feign.client.config.default.loggerLevel: HEADERS

logging.level.com.example.feigndemo.ManagementApiService: DEBUG

eureka:
  client:
    enabled: false

management-api:
  ribbon:
    listOfServers: localhost:8080

当我看到传递的请求 header 时，feign 正在传递两个 header 。

Accept-Encoding: deflate
Accept-Encoding: gzip

gradle 文件

plugins {
        id 'org.springframework.boot' version '2.1.8.RELEASE'
        id 'io.spring.dependency-management' version '1.0.8.RELEASE'
        id 'java'
    }

    group = 'com.example'
    version = '0.0.1-SNAPSHOT'
    sourceCompatibility = '1.8'

    configurations {
        compileOnly {
            extendsFrom annotationProcessor
        }
    }

    repositories {
        mavenCentral()
    }

    ext {
        set('springCloudVersion', "Greenwich.SR2")
    }

    dependencies {
        implementation 'org.springframework.boot:spring-boot-starter-web'
        compile ('org.springframework.cloud:spring-cloud-starter-netflix-ribbon')
        compile('org.springframework.cloud:spring-cloud-starter-openfeign')
    // https://mvnrepository.com/artifact/io.github.openfeign/feign-httpclient
    // https://mvnrepository.com/artifact/io.github.openfeign/feign-httpclient
        //compile group: 'io.github.openfeign', name: 'feign-httpclient', version: '9.5.0'

        compileOnly 'org.projectlombok:lombok'
        annotationProcessor 'org.projectlombok:lombok'
        testImplementation 'org.springframework.boot:spring-boot-starter-test'
    }

    dependencyManagement {
        imports {
            mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
        }
    }

响应未压缩。我所看到的是 Spring feign 将“接受编码”作为两个不同的值发送

让我知道这里是否有问题

Answer 1

几周前我遇到了同样的问题，我开始知道没有富有成效/直接的方法可以做到这一点。我也知道，当@patan 向 spring 社区报告问题时@patan reported issue1和 @patan reported issue2为 tomcat 端创建了一张票来尝试解决该问题 ( issue link )。 Jetty 端也有一张票( ticket link )与此相关。最初，我计划使用 github 中建议的方法。但是后来才知道这个库已经被合并到spring-cloud-openfeign-core了 jar 下 org.springframework.cloud.openfeign.encoding包裹。然而，我们无法按预期实现压缩，并面临以下两个挑战:

当我们通过设置启用伪装压缩时 org.springframework.cloud.openfeign.encoding.FeignAcceptGzipEncodingInterceptor ( code-link ) 类添加了 Accept-Encoding标题为 gzip和 deflate但由于问题( ticket )，tomcat 服务器无法将其解释为压缩信号的标志。作为解决方案，我们必须添加手动 Feign 解释器来覆盖FeignAcceptGzipEncodingInterceptor功能并连接标题。

Feign 的默认压缩设置在最简单的场景中完美地工作，但是当出现以下情况时 Client calling microservice and that microservice calling another microservice through feign然后 feign 无法处理压缩的响应，因为 Spring cloud open feign 解码器默认不解压响应( default spring open feign decoder )，最终以问题( issue link )结束。所以我们必须自己编写解码器来实现解压。

我终于找到了基于各种 的解决方案可用资源所以只需按照 Spring 假装压缩的步骤:
application.yml

spring:
  http:
    encoding:
      enabled: true

#to enable server side compression
server:
  compression:
    enabled: true
    mime-types:
      - application/json
    min-response-size: 2048

#to enable feign side request/response compression
feign:
  httpclient:
    enabled: true
  compression:
    request:
      enabled: true
      mime-types:
        - application/json
      min-request-size: 2048
    response:
      enabled: true

注意 : 上面的feign配置我默认启用对所有feign客户端的压缩。
CustomFeignDecoder

import feign.Response;
import feign.Util;
import feign.codec.Decoder;
import org.springframework.cloud.openfeign.encoding.HttpEncoding;

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.lang.reflect.Type;
import java.nio.charset.StandardCharsets;
import java.util.Collection;
import java.util.Objects;
import java.util.zip.GZIPInputStream;

public class CustomGZIPResponseDecoder implements Decoder {

    final Decoder delegate;

    public CustomGZIPResponseDecoder(Decoder delegate) {
        Objects.requireNonNull(delegate, "Decoder must not be null. ");
        this.delegate = delegate;
    }

    @Override
    public Object decode(Response response, Type type) throws IOException {
        Collection<String> values = response.headers().get(HttpEncoding.CONTENT_ENCODING_HEADER);
        if(Objects.nonNull(values) && !values.isEmpty() && values.contains(HttpEncoding.GZIP_ENCODING)){
            byte[] compressed = Util.toByteArray(response.body().asInputStream());
            if ((compressed == null) || (compressed.length == 0)) {
               return delegate.decode(response, type);
            }
            //decompression part
            //after decompress we are delegating the decompressed response to default 
            //decoder
            if (isCompressed(compressed)) {
                final StringBuilder output = new StringBuilder();
                final GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(compressed));
                final BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8));
                String line;
                while ((line = bufferedReader.readLine()) != null) {
                    output.append(line);
                }
                Response uncompressedResponse = response.toBuilder().body(output.toString().getBytes()).build();
                return delegate.decode(uncompressedResponse, type);
            }else{
                return delegate.decode(response, type);
            }
        }else{
            return delegate.decode(response, type);
        }
    }

    private static boolean isCompressed(final byte[] compressed) {
        return (compressed[0] == (byte) (GZIPInputStream.GZIP_MAGIC)) && (compressed[1] == (byte) (GZIPInputStream.GZIP_MAGIC >> 8));
    }
}

Feign自定义配置

import feign.RequestInterceptor;
import feign.RequestTemplate;
import feign.optionals.OptionalDecoder;
import org.springframework.beans.factory.ObjectFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.http.HttpMessageConverters;
import org.springframework.cloud.openfeign.support.ResponseEntityDecoder;
import org.springframework.cloud.openfeign.support.SpringDecoder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class CustomFeignConfiguration {


    @Autowired
    private ObjectFactory<HttpMessageConverters> messageConverters;

    //concatenating headers because of https://github.com/spring-projects/spring-boot/issues/18176
    @Bean
    public RequestInterceptor gzipInterceptor() {
        return new RequestInterceptor() {
            @Override
            public void apply(RequestTemplate template) {
                template.header("Accept-Encoding", "gzip, deflate");
            }
        };
    }

    @Bean
    public CustomGZIPResponseDecoder customGZIPResponseDecoder() {
        OptionalDecoder feignDecoder = new OptionalDecoder(new ResponseEntityDecoder(new SpringDecoder(this.messageConverters)));
        return new CustomGZIPResponseDecoder(feignDecoder);
    }
}

其他提示
如果您打算仅使用 feign-core 构建 CustomDecoder图书馆

import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.type.TypeFactory;
import feign.Response;
import feign.Util;
import feign.codec.DecodeException;
import feign.codec.Decoder;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.ClientHttpResponse;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.util.StringUtils;
import org.springframework.web.client.HttpMessageConverterExtractor;

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;
import java.lang.reflect.WildcardType;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.LinkedList;
import java.util.Map;
import java.util.Objects;
import java.util.zip.GZIPInputStream;

import static java.util.zip.GZIPInputStream.GZIP_MAGIC;

public class CustomGZIPResponseDecoder implements Decoder {

    private final Decoder delegate;

    public CustomGZIPResponseDecoder(Decoder delegate) {
        Objects.requireNonNull(delegate, "Decoder must not be null. ");
        this.delegate = delegate;
    }

    @Override
    public Object decode(Response response, Type type) throws IOException {
        Collection<String> values = response.headers().get("Content-Encoding");
        if (Objects.nonNull(values) && !values.isEmpty() && values.contains("gzip")) {
            byte[] compressed = Util.toByteArray(response.body().asInputStream());
            if ((compressed == null) || (compressed.length == 0)) {
                return delegate.decode(response, type);
            }
            if (isCompressed(compressed)) {
                Response uncompressedResponse = getDecompressedResponse(response, compressed);
                return getObject(type, uncompressedResponse);
            } else {
                return getObject(type, response);
            }
        } else {
            return getObject(type, response);
        }
    }

    private Object getObject(Type type, Response response) throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        if (response.status() == 404 || response.status() == 204)
            return Util.emptyValueOf(type);
        if (Objects.isNull(response.body()))
            return null;
        if (byte[].class.equals(type))
            return Util.toByteArray(response.body().asInputStream());
        if (isParameterizeHttpEntity(type)) {
            type = ((ParameterizedType) type).getActualTypeArguments()[0];
            if (type instanceof Class || type instanceof ParameterizedType
                    || type instanceof WildcardType) {
                @SuppressWarnings({"unchecked", "rawtypes"})
                HttpMessageConverterExtractor<?> extractor = new HttpMessageConverterExtractor(
                        type, Collections.singletonList(new MappingJackson2HttpMessageConverter(mapper)));
                Object decodedObject = extractor.extractData(new FeignResponseAdapter(response));
                return createResponse(decodedObject, response);
            }
            throw new DecodeException(HttpStatus.INTERNAL_SERVER_ERROR.value(),
                    "type is not an instance of Class or ParameterizedType: " + type);
        } else if (isHttpEntity(type)) {
            return delegate.decode(response, type);
        } else if (String.class.equals(type)) {
            String responseValue = Util.toString(response.body().asReader());
            return StringUtils.isEmpty(responseValue) ? Util.emptyValueOf(type) : responseValue;
        } else {
            String s = Util.toString(response.body().asReader());
            JavaType javaType = TypeFactory.defaultInstance().constructType(type);
            return !StringUtils.isEmpty(s) ? mapper.readValue(s, javaType) : Util.emptyValueOf(type);
        }
    }

    public static boolean isCompressed(final byte[] compressed) {
        return (compressed[0] == (byte) (GZIP_MAGIC)) && (compressed[1] == (byte) (GZIP_MAGIC >> 8));
    }

    public static Response getDecompressedResponse(Response response, byte[] compressed) throws IOException {
        final StringBuilder output = new StringBuilder();
        final GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(compressed));
        final BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8));
        String line;
        while ((line = bufferedReader.readLine()) != null) {
            output.append(line);
        }
        return response.toBuilder().body(output.toString().getBytes()).build();
    }

    public static String getDecompressedResponseAsString(byte[] compressed) throws IOException {
        final StringBuilder output = new StringBuilder();
        final GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(compressed));
        final BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8));
        String line;
        while ((line = bufferedReader.readLine()) != null) {
            output.append(line);
        }
        return output.toString();
    }

    private boolean isParameterizeHttpEntity(Type type) {
        if (type instanceof ParameterizedType) {
            return isHttpEntity(((ParameterizedType) type).getRawType());
        }
        return false;
    }

    private boolean isHttpEntity(Type type) {
        if (type instanceof Class) {
            Class c = (Class) type;
            return HttpEntity.class.isAssignableFrom(c);
        }
        return false;
    }

    private <T> ResponseEntity<T> createResponse(Object instance, Response response) {

        MultiValueMap<String, String> headers = new LinkedMultiValueMap<>();
        for (String key : response.headers().keySet()) {
            headers.put(key, new LinkedList<>(response.headers().get(key)));
        }

        return new ResponseEntity<>((T) instance, headers, HttpStatus.valueOf(response
                .status()));
    }

    private class FeignResponseAdapter implements ClientHttpResponse {

        private final Response response;

        private FeignResponseAdapter(Response response) {
            this.response = response;
        }

        @Override
        public HttpStatus getStatusCode() throws IOException {
            return HttpStatus.valueOf(this.response.status());
        }

        @Override
        public int getRawStatusCode() throws IOException {
            return this.response.status();
        }

        @Override
        public String getStatusText() throws IOException {
            return this.response.reason();
        }

        @Override
        public void close() {
            try {
                this.response.body().close();
            } catch (IOException ex) {
                // Ignore exception on close...
            }
        }

        @Override
        public InputStream getBody() throws IOException {
            return this.response.body().asInputStream();
        }

        @Override
        public HttpHeaders getHeaders() {
            return getHttpHeaders(this.response.headers());
        }

        private HttpHeaders getHttpHeaders(Map<String, Collection<String>> headers) {
            HttpHeaders httpHeaders = new HttpHeaders();
            for (Map.Entry<String, Collection<String>> entry : headers.entrySet()) {
                httpHeaders.put(entry.getKey(), new ArrayList<>(entry.getValue()));
            }
            return httpHeaders;
        }
    }

}

如果您打算构建自己的 Feign 构建器，那么您可以像下面这样配置

 Feign.builder().decoder(new CustomGZIPResponseDecoder(new feign.optionals.OptionalDecoder(new feign.codec.StringDecoder())))
                 .target(SomeFeignClient.class, "someurl");

更新上述答案:
如果您计划更新 spring-cloud-openfeign-core 的依赖版本至 'org.springframework.cloud:spring-cloud-openfeign-core:2.2.5.RELEASE'然后意识到 FeignContentGzipEncodingAutoConfiguration class 中的以下变化.
在 FeignContentGzipEncodingAutoConfiguration类的签名 ConditionalOnProperty注释从 @ConditionalOnProperty("feign.compression.request.enabled", matchIfMissing = false)至 @ConditionalOnProperty(value = "feign.compression.request.enabled") , 所以默认情况下 FeignContentGzipEncodingInterceptor如果您有应用程序属性 feign.request.compression=true，bean 将被注入(inject)到 spring 容器中如果超出默认/配置的大小限制，则在您的环境中压缩请求正文。如果您的服务器没有处理压缩请求的机制，这会导致问题，在这种情况下，添加/修改属性为 feign.request.compression=false