Curl 不会在 HTTP POST 请求中发送整个表单数据

Question

编辑:

问题:2 和问题:3 通过以下@melpomene 评论解决，即通过使用读取的字节数来打印缓冲区。

但仍然对问题:1感到震惊。

---

我写了一个 TCP 服务器-客户端程序。后来出于好奇，想了解一下HTTP服务器。

我之前的问题:Simple TCP server can't output to web browser

现在，我只是通过使用 GET 和 POST( form-data 和x-www-form-urlencoded 现在)。

我正在关注 How to cURL POST from the Command Line发送 POST 请求。

当我将 x-www-form-urlencoded 发送为:

curl -d "data=example1&data2=example2" localhost:8080

服务器输出:

POST / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.54.0
Accept: */*
Content-Length: 28
Content-Type: application/x-www-form-urlencoded

data=example1&data2=example2

这符合预期。

---

问题:1

问题来了。当我尝试发送 form-data 时，输出不是预期的。

当我发送 form-data 时:

curl -X POST -F "name=user" -F "password=test" localhost:8080

服务器输出:

POST / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.54.0
Accept: */*
Content-Length: 244
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------78b7f8917ad1992c

我得到了边界，但我没有得到下一部分，比如我正在发送的数据。

问题:2

一件更奇怪的事情是当我尝试在发送 form-data 之后发送 x-www-form-urlencoded 时。

当我在 form-data 之后发送 x-www-form-urlencoded 时:

curl -d "data=example1&data2=example2" localhost:8080

服务器输出:

POST / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.54.0
Accept: */*
Content-Length: 28
Content-Type: application/x-www-form-urlencoded

data=example1&data2=example2------------78b7f8917ad1992c

为什么我在这里得到边界？

问题:3

同时发送 GET 为:

curl localhost:8080

服务器输出:

GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.54.0
Accept: */*

ontent-Length: 28
Content-Type: application/x-www-form-urlencoded

data=example1&data2=example2------------78b7f8917ad1992c

我正在获取 Content-Type 和 x-www-form-urlencoded 数据以及 boundary。

我做错了什么？我的代码或我的理解有问题吗？

Server.c:

// Server side C program to demonstrate Socket programming
#include <stdio.h>
#include <sys/socket.h>
#include <unistd.h>
#include <stdlib.h>
#include <netinet/in.h>
#include <string.h>

#define PORT 8080
int main(int argc, char const *argv[])
{
    int server_fd, new_socket; long valread;
    struct sockaddr_in address;
    int addrlen = sizeof(address);
    char buffer[1024] = {0};
    char *hello = "HTTP/1.1 200 OK\nContent-Type: text/plain\nContent-Length: 12\n\nHello world!";

    // Creating socket file descriptor
    if ((server_fd = socket(AF_INET, SOCK_STREAM, 0)) == 0)
    {
        perror("In socket");
        exit(EXIT_FAILURE);
    }


    address.sin_family = AF_INET;
    address.sin_addr.s_addr = INADDR_ANY;
    address.sin_port = htons( PORT );


    if (bind(server_fd, (struct sockaddr *)&address, sizeof(address))<0)
    {
        perror("In bind");
        exit(EXIT_FAILURE);
    }
    if (listen(server_fd, 10) < 0)
    {
        perror("In listen");
        exit(EXIT_FAILURE);
    }
    while(1)
    {
        printf("\n+++++++ Waiting for new connection ++++++++\n\n");
        if ((new_socket = accept(server_fd, (struct sockaddr *)&address, (socklen_t*)&addrlen))<0)
        {
            perror("In accept");
            exit(EXIT_FAILURE);
        }
        valread = read( new_socket , buffer, 1024);
        printf("%s\n",buffer );
        write(new_socket , hello , strlen(hello));
        printf("------------------Hello message sent-------------------\n");
        close(new_socket);
    }
    return 0;
}

Answer 1

问题一

我检查了请求 header 并找到了 Expect: 100-continue header 。这是我第一次看到这个标题。

在 Google 中的简单搜索表明这是导致问题的原因。

Expect: 100-Continue' Issues and Risks (我只是要粘贴所有内容以避免死链接)

How the Expect: 100-Continue Header Works

When Expect: 100-Continue is NOT present, HTTP follows approximately the following flow (from the client's point of view):

1. The request initiates a TCP connection to the server.

2. When the connection to the server is established, the full request--which includes both the request headers and the request body--is transmitted to the server.

3. The client waits for a response from the server (comprised of response headers and a response body).

4. If HTTP keep-alives are supported, the request is optionally repeated from step 2.

When the client is using the Expect: 100-Continue feature, the following events occur:

1. The request initiates a TCP connection to the server.

2. When the connection to the server is established, the request--including the headers, the Expect: 100-Continue header, without the request body--is then transmitted to the server.

3. The client then waits for a response from the server. If the status code is a final status code, using the prior steps above the client retries the request without Expect: 100-Continue header. If the status code is 100-Continue, the request body is sent to the server.

4. The client will then wait for a response from the server (comprised of response headers and a response body).

5. If HTTP keep-alives are supported, the request is optionally repeated from step 2.

Why use Expect: 100-Continue?

API POST requests that include the Expect: 100-Continue header save bandwidth between the client and the server, because the server can reject the API request before the request body is even transmitted. For API POST requests with very large request bodies (such as file uploads), the server can, for example, check for invalid authentication and reject the request before the push body was sent, resulting in significant bandwidth savings.

Without Expect: 100-Continue:

Without the Expect: 100-Continue feature, the entire API request, including the (potentially large) push body would have to be transmitted before the server could even determine if the syntax or authentication is valid. However, since the majority of our API requests have small POST bodies, the benefits of separating the request header from the request body is negligible.

Problems when the request header and body are sent separately

Because of the high volume of requests that Urban Airship handles, many levels of complexity exist between our clients and the servers responsible for responding to API requests. This is not an abnormal phenomenon for most server configurations and strategies, but it does introduce a risk of elevated request failures to any API POST requests using the Expect: 100-Continue header. This is due to the fact that the request header and the request body are sent separately from one another, and must travel through the same connection throughout the entire API server infrastructure.

With the number of proxies, load-balancing servers, and back-end request processing servers that are implemented, requests with the Expect: 100-Continue header have an increased probability of becoming separated from one another, and hence returning with an error.

What To Expect:

We've always attempted to support Expect: 100-Continue. However, we have determined that our customers that use Expect: 100-Continue are receiving a sub-optimal quality of service due to elevated request failures.

Additionally, the majority of our API requests have small POST bodies, and as a result the benefits of separating the request header from the request body are negligible. These reasons have motivated us to disable support for Expect: 100-Continue service-wide.

Our Recommendations:

We recommend against the use of Expect: 100-Continue. If you receive an HTTP Error 417 (Expectation failed), retry the request without Expect: 100-Continue.

因此，要防止 Expect: 100-continue header 出现在 POST form-data 中，请包含 -H 'Expect:' 在你的 curl 中

curl -X POST -F "name=user" -F "password=test" localhost:8080 -H 'Expect:'

现在您可以像您在评论中所说的那样一次性接收所有数据(就像 Postman 一样)。

---

问题 2 和 3

正如@melpomene 在评论中所说，read() 不会在阅读后放置\0。这就是您看到以前请求的数据的原因。

因此，只需使用 valread 遍历要打印的字符串，或者像我在评论中所说的那样在 while 循环中声明变量。

代码:

while(1)
    {
        printf("\n+++++++ Waiting for new connection ++++++++\n\n");
        if ((new_socket = accept(server_fd, (struct sockaddr *)&address, (socklen_t*)&addrlen))<0)
        {
            perror("In accept");
            exit(EXIT_FAILURE);
        }

        char buffer[30000] = {0};  // This way you get new variable everytime. So, there is no need to iterate over the string using valread value.
        valread = read( new_socket , buffer, 30000);
        printf("%s\n",buffer );
        write(new_socket , hello , strlen(hello));
        printf("------------------Hello message sent-------------------%lu\n", valread);
        close(new_socket);
    }