net/http: support streaming request body in wasm

[mihaiav]

Go version

go1.22.5 darwin/arm64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/mihai/Library/Caches/go-build'
GOENV='/Users/mihai/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/mihai/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/mihai/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.5'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/gs/rghtsvt5531bcqtkx6zlphc40000gn/T/go-build4222966030=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

I've tried to upload a large file (1.2 GB) to a remote server. Obviously I've posted the body as an io.Reader.

What did you see happen?

I've got an out of memory error:
runtime: out of memory: cannot allocate 918552576-byte block (3715989504 in use) [wasm_exec.js:22:14](https://hopz.com/static/wasm/wasm_exec.js) fatal error: out of memory

After some investigation I've found this line which seems to read the whole body before posting it.
https://github.com/golang/go/blob/master/src/net/http/roundtrip_js.go#L116

What did you expect to see?

I expected to be able to post large file using the io.Reader supported by the net/http package.

[gabyhelp]

Related Issues and Documentation

• net/http: HTTP large file upload fails with "bufio: buffer full" #26707 (closed)
• net/http,mime/multipart: Post big file by http client #28774 (closed)
• net/http: request.Body.Close() can be called after RoundTrip() returns #58521 (closed)
• net/http: http.MaxBytesReader has a bug #14981 (closed)
• net/http: MaxBytesReader's behavior does not match the comments #10884 (closed)
• net/http: add MaxBytesError #30715 (closed)
• net/http: improve performance for parsePostForm #14655 (closed)
• net/http: Transport hangs when io.Pipe is used as request body #29246 (closed)
• http: unexpected behavior of http.Server with http.MaxBytesReader and streaming response #46826 (closed)
• net/http: request canceled (Client.Timeout exceeded while reading body) #37916 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

this was previously implemented as CL 458395 and reverted for #61889

[ianlancetaylor]

CC @golang/wasm