Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ngtcp2 use-after-free crash on HTTP/3 connection failure #5565

Closed
Lekensteyn opened this issue Jun 14, 2020 · 1 comment
Closed

ngtcp2 use-after-free crash on HTTP/3 connection failure #5565

Lekensteyn opened this issue Jun 14, 2020 · 1 comment
Assignees
@bagder
Labels
crash HTTP/3 h3 or quic related

Comments

@Lekensteyn
Copy link
Contributor

I did this

$ src/curl https://blog.cloudflare.com --http3 -v
*   Trying 104.18.26.46:443...
* Connect socket 3 over QUIC to 104.18.26.46:443
* connect to 104.18.26.46 port 443 failed: Resource temporarily unavailable
*   Trying 104.18.27.46:443...
* Connect socket 6 over QUIC to 104.18.27.46:443
=================================================================
==1307808==ERROR: AddressSanitizer: heap-use-after-free on address 0x623000008a58 at pc 0x7fe07a03af2e bp 0x7ffe25912cd0 sp 0x7ffe25912cc0
READ of size 8 at 0x623000008a58 thread T0
    #0 0x7fe07a03af2d in SSL_free ../../openssl/ssl/ssl_lib.c:1150
    #1 0x7fe07a7ce4b7 in quic_init_ssl lib/vquic/ngtcp2.c:345
    #2 0x7fe07a7cfb7b in Curl_quic_connect lib/vquic/ngtcp2.c:803
    #3 0x7fe07a590cda in singleipconnect lib/connect.c:1261
    #4 0x7fe07a589862 in trynextip lib/connect.c:594
    #5 0x7fe07a58ed4e in Curl_is_connected lib/connect.c:974
    #6 0x7fe07a6c0288 in multi_runsingle lib/multi.c:1818
    #7 0x7fe07a6c4a39 in curl_multi_perform lib/multi.c:2405
    #8 0x7fe07a5c1fe4 in easy_transfer lib/easy.c:603
    #9 0x7fe07a5c28c7 in easy_perform lib/easy.c:693
    #10 0x7fe07a5c299f in curl_easy_perform lib/easy.c:712
    #11 0x55fceb598a9f in serial_transfers src/tool_operate.c:2291
    #12 0x55fceb599f5c in run_all_transfers src/tool_operate.c:2475
    #13 0x55fceb59a8f7 in operate src/tool_operate.c:2590
    #14 0x55fceb578153 in main src/tool_main.c:323
    #15 0x7fe078cd9022 in __libc_start_main (/usr/lib/libc.so.6+0x27022)
    #16 0x55fceb535a6d in _start (/tmp/curl-ngtcp2/src/curl+0xd1a6d)

0x623000008a58 is located 6488 bytes inside of 6552-byte region [0x623000007100,0x623000008a98)
freed by thread T0 here:
    #0 0x7fe07ae8d720 in __interceptor_free /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cc:122
    #1 0x7fe079c594d7 in CRYPTO_free ../../openssl/crypto/mem.c:312
    #2 0x7fe07a03bcb5 in SSL_free ../../openssl/ssl/ssl_lib.c:1247
    #3 0x7fe07a7d0bc9 in qs_disconnect lib/vquic/ngtcp2.c:888
    #4 0x7fe07a7d8f10 in Curl_quic_is_connected lib/vquic/ngtcp2.c:1698
    #5 0x7fe07a58c913 in Curl_is_connected lib/connect.c:865
    #6 0x7fe07a6c0288 in multi_runsingle lib/multi.c:1818
    #7 0x7fe07a6c4a39 in curl_multi_perform lib/multi.c:2405
    #8 0x7fe07a5c1fe4 in easy_transfer lib/easy.c:603
    #9 0x7fe07a5c28c7 in easy_perform lib/easy.c:693
    #10 0x7fe07a5c299f in curl_easy_perform lib/easy.c:712
    #11 0x55fceb598a9f in serial_transfers src/tool_operate.c:2291
    #12 0x55fceb599f5c in run_all_transfers src/tool_operate.c:2475
    #13 0x55fceb59a8f7 in operate src/tool_operate.c:2590
    #14 0x55fceb578153 in main src/tool_main.c:323
    #15 0x7fe078cd9022 in __libc_start_main (/usr/lib/libc.so.6+0x27022)

previously allocated by thread T0 here:
    #0 0x7fe07ae8db3a in __interceptor_malloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0x7fe079c59286 in CRYPTO_malloc ../../openssl/crypto/mem.c:222
    #2 0x7fe079c592b9 in CRYPTO_zalloc ../../openssl/crypto/mem.c:230
    #3 0x7fe07a03741f in SSL_new ../../openssl/ssl/ssl_lib.c:691
    #4 0x7fe07a7ce53c in quic_init_ssl lib/vquic/ngtcp2.c:347
    #5 0x7fe07a7cfb7b in Curl_quic_connect lib/vquic/ngtcp2.c:803
    #6 0x7fe07a590cda in singleipconnect lib/connect.c:1261
    #7 0x7fe07a591d12 in Curl_connecthost lib/connect.c:1349
    #8 0x7fe07a628c3c in Curl_setup_conn lib/url.c:3906
    #9 0x7fe07a64bb42 in Curl_once_resolved lib/hostip.c:1102
    #10 0x7fe07a6bfb10 in multi_runsingle lib/multi.c:1753
    #11 0x7fe07a6c4a39 in curl_multi_perform lib/multi.c:2405
    #12 0x7fe07a5c1fe4 in easy_transfer lib/easy.c:603
    #13 0x7fe07a5c28c7 in easy_perform lib/easy.c:693
    #14 0x7fe07a5c299f in curl_easy_perform lib/easy.c:712
    #15 0x55fceb598a9f in serial_transfers src/tool_operate.c:2291
    #16 0x55fceb599f5c in run_all_transfers src/tool_operate.c:2475
    #17 0x55fceb59a8f7 in operate src/tool_operate.c:2590
    #18 0x55fceb578153 in main src/tool_main.c:323
    #19 0x7fe078cd9022 in __libc_start_main (/usr/lib/libc.so.6+0x27022)

SUMMARY: AddressSanitizer: heap-use-after-free ../../openssl/ssl/ssl_lib.c:1150 in SSL_free
Shadow bytes around the buggy address:
  0x0c467fff90f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c467fff9100: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c467fff9110: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c467fff9120: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c467fff9130: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c467fff9140: fd fd fd fd fd fd fd fd fd fd fd[fd]fd fd fd fd
  0x0c467fff9150: fd fd fd fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c467fff9160: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c467fff9170: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c467fff9180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c467fff9190: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==1307808==ABORTING

I expected the following

(no crash, hopefully a successful HTTP/3 request)

curl/libcurl version

curl 7.71.0-DEV (Linux) libcurl/7.71.0-DEV OpenSSL/1.1.1g zlib/1.2.11 libssh2/1.9.0 ngtcp2/0.1.90 nghttp3/0.1.90
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS HTTP3 HTTPS-proxy IPv6 Largefile libz NTLM SSL UnixSockets

Extra information

$ for i in curl ngtcp2 nghttp3 openssl; do echo $i $(git -C $i branch --show-current) $(git -C $i describe --always); done
curl master curl-7_70_0-218-g350a99b21
ngtcp2 master 1f0cd40
nghttp3 master 9df2310
openssl OpenSSL_1_1_1g-quic-draft-29 OpenSSL_1_1_1g-8-gdac2d09523

Client tries to negotiate draft 29, server responds with a Version Negotiation advertising draft 27 only.
@Lekensteyn Lekensteyn added the HTTP/3 h3 or quic related label Jun 14, 2020
@bagder bagder added the crash label Jun 15, 2020
@bagder bagder self-assigned this Jun 15, 2020
@bagder
Copy link
Member

bagder commented Jun 15, 2020 •
edited
Loading

The crash was easy to find and fix, but it then revealed a happy eyeballs bug that has given me a little more head-scratching... Stay tuned!

@bagder bagder mentioned this issue Jun 15, 2020
Closed
bagder added a commit that referenced this issue Jun 16, 2020
@bagder
ngtcp2: fix happy eyeballs quic connect crash
03b786a 
Reported-by: Peter Wu
Fixes #5565
Closes #5568
@bagder bagder closed this as completed in b6af4c2 Jun 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bagder bagder

Labels
crash HTTP/3 h3 or quic related
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Lekensteyn @bagder