Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

curl verbose output does not show handshake information #6783

Closed
ghost opened this issue Mar 24, 2021 · 10 comments
Closed

curl verbose output does not show handshake information #6783

ghost opened this issue Mar 24, 2021 · 10 comments
Labels
TLS

Comments

@ghost
Copy link

ghost commented Mar 24, 2021 •
edited by ghost
Loading

I did this

I have compiled curl from source as of Mar 24 2021 GMT. I tried to produce verbose output with curl -v <url> however the output only shows "Trying......" "Connecting "and "ALPN......" instead of "Successfully set certificate verify locations...... CAFile...... CAPath......" and "TLSv1.3, TLS handshake, Client Hello (1):"

I expected the following

Showing verbose output including handshake information before the HTTP request.

However, the verbose output jumps straight from "Connecting......" to GET / HTTP/2

curl/libcurl version

curl 7.76.0-DEV (x86_64-pc-linux-gnu) libcurl/7.76.0-DEV
[curl -V output]

curl 7.76.0-DEV (x86_64-pc-linux-gnu) libcurl/7.76.0-DEV BoringSSL zlib/1.2.11 brotli/1.0.9 zstd/1.4.9 libidn2/2.3.0 libpsl/0.21.1 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.44.0-DEV quiche/0.7.0 librtmp/2.3 libgsasl/1.8.1
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli gsasl HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Largefile libz Metalink NTLM NTLM_WB PSL SSL UnixSockets zstd

operating system

Linux fedora 5.11.8-300.fc34.x86_64 #1 SMP Mon Mar 22 01:33:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
@ghost
Copy link
Author

ghost commented Mar 24, 2021

Here is an example output when the issue occurs

*   Trying 172.217.14.238:443...
* Connected to google.com (172.217.14.238) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=*.google.com
*  start date: Feb 23 15:36:56 2021 GMT
*  expire date: May 18 15:36:55 2021 GMT
*  subjectAltName: host "google.com" matched cert's "google.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0xc03410)
> GET / HTTP/2
> Host: google.com
> user-agent: curl/7.76.0-DEV
> accept: */*
>

Notice where the TLS handshake information are missing

@bagder bagder added the TLS label Mar 24, 2021
@bagder
Copy link
Member

bagder commented Mar 24, 2021

I believe this is to be expected when you use BoringSSL: it doesn't provide the SSL_CTX_set_msg_callback that curl needs for this.

$ cd $BORINGSSL
$ grep -r SSL_CTRL_SET_MSG_CALLBACK
openssl/ssl.h:#define SSL_CTRL_SET_MSG_CALLBACK doesnt_exist
openssl/ssl.h:#define SSL_CTRL_SET_MSG_CALLBACK_ARG doesnt_exist

@ghost
Copy link
Author

ghost commented Mar 24, 2021 •
edited by ghost
Loading

OK. I will recompile with GnuTLS+nghttp3+ngtcp2 and see if the issue persists.

@bagder
Copy link
Member

bagder commented Mar 24, 2021

Just note that then you can't use quiche...

@bagder
Copy link
Member

bagder commented Mar 24, 2021

Oops. Maybe you can... they can probably coexist

@ghost
Copy link
Author

ghost commented Mar 24, 2021

Just note that then you can't use quiche...

nghttp3 is also fine for me

@ghost
Copy link
Author

ghost commented Mar 24, 2021

@bagder I experienced the very same issue as #6493

The ngtcp2 pkgconfig file exists and configurations are all correct, but ./configure refuses to continue because of error: --with-ngtcp2 is specified but could not find ngtcp2_crypto_gnutls pkg-config file. I am completely frustrated. I spent nearly an hour building prerequisites (GnuTLS+nghttp3+ngtcp2) but only get a failed build :-(

@jay
Copy link
Member

jay commented Mar 25, 2021

I don't see any handshake output in Windows. The first message I see is after the handshake is over and the request has been sent, * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):.

curl 7.76.0-DEV (i386-pc-win32) libcurl/7.76.0-DEV OpenSSL/1.1.1j nghttp2/1.40.0

@bagder
Copy link
Member

bagder commented Mar 25, 2021 •
edited
Loading

Wow, yes now I spotted it too! PR coming up

bagder added a commit that referenced this issue Mar 25, 2021
@bagder
openssl: set the transfer pointer for logging early
201510c 
Otherwise, the transfer will be NULL in the trace function when the
early handshake details arrive and then curl won't show them.

Reported-by: David Hu
Fixes #6783
@bagder bagder mentioned this issue Mar 25, 2021
Closed
@ghost
Copy link
Author

ghost commented Mar 25, 2021

Thank you SO much Daniel!

@bagder bagder closed this as completed in db4e0bd Mar 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
TLS
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

openssl: set the transfer pointer for logging early curl/curl
2 participants
@bagder @jay