Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing APPDATA and USERPROFILE environment variables cause error "curl: (27) Out of memory" #4644

Closed
vovcacik opened this issue Nov 27, 2019 · 7 comments
Closed

Missing APPDATA and USERPROFILE environment variables cause error "curl: (27) Out of memory" #4644

vovcacik opened this issue Nov 27, 2019 · 7 comments
Labels
cmdline tool needs-info Windows Windows-specific

Comments

@vovcacik
Copy link

I did this

C:\Users\Admin\Desktop>curl.exe https://[redacted].[redacted] -i -T "C:\Users\Admin\AppData\Local\Temp\test.txt
curl: (27) Out of memory

I expected the following

C:\Users\Admin\Desktop>curl.exe https://[redacted].[redacted] -i -T "C:\Users\Admin\AppData\Local\Temp\test.txt
HTTP/2 200
content-location: /a
content-type: text/plain
server: [redacted]
content-length: 3
date: Wed, 27 Nov 2019 17:49:03 GMT

/a

curl/libcurl version

C:\Users\Admin>curl --version
curl 7.62.0 (x86_64-pc-win32) libcurl/7.62.0 OpenSSL/1.1.1 (WinSSL) zlib/1.2.11 brotli/1.0.7 WinIDN libssh2/1.8.0 nghttp2/1.34.0
Release-Date: 2018-10-31
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz brotli TLS-SRP HTTP2 HTTPS-proxy MultiSSL

operating system

Windows 10

description

The problem is curl depends on USERPROFILE and APPDATA environment variables and when both are unset it quickly fails with curl: (27) Out of memory.

Points to address:

  • is curl expected to fail when the env vars are missing?
  • should it fail with out of memory error despite of this seems more like configuration/context/file access problem?
  • the error does not occur when I run curl --version or e.g. curl --help
  • issue curl error: out of memory (27) #3117 has exactly the same symptoms
  • difficult to debug as the error is misleading and afaik this dependency is not documented

The solution is

  • to set one of the variables or both to its appropriate value
  • to make sure curl is invoked with all environment variables

Feel free to close.
@jay
Copy link
Member

jay commented Nov 27, 2019

homedir function looks in this order for win32
%CURL_HOME%
%HOME%
%APPDATA%
%USERPROFILE%\Application Data

curl/src/tool_homedir.c

Lines 88 to 94 in 2e9b725

#ifdef WIN32
home = GetEnv("APPDATA", TRUE);
if(!home)
home = GetEnv("%USERPROFILE%\\Application Data", TRUE); /* Normally only
on Win-2K/XP */
#endif /* WIN32 */
return home;

CURLE_OUT_OF_MEMORY could happen but only if built with ssh support:

curl/src/tool_operate.c

Lines 1520 to 1540 in 2e9b725

if(built_in_protos & (CURLPROTO_SCP|CURLPROTO_SFTP)) {
if(!config->insecure_ok) {
char *home;
char *file;
result = CURLE_OUT_OF_MEMORY;
home = homedir();
if(home) {
file = aprintf("%s/.ssh/known_hosts", home);
if(file) {
/* new in curl 7.19.6 */
result = res_setopt_str(curl, CURLOPT_SSH_KNOWNHOSTS, file);
curl_free(file);
if(result == CURLE_UNKNOWN_OPTION)
/* libssh2 version older than 1.1.1 */
result = CURLE_OK;
}
Curl_safefree(home);
}
if(result)
break;
}

Broken (albeit inadvertently) in fb3845a which was over 9 years ago. Prior to that it didn't work right anyway without a homedir, it appears it would set it to /_ssh/known_hosts.

We could fix this by moving the if(result) break; to inside the if(file) block. However there may be other places that curl expects a homedir. Do you know why those variables are not set for you?

@jay jay added cmdline tool Windows Windows-specific labels Nov 27, 2019
@bagder
Copy link
Member

bagder commented Nov 28, 2019

homedir() is only called in two places and the invoke in tool_operate.c looks fine.

This "out of memory" error is in this case just a wrong error message since curl couldn't continue anyway. The reason for the wrong error message is of course that homedir() cannot specify the reason why it failed so the tool_operate.c code "guesses" what the reason is...

@bagder
Copy link
Member

bagder commented Nov 28, 2019

Maybe the homedir function should be adjusted to use SHGetFolderPath on Windows?

@jay
Copy link
Member

jay commented Nov 30, 2019

I think shell32 may have some problems if run from a service (ie curl is run by a service and not a normal user account), so I'm not sure we'd want curl/libcurl to depend on it. I think it's possible to try for appdata location from the registry as a backup if env var is empty, maybe that would be better.. @captain-caveman2k opinion?

@bagder
Copy link
Member

bagder commented Nov 30, 2019

But if we'd do that function call as a last resort when no env variable exists, shouldn't that at least cover @vovcacik's case?

@jay
Copy link
Member

jay commented Nov 30, 2019

Yes it will cover his case but I'm kind of interested why his case, I've just never heard of it happening before. It seems like something exclusively for him (until proven otherwise).

@jay jay added the needs-info label Nov 30, 2019
@vovcacik
Copy link
Author

vovcacik commented Dec 2, 2019

Do you know why those variables are not set for you?

Yes, the curl is executed in cgi script.

This "out of memory" error is in this case just a wrong error message

Agreed, I was able to fix this on my end rather easily. Actually my first thought was that the environment is broken because of the cgi, but the error message indicating far more severe error state held me from digging in that first. Hence this issue.

I think shell32 may have some problems if run from a service

Not in my case, but yeah...

But if we'd do that function call as a last resort when no env variable exists, shouldn't that at least cover @vovcacik's case?

I think it would, the cgi host was purging envvars only and not e.g. registry read permissions. Other CGI servers could though.

Yes it will cover his case but I'm kind of interested why his case, I've just never heard of it happening before. It seems like something exclusively for him (until proven otherwise).

I am surprised myself, this should have surfaced many times before. However I am quite convinced that #3117 issue had the same culprit, same symptoms and I wouldn't be surprised if his java server or his mingw stripped the envvars. Just a speculation though.

I would be quite happy with simply more descriptive error message. The CURLE_FAILED_INIT sounds right.

bagder added a commit that referenced this issue Dec 2, 2019
@bagder
curl: show better error message when no homedir is found
5fdbda2 
Reported-by: Vlastimil Ovčáčík
Fixes #4644
@bagder bagder mentioned this issue Dec 2, 2019
Closed
@bagder bagder closed this as completed in 7dffc2b Dec 3, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Mar 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
cmdline tool needs-info Windows Windows-specific
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

curl: show better error message when no homedir is found curl/curl
3 participants
@bagder @vovcacik @jay