Linker broken after IP parser changes

[Arathen]

Please provide the following information

qBittorrent version and Operating System:

qBittorrrent version - commit 67f44e0 from branch 'master'
OS is Ubuntu 16.04.2 LTS armhf running on a Raspberry Pi 2

If on linux, libtorrent and Qt version:

libtorrent 1.0.11 (commit bda4fd60c4c798f7eb0078f727e08c3e4d228669 from branch RC_1_0 compiled from source)
Qt 5 v5.5.1

What is the problem: After compiling successfully, linking fails with the following error -

linking qbittorrent-nox
filterparserthread.o: In function `(anonymous namespace)::parseIPAddress(char const*, boost::asio::ip::address&) [clone .constprop.101]':
/home/dev/src/qbittorrent-git/src/base/bittorrent/private/filterparserthread.cpp:99: undefined reference to `boost::asio::ip::address_v4::address_v4(std::array<unsigned char, 4u> const&)'
collect2: error: ld returned 1 exit status
Makefile:508: recipe for target 'qbittorrent-nox' failed
make[1]: *** [qbittorrent-nox] Error 1
make[1]: Leaving directory '/home/dev/src/qbittorrent-git/src'
Makefile:42: recipe for target 'sub-src-make_first' failed
make: *** [sub-src-make_first] Error 2

What is the expected behavior: Linked binary produced.

Steps to reproduce: git clone // configure // make on the appropriate platform with appropriate libs.

Extra info(if any): This seems to have been introduced with the IP parser changes of 25th April. qBittorrent from git source compiled and linked successfully before then.

[fbriere]

Thanks for the info. As you correctly guessed, this symbol was introduced by 8f6e04d. It should, however, be provided by libtorrent; as least, it is in my case (with 4ul instead of 4u, but I suppose this is due to the difference between amd64 and ARM). Could you check if it is?

objdump -T -C .../libtorrent-rasterbar.so.8 | grep 'boost::asio::ip::address_v4::address_v4(std::'

In your case, the symbol must either be missing, or its signature doesn't match.

[Arathen]

Done. Happy to help. It looks like its missing as the grepped objdump comes up empty. Widening the grep a little gives this:

dev@pi3 ~ $ objdump -T -C /usr/lib/libtorrent-rasterbar.so.8 | grep 'boost::asio::ip::address_v4::address_v4'
0005dcb8 g    DF .text  00000006  Base        boost::asio::ip::address_v4::address_v4(boost::array<unsigned char, 4u> const&)
0005dcc0 g    DF .text  00000010  Base        boost::asio::ip::address_v4::address_v4(unsigned long)
0005dcc0 g    DF .text  00000010  Base        boost::asio::ip::address_v4::address_v4(unsigned long)
0005dcb8 g    DF .text  00000006  Base        boost::asio::ip::address_v4::address_v4(boost::array<unsigned char, 4u> const&)

Let me know if you need anything else.

Cheers.

[fbriere]

Thanks! That was indeed the missing piece of the puzzle.

It turns out that Boost defines boost::asio::ip::address_v4::bytes_type as either std::array (introduced in C++11) if available, or boost::array if not. This means that libtorrent and qBittorrent can disagree on this matter if they were compiled with different compilers, or with different compiler options.

For qBittorrent, C++11 support is explicitly enabled. For libtorrent, this is not the case. Yet std::array is nevertheless chosen on my system, so I guess the Boost header files perform some magic to figure this out. Which doesn't yield the same result in your case. (Could you be using clang, by any chance?)

If we cannot avoid using bytes_type, then I guess both qBittorrent and libtorrent should explicitly enable or disable std::array support (http://www.boost.org/doc/libs/1_64_0/doc/html/boost_asio/overview/cpp2011/array.html). If enabled, this would make libtorrent require C++11 support. If disabled, this would probably require recompiling on most GNU/Linux distributions.

@evsh, @arvidn, your thoughts?

[Arathen]

For what its worth, I recompiled libtorrent and qbittorrent both using CXXFLAGS="-std=c++0x" and that has (after a HEAP of warnings while compiling libtorrent) produced binaries that link and run semi-happily.

I say semi-happily because the latest commit has broken daemon mode (running without -d still works fine). But i'll raise a seperate issue for that.

[arvidn]

the next major release of libtorrent (1.2) will require C++11.

[Arathen]

Turns out, compiling libtorrent/qbittorrent both with CXXFLAGS="-std=c++0x" to force std::array isnt really a solution. Despite compiling successfully, it looks like there was something in all those warnings which was significant, because it seems that is what was causing the qbittorrent-nox binary to seg fault when run with the -d flag set. I dont really understand what it is about the -d which is significant.

[fbriere]

@Arathen: In your case, adding BOOST_ASIO_DISABLE_STD_ARRAY to DEFINES in qBittorrent's src/src.pro should be all you need. (There's no need to recompile your version of libtorrent, since it already uses boost::array.)

[zeule]

I doubt qBt may link successfully to a libtorrent, compiled not in C++11 mode (see #5265 and reference therein).

[fbriere]

Thanks for the link, @evsh. If I understand correctly, these two issues are the same, just over different symbols.

Given how there are 10 different BOOST_ASIO_* options (and probably more like those across the entire Boost library), and no obvious way to enable/disable them all at once, yeah, it's probably best to just pick a standard (in this case, c++11) and stick with it.

[zeule]

With each C++ version step, libtorrent migrates from boost:: classes to std:: ones (and chrono is one of the examples in #5265, another recent one is string_view). So the problem is not in boost::asio defines only.

[sledgehammer999]

This is "documented" requirement in my PR #5310 doing the IPFilter parser refactoring.
In any case you can either use the libtorrent from our PPA, which is compiled with c++11 mode
or compile libtorrent 1.0.x series yourself:

1. By adding CXXFLAGS=-std=c++11 at the end of the configure command
2. By doing:

extra_compile_args = parse_cmd(extra_cmd, '-D', True) + arch() \
				+ target_specific() + ['-std=c++11'],

in bindings/python/setup.py

The 2nd part may or may not affect only the python bindings.

PS: If you're on the 1.1.x series then I haven't tested them yet.

[ChrisBlizz]

Same thing happening to me on Linux Mint 18.03 x86_64 using qBittorrent-release-4.0.4 (also confirmed on 4.0.3). Adding BOOST_ASIO_DISABLE_STD_ARRAY to DEFINES in qBittorrent's src/src.pro as suggested by @fbriere did it for me.

[WolfganP]

I had no issues with latest master linked with libtorrent RC_1_1, but as devs started to move to v1.2 and next beta will be using that version, I cleaned up my environment and went to full rebuild of qBt master against libtorrent RC_1_2.
Just in case, I made sure my setup followed https://github.com/qbittorrent/qBittorrent/wiki/Compiling-qBittorrent-nox-4.x-on-DietPi-or-Raspbian-(Debian-9.0)-for-ARM-devices for my RasPi environment just in case something changed lately.

Problem is that it fails at linking stage, as:

$ make
cd src/ && ( test -e Makefile || /usr/lib/qt5/bin/qmake -o Makefile /mnt/usb_12/wTools/qBittorrent/src/src.pro -qtconf /usr/lib/arm-linux-gnueabihf/qt5/qt.conf -early QMAKE_CC=arm-linux-gnueabihf-gcc QMAKE_CXX=arm-linux-gnueabihf-g++ QMAKE_LINK=arm-linux-gnueabihf-g++ QMAKE_QMAKE=/usr/bin/arm-linux-gnueabihf-qmake PKG_CONFIG=arm-linux-gnueabihf-pkg-config -before QMAKE_LRELEASE= ) && make -f Makefile
make[1]: Entering directory '/mnt/usb_12/wTools/qBittorrent/src'
linking qbittorrent-nox
torrenthandle.o: In function `int libtorrent::detail::bencode_recursive<std::back_insert_iterator<QVector<char> > >(std::back_insert_iterator<QVector<char> >&, libtorrent::entry const&)':
/usr/local/include/libtorrent/bencode.hpp:165: undefined reference to `libtorrent::entry::dict[abi:cxx11]() const'
session.o: In function `int libtorrent::detail::bencode_recursive<std::back_insert_iterator<QByteArray> >(std::back_insert_iterator<QByteArray>&, libtorrent::entry const&)':
/usr/local/include/libtorrent/bencode.hpp:165: undefined reference to `libtorrent::entry::dict[abi:cxx11]() const'
torrentcreatorthread.o: In function `int libtorrent::detail::bencode_recursive<std::ostream_iterator<char, char, std::char_traits<char> > >(std::ostream_iterator<char, char, std::char_traits<char> >&, libtorrent::entry const&)':
/usr/local/include/libtorrent/bencode.hpp:165: undefined reference to `libtorrent::entry::dict[abi:cxx11]() const'
tracker.o: In function `BitTorrent::Tracker::prepareAnnounceResponse(BitTorrent::Tracker::TrackerAnnounceRequest const&)':
/home/pi/h/wTools/qBittorrent/src/base/bittorrent/tracker.cpp:440: undefined reference to `libtorrent::entry::entry(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, libtorrent::entry, libtorrent::aux::strview_less, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libtorrent::entry> > >)'
tracker.o: In function `void __gnu_cxx::new_allocator<libtorrent::entry>::construct<libtorrent::entry, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, libtorrent::entry, libtorrent::aux::strview_less, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libtorrent::entry> > >&>(libtorrent::entry*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, libtorrent::entry, libtorrent::aux::strview_less, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libtorrent::entry> > >&)':
/usr/include/c++/6/ext/new_allocator.h:120: undefined reference to `libtorrent::entry::entry(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, libtorrent::entry, libtorrent::aux::strview_less, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libtorrent::entry> > >)'
tracker.o: In function `BitTorrent::Tracker::processRequest(Http::Request const&, Http::Environment const&)':
/home/pi/h/wTools/qBittorrent/src/base/bittorrent/tracker.cpp:259: undefined reference to `libtorrent::entry::entry(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, libtorrent::entry, libtorrent::aux::strview_less, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libtorrent::entry> > >)'
tracker.o: In function `void __gnu_cxx::new_allocator<libtorrent::entry>::construct<libtorrent::entry, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, libtorrent::entry, libtorrent::aux::strview_less, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libtorrent::entry> > >&>(libtorrent::entry*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, libtorrent::entry, libtorrent::aux::strview_less, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libtorrent::entry> > >&)':
/usr/include/c++/6/ext/new_allocator.h:120: undefined reference to `libtorrent::entry::entry(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, libtorrent::entry, libtorrent::aux::strview_less, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libtorrent::entry> > >)'
collect2: error: ld returned 1 exit status
Makefile:600: recipe for target 'qbittorrent-nox' failed
make[1]: *** [qbittorrent-nox] Error 1
make[1]: Leaving directory '/mnt/usb_12/wTools/qBittorrent/src'
Makefile:44: recipe for target 'sub-src-make_first' failed
make: *** [sub-src-make_first] Error 2

For those that are already using libtorrent 1.2, any clues?
Thanks in advance, Wolf

[arvidn]

my guess is that you build libtorrent and qBittorrent with different versions of C++.

[sledgehammer999]

@WolfganP current qbt master (aka 4.2.x) requires at least c++14. And from the errors above it seems you have libtorrent built with c++11. This is a problem because some symbols exposed by libtorrent change type between c++ versions. You should rebuild libtorrent using c++14. I think doing

./configure <your options> CXXFLAGS=-std=c++14

when configuring libtorrent is enough.