Circular dependencies when building NuttX as a library

[sastel]

@acassis in your 2020 blog post you described how to export NuttX as a library. This is also documented in C++ Example using CMake. In both of these cases, and in the PX4 codebase, the link command includes the NuttX library collection inside --start-group --end-group.

I'm trying to package the NuttX libraries with Conan 2, and I haven't been able to figure out how to communicate to package consumers that the NuttX libs have to be within a link group. The details are in conan-io/conan#16283. The circular dependencies create problems for consumers.

Are the inter dependencies of the NuttX libraries simple enough that they can be spelled out manually in CMake, or are they not resolvable with anything other than the link group?

Alternatively, has anyone tried combining all of the NuttX static libs into one static lib prior to export?

[acassis]

Hi @sastel I never heard about Conan before (only the movie), seems like a nice package manager to integrate external libraries.
I never faced this issue because normally in my projects I always use the NuttX build system directly, the example I posted in my blog was to help people willing to use NuttX as library to avoid using the NuttX build system.
Probably your idea to merge the libraries into a single lib could work, did you try it: https://stackoverflow.com/questions/3821916/how-to-merge-two-ar-static-libraries-into-one ?

Also @davids5 could have some hints about how they are solving this issue on PX4 project, because their they also use NuttX as a library.

[sastel]

@acassis thank you for your help. I will try merging the static libraries next.

PX4 is using cmake, but they call make for each directory in NuttX, then use the lib*.a files as imported targets in cmake. Since they are using cmake they were able to link the imported targets in --start-group and --end-group with no issues.

Really this is a Conan issue, I was just curious if there was a way to eliminate the circular dependencies between the libs that make export produces.

[leducp]

I'm not really sure that this makes sense to generate NuttX with conan: NuttX as an operating system is the root of your dependencies. As such it provides the toolchain description and the C library. Conan need this as an input to generate its outputs (through the conan profile file).
Even the configuration of NuttX (or any OS) seems really complicated to do through conan.

We are using conan and NuttX intensively at work and our approach is to consider the root system as a first - bootstrap - step (our code base can run on Linux, NuttX, PikeOS and partially on Windows). Then we call conan to generate the intermediate dependencies: at this step conan consume the OS toolchain file to generate its outputs. At the end we can build our application by consuming the OS toolchain file/libs and conan libs.

[sastel]

@leducp are you using make export with NuttX? And if so, how are you getting Conan to consume the resulting toolchain in the export? Are you simply treating it as a customer toolchain file as described here? And how do you switch between different NuttX configurations?

I've been using Conan and CMake to do what make export does - compile NuttX as a set of libraries, then package them. I have out-of-tree board configs which I select between based on a variable in the Conan profile. Conan helps ensure I have the right version of NuttX, and the toolchain. The toolchain doesn't need to be locally installed, because it is also a Conan package. The drawback is that I have to ensure that the Conan generated toolchain file sets the right flags and variables. But this is manageable since I only have to do this comparison when I change the NuttX version, which is rare.

[trns1997]

Hi @sastel. We more or less do the following:

1. We fetch nuttx and nuttx-app of set version
2. We then configure our board (this can also be a out-of-tree board as well) using configure.sh
3. Then we call make export to generate the archive
4. And then we use the exported archive to build our applications.
5. Now once we have the archive we have the toolchain generated by the export in the scripts folder (not sure need to check)
6. Then you need create a conan profile with the right settings, buildenv and most importantly you need to set the conf to inject the nuttx generated toolchain (details can be found here: https://docs.conan.io/2/reference/config_files/global_conf.html#reference-config-files-global-conf , i think it was either tools.cmake.cmaketoolchain:toolchain_file: Use other existing file rather than conan_toolchain.cmake one or tools.cmake.cmaketoolchain:user_toolchain: Inject existing user toolchains at the beginning of conan_toolchain.cmake @leducp can you confirm which one to use).
7. From there you can conan install packages you need with your application.

If you want i could create a minimal example for you and future users who wish to use conan as a package manager with nuttx. But to be clear we do not in any way use conan to build nuttx.

[sastel]

@trns1997 I might modify my approach to do what you are suggesting. A minimal example would be welcome!

I'm especially interested in how you work with the --start-group <list of nuttx libs> --end-group link flags in the Conan packages which consume the NuttX export.

[trns1997]

@sastel here is what the <NUTTX_EXPORT>/scripts/toolchain.cmake looks like:

set(CMAKE_SYSTEM_NAME Generic)
set(CMAKE_SYSTEM_VERSION 1)
set(NUTTX 1)

set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)

set(NUTTX_PATH ${CMAKE_CURRENT_LIST_DIR}/..)
include(${NUTTX_PATH}/scripts/target.cmake)

set(LINKER_SCRIPT ${NUTTX_PATH}/scripts/${LDNAME})

set(CMAKE_C_FLAGS "${ARCHCPUFLAGS} ${ARCHCFLAGS} -D__NuttX__")
set(CMAKE_CXX_FLAGS "${ARCHCPUFLAGS} ${ARCHCXXFLAGS} -D__NuttX__")

set(CMAKE_C_STANDARD_INCLUDE_DIRECTORIES ${NUTTX_PATH}/include
                                         ${NUTTX_PATH}/arch/chip)

set(CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES
    ${NUTTX_PATH}/include/${NUTTX_CXX} ${NUTTX_PATH}/include
    ${NUTTX_PATH}/arch/chip)

file(GLOB STARTUP_OBJS ${NUTTX_PATH}/startup/*)

add_compile_options(-nostdlib)
add_compile_options(-ffunction-sections -fdata-sections)

# same entry used for all build modes in crt0.c and arch/.../xxx_start.c

set(ENTRY_NAME "__start")

set(CMAKE_C_LINK_EXECUTABLE
    "<CMAKE_LINKER> ${LDFLAGS} --entry=${ENTRY_NAME} -T${LINKER_SCRIPT} <OBJECTS> ${STARTUP_OBJS} -o <TARGET> <LINK_LIBRARIES> -L${NUTTX_PATH}/libs --start-group ${LDLIBS} ${EXTRA_LIBS} --end-group"
)
set(CMAKE_CXX_LINK_EXECUTABLE
    "<CMAKE_LINKER> ${LDFLAGS} --entry=${ENTRY_NAME} -T${LINKER_SCRIPT} <OBJECTS> ${STARTUP_OBJS} -o <TARGET> <LINK_LIBRARIES> -L${NUTTX_PATH}/libs --start-group ${LDLIBS} ${EXTRA_LIBS} --end-group"
)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

This is automatically generated during the make export process, you need to inject this toolchain to conan for it to work. There might be some subtleties that i might be missing maybe @leducp could clarify if i am. I will try to create a minimal example for you and potential future users with the following board: xmc4800 and eigen/zstd from conan. Let me know what you think.

[leducp]

Here after our conan profile for NuttX:

[settings]
os=baremetal
arch=armv7
compiler=gcc
compiler.version=12
compiler.libcxx=libstdc++11
compiler.cppstd=17

[options]

[tool_requires]

[conf]
tools.cmake.cmaketoolchain:user_toolchain=["/src/wdc_build/nuttx.cmake"]

[buildenv]
CONAN_CMAKE_GENERATOR=Ninja
CONAN_CMAKE_TOOLCHAIN_FILE=/build/nuttx/scripts/toolchain.cmake
CC=@CMAKE_C_COMPILER@
CXX=@CMAKE_CXX_COMPILER@
CFLAGS=@ARCHCPUFLAGS@ @ARCHCFLAGS@ -DCONFIG_WCHAR_BUILTIN -D__NuttX__ -isystem /build/nuttx/include
CXXFLAGS=@ARCHCPUFLAGS@ @ARCHCXXFLAGS@ -DCONFIG_WCHAR_BUILTIN -D__NuttX__ -isystem /build/nuttx/include/cxx -isystem /build/nuttx/include

This file is a template that can be filled with a CMake script like this one:

include(/build/nuttx/scripts/target.cmake)

configure_file(
    /src/conan/profiles/nuttx.in
    /build/nuttx/conan_profile
)

This template could be generated directly by NuttX during the export phase, but I didn't have the time to do it yet. We also lazily copy a few variables of the export that are not exposed through CMake yet (the includes paths). Theses are mandatory for conan package that do not consume CMake toolchain directly.

So as you can see, we are feeding conan with mainly the NuttX export :)

[trns1997]

@acassis quick question for you, do you think it would be an interesting feature to add for the nuttx community. I was thinking of exporting a nuttx_conan_profile.txt in the export, so that users can use the generated profile and the toolchain to build conan packages?

[acassis]

Hi @trns1997 yes! I think it could be nice to have this option, since more people could decide to use NuttX with conan to control packages integration. If you are going to submit support to it, please also create a Documentation similar to https://nuttx.apache.org/docs/latest/guides/cpp_cmake.html otherwise it will generate more support for us.

[trns1997]

@acassis noted, once i have a somethign functional i'll create a doc. Also just a remark regarding the doc https://nuttx.apache.org/docs/latest/guides/cpp_cmake.html , i noticed we have provide an example of the toolchain file, i was just wonder now that we autogenerate and provide the toolchain with the export, would it not be a good idea to update the doc to use the generated toolchain file instead of copying the content from the doc?

[trns1997]

@sastel here is a concrete example that i created: https://github.com/trns1997/nuttx_conan_example i will see how to go about dynamically generating the profile when we call make export, but this example has everything you need to build and consume conan packages with the nuttx.

[sastel]

@trns1997 thank you very much for the example. That clarifies what you described.

[sastel]

Hey guys, I did some more thinking and took this a step further with the help of one of the conan developers. See conan-io/conan#16340. With this approach you can package NuttX, treat it as a tool_requires in your apps/libs, and still inject the scripts/toolchain.cmake into the conan_toolchain.cmake file. I like it because it means that I can automate which compiler package is used to build NuttX in the conan build virtual environment, as well as control the board/config for NuttX all from Conan.

[trns1997]

@sastel niceeeee. Honestly I'd like to give it a try. But a quick question. How do you go about installing other Conan packages ? From what I can see you are building nuttx using Conan but what if your application needs other Conan packages like eigen which requires the nuttx toolchain otherwise your final nuttx application that requires nuttx and eigen won't really work. How do you inject the nuttx toolchain in the Conan profile on the fly? Unless it is not really your use case and that you just require the nuttx package via Conan.

[leducp]

@sastel
Thanks for the feedback, we may git it a shot at our workplace :D

[sastel]

How do you go about installing other Conan packages ? From what I can see you are building nuttx using Conan but what if your application needs other Conan packages like eigen which requires the nuttx toolchain otherwise your final nuttx application that requires nuttx and eigen won't really work. How do you inject the nuttx toolchain in the Conan profile on the fly?

I tried including Eigen 3.4.0 from conancenter just now, and the answer is that you specify Eigen in the requires section of the recipe that needs it, and ensure that your host profile mentions nuttxpkg as a tool_require. The host profile gets applied to all of the dependencies including Eigen. Because of the following snippet in the nuttxpkg recipe, the toolchain.cmake file gets applied to any project that uses nuttxpkg as a tool require.

    def package_info(self):
        # Later packages get access to the NuttX export not by find_package() in CMake,
        # but through the conan_toolchain.cmake file which includes the toolchain.cmake
        # from NuttX. This means that all builds which mark this package as a
        # tool_requires will include the NuttX headers and link against the NuttX libs.
        toolchain = os.path.join(self.package_folder, "nuttx-export-12.5.0/scripts/toolchain.cmake")
        self.conf_info.append("tools.cmake.cmaketoolchain:user_toolchain", toolchain)

I didn't have to modify the Eigen package at all, and it is including the POSIX headers from nuttxpkg. See this graph from the conan graph info command. (Note that the dependency tree isn't quite complete, because nuttxpkg should be showing a dependency on my xpack compiler package, but isn't yet because I didn't form the xpack package correctly).