From 26affe51cf5a47f1908d9f65f61745c5afe80d6f Mon Sep 17 00:00:00 2001 From: Lephenixnoir Date: Fri, 19 Aug 2022 15:23:21 +0200 Subject: [PATCH] new build system for fxSDK sysroot and libstdc++-v3 The main change is making the scripts a two-stage process where we first build GCC, leave the user to install the libc, and then come back to install libstdc++-v3. * Detect whether we are in the first or second stage * Don't clean files after first stage install, and even then only do it if :clean is specified * Update README except for the manual install tutorial * Patch the source using a backported GCC 12.1 commit to make the configure script for libstdc++-v3 skip checking for dlopen, which is impossible is our not-really-hosted setup Details: * Proper .gitignore --- .gitignore | 16 +- README.md | 66 +++++--- build.sh | 16 +- configure.sh | 156 +++++++++++++----- giteapc-config-clean.make | 4 + giteapc-config-noclean.make | 4 - install.sh | 38 ++--- .../gcc-11.1.0-libstdc++-v3-skip-dlopen.patch | 65 ++++++++ uninstall.sh | 18 +- util.sh | 16 ++ 10 files changed, 280 insertions(+), 119 deletions(-) create mode 100644 giteapc-config-clean.make delete mode 100644 giteapc-config-noclean.make create mode 100644 patches/gcc-11.1.0-libstdc++-v3-skip-dlopen.patch diff --git a/.gitignore b/.gitignore index a323bde..68c2494 100644 --- a/.gitignore +++ b/.gitignore @@ -1,12 +1,4 @@ -# Everything but the scripts -* -!giteapc.make -!giteapc-config-any.make -!giteapc-config-noclean.make -!configure.sh -!build.sh -!install.sh -!uninstall.sh -!util.sh -!.gitignore -!README.md +/gcc-*/ +/gcc-*.tar.* +/build*/ +/giteapc-config.make diff --git a/README.md b/README.md index c096742..abb1da0 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,54 @@ -# Automatic `sh-elf-gcc` installer +# SuperH toolchain: `sh-elf-gcc` -This script can be used to automatically compile and install a GCC cross-compiler targeting SH3 and SH4 calculators. The normal use is with GiteaPC: +This repository provides scripts to automatically compile and install an SH3/SH4-compatible [GCC cross-compiler](https://gcc.gnu.org/). GCC is a collection of compilers most commonly used for C/C++. -``` +The following three methods can be used to install the compiler with different levels of automation. + +Note that this repository should usually be built twice: first to build the compiler, and then after the libc is installed to build the C++ library. + +## Method 1: Using GiteaPC + +The most common way to install this compiler is for the fxSDK, and it can be automated with [GiteaPC](/Lephenixnoir/GiteaPC): + +```bash % giteapc install Lephenixnoir/sh-elf-gcc ``` -You can also install manually. First install [`sh-elf-binutils`](https://gitea.planet-casio.com/Lephenixnoir/sh-elf-binutils), then run the GiteaPC Makefile with a manually-specified install prefix: +This installs GCC (and binutils if missing) in the fxSDK's SuperH system root. Note that at first it will *not* install the C++ standard library libstdc++, because it requires the C standard library which is not available at this stage. After you install [fxlibc](/Vhex-Kernel-Core/fxlibc/) you should run GiteaPC's install command again, and this time the scripts will build libstdc++. The GiteaPC tutorial has more detailed instructions about this two-stage process. -``` -% make -f giteapc.make configure build install PREFIX=$HOME/.local -``` +A `:any` configuration is provided in case you already have another version of GCC installed in the fxSDK sysroot and want to keep using it (ie. skip a version upgrade). This will mark this repository as installed, so other repositories depending on it can build, without actually compiling binutils. -An `any` configuration is provided in case GCC is already installed externally, to have this package installed without rebuilding it. - -``` +```bash % giteapc install Lephenixnoir/sh-elf-gcc:any ``` -## Notes on building libstdc++-v3 +A `:clean` configuration is also provided if you want to clean up the source and build files automatically after the second pass. This frees up some disk space. + +```bash +% giteapc install Lephenixnoir/sh-elf-gcc:clean +``` + +## Method 2: Manually running the scripts + +Make sure to previously install: + +* [fxSDK](https://gitea.planet-casio.com/Lephenixnoir/fxsdk) ≥ 2.8.1 (provides the sysroot) +* [binutils for SuperH](https://gitea.planet-casio.com/Lephenixnoir/sh-elf-binutils) +* Most of the [requirements for GCC](https://gcc.gnu.org/install/prerequisites.html) are checked in the binutils `configure.sh` + +Follow the same procedure as for binutils; preferably use the same `PREFIX`. + +```bash +% make -f giteapc.make configure build install PREFIX="$HOME/.local" +``` + +## Method 3: Fully manually + +TODO: Manual install tutorial and link to the Planète Casio one + +TODO: The stuff below about libstdc++ is outdated, we can now build the entire thing + +**Notes on building libstdc++-v3** These are experimental notes on attempts at building the C++ standard library implementation bundled with GCC, `libstdc++-v3`. For the official manual, see [libstdc++ info manual, Chapter 2: Setup](https://gcc.gnu.org/onlinedocs/libstdc++/manual/setup.html) (gcc.gnu.org). @@ -81,17 +111,3 @@ If it succeeds, install. ``` % make install-strip-target-libstdc++-v3 ``` - -### Current problems - -Hard problems: - -* None. The free-standing subset compiles. - -Things that look like they could be involved in problems: - -* Anything that is not in the fxlibc can fail to link. - -* The conftest programs are built without `-ffreestanding`, which means autoconf cannot really link stuff. This is probably not too much of a problem, because it's cross-compiled anyway so there's nothing to do with a linked program, but who knows. - -* autoconf compiles conftest programs as if on a fully-featured dynamic OS, which is nowhere near true (eg. it builds with `-shared-libgcc`). diff --git a/build.sh b/build.sh index 08b9eda..08d9d29 100755 --- a/build.sh +++ b/build.sh @@ -1,7 +1,7 @@ #! /usr/bin/env bash # Avoid rebuilds of the same version -[[ ! -d build ]] && exit 0 +[[ -e "build/giteapc-skip-rebuild.txt" ]] && exit 0 source util.sh cd build @@ -13,12 +13,12 @@ else cores=$(nproc) fi -if command -v gmake >/dev/null 2>&1; then - make_command=gmake +if [[ -e "giteapc-build-libstdcxx.txt" ]]; then + echo "$TAG Compiling libstdc++-v3..." + run_quietly giteapc-build-libstdcxx.log \ + $MAKE_COMMAND -j"$cores" all-target-libstdc++-v3 else - make_command=make + echo "$TAG Compiling gcc (usually 10-20 minutes)..." + run_quietly giteapc-build.log \ + $MAKE_COMMAND -j"$cores" all-gcc all-target-libgcc fi - -echo "$TAG Compiling gcc (usually 10-20 minutes)..." -run_quietly giteapc-build.log \ -$make_command -j"$cores" all-gcc all-target-libgcc diff --git a/configure.sh b/configure.sh index cd70f1c..2b41799 100755 --- a/configure.sh +++ b/configure.sh @@ -7,34 +7,70 @@ PREFIX="$2" URL="https://ftp.gnu.org/gnu/gcc/gcc-$VERSION/gcc-$VERSION.tar.xz" ARCHIVE=$(basename "$URL") -# Avoid rebuilds of the same version +# Final location of gcc in the sysroot +SYSROOT_GCC="$SYSROOT/bin/sh-elf-gcc" +# Version string of any existing sh-elf-gcc in the sysroot +SYSROOT_GCC_VERSION= +# "1" if libstdc++-v3 is already in the install folder +SYSROOT_HAS_LIBSTDCXX= +# "1" if the conditions for building libstdc++-v3 are met +CAN_BUILD_LIBSTDCXX= -existing_gcc="$PREFIX/bin/sh-elf-gcc" +#--- +# Determine what parts of GCC are already installed and decide whether to build +#--- -if [[ -f "$existing_gcc" ]]; then - existing_version=$(sh-elf-gcc --version | head -n 1 | grep -Eo '[0-9.]+$') - if [[ $existing_version == $VERSION ]]; then - echo "$TAG Version $VERSION already installed, skipping rebuild" - if [[ -e build ]]; then - rm -rf build - fi +if [[ -f "$SYSROOT_GCC" ]]; then + SYSROOT_GCC_VERSION=$("$SYSROOT_GCC" --version | head -n 1 | grep -Eo '[0-9.]+$') + + # TODO: Once we start using the full C++ library, check for libstdc++.a instead + LIBSTDCXX_FILE=$("$SYSROOT_GCC" -print-file-name=libsupc++.a) + LIBC_FILE=$("$SYSROOT_GCC" -print-file-name=libc.a) + + if [[ "$LIBSTDCXX_FILE" != "libsupc++.a" ]]; then + SYSROOT_HAS_LIBSTDCXX=1 + echo "$TAG libsupc++.a found, libstdc++-v3 is already built" + fi + + if [[ "$LIBC_FILE" != "libc.a" ]]; then + CAN_BUILD_LIBSTDCXX=1 + [[ "$SYSROOT_HAS_LIBSTDCXX" != "1" ]] && \ + echo "$TAG libc.a found, we can build libstdc++-v3" + else + echo "$TAG libc.a not found, we cannot build libstdc++-v3" + fi +else + echo "$TAG After installing the libc, rebuild here for libstdc++-v3" +fi + +if [[ "$SYSROOT_GCC_VERSION" = "$VERSION" ]]; then + if [[ "$SYSROOT_HAS_LIBSTDCXX" = "1" ]]; then + echo "$TAG GCC $VERSION with libstdc++-v3 already there; skipping rebuild" + mkdir -p build + touch build/giteapc-skip-rebuild.txt exit 0 + elif [[ "$CAN_BUILD_LIBSTDCXX" != "1" ]]; then + echo "$TAG Cannot build libstdc++-v3 yet; skipping rebuild until next time" + mkdir -p build + touch build/giteapc-skip-rebuild.txt + exit 0 + else + echo "$TAG We will proceed with the second stage and build libstdc++-v3" fi fi -# Aggressive parameter to avoid rebuilds - -if [[ ! -z "$ACCEPT_ANY" ]]; then - if command -v sh-elf-gcc >/dev/null 2>&1; then - echo "$TAG sh-elf-gcc in PATH and ACCEPT_ANY is set, skipping build" - if [[ -e build ]]; then - rm -rf build - fi - exit 0 - fi +if [[ ! -z "$ACCEPT_ANY" && ! -z "$SYSROOT_GCC_VERSION" ]]; then + echo "$TAG GCC $VERSION already available; skipping rebuild as per ACCEPT_ANY" + mkdir -p build + touch build/giteapc-skip-rebuild.txt + exit 0 fi -# Download archive +[[ -e "build/giteapc-skip-rebuild.txt" ]] && rm build/giteapc-skip-rebuild.txt + +#--- +# Get sources +#--- if [[ -f "$ARCHIVE" ]]; then echo "$TAG Found $ARCHIVE, skipping download" @@ -50,44 +86,76 @@ else fi fi -# Extract archive (openBSD-compliant version) +# Extract archive (OpenBSD-compliant version) +if [[ -d "${ARCHIVE%.tar.*}" ]]; then + echo "$TAG Found ${ARCHIVE%.tar.*}, skipping extraction" +else + echo "$TAG Extracting $ARCHIVE..." + unxz -c < $ARCHIVE | tar -xf - -echo "$TAG Extracting $ARCHIVE..." -unxz -c < $ARCHIVE | tar -xf - + if [[ "$VERSION" = "11.1.0" ]]; then + echo "$TAG Applying patches/gcc-11.1.0-libstdc++-v3-skip-dlopen.patch..." + patch -u -N -p0 < patches/gcc-11.1.0-libstdc++-v3-skip-dlopen.patch + fi +fi # Download prerequisites - cd gcc-$VERSION ./contrib/download_prerequisites cd .. -# Create build folder +#--- +# Configure +#--- -[[ -d "build" ]] && rm -rf build -mkdir build - -# Configure. GCC does not support make uninstall so we install in this -# directory and later symlink executables to $PREFIX/bin. - -PREFIX="$(pwd)" +# We install in the sysroot and then symlink to the PREFIX folder in the path. # Symlink as, ld, ar and ranlib, which gcc will not find by itself (we renamed # them from sh3eb-elf-* to sh-elf-* with --program-prefix). -mkdir -p sh3eb-elf/bin -ln -sf $(command -v sh-elf-as) sh3eb-elf/bin/as -ln -sf $(command -v sh-elf-ld) sh3eb-elf/bin/ld -ln -sf $(command -v sh-elf-ar) sh3eb-elf/bin/ar -ln -sf $(command -v sh-elf-ranlib) sh3eb-elf/bin/ranlib - -cd build +mkdir -p "$SYSROOT/sh3eb-elf/bin" +for TOOL in as ld ar ranlib; do + if ! command -v sh-elf-$TOOL >/dev/null 2>&1; then + echo "error: sh-elf-$TOOL not found in PATH!" >&2 + exit 1 + fi + ln -sf $(command -v sh-elf-$TOOL) "$SYSROOT/sh3eb-elf/bin/$TOOL" +done # OpenBSD apparently installs these in /usr/local if [[ $(uname) == "OpenBSD" ]]; then - extra_args="--with-gmp=/usr/local --with-mpfr=/usr/local --with-mpc=/usr/local" + EXTRA_ARGS="--with-gmp=/usr/local --with-mpfr=/usr/local --with-mpc=/usr/local" else - extra_args= + EXTRA_ARGS= fi -echo "$TAG Configuring gcc..." -run_quietly giteapc-configure.log \ -../gcc-$VERSION/configure --prefix="$PREFIX" --target=sh3eb-elf --with-multilib-list=m3,m4-nofpu --enable-languages=c,c++ --without-headers --with-newlib --program-prefix=sh-elf- --enable-libssp --enable-lto $extra_args +# In libstdc++-v3 mode, don't reconfigure to avoid recompiling +if [[ "$SYSROOT_GCC_VERSION" = "$VERSION" && "$SYSROOT_HAS_LIBSTDCXX" != "1" && "$CAN_BUILD_LIBSTDCXX" = "1" ]]; then + touch build/giteapc-build-libstdcxx.txt +elif [[ "$SYSROOT_GCC_VERSION" != "$VERSION" ]]; then + echo "$TAG Configuring gcc..." + [[ -d build ]] && rm -rf build + mkdir build + + cd build + run_quietly giteapc-configure.log \ + ../gcc-$VERSION/configure \ + --prefix="$SYSROOT" \ + --target="sh3eb-elf" \ + --with-multilib-list="m3,m4-nofpu" \ + --enable-languages="c,c++" \ + --without-headers \ + --program-prefix="sh-elf-" \ + --enable-libssp \ + --enable-lto \ + --enable-clocale="generic" \ + --enable-libstdcxx-allocator \ + --disable-threads \ + --disable-libstdcxx-verbose \ + --enable-cxx-flags="-fno-exceptions" \ + $EXTRA_ARGS + cd .. +else + echo "$TAG GCC already built, can't build libstdc++-v3; skipping!" + touch build/giteapc-skip-rebuild.txt + exit 0 +fi diff --git a/giteapc-config-clean.make b/giteapc-config-clean.make new file mode 100644 index 0000000..572345d --- /dev/null +++ b/giteapc-config-clean.make @@ -0,0 +1,4 @@ +# Clean the build products after install +CONFIG_CLEAN=1 + +export CONFIG_CLEAN diff --git a/giteapc-config-noclean.make b/giteapc-config-noclean.make deleted file mode 100644 index 6fc1db7..0000000 --- a/giteapc-config-noclean.make +++ /dev/null @@ -1,4 +0,0 @@ -# Don't clean build folder after a build -NO_CLEAN=1 - -export NO_CLEAN diff --git a/install.sh b/install.sh index f9effdd..cecdd26 100755 --- a/install.sh +++ b/install.sh @@ -1,33 +1,33 @@ #! /usr/bin/env bash # Avoid rebuilds of the same version -[[ ! -d build ]] && exit 0 +[[ -e "build/giteapc-skip-rebuild.txt" ]] && exit 0 source util.sh PREFIX="$1" +cd build -if command -v gmake >/dev/null 2>&1; then - make_command=gmake +if [[ -e "giteapc-build-libstdcxx.txt" ]]; then + echo "$TAG Installing libstdc++-v3 to the SuperH sysroot..." + run_quietly giteapc-install-libstdcxx.log \ + $MAKE_COMMAND -j"$cores" install-strip-target-libstdc++-v3 else - make_command=make + echo "$TAG Installing GCC to the SuperH sysroot..." + run_quietly giteapc-install.log \ + $MAKE_COMMAND install-strip-gcc install-strip-target-libgcc + + # Symbolic link executables to $PREFIX/bin + echo "$TAG Symlinking sysroot binaries to $PREFIX/bin..." + mkdir -p "$PREFIX/bin" + for f in "$SYSROOT/bin"/*; do + ln -sf "$f" "$PREFIX/${f#$SYSROOT/}" + done fi -cd build -echo "$TAG Installing to local folder..." -run_quietly giteapc-install.log \ -$make_command install-strip-gcc install-strip-target-libgcc cd .. -# Symbolic link executables to $PREFIX/bin -echo "$TAG Symlinking binaries..." -mkdir -p "$PREFIX/bin" -for x in bin/*; do - ln -sf "$(pwd)/$x" "$PREFIX/$x" -done - -# Cleanup build files -if [[ -z $NO_CLEAN ]]; then +# Cleanup build files after installing libstdc++ +if [[ ! -z "$CONFIG_CLEAN" && -e "build/giteapc-build-libstdcxx.txt" ]]; then echo "$TAG Cleaning up build files..." - rm -rf gcc-*/ gcc-*.tar.* - rm -rf build/ + rm -rf gcc-*/ gcc-*.tar.* build/ fi diff --git a/patches/gcc-11.1.0-libstdc++-v3-skip-dlopen.patch b/patches/gcc-11.1.0-libstdc++-v3-skip-dlopen.patch new file mode 100644 index 0000000..1065424 --- /dev/null +++ b/patches/gcc-11.1.0-libstdc++-v3-skip-dlopen.patch @@ -0,0 +1,65 @@ +# Building libstdc++-v3 for a bare-bones compiler is somewhat in a gray area +# since we don't have a full hosted system to target. From a version of GCC to +# the next this might break in minor ways. +# +# Here, the issue is the reliance on a dlopen() check which cannot be +# completed because we don't have link tests. The intent of the configure +# script is to skip it for cross-compilers, which is materialized by the check +# for --with-newlib (which is somewhat abused to mean "no libc"). +# +# However, building with --with-newlib is not quite right since we have our +# own libc and fails due to it enabling small newlib-specific bits of code +# that don't compile with the fxlibc. +# +# Instead, we disable the test when --without-headers is specified, which is +# the case in our configuration. This extra condition is already part of the +# upstreamed source in GCC 12.1.0, so this is basically a backport. We also +# backport disabling the GLIBCXX_CROSSCONFIG which would block the build. +# +# Explanations about the gray area: +# +# Commit introducing this test in GCC 12.1.0: +# +# GCC bug motivating the commit: +# + +--- gcc-11.1.0/libstdc++-v3/configure.ac 2021-04-27 12:00:16.000000000 +0200 ++++ gcc-11.1.0/libstdc++-v3/configure2.ac 2022-08-15 22:53:21.730140593 +0200 +@@ -90,7 +90,7 @@ + GLIBCXX_CONFIGURE + + # Libtool setup. +-if test "x${with_newlib}" != "xyes" && test "x${with_avrlibc}" != "xyes"; then ++if test "x${with_newlib}" != "xyes" && test "x${with_avrlibc}" != "xyes" && test "x${with_headers}" != "xno"; then + AC_LIBTOOL_DLOPEN + fi + AM_PROG_LIBTOOL +@@ -353,7 +353,7 @@ + + AC_DEFINE(HAVE_ICONV) + AC_DEFINE(HAVE_MEMALIGN) +- else ++ elif test "x${with_headers}" != "xno"; then + GLIBCXX_CROSSCONFIG + fi + +--- gcc-11.1.0/libstdc++-v3/configure 2022-08-15 22:37:29.043460208 +0200 ++++ gcc-11.1.0/libstdc++-v3/configure2 2022-08-15 22:37:03.403459840 +0200 +@@ -5903,7 +5903,7 @@ + + + # Libtool setup. +-if test "x${with_newlib}" != "xyes" && test "x${with_avrlibc}" != "xyes"; then ++if test "x${with_newlib}" != "xyes" && test "x${with_avrlibc}" != "xyes" && test "x${with_headers}" != "xno"; then + enable_dlopen=yes + + +@@ -29266,7 +29266,7 @@ + + $as_echo "#define HAVE_MEMALIGN 1" >>confdefs.h + +- else ++ elif test "x${with_headers}" != "xno"; then + + # Base decisions on target environment. + case "${host}" in diff --git a/uninstall.sh b/uninstall.sh index e186054..a4518e2 100755 --- a/uninstall.sh +++ b/uninstall.sh @@ -3,12 +3,16 @@ source util.sh PREFIX="$1" -# Remove symlinks -echo "$TAG Removing symlinks to binaries..." -for x in bin/*; do - rm "$PREFIX/$x" +echo "$TAG Removing symlinks to $PREFIX/bin..." +TOOLS="c++ cpp g++ gcc gcc-ar gcc-nm gcc-ranlib gcov gcov-dump gcov-tool \ + lto-dump" + +for t in $TOOLS; do + [[ -L "$PREFIX/bin/sh-elf-$t" ]] && rm "$PREFIX/bin/sh-elf-$t" +done +for l in "$PREFIX"/bin/sh3eb-elf-gcc-*; do + [[ -L "$l" ]] && rm "$l" done -# Remove local files -echo "$TAG Removing installed files..." -rm -rf bin/ include/ lib/ libexec/ share/ +echo "$TAG Other files are managed by the fxSDK's SuperH sysroot" +echo "$TAG Uninstall Lephenixnoir/fxsdk to clean up the sysroot" diff --git a/util.sh b/util.sh index 746ad23..999bf10 100644 --- a/util.sh +++ b/util.sh @@ -1,5 +1,21 @@ TAG="" +if command -v gmake >/dev/null 2>&1; then + MAKE_COMMAND=gmake +else + MAKE_COMMAND=make +fi + +if ! command -v fxsdk >/dev/null 2>&1; then + echo "$TAG error: fxSDK is not installed" + exit 1 +elif ! fxsdk path sysroot >/dev/null 2>&1; then + echo "$TAG error: need fxSDK ≥ 2.9 with 'path' command" + exit 1 +fi + +SYSROOT="$(fxsdk path sysroot)" + run_quietly() { out="$1" shift 1