OpenJDK8 编译构建基础设施详解(1) - A New OpenJDK Build-Infra Detail With GNU MAKE And AutoConf 置顶!
你也可以在知乎阅读此文: https://zhuanlan.zhihu.com/p/518013598
从OpenJDK8 开始 , OpenJDK的编译构建过程趋于标准. 从此版本的JDK开始, JDK使用GNU MAKE 标准的两步编译风格来构建OpenJDK的源代码. 一切看起来就像一个最简单的开源软件的编译构建一样简单.
./configure
make
使用如上两步就能够完成JDK的编译构建. 比起以往的JDK复杂的构建方式. 以及难以处理的环境依赖. 简直是一个质的飞跃. 12年发布的openjdk8十年过去了, 此版本的长期维护版本还能够非常容易的在最新(注: 接近最新)的MacOS上编译运行. 且基本做到了零配置, 编译过程极其丝滑 (如需了解Mac环境下的 OpenJDK
的编译详情,可以参考: 基于Mac OS Bigsur 编译OpenJDK8 , 如果想了解Linux环境的编译过程,可以参考: Hotspot调试环境搭建-基于Ubuntu16.04.7-OpenJDK8u-Clion). 这些你便利的可维护性,是因为OpenJDK 采用了 AutoConf
这样的工具来管理编译过程. 用它生成的 Configure
文件能够最大限度的兼容各种各样的操作系统环境,而不需要过多的人工处理.
下面我们一步步的分析下整个过程都发生了些什么.
./configure
生成编译环境
我们先看一下 OpenJDK
的整体目录情况
openjdk 整体包含了特别多的子项目.每一个目录基本都是一个子项目. 比如 hotspot
虚拟机的目录. 还有jdk的目录等.
注: 这些子项目都是以子 仓库的形式单独在
Mercurial
上管理的. 如果看过之前的编译JDK的文章, 可以知道这些子仓库是通过根仓库的脚本文件:get_source.sh
下载的.
文件位置: http://hg.openjdk.java.net/jdk8u/jdk8u/file/a323800a7172/get_source.sh
# 文件位置: http://hg.openjdk.java.net/jdk8u/jdk8u/file/a323800a7172/get_source.sh
# get_source.sh
#
#!/bin/sh
#
# Copyright (c) 2010, 2014, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
# ....
# Get clones of all absent nested repositories (harmless if already exist)
sh ./common/bin/hgforest.sh clone "$@" || exit $?
# Update all existing repositories to the latest sources
sh ./common/bin/hgforest.sh pull -u
./confgure
的内容如下, 我们发现 configure
文件只是一个wrapper文件. 实际是会直接调用: $this_script_dir/common/autoconf/configure
脚本.
#!/bin/bash
# This is a thin wrapper which will call the real configure script, and
# make sure that is called using bash.
# Get an absolute path to this script, since that determines the top-level directory.
this_script_dir=`dirname $0`
this_script_dir=`cd $this_script_dir > /dev/null && pwd`
# Delegate to wrapper, forcing wrapper to believe $0 is this script by using -c.
# This trick is needed to get autoconf to co-operate properly.
bash -c ". $this_script_dir/common/autoconf/configure" $this_script_dir/configure CHECKME $this_script_dir "$@"
而位于位置:$this_script_dir/common/autoconf/configure
的脚本内容如下:
# 检查运行模式是否是从外层的wrapper使用.如果不是发出警告.
if test "x$1" != xCHECKME; then
echo "WARNING: Calling the wrapper script directly is deprecated and unsupported."
echo "Not all features of configure will be available."
echo "Use the 'configure' script in the top-level directory instead."
TOPDIR=$(cd $(dirname $0)/../.. > /dev/null && pwd)
else
# Now the next argument is the absolute top-level directory path.
# The TOPDIR variable is passed on to configure.ac.
TOPDIR="$2"
# Remove these two arguments to get to the user supplied arguments
shift
shift
fi
# 强制 autoconf 使用bash
# Force autoconf to use bash. This also means we must disable autoconf re-exec.
export CONFIG_SHELL=$BASH
export _as_can_reexec=no
# 配置脚本文件的目录是 这个autoconf目录
conf_script_dir="$TOPDIR/common/autoconf"
# 自定义的配置目录一般都为空
if [ "$CUSTOM_CONFIG_DIR" = "" ]; then
conf_custom_script_dir="$TOPDIR/jdk/make/closed/autoconf"
else
conf_custom_script_dir="$CUSTOM_CONFIG_DIR"
fi
###
### 用于测试生成的configure文件是否是最新的.
### Test that the generated configure is up-to-date
###
run_autogen_or_fail() {
if test "x`which autoconf 2> /dev/null`" = x; then
echo "Cannot locate autoconf, unable to correct situation."
echo "Please install autoconf and run 'bash autogen.sh' to update the generated files."
echo "Error: Cannot continue" 1>&2
exit 1
else
echo "Running autogen.sh to correct the situation"
bash $conf_script_dir/autogen.sh
fi
}
# 检查配置脚本目录下的 文件: 1. configure.ac 2. configure.ac目录下的所有的 m4 文件
# 看看这些文件的更新时间是否 更新于: generated-configure.sh , 如果是更新的. 就直接
# 先运行一次 autogen.sh , 用于重新生成 configure 文件
check_autoconf_timestamps() {
for file in $conf_script_dir/configure.ac $conf_script_dir/*.m4 ; do
if test $file -nt $conf_script_dir/generated-configure.sh; then
echo "Warning: The configure source files is newer than the generated files."
run_autogen_or_fail
fi
done
# 此时检查是否存在 自定义的 generated-configure.sh 文件.
if test -e $conf_custom_script_dir/generated-configure.sh; then
# If custom source configure is available, make sure it is up-to-date as well.
# 根据 configure.ac , 以及configure.ac 同目录的m4文件 和 自定义目录下的m4文件是否有更新. 有也重新生成一次
for file in $conf_script_dir/configure.ac $conf_script_dir/*.m4 $conf_custom_script_dir/*.m4; do
if test $file -nt $conf_custom_script_dir/generated-configure.sh; then
echo "Warning: The configure source files is newer than the custom generated files."
run_autogen_or_fail
fi
done
fi
}
check_hg_updates() {
if test "x`which hg 2> /dev/null`" != x; then
conf_updated_autoconf_files=`cd $conf_script_dir && hg status -mard 2> /dev/null | grep autoconf`
if test "x$conf_updated_autoconf_files" != x; then
echo "Configure source code has been updated, checking time stamps"
check_autoconf_timestamps
fi
if test -e $conf_custom_script_dir; then
# If custom source configure is available, make sure it is up-to-date as well.
conf_custom_updated_autoconf_files=`cd $conf_custom_script_dir && hg status -mard 2> /dev/null | grep autoconf`
if test "x$conf_custom_updated_autoconf_files" != x; then
echo "Configure custom source code has been updated, checking time stamps"
check_autoconf_timestamps
fi
fi
fi
}
# Check for local changes
check_hg_updates
if test -e $conf_custom_script_dir/generated-configure.sh; then
# Test if open configure is newer than custom configure, if so, custom needs to
# be regenerated. This test is required to ensure consistency with custom source.
conf_open_configure_timestamp=`grep DATE_WHEN_GENERATED= $conf_script_dir/generated-configure.sh | cut -d"=" -f 2`
conf_custom_configure_timestamp=`grep DATE_WHEN_GENERATED= $conf_custom_script_dir/generated-configure.sh | cut -d"=" -f 2`
if test $conf_open_configure_timestamp -gt $conf_custom_configure_timestamp; then
echo "Warning: The generated configure file contains changes not present in the custom generated file."
run_autogen_or_fail
fi
fi
# Autoconf calls the configure script recursively sometimes.
# Don't start logging twice in that case
if test "x$conf_debug_configure" = xtrue; then
conf_debug_configure=recursive
fi
###
### Process command-line arguments
###
# Returns a shell-escaped version of the argument given.
function shell_quote() {
if [[ -n "$1" ]]; then
# Uses only shell-safe characters? No quoting needed.
# '=' is a zsh meta-character, but only in word-initial position.
if [[ "$1" =~ ^[ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789.:,%/+=_-]+$ && ! "$1" =~ ^= ]]; then
quoted="$1"
else
if [[ "$1" =~ [\'!] ]]; then
# csh does history expansion within single quotes, but not
# when backslash-escaped!
local quoted_quote="'\\''" quoted_exclam="'\\!'"
word="${1//\'/${quoted_quote}}"
word="${1//\!/${quoted_exclam}}"
fi
quoted="'$1'"
fi
echo "$quoted"
fi
}
conf_processed_arguments=()
conf_quoted_arguments=()
conf_openjdk_target=
for conf_option
do
# Process (and remove) our own extensions that will not be passed to autoconf
case $conf_option in
--openjdk-target=*)
conf_openjdk_target=`expr "X$conf_option" : '[^=]*=\(.*\)'`
;;
--debug-configure)
if test "x$conf_debug_configure" != xrecursive; then
conf_debug_configure=true
export conf_debug_configure
fi
;;
*)
conf_processed_arguments=("${conf_processed_arguments[@]}" "$conf_option")
;;
esac
# Store all variables overridden on the command line
case $conf_option in
[^-]*=*)
# Add name of variable to CONFIGURE_OVERRIDDEN_VARIABLES list inside !...!.
conf_env_var=`expr "x$conf_option" : 'x\([^=]*\)='`
CONFIGURE_OVERRIDDEN_VARIABLES="$CONFIGURE_OVERRIDDEN_VARIABLES!$conf_env_var!"
;;
esac
# Save the arguments, intelligently quoted for CONFIGURE_COMMAND_LINE.
case $conf_option in
*=*)
conf_option_name=`expr "x$conf_option" : 'x\([^=]*\)='`
conf_option_name=$(shell_quote "$conf_option_name")
conf_option_value=`expr "x$conf_option" : 'x[^=]*=\(.*\)'`
conf_option_value=$(shell_quote "$conf_option_value")
conf_quoted_arguments=("${conf_quoted_arguments[@]}" "$conf_option_name=$conf_option_value")
;;
*)
conf_quoted_arguments=("${conf_quoted_arguments[@]}" "$(shell_quote "$conf_option")")
;;
esac
# Check for certain autoconf options that require extra action
case $conf_option in
-build | --build | --buil | --bui | --bu |-build=* | --build=* | --buil=* | --bui=* | --bu=*)
conf_legacy_crosscompile="$conf_legacy_crosscompile $conf_option" ;;
-target | --target | --targe | --targ | --tar | --ta | --t | -target=* | --target=* | --targe=* | --targ=* | --tar=* | --ta=* | --t=*)
conf_legacy_crosscompile="$conf_legacy_crosscompile $conf_option" ;;
-host | --host | --hos | --ho | -host=* | --host=* | --hos=* | --ho=*)
conf_legacy_crosscompile="$conf_legacy_crosscompile $conf_option" ;;
-help | --help | --hel | --he | -h)
conf_print_help=true ;;
esac
done
# Save the quoted command line
CONFIGURE_COMMAND_LINE="${conf_quoted_arguments[@]}"
if test "x$conf_legacy_crosscompile" != "x"; then
if test "x$conf_openjdk_target" != "x"; then
echo "Error: Specifying --openjdk-target together with autoconf"
echo "legacy cross-compilation flags is not supported."
echo "You specified: --openjdk-target=$conf_openjdk_target and $conf_legacy_crosscompile."
echo "The recommended use is just --openjdk-target."
exit 1
else
echo "Warning: You are using legacy autoconf cross-compilation flags."
echo "It is recommended that you use --openjdk-target instead."
echo ""
fi
fi
if test "x$conf_openjdk_target" != "x"; then
conf_build_platform=`sh $conf_script_dir/build-aux/config.guess`
conf_processed_arguments=("--build=$conf_build_platform" "--host=$conf_openjdk_target" "--target=$conf_openjdk_target" "${conf_processed_arguments[@]}")
fi
# Make configure exit with error on invalid options as default.
# Can be overridden by --disable-option-checking, since we prepend our argument
# and later options override earlier.
conf_processed_arguments=("--enable-option-checking=fatal" "${conf_processed_arguments[@]}")
###
### Call the configure script
###
if test -e $conf_custom_script_dir/generated-configure.sh; then
# Custom source configure available; run that instead
echo "Running custom generated-configure.sh"
conf_script_to_run=$conf_custom_script_dir/generated-configure.sh
else
echo "Running generated-configure.sh"
conf_script_to_run=$conf_script_dir/generated-configure.sh
fi
if test "x$conf_debug_configure" != x; then
# Turn on shell debug output if requested (initial or recursive)
set -x
fi
# =================== 主流程 ==============================
# 这里一般执行脚本: $conf_script_dir/generated-configure.sh
# =======================================================
# Now transfer control to the script generated by autoconf. This is where the
# main work is done.
conf_logfile=./configure.log
(exec 3>&1 ; (. $conf_script_to_run "${conf_processed_arguments[@]}" 2>&1 1>&3 ) | tee -a $conf_logfile 1>&2 ; exec 3>&-) | tee -a $conf_logfile
conf_result_code=$?
###
### Post-processing
###
if test $conf_result_code -eq 0; then
if test "x$conf_print_help" = xtrue; then
cat <<EOT
Additional (non-autoconf) OpenJDK Options:
--openjdk-target=TARGET cross-compile with TARGET as target platform
(i.e. the one you will run the resulting binary on).
Equivalent to --host=TARGET --target=TARGET
--build=<current platform>
--debug-configure Run the configure script with additional debug
logging enabled.
EOT
# Print list of toolchains. This must be done by the autoconf script.
( CONFIGURE_PRINT_TOOLCHAIN_LIST=true . $conf_script_to_run PRINTF=printf )
cat <<EOT
Please be aware that, when cross-compiling, the OpenJDK configure script will
generally use 'target' where autoconf traditionally uses 'host'.
Also note that variables must be passed on the command line. Variables in the
environment will generally be ignored, unlike traditional autoconf scripts.
EOT
fi
else
echo configure exiting with result code $conf_result_code
fi
exit $conf_result_code
上面脚本的核心逻辑是执行脚本: $conf_script_dir/generated-configure.sh
在执行前会检查此脚本的新旧程度.如果生成此文件的源文件比它新的话会自动更新此文件.这个比较逻辑有两层
- 比较mercurial仓库的文件是否有更新.
- 比较本地的所有的m4文件以及:
configure.ac
文件是否比它新.如果是则自动更新此脚本.
执行逻辑核心代码接抄:
# =================== 主流程 ==============================
# 这里一般执行脚本: $conf_script_dir/generated-configure.sh
# =======================================================
# Now transfer control to the script generated by autoconf. This is where the
# main work is done.
conf_logfile=./configure.log
(exec 3>&1 ; (. $conf_script_to_run "${conf_processed_arguments[@]}" 2>&1 1>&3 ) | tee -a $conf_logfile 1>&2 ; exec 3>&-) | tee -a $conf_logfile
通过以上的分析, 我们知道平时运行的 ./configure
命令实际运行的是: generated-configure.sh 文件. 而这个文件从上面的脚本的更新逻辑看:
###
### Test that the generated configure is up-to-date
###
run_autogen_or_fail() {
if test "x`which autoconf 2> /dev/null`" = x; then
echo "Cannot locate autoconf, unable to correct situation."
echo "Please install autoconf and run 'bash autogen.sh' to update the generated files."
echo "Error: Cannot continue" 1>&2
exit 1
else
echo "Running autogen.sh to correct the situation"
bash $conf_script_dir/autogen.sh
fi
}
这个 generated-configure.sh
自动生成的文件又是通过脚本: bash $conf_script_dir/autogen.sh
来生成的. 这个脚本主要是调用
AutoConf
来生成脚本文件: generated-configure.sh
, 具体见下一小节的分析.
generated-configure.sh
这个文件是
AutoConf
生成的与环境无关的配置check文件. 平时在编译前进行的配置命令:./Configure
就是运行此文件. 此文件是AutoConf
基于输入源文件:
- configure.ac
- *.m4
这两类原始的配置输入文件生成的与AutoConf
无关的环境检查配置脚本.
$conf_script_dir/autogen.sh
我们看一下生成这个脚本文件(generated-configure.sh
)的脚本($conf_script_dir/autogen.sh
):
generate_configure_script() {
# First create a header
cat > $1 << EOT
#!/bin/bash
#
# ##########################################################
# ### THIS FILE IS AUTOMATICALLY GENERATED. DO NOT EDIT. ###
# ##########################################################
#
EOT
# Then replace "magic" variables in configure.ac and append the output
# from autoconf. $2 is either cat (just a no-op) or a filter.
cat $script_dir/configure.ac | sed -e "s|@DATE_WHEN_GENERATED@|$TIMESTAMP|" | \
eval $2 | ${AUTOCONF} -W all -I$script_dir - >> $1
rm -rf autom4te.cache
}
script_dir=`dirname $0`
custom_hook=$custom_script_dir/custom-hook.m4
AUTOCONF="`which autoconf 2> /dev/null | grep -v '^no autoconf in'`"
autoconf_version=`$AUTOCONF --version | head -1`
echo "Using autoconf at ${AUTOCONF} [$autoconf_version]"
echo "Generating generated-configure.sh"
generate_configure_script "$script_dir/generated-configure.sh" 'cat'
以上关键命令是;
cat $script_dir/configure.ac | sed -e "s|@DATE_WHEN_GENERATED@|$TIMESTAMP|" | \
eval $2 | ${AUTOCONF} -W all -I$script_dir - >> $1
简单解释就是使用 configure.ac
文件作为输入, 把文件内容传递给: AutoConf
进行宏展开. 然后把生成的内容输出到参数 $1
, 也就是上面调用此函数的参数: "$script_dir/generated-configure.sh"
也就是我们的 generated-configure.sh
了.
configure.ac
具体源文件见: http://hg.openjdk.java.net/jdk8u/jdk8u/file/a323800a7172/common/autoconf/configure.ac
这里只列出部分.
# 标准 auto_conf输入文件的开始方式.
# 定义了需要的最低版本的AutoConf
# 必须引用宏: AC_INIT ,定义软件版本信息
AC_PREREQ([2.69])
AC_INIT(OpenJDK, jdk8, build-dev@openjdk.java.net,,http://openjdk.java.net)
# 定义autoconf工作时的辅助脚本的位置
AC_CONFIG_AUX_DIR([$TOPDIR/common/autoconf/build-aux])
m4_include([build-aux/pkg.m4])
# 引入一堆的自定义宏.(引入宏的定义.)
# Include these first...
m4_include([basics.m4])
m4_include([basics_windows.m4])
m4_include([builddeps.m4])
# ... then the rest
m4_include([boot-jdk.m4])
m4_include([build-performance.m4])
m4_include([flags.m4])
m4_include([help.m4])
m4_include([jdk-options.m4])
m4_include([libraries.m4])
m4_include([platform.m4])
m4_include([source-dirs.m4])
m4_include([toolchain.m4])
m4_include([toolchain_windows.m4])
# 调用一堆只展开一次的宏
AC_DEFUN_ONCE([CUSTOM_EARLY_HOOK])
AC_DEFUN_ONCE([CUSTOM_LATE_HOOK])
AC_DEFUN_ONCE([CUSTOM_CONFIG_OUTPUT_GENERATED_HOOK])
# This line needs to be here, verbatim, after all includes and the dummy hook
# definitions. It is replaced with custom functionality when building
# custom sources.
#CUSTOM_AUTOCONF_INCLUDE
# 文件生成的时间变量.
# Do not change or remove the following line, it is needed for consistency checks:
DATE_WHEN_GENERATED=@DATE_WHEN_GENERATED@
###############################################################################
#
# Initialization / Boot-strapping
#
# The bootstrapping process needs to solve the "chicken or the egg" problem,
# thus it jumps back and forth, each time gaining something needed later on.
#
###############################################################################
# If we are requested to print additional help, do that and then exit.
# This must be the very first call.
HELP_PRINT_ADDITIONAL_HELP_AND_EXIT
里面是一个标准的 Autoconf
输出文件的开局: configure.ac
,然后引入了一大堆的自定义宏: 我们先挑一个看看定义.:
HELP_PRINT_ADDITIONAL_HELP_AND_EXIT
文件位置: http://hg.openjdk.java.net/jdk8u/jdk8u/file/a323800a7172/common/autoconf/help.m4
# This function will check if we're called from the "configure" wrapper while
# printing --help. If so, we will print out additional information that can
# only be extracted within the autoconf script, and then exit. This must be
# called at the very beginning in configure.ac.
AC_DEFUN_ONCE([HELP_PRINT_ADDITIONAL_HELP_AND_EXIT],
[
if test "x$CONFIGURE_PRINT_TOOLCHAIN_LIST" != x; then
$PRINTF "The following toolchains are available as arguments to --with-toolchain-type.\n"
$PRINTF "Which are valid to use depends on the build platform.\n"
for toolchain in $VALID_TOOLCHAINS_all; do
# Use indirect variable referencing
toolchain_var_name=TOOLCHAIN_DESCRIPTION_$toolchain
TOOLCHAIN_DESCRIPTION=${!toolchain_var_name}
$PRINTF " %-10s %s\n" $toolchain "$TOOLCHAIN_DESCRIPTION"
done
# And now exit directly
exit 0
fi
])
上面列举的是 help.m4
文件中的定义. 而在 configure.ac
中调用这个宏的时候就是把这个宏的内容展开作为输出.最终写到了 generated-configure.sh
, 也就是我们平时在
运行 ./congiure
命令的时候. 可以执行到这一段脚本.
boot-jdk.m4
看一下我们平时编译时会指定的参数:
# mac环境下的编译参数
./configure MAKE=/usr/bin/make --with-toolchain-type=clang --with-debug-level=slowdebug --disable-zip-debug-info --with-target-bits=64 --with-boot-jdk=/Library/Java/JavaVirtualMachines/jdk1.7.0_80.jdk/Contents/Home/ --with-freetype-include=/usr/local/Cellar/freetype/2.12.0/include/freetype2 --with-freetype-lib=/usr/local/Cellar/freetype/2.12.0/lib/
里面有一个配置项就是指定 BOOTJDK
的路径. 参数名为:--with-boot-jdk
, 我们看一下这个配置项是怎么引入进来并且生效的.在 configure.ac
里面有如下几行代码:
###############################################################################
#
# Setup BootJDK, used to bootstrap the build.
#
###############################################################################
BOOTJDK_SETUP_BOOT_JDK
BOOTJDK_SETUP_BOOT_JDK_ARGUMENTS
这里是对两个宏定义的调用(展开),我们关心的应该是第一个,第二个宏看名称应该是处理JDK启动参数的宏. 所以我们先只看下第一个宏:
代码位置: common/autoconf/boot-jdk.m4 : 214行
仓库链接: http://hg.openjdk.java.net/jdk8u/jdk8u/file/a323800a7172/common/autoconf/boot-jdk.m4
###############################################################################
#
# We need a Boot JDK to bootstrap the build.
#
AC_DEFUN_ONCE([BOOTJDK_SETUP_BOOT_JDK],
[
BOOT_JDK_FOUND=no
AC_ARG_WITH(boot-jdk, [AS_HELP_STRING([--with-boot-jdk],
[path to Boot JDK (used to bootstrap build) @<:@probed@:>@])])
# We look for the Boot JDK through various means, going from more certain to
# more of a guess-work. After each test, BOOT_JDK_FOUND is set to "yes" if
# we detected something (if so, the path to the jdk is in BOOT_JDK). But we
# must check if this is indeed valid; otherwise we'll continue looking.
# Test: Is bootjdk explicitely set by command line arguments?
BOOTJDK_DO_CHECK([BOOTJDK_CHECK_ARGUMENTS])
if test "x$with_boot_jdk" != x && test "x$BOOT_JDK_FOUND" = xno; then
# Having specified an argument which is incorrect will produce an instant failure;
# we should not go on looking
AC_MSG_ERROR([The path given by --with-boot-jdk does not contain a valid Boot JDK])
fi
# Test: Is bootjdk available from builddeps?
BOOTJDK_DO_CHECK([BOOTJDK_CHECK_BUILDDEPS])
# Test: Is $JAVA_HOME set?
BOOTJDK_DO_CHECK([BOOTJDK_CHECK_JAVA_HOME])
# Test: Is there a /usr/libexec/java_home? (Typically on MacOSX)
BOOTJDK_DO_CHECK([BOOTJDK_CHECK_LIBEXEC_JAVA_HOME])
# Test: Is there a java or javac in the PATH, which is a symlink to the JDK?
BOOTJDK_DO_CHECK([BOOTJDK_CHECK_JAVA_IN_PATH_IS_SYMLINK])
# Test: Is there a JDK installed in default, well-known locations?
BOOTJDK_DO_CHECK([BOOTJDK_CHECK_WELL_KNOWN_LOCATIONS])
# If we haven't found anything yet, we've truly lost. Give up.
if test "x$BOOT_JDK_FOUND" = xno; then
HELP_MSG_MISSING_DEPENDENCY([openjdk])
AC_MSG_NOTICE([Could not find a valid Boot JDK. $HELP_MSG])
AC_MSG_NOTICE([This might be fixed by explicitely setting --with-boot-jdk])
AC_MSG_ERROR([Cannot continue])
fi
# Setup proper paths for what we found
BOOT_RTJAR="$BOOT_JDK/jre/lib/rt.jar"
if test ! -f "$BOOT_RTJAR"; then
# On MacOSX it is called classes.jar
BOOT_RTJAR="$BOOT_JDK/../Classes/classes.jar"
if test -f "$BOOT_RTJAR"; then
# Remove the ..
BOOT_RTJAR="`cd ${BOOT_RTJAR%/*} && pwd`/${BOOT_RTJAR##*/}"
fi
fi
BOOT_TOOLSJAR="$BOOT_JDK/lib/tools.jar"
BOOT_JDK="$BOOT_JDK"
AC_SUBST(BOOT_RTJAR)
AC_SUBST(BOOT_TOOLSJAR)
AC_SUBST(BOOT_JDK)
# Setup tools from the Boot JDK.
BOOTJDK_CHECK_TOOL_IN_BOOTJDK(JAVA,java)
BOOTJDK_CHECK_TOOL_IN_BOOTJDK(JAVAC,javac)
BOOTJDK_CHECK_TOOL_IN_BOOTJDK(JAVAH,javah)
BOOTJDK_CHECK_TOOL_IN_BOOTJDK(JAVAP,javap)
BOOTJDK_CHECK_TOOL_IN_BOOTJDK(JAR,jar)
BOOTJDK_CHECK_TOOL_IN_BOOTJDK(RMIC,rmic)
BOOTJDK_CHECK_TOOL_IN_BOOTJDK(NATIVE2ASCII,native2ascii)
# Finally, set some other options...
# When compiling code to be executed by the Boot JDK, force jdk7 compatibility.
BOOT_JDK_SOURCETARGET="-source 7 -target 7"
AC_SUBST(BOOT_JDK_SOURCETARGET)
AC_SUBST(JAVAC_FLAGS)
# Check if the boot jdk is 32 or 64 bit
if "$JAVA" -d64 -version > /dev/null 2>&1; then
BOOT_JDK_BITS="64"
else
BOOT_JDK_BITS="32"
fi
AC_MSG_CHECKING([if Boot JDK is 32 or 64 bits])
AC_MSG_RESULT([$BOOT_JDK_BITS])
AC_SUBST(BOOT_JDK_BITS)
])
里面用到了一个宏: AC_ARG_WITH
, 参考https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/autoconf.html#External-Software
Normally
configure
scripts complain about --with-package options that they do not support. See Controlling Checking ofconfigure
Options, for details, and for how to override the defaults.通常,configure 脚本会抱怨 -- with-package 选项,它们不支持这些选项。有关详细信息和如何覆盖缺省值,请参阅控制配置选项的检查。
For each external software package that may be used, configure.ac should call
AC_ARG_WITH
to detect whether theconfigure
user asked to use it. Whether each package is used or not by default, and which arguments are valid, is up to you.对于可能使用的外部软件包,
configure.ac
应该调用AC_ARG_WITH
来检测configure
用户是否请求使用它。默认情况下是否使用每个包,以及哪些参数是有效的,这取决于您。
从上面的文档我们可以得知: 我们现在在考查的 --with-boot-jdk
是一个扩展的选项,默认情况下 configure
是不支持此参数的. 可以做一个实验:
# 文件名: configure.ac
# 文件内容: 只定义一个必须包含的宏: AC_INIT
# Macro: AC_INIT (package, version, [bug-report], [tarname], [url])
AC_INIT([MyHelloSoft],[0.0.1],[lijianhongfir@foxmail.com],,[https://firfor.cn])
使用如下命令进行生成配置命令文件:configure
,并启动配置:
autoconf configure.ac > ./configure && \
chmod u+x ./configure && \
./configure --with-boot-jdk=xxx`
运行后得到如下的输出:
configure: WARNING: unrecognized options: --with-boot-jdk
所以在上面的检查配置项: --with-boot-jdk
的宏定义里面会嵌套一个宏: AC_ARG_WITH
, 它正是用来扩展处理参数读取的. 其定义如下:
Macro: AC_ARG_WITH (package, help-string, [action-if-given], [action-if-not-given])¶
再看一下上面检查bootjdk的代码是这样的:
AC_ARG_WITH(
boot-jdk,
[
AS_HELP_STRING(
[--with-boot-jdk],
[path to Boot JDK (used to bootstrap build) @<:@probed@:>@]
)
]
)
这个宏里面只传递了前两个参数. 后面的两个可选参数是空的. 同时对于第二个参数的:help-string,这里使用了另外一个宏定义来处理.: AS_HELP_STRING
, 再看一下这个宏的定义:
Macro: AS_HELP_STRING (left-hand-side, right-hand-side [indent-column = ‘26’], [wrap-column = ‘79’])¶
Expands into a help string that looks pretty when the user executes ‘configure --help’. It is typically used in
AC_ARG_WITH
(see Working With External Software) orAC_ARG_ENABLE
(see Choosing Package Options). The following example makes this clearer.
这个宏在这里没有太多的作用. 只是定义了这个参数项的说明信息.主要还是由 AC_ARG_WITH
来生成一个变量: withval
上面那串:AC_ARG_WITH
展开后的内容如下:
# Check whether --with-boot-jdk was given.
if test "${with_boot_jdk+set}" = set; then :
withval=$with_boot_jdk;
fi
注: shell中的:
${strname+xxx}
的作用是,这个引用值是: 如果变量:strname
已经定义,那这个引用的结果值就是后面的:xxx
值 ,如果没有定义,就是null. 参考: shell字符串处理-从批量替换文件名说起
这个值会有三种情况:
- 如果我是引用的:
./configure --with-boot-jdk=xxx
, 那这个变量的内容为:xxx
- 如果传递的参数:
./configure --with-boot-jdk
, 那变量的值是:yes
- 如果我用的是:
./configure --without-boot-jdk
, 那变量的值就是no
下图是我使用demo来验证的这个配置的结果输出.
接着往下, 这个 BOOTJDK_SETUP_BOOT_JDK
宏里面,接下来的内容是;
# Test: Is bootjdk explicitely set by command line arguments?
BOOTJDK_DO_CHECK([BOOTJDK_CHECK_ARGUMENTS])
这里调用了一个宏:BOOTJDK_DO_CHECK
,同时这个宏的参数又是另外一个宏:BOOTJDK_CHECK_ARGUMENTS
, 从直接定义看 , 参数应该要先展开. 我们先看看参数的定义:
# Test: Is bootjdk explicitely set by command line arguments?
AC_DEFUN([BOOTJDK_CHECK_ARGUMENTS],
[
if test "x$with_boot_jdk" != x; then
BOOT_JDK=$with_boot_jdk
BOOT_JDK_FOUND=maybe
AC_MSG_NOTICE([Found potential Boot JDK using configure arguments])
fi
])
这个参数定义宏,不是生成一个参数的返回值. 其也是生成一段代码. 代码的作用就是对全局变量:BOOT_JDK
和 BOOT_JDK_FOUND
两者进行赋值. 同时打印一个消息.(打印消息是调用了另外一个宏: AC_MSG_NOTICE
,这是一个AutoConf的内建宏. 所以暂时就不纠结其定义了), 这个宏至于怎么生效.怎么作用于下面的
这个宏至于怎么生效.怎么作用于外面的调用,接下来看一下: BOOTJDK_DO_CHECK
的定义.
# Execute the check given as argument, and verify the result
# If the Boot JDK was previously found, do nothing
# $1 A command line (typically autoconf macro) to execute
# 注: 这里可以看到是把参数1直接的嵌套到了函数代码段里面进行执行.
AC_DEFUN([BOOTJDK_DO_CHECK],
[
if test "x$BOOT_JDK_FOUND" = xno; then
# Now execute the test
$1
# If previous step claimed to have found a JDK, check it to see if it seems to be valid.
if test "x$BOOT_JDK_FOUND" = xmaybe; then
# Do we have a bin/java?
if test ! -x "$BOOT_JDK/bin/java"; then
AC_MSG_NOTICE([Potential Boot JDK found at $BOOT_JDK did not contain bin/java; ignoring])
BOOT_JDK_FOUND=no
else
# Do we have a bin/javac?
if test ! -x "$BOOT_JDK/bin/javac"; then
AC_MSG_NOTICE([Potential Boot JDK found at $BOOT_JDK did not contain bin/javac; ignoring])
AC_MSG_NOTICE([(This might be an JRE instead of an JDK)])
BOOT_JDK_FOUND=no
else
# Do we have an rt.jar? (On MacOSX it is called classes.jar)
if test ! -f "$BOOT_JDK/jre/lib/rt.jar" && test ! -f "$BOOT_JDK/../Classes/classes.jar"; then
AC_MSG_NOTICE([Potential Boot JDK found at $BOOT_JDK did not contain an rt.jar; ignoring])
BOOT_JDK_FOUND=no
else
# Oh, this is looking good! We probably have found a proper JDK. Is it the correct version?
BOOT_JDK_VERSION=`"$BOOT_JDK/bin/java" -version 2>&1 | head -n 1`
# Extra M4 quote needed to protect [] in grep expression.
[FOUND_VERSION_78=`echo $BOOT_JDK_VERSION | grep '\"1\.[78]\.'`]
if test "x$FOUND_VERSION_78" = x; then
AC_MSG_NOTICE([Potential Boot JDK found at $BOOT_JDK is incorrect JDK version ($BOOT_JDK_VERSION); ignoring])
AC_MSG_NOTICE([(Your Boot JDK must be version 7 or 8)])
BOOT_JDK_FOUND=no
else
# We're done! :-)
BOOT_JDK_FOUND=yes
BASIC_FIXUP_PATH(BOOT_JDK)
AC_MSG_CHECKING([for Boot JDK])
AC_MSG_RESULT([$BOOT_JDK])
AC_MSG_CHECKING([Boot JDK version])
BOOT_JDK_VERSION=`"$BOOT_JDK/bin/java" -version 2>&1 | $TR '\n\r' ' '`
AC_MSG_RESULT([$BOOT_JDK_VERSION])
fi # end check jdk version
fi # end check rt.jar
fi # end check javac
fi # end check java
fi # end check boot jdk found
fi
])
这个就是对传入的参数在没有找到JDK的时候. 使用传入的JDK的参数进行检查.:这个代码展开后的片段如下:
这个参数的处理就这么一些,比较简单. 接着看.BOOTJDK_DO_CHECK
的内容, 这个内容如上面的源代码所示. 里面会进行可能位置的JDK的一些check:
- 检查是否存在
javac
, 这个是用于java 的class编译的命令; - 是否存储
rt.jar
这个是jdk的运行时的主要文件. - 检查JDK的版本是否是我们想要的.
- 最后设定JDK查找的结果. 最后对路径进行一次修正:调用
BASIC_FIXUP_PATH
进行修复.
这个修复宏如下:
# This will make sure the given variable points to a full and proper
# path. This means:
# 1) There will be no spaces in the path. On posix platforms,
# spaces in the path will result in an error. On Windows,
# the path will be rewritten using short-style to be space-free.
# 2) The path will be absolute, and it will be in unix-style (on
# cygwin).
# $1: The name of the variable to fix
AC_DEFUN([BASIC_FIXUP_PATH],
[
if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then
BASIC_FIXUP_PATH_CYGWIN($1)
elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
BASIC_FIXUP_PATH_MSYS($1)
else
# We're on a posix platform. Hooray! :)
path="[$]$1"
has_space=`$ECHO "$path" | $GREP " "`
if test "x$has_space" != x; then
AC_MSG_NOTICE([The path of $1, which resolves as "$path", is invalid.])
AC_MSG_ERROR([Spaces are not allowed in this path.])
fi
# Use eval to expand a potential ~
eval path="$path"
if test ! -f "$path" && test ! -d "$path"; then
AC_MSG_ERROR([The path of $1, which resolves as "$path", is not found.])
fi
$1="`cd "$path"; $THEPWDCMD -L`"
fi
])
可以看到,这个脚本实际只是对路径String的一个风格的修复. 无它.下面再来看一下这个 ./configure
文件的运行结果:
这里打印的消息,基本与上面的check后面的消息输出是一一对应上的. 所以对于 ./configure
流程的原理与过程就基本讲清楚了.下面给出一张图来描述这个过程.
注: 上面讲解的步骤.主要是两个线,一个是用户调用
./configure
最终要执行哪些脚本 . 这个基本只是说明了执行流程. 最终是执行autoconf生成的:generated-configure.sh
脚本. 但是对于这个脚本的输出的build
目录的构建过程.没有太多展开. 我们上面的内容主要是介绍了:
generate-configure.sh
脚本是怎么生成的.- 它的内容是哪些. 这些内容是如何生成的.
- 重点展开讲了一个JDK相关的配置检查的脚本内容的生成过程.
我们可以看一下运行 ./configure
后的build 目录的基本情况:
./configure
命令产物:build/macosx-x86_64-normal-server-slowdebug
目录文件详解
配置目录产生的内容列表如下:
MacBook-Pro ~/IdeaProjects/jdk8u/build/macosx-x86_64-normal-server-slowdebug]$ls -1
Makefile
bootcycle-spec.gmk
compare.sh
config.h
config.log
config.status
configure.log
configure.log.old
hotspot-spec.gmk
spec.gmk
spec.sh
我们看一下 configure.ac
引用的: basics.m4
中有如下定义:
宏名: BASIC_SETUP_OUTPUT_DIR (基础_设置_输出目录)
AC_DEFUN_ONCE([BASIC_SETUP_OUTPUT_DIR],
[
# 说明: 定义一个配置名称, 可以用于覆盖默认的build下的配置目录名称
AC_ARG_WITH(conf-name, [AS_HELP_STRING([--with-conf-name],
[use this as the name of the configuration @<:@generated from important configuration options@:>@])],
[ CONF_NAME=${with_conf_name} ])
# 测试我们从哪开始运行的 confiugre .是在源代码目录的里面或者是外面
# Test from where we are running configure, in or outside of src root.
AC_MSG_CHECKING([where to store configuration])
if test "x$CURDIR" = "x$SRC_ROOT" || test "x$CURDIR" = "x$SRC_ROOT/common" \
|| test "x$CURDIR" = "x$SRC_ROOT/common/autoconf" \
|| test "x$CURDIR" = "x$SRC_ROOT/make" ; then
# We are running configure from the src root.
# Create a default ./build/target-variant-debuglevel output root.
# 创建构建的输出目录.如果
# 1. 没有传入的CONF_NAME参数.则使用默认的目录名称. 操作系统类别_目标CPU_JDK类型_JVM类型_调试级别
# 2. 如果传入了CONF_NAME , 直接使用传入的配置输出目录.
if test "x${CONF_NAME}" = x; then
AC_MSG_RESULT([in default location])
CONF_NAME="${OPENJDK_TARGET_OS}-${OPENJDK_TARGET_CPU}-${JDK_VARIANT}-${ANDED_JVM_VARIANTS}-${DEBUG_LEVEL}"
else
AC_MSG_RESULT([in build directory with custom name])
fi
OUTPUT_ROOT="$SRC_ROOT/build/${CONF_NAME}"
# 创建输出build的目录
$MKDIR -p "$OUTPUT_ROOT"
if test ! -d "$OUTPUT_ROOT"; then
AC_MSG_ERROR([Could not create build directory $OUTPUT_ROOT])
fi
else
# We are running configure from outside of the src dir.
# Then use the current directory as output dir!
# If configuration is situated in normal build directory, just use the build
# directory name as configuration name, otherwise use the complete path.
if test "x${CONF_NAME}" = x; then
CONF_NAME=`$ECHO $CURDIR | $SED -e "s!^${SRC_ROOT}/build/!!"`
fi
OUTPUT_ROOT="$CURDIR"
AC_MSG_RESULT([in current directory])
# WARNING: This might be a bad thing to do. You need to be sure you want to
# have a configuration in this directory. Do some sanity checks!
if test ! -e "$OUTPUT_ROOT/spec.gmk"; then
# If we have a spec.gmk, we have run here before and we are OK. Otherwise, check for
# other files
files_present=`$LS $OUTPUT_ROOT`
# Configure has already touched config.log and confdefs.h in the current dir when this check
# is performed.
filtered_files=`$ECHO "$files_present" \
| $SED -e 's/config.log//g' \
-e 's/configure.log//g' \
-e 's/confdefs.h//g' \
-e 's/ //g' \
| $TR -d '\n'`
if test "x$filtered_files" != x; then
AC_MSG_NOTICE([Current directory is $CURDIR.])
AC_MSG_NOTICE([Since this is not the source root, configure will output the configuration here])
AC_MSG_NOTICE([(as opposed to creating a configuration in <src_root>/build/<conf-name>).])
AC_MSG_NOTICE([However, this directory is not empty. This is not allowed, since it could])
AC_MSG_NOTICE([seriously mess up just about everything.])
AC_MSG_NOTICE([Try 'cd $SRC_ROOT' and restart configure])
AC_MSG_NOTICE([(or create a new empty directory and cd to it).])
AC_MSG_ERROR([Will not continue creating configuration in $CURDIR])
fi
fi
fi
AC_MSG_CHECKING([what configuration name to use])
AC_MSG_RESULT([$CONF_NAME])
BASIC_FIXUP_PATH(OUTPUT_ROOT)
##
# 这里设置好了三个用于替换的变量. 用于后面的 *.in 文件的替换
##
AC_SUBST(SPEC, $OUTPUT_ROOT/spec.gmk)
AC_SUBST(CONF_NAME, $CONF_NAME)
AC_SUBST(OUTPUT_ROOT, $OUTPUT_ROOT)
# Most of the probed defines are put into config.h
AC_CONFIG_HEADERS([$OUTPUT_ROOT/config.h:$AUTOCONF_DIR/config.h.in])
# The spec.gmk file contains all variables for the make system.
AC_CONFIG_FILES([$OUTPUT_ROOT/spec.gmk:$AUTOCONF_DIR/spec.gmk.in])
# The hotspot-spec.gmk file contains legacy variables for the hotspot make system.
AC_CONFIG_FILES([$OUTPUT_ROOT/hotspot-spec.gmk:$AUTOCONF_DIR/hotspot-spec.gmk.in])
# The bootcycle-spec.gmk file contains support for boot cycle builds.
AC_CONFIG_FILES([$OUTPUT_ROOT/bootcycle-spec.gmk:$AUTOCONF_DIR/bootcycle-spec.gmk.in])
# The compare.sh is used to compare the build output to other builds.
AC_CONFIG_FILES([$OUTPUT_ROOT/compare.sh:$AUTOCONF_DIR/compare.sh.in])
# Spec.sh is currently used by compare-objects.sh
AC_CONFIG_FILES([$OUTPUT_ROOT/spec.sh:$AUTOCONF_DIR/spec.sh.in])
# The generated Makefile knows where the spec.gmk is and where the source is.
# You can run make from the OUTPUT_ROOT, or from the top-level Makefile
# which will look for generated configurations
AC_CONFIG_FILES([$OUTPUT_ROOT/Makefile:$AUTOCONF_DIR/Makefile.in])
])
前面的内容暂且不表,看一下最后的定义内容: 这里引用了两个宏:
-
AC_CONFIG_HEADERS:
- Macro: AC_CONFIG_HEADERS (header …, [cmds], [init-cmds])
-
AC_CONFIG_FILES:
Make
AC_OUTPUT
create each file by copying an input file (by default file.in), substituting the output variable values. This macro is one of the instantiating macros; see Performing Configuration Actions. See Substitutions in Makefiles, for more information on using output variables. See Setting Output Variables, for more information on creating them. This macro creates the directory that the file is in if it doesn’t exist. Usually, makefiles are created this way, but other files, such as .gdbinit, can be specified as well.
然后直接看一下这些个宏展开生成的shell:
#
# 文件名: common/autoconf/generated-configure.sh
#
# Most of the probed defines are put into config.h
ac_config_headers="$ac_config_headers $OUTPUT_ROOT/config.h:$AUTOCONF_DIR/config.h.in"
# The spec.gmk file contains all variables for the make system.
ac_config_files="$ac_config_files $OUTPUT_ROOT/spec.gmk:$AUTOCONF_DIR/spec.gmk.in"
# The hotspot-spec.gmk file contains legacy variables for the hotspot make system.
ac_config_files="$ac_config_files $OUTPUT_ROOT/hotspot-spec.gmk:$AUTOCONF_DIR/hotspot-spec.gmk.in"
# The bootcycle-spec.gmk file contains support for boot cycle builds.
ac_config_files="$ac_config_files $OUTPUT_ROOT/bootcycle-spec.gmk:$AUTOCONF_DIR/bootcycle-spec.gmk.in"
# The compare.sh is used to compare the build output to other builds.
ac_config_files="$ac_config_files $OUTPUT_ROOT/compare.sh:$AUTOCONF_DIR/compare.sh.in"
# Spec.sh is currently used by compare-objects.sh
ac_config_files="$ac_config_files $OUTPUT_ROOT/spec.sh:$AUTOCONF_DIR/spec.sh.in"
# The generated Makefile knows where the spec.gmk is and where the source is.
# You can run make from the OUTPUT_ROOT, or from the top-level Makefile
# which will look for generated configurations
ac_config_files="$ac_config_files $OUTPUT_ROOT/Makefile:$AUTOCONF_DIR/Makefile.in"
可以看到这些个宏在直接展开的时候,都是直接收集了相应的要拷贝的文件的输入和输了的相关路径的配置 .所有的文件列表的名称以空格分隔. 最后应该是由如下命令(不是很确定,如有错误,请指正):
eval set X " :F $CONFIG_FILES :H $CONFIG_HEADERS "
AC_CONFIG_FILES
结合着AC_SUBST
,这样就可以完成 *.in
文件的变量替换,并且把输出写入到指定的目标文件.举一个例子:我们看一下common/autoconf/Makefile.in
的文件的内容:
# fileName: common/autoconf/Makefile.in
SPEC:=@OUTPUT_ROOT@/spec.gmk
include @TOPDIR@/Makefile
我们看一下输出目录下的文件内容, 我们可以看到: 以@开始和@结束包裹起来的变量被替换为了实际的值
# 文件路径: build/macosx-x86_64-normal-server-slowdebug/Makefile
# This Makefile was generated by configure Sun May 22 20:08:01 CST 2022
# GENERATED FILE, DO NOT EDIT
SPEC:=/Users/lijianhong/IdeaProjects/jdk8u/build/macosx-x86_64-normal-server-slowdebug/spec.gmk
include /Users/lijianhong/IdeaProjects/jdk8u/Makefile
上面的列举是普通文件的替换. .h文件有一点不一样. 具体细节可以参考: autoconf-2.71/autoconf.html#Configuration-Headers
下面以上面提到的hotsport-spec.gmk
的输入与输入的部分对比作为这篇文章的结尾:
简单总结一下:
- 我们整体看了一下
./configure
命中的执行过程. - 以及我们执行
./configure
命令,实际上是在执行 generated-configure.sh - 然后我们又看了一下
generated-configure.sh
脚本是怎么来的. 顺便解释了一下AutoConf 处理的流程与原理. - 最后看了一下
build/xxx
目录的内容.以及这些内容是怎样生成的.
由于内容过多, Makefile的编译过程的梳理放到另外一篇新的文章中进行分析. 此文章到此就基本更新完成了. 如果你觉得还有点帮助的话. 请帮点个赞谢谢.