8264449: Enable reproducible builds with SOURCE_DATE_EPOCH #446

jgneff · 2021-03-30T16:20:21Z

This pull request allows for reproducible builds of JavaFX on Linux, macOS, and Windows by defining the SOURCE_DATE_EPOCH environment variable. For example, the following commands create a reproducible build:

$ export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
$ bash gradlew sdk jmods javadoc
$ strip-nondeterminism -v -T $SOURCE_DATE_EPOCH build/jmods/*.jmod

The three commands:

set the build timestamp to the date of the latest source code change,
build the JavaFX SDK libraries, JMOD archives, and API documentation, and
recreate the JMOD files with stable file modification times and ordering.

The third command won't be necessary once Gradle can build the JMOD archives or the jmod tool itself has the required support. For more information on the environment variable, see the SOURCE_DATE_EPOCH page. For more information on the command to recreate the JMOD files, see the strip-nondeterminism repository. I'd like to propose that we allow for reproducible builds in JavaFX 17 and consider making them the default in JavaFX 18.

Fixes

There are at least four sources of non-determinism in the JavaFX builds:

Build timestamp

The class com.sun.javafx.runtime.VersionInfo in the JavaFX Base module stores the time of the build. Furthermore, for builds that don't run on the Hudson continuous integration tool, the class adds the build time to the system property javafx.runtime.version.
Modification times

The JAR, JMOD, and ZIP archives store the modification time of each file.
File ordering

The JAR, JMOD, and ZIP archives store their files in the order returned by the file system. The native shared libraries also store their object files in the order returned by the file system. Most file systems, though, do not guarantee the order of a directory's file listing.
Build path

The class com.sun.javafx.css.parser.Css2Bin in the JavaFX Graphics module stores the absolute path of its .css input file in the corresponding .bss output file, which is then included in the JavaFX Controls module.

This pull request modifies the Gradle and Groovy build files to fix the first three sources of non-determinism. A later pull request can modify the Java files to fix the fourth.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issues

JDK-8264449: Enable reproducible builds with SOURCE_DATE_EPOCH (Enhancement - P4)
JDK-8238650: Allow to override buildDate with SOURCE_DATE_EPOCH (Enhancement - P4)

Reviewers

Alexander Matveev (@sashamatveev - Reviewer) 🔄 Re-review required (review applies to afeb3fe0)
Arun Joseph (@arun-joseph - Reviewer) 🔄 Re-review required (review applies to cb8510c6)
Kevin Rushforth (@kevinrushforth - Reviewer)
Johan Vos (@johanvos - Reviewer)

Contributors

Bernhard M. Wiedemann <javabmw@lsmod.de>

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jfx.git pull/446/head:pull/446
$ git checkout pull/446

Update a local copy of the PR:
$ git checkout pull/446
$ git pull https://git.openjdk.org/jfx.git pull/446/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 446

View PR using the GUI difftool:
$ git pr show -t 446

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jfx/pull/446.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2021-03-30T16:20:44Z

👋 Welcome back jgneff! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

mlbridge · 2021-03-30T16:24:33Z

Webrevs

jgneff · 2021-03-30T16:30:25Z

I think there might be a Skara bug. The pre-submit builds on Linux, macOS, and Windows completed immediately. I think that's because the first of the two commits in this pull request includes the Java Bug ID from another pending pull request, because this pull request is a continuation of that one. I can squash the two commits and force-push the changes, if that would help.

jgneff · 2021-03-30T19:07:31Z

I think there might be a Skara bug.

No, no bug. Sorry about that. I just don't know how GitHub works. ☹️ The pre-submit tests ran two days ago when I pushed the branch to GitHub, so by the time I submitted the pull request, they had finished long ago.

kevinrushforth · 2021-03-31T13:25:48Z

/reviewers 2

openjdk · 2021-03-31T13:26:05Z

@kevinrushforth
The number of required reviews for this PR is now set to 2 (with at least 1 of role reviewers).

kevinrushforth · 2021-03-31T13:59:12Z

I recommend trying this with the following gradle flags, to match the settings for production builds:

-PCONF=Release -PPROMOTED_BUILD_NUMBER=NNN -PHUDSON_BUILD_NUMBER=MMM -PHUDSON_JOB_NAME=jfx -PCOMPILE_WEBKIT=true -PCOMPILE_MEDIA=true -PBUILD_LIBAV_STUBS=true

where NNN is the promoted build number that is being built (usually taken from the repo tag) and MMM is the CI build number. You can just pick any two positive numbers for your test builds.

Note that this will build the native media libraries and the native webkit library.

jgneff · 2021-03-31T23:08:38Z

I recommend trying this with the following gradle flags, to match the settings for production builds:

Thanks, Kevin. Good news so far. I'm posting the Linux results while I run the macOS and Windows builds.

Linux

I ran the following commands twice, moving the build directory to build1 and then build2 to save their output:

$ bash gradlew clean
$ bash gradlew -PCONF=Release -PPROMOTED_BUILD_NUMBER=5 \
    -PHUDSON_BUILD_NUMBER=101 -PHUDSON_JOB_NAME=jfx \
    -PCOMPILE_WEBKIT=true -PCOMPILE_MEDIA=true \
    -PBUILD_LIBAV_STUBS=true sdk jmods javadoc

Then I changed the Hudson job number with -PHUDSON_BUILD_NUMBER=102, ran the build a third time, and moved build to build3. I also ran strip-nondeterminism as shown in the first comment of this pull request.

The first result is as hoped, and the second result is as expected:

$ diff -qr build1 build2
$ diff -qr build2 build3
Files build2/jmods/javafx.base.jmod
   and build3/jmods/javafx.base.jmod
   differ
Files build2/modular-sdk/modules/javafx.base/com/sun/javafx/runtime/VersionInfo.class
  and build3/modular-sdk/modules/javafx.base/com/sun/javafx/runtime/VersionInfo.class
  differ
Files build2/modular-sdk/modules_src/javafx.base/com/sun/javafx/runtime/VersionInfo.java
  and build3/modular-sdk/modules_src/javafx.base/com/sun/javafx/runtime/VersionInfo.java
  differ
Files build2/publications/javafx.base-linux.jar
  and build3/publications/javafx.base-linux.jar
  differ
Files build2/sdk/lib/javafx.base.jar
  and build3/sdk/lib/javafx.base.jar
  differ
Files build2/sdk/lib/src.zip
  and build3/sdk/lib/src.zip
  differ

You have to appreciate the irony of adding all this information to the build — the time, the path, even the job number — so that we can uniquely identify a build by its output. Meanwhile, if we didn't add this information, our builds could be uniquely identified by a single Git tag.

jgneff · 2021-04-01T15:45:15Z

There's good news and bad news. Good news first.

macOS

The two comparisons of the three builds on macOS were similar to those on Linux:

$ diff -qr build1 build2
$ diff -qr build2 build3
Files build2/jmods/javafx.base.jmod
  and build3/jmods/javafx.base.jmod
  differ
Files build2/modular-sdk/modules/javafx.base/com/sun/javafx/runtime/VersionInfo.class
  and build3/modular-sdk/modules/javafx.base/com/sun/javafx/runtime/VersionInfo.class
  differ
Files build2/modular-sdk/modules_src/javafx.base/com/sun/javafx/runtime/VersionInfo.java
  and build3/modular-sdk/modules_src/javafx.base/com/sun/javafx/runtime/VersionInfo.java
  differ
Files build2/publications/javafx.base-mac.jar
  and build3/publications/javafx.base-mac.jar
  differ
Files build2/sdk/lib/javafx.base.jar
  and build3/sdk/lib/javafx.base.jar
  differ
Files build2/sdk/lib/src.zip
  and build3/sdk/lib/src.zip
  differ

Windows

The Windows build, on the other hand, failed to produce identical copies of the media and WebKit shared libraries:

$ diff -qr build1 build2
Files build1/jmods/javafx.media.jmod
  and build2/jmods/javafx.media.jmod
  differ
Files build1/jmods/javafx.web.jmod
  and build2/jmods/javafx.web.jmod
  differ
Files build1/modular-sdk/modules_libs/javafx.media/fxplugins.dll
  and build2/modular-sdk/modules_libs/javafx.media/fxplugins.dll
  differ
Files build1/modular-sdk/modules_libs/javafx.media/glib-lite.dll
  and build2/modular-sdk/modules_libs/javafx.media/glib-lite.dll
  differ
Files build1/modular-sdk/modules_libs/javafx.media/gstreamer-lite.dll
  and build2/modular-sdk/modules_libs/javafx.media/gstreamer-lite.dll
  differ
Files build1/modular-sdk/modules_libs/javafx.media/jfxmedia.dll
  and build2/modular-sdk/modules_libs/javafx.media/jfxmedia.dll
  differ
Files build1/modular-sdk/modules_libs/javafx.web/jfxwebkit.dll
  and build2/modular-sdk/modules_libs/javafx.web/jfxwebkit.dll
  differ
Files build1/publications/javafx.media-win.jar
  and build2/publications/javafx.media-win.jar
  differ
Files build1/publications/javafx.web-win.jar
  and build2/publications/javafx.web-win.jar
  differ
Files build1/sdk/bin/fxplugins.dll
  and build2/sdk/bin/fxplugins.dll
  differ
Files build1/sdk/bin/glib-lite.dll
  and build2/sdk/bin/glib-lite.dll
  differ
Files build1/sdk/bin/gstreamer-lite.dll
  and build2/sdk/bin/gstreamer-lite.dll
  differ
Files build1/sdk/bin/jfxmedia.dll
  and build2/sdk/bin/jfxmedia.dll
  differ
Files build1/sdk/bin/jfxwebkit.dll
  and build2/sdk/bin/jfxwebkit.dll
  differ

I didn't bother running the third build with -PHUDSON_BUILD_NUMBER=102. I assume CMake would be the same across platforms. I'm hoping it's just a missing /experimental:deterministic somewhere for the Windows linker. I'll track it down.

jgneff · 2021-04-01T16:50:31Z

@kevinrushforth I found the Makefiles building the media libraries for Windows. I can fix those, but there's also libxml and libxslt that invoke the Windows linker here:

modules/javafx.web/src/main/native/Source/ThirdParty/libxml/src/win32/Makefile.msvc
modules/javafx.web/src/main/native/Source/ThirdParty/libxslt/src/win32/Makefile.msvc

They both state at the top, "There should never be a need to modify anything below this line." Are those files directly from their upstream sources? If so, what's our policy on patching them while waiting for fixes?

kevinrushforth · 2021-04-01T17:22:16Z

The WebKit build is complicated (to say the least). All of the WebKit components, including the additional third-party dependencies in /Source/ThirdParty, are built using cmake. I think any changes would involve passing flags to cmake in build.gradle.

@arun-joseph might be able to comment on that.

kevinrushforth · 2021-04-01T17:33:35Z

I should add that if changes are needed to the Makefile.msvc files to accept extra link flags to be passed in, then it would be fine to modify them.

arun-joseph · 2021-04-01T17:44:58Z

It would be better to add the flag in libxml/CMakeLists.txt or in build.gradle via cmakeArgs.

kevinrushforth · 2021-04-01T17:50:47Z

Ideally the latter, if feasible, so that the logic to handle SOURCE_DATE_EPOCH is more localized.

Enable reproducible builds of the native media shared libraries for Windows when SOURCE_DATE_EPOCH is defined. The libraries are: fxplugins.dll glib-lite.dll gstreamer-lite.dll jfxmedia.dll

bmwiedemann · 2021-04-01T19:38:18Z

IMHO, it would make sense to merge any clear improvement of the status-quo and debug+fix more in a later PR. "Perfect is the enemy of good."

Regarding Webkit, I know of
https://bugs.webkit.org/show_bug.cgi?id=174540
https://bugs.webkit.org/show_bug.cgi?id=188738
https://build.opensuse.org/request/show/667729
but your version is probably not that old.

kevinrushforth · 2021-04-01T20:07:12Z

Given the level of effort to test this, I would recommend minimizing the number of separate fixes for this enhancement. If WebKit turns out to be too big a can of worms, then it might make sense to do everything else with this fix and file a follow-on for WebKit.

Regarding Webkit...but your version is probably not that old.

Right, we aren't nearly that old. We just updated WebKit a little over 2 months ago with recent sources (we update WebKit every six months).

jgneff · 2021-04-01T20:18:03Z

IMHO, it would make sense to merge any clear improvement of the status-quo and debug+fix more in a later PR.

I agree that we will have to draw the line somewhere, but right now I'm down to just one file on one operating system out of 10,633 files in the build output! (It's just jfxwebkit.dll on Windows.) It would be a shame to give up now.

I would, though, like to postpone tests of static builds and builds for Android and iOS. My priorities for this pull request are:

Linux distributions building JavaFX
Developers building JavaFX from the repository source with the build defaults
Oracle, Gluon, and BellSoft building JavaFX for their customers and for downloading

I would be happy to satisfy just the first group, but we might be close to getting all three.

bmwiedemann · 2021-04-01T20:24:33Z

/contributor add Bernhard M. Wiedemann javabmw@lsmod.de

openjdk · 2021-04-01T20:25:32Z

@bmwiedemann Only the author (@jgneff) is allowed to issue the contributor command.

kevinrushforth · 2021-04-01T21:26:44Z

@sashamatveev Can you look at the changes to the media Makefiles?

sashamatveev

Media makefiles look fine.

mlbridge · 2021-04-02T13:22:06Z

Mailing list message from Eric Bresie on openjfx-dev:

Silly question, is the difference with Windows just the nature of the native support on each platform (Unix based vs Windows) and libraries used as part of that?

Eric Bresie
Ebresie at gmail.com (mailto:Ebresie at gmail.com)

jgneff · 2023-04-07T06:24:02Z

I reverted the version timestamp to the original format. The version looks like this for "release" early-access builds:

$ cat build5/sdk/lib/javafx.properties
javafx.version=21-ea
javafx.runtime.version=21-ea+12
javafx.runtime.build=12

and like this for developer builds:

$ cat build1/sdk/lib/javafx.properties
javafx.version=21-internal
javafx.runtime.version=21-internal+0-2023-04-06-232926
javafx.runtime.build=0

The only difference is that the timestamp is localized to UTC, while before it was in the local time of the build machine.

The jmod --date option requires the ISO 8601 extended format, so we need at least two different formats for the timestamp. Meanwhile, Java has no built-in support for parsing an ISO 8601 string in basic format, so both the current non-standard format and the standard basic format need a DateTimeFormatter with a custom pattern. There's no advantage from a parsing perspective in using the standard format:

    // Parses the format in the current JavaFX release
    var currentFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd-HHmmss");
    var localTime = LocalDateTime.parse(timestamp, currentFormatter);

    // Parses the ISO 8601 basic format
    var basicFormatter = DateTimeFormatter.ofPattern("yyyyMMdd'T'HHmmssX");
    var zonedTime = ZonedDateTime.parse(timestamp, basicFormatter);

jgneff · 2023-04-07T16:05:28Z

For the record, here's the latest version of my program that tests the timestamp formats:

import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneOffset;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;

public class Timestamp {

    private static final String OPT = "([-a-zA-Z0-9.]+)";
    private static final String OK = " (OK)";
    private static final String NOT_OK = " (FAILED)";

    public static void main(String[] args) {
        var buildInstant = Instant.now().truncatedTo(ChronoUnit.SECONDS);

        // Creates the timestamp in the ISO 8601 extended format
        String extended = buildInstant.toString();

        // Creates the timestamp in the current non-stnadard format
        var zonedTime = ZonedDateTime.ofInstant(buildInstant, ZoneOffset.UTC);
        var currentFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd-HHmmss");
        String current = zonedTime.format(currentFormatter);

        // Creates the timestamp in the ISO 8601 basic format
        zonedTime = ZonedDateTime.ofInstant(buildInstant, ZoneOffset.UTC);
        var basicFormatter = DateTimeFormatter.ofPattern("yyyyMMdd'T'HHmmssX");
        String basic = zonedTime.format(basicFormatter);

        // Prints the timestamps and checks them for the version OPT field
        System.out.print("ISO 8601 extended format = " + extended);
        System.out.println(extended.matches(OPT) ? OK : NOT_OK);
        System.out.print("Current JavaFX format    = " + current);
        System.out.println(current.matches(OPT) ? OK : NOT_OK);
        System.out.print("ISO 8601 basic format    = " + basic);
        System.out.println(basic.matches(OPT) ? OK : NOT_OK);
        System.out.println();

        // Parses the timestamp in the ISO 8601 extended format
        zonedTime = ZonedDateTime.parse(extended);
        System.out.println("Parsed extended time = " + zonedTime);

        // Parses the timestamp in the current non-standard format
        var localTime = LocalDateTime.parse(current, currentFormatter);
        System.out.println("Parsed current time  = " + localTime);

        // Parses the timestamp in the ISO 8601 basic format
        zonedTime = ZonedDateTime.parse(basic, basicFormatter);
        System.out.println("Parsed basic time    = " + zonedTime);
    }
}

Below is a sample of its output:

$ java Timestamp
ISO 8601 extended format = 2023-04-07T16:01:16Z (FAILED)
Current JavaFX format    = 2023-04-07-160116 (OK)
ISO 8601 basic format    = 20230407T160116Z (OK)

Parsed extended time = 2023-04-07T16:01:16Z
Parsed current time  = 2023-04-07T16:01:16
Parsed basic time    = 2023-04-07T16:01:16Z

kevinrushforth · 2023-04-07T21:01:29Z

Your solution using two formatted date strings looks good to me. Thanks for the additional detail.

jgneff · 2023-04-26T03:59:36Z

I solved the build failure occurring on Windows by downgrading to Visual Studio 2022 Build Tools version 17.1.0, dated February 15, 2022, from the prior version 17.5.4 that I was using. I opened a bug report, linked below, as a heads-up for when we upgrade Visual Studio.

I'm now using the following versions of the tools:

Toolchains

Linux: gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
macOS: Command Line Tools for Xcode 14.2 version 14.2.0.0.1.1668646533
Windows: Visual Studio Build Tools 2022 version 17.1.0, build number 17.1.32210.238

Build Tools

JDK: OpenJDK Runtime Environment (build 19.0.2+7-44)
Ant: Apache Ant(TM) version 1.10.13 compiled on January 4 2023
CMake: cmake version 3.26.3

My latest test results are:

System	Develop	Actions	Release	Notes
Linux	✔	✔	❌	`libjfxwebkit.so` differs
macOS	✔	✔	✔	Just luck, I suspect.
Windows	✔	✔	❌	`jfxwebkit.dll` differs

where:

✔ means that all of the files in the build directory were identical between the two consecutive builds, and the unit tests ran successfully, while
❌ means that there were differences in the native libraries between the two builds.

I created bug reports to track each of the remaining potential problems:

JDK-8306884: Building WebKit on Linux is not deterministic
JDK-8306885: Building WebKit on Windows is not deterministic
JDK-8306886: Building macOS libraries can be non-deterministic
JDK-8306887: Error C2327 while compiling WebKit on Windows

jgneff · 2023-04-28T01:24:51Z

I created bug reports to track each of the remaining potential problems:

I forgot the original remaining problem! It's listed as item 4 under Fixes in the description at the top of this pull request and now described by the following bug report:

JDK-8307082: Build path is recorded in JavaFX Controls module

bridgekeeper · 2023-05-26T06:42:45Z

@jgneff This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

kevinrushforth · 2023-06-02T16:34:20Z

Now that the compiler updates are in, I'll resume reviewing and testing this. I'd like to get it in soon.

@jgneff Can you merge in the latest master? This will pick up the recent compiler updates along with any other recent changes. I'll do that anyway when testing, but that way we'll get a GHA run on Mac and Linux with the latest compilers.

kevinrushforth

I finished all my testing. This looks good to me, and I have no further questions or concerns. There is some follow-up work that is needed, but that can be done after we get this in.

Here are the results of my testing. I did 3 CI builds as well as a few local builds on all three desktop OS platforms. All of my testing was done using the latest compiler devkits and the kevinrushforth:test-pr-446 branch, which is the latest branch from this PR with the latest upstream master merged in.

For the first test, I ran without setting SOURCE_DATE_EPOCH and compared the results of a build from test-pr-446 with a build from master. All is as expected.

For the second test, I compared the results of two builds using the same SOURCE_DATE_EPOCH timestamp on the same platform. Except for the native WebKit library on Windows and macOS, everything was identical.

For the third test, I compared the results of a local build with the results of a CI build using the same SOURCE_DATE_EPOCH timestamp. There were some differences on Windows and Linux in the native binaries that should be followed-up on.

macOS

Two different CI builds on the same system:

Only libjfxwebkit.dylib is different

Two different builds (CI and local) each on a different macOS 13.x system:

libjfxwebkit.dylib is different
.bss files differ due to known issue where the absolute file path is recorded in the .bss files

Linux

Two different CI builds on the same system:
* All artifacts are identical

Two different builds (CI and local) each on a different OS platform (Oracle Linux 7.x versus Ubuntu 16.04) with devkit:
* All .so files are different
* .bss files differ due to file path issues

Windows

Two different builds on the same system:
* Only jfxwebkit.dll is different

Two different builds (CI and local) each on a different OS platform (Windows 10 versus Windows Server 2016) with devkit:
* All .dll files are different
* .bss files differ due to file path issues

NOTE: Please merge in the latest master and I'll reapprove.

jgneff · 2023-06-13T16:54:18Z

@jgneff Can you merge in the latest master? This will pick up the recent compiler updates along with any other recent changes.

Thank you, Kevin, for your patience while I was away on vacation. I'll also give this another test, at least on Linux, as my own final review.

jgneff · 2023-06-15T22:04:53Z

I upgraded Visual Studio 2022 to version 17.5.5 on Windows and CMake to version 3.26.4 on all of the platforms, so now I'm using the following tools:

Toolchains

Linux: gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0
macOS: Command Line Tools for Xcode 14.2 version 14.2.0.0.1.1668646533
Windows: Visual Studio Build Tools 2022 version 17.5.5, build number 17.5.33627.172

Build Tools

JDK: OpenJDK Runtime Environment (build 19.0.2+7-44)
Ant: Apache Ant(TM) version 1.10.13 compiled on January 4 2023
CMake: cmake version 3.26.4

I ran the builds as described in the jgneff/jfxbuild repository. The results are shown below.

Linux

Only the file libjfxwebkit.so differs between builds:

ubuntu@openjfx:~/src/jfx$ for n in 1,2 3,4 5,6; do (set -x; diff -qr build[$n]); done
+ diff -qr build1 build2
+ diff -qr build3 build4
+ diff -qr build5 build6
Files build5/jmods/javafx.web.jmod and build6/jmods/javafx.web.jmod differ
Files build5/modular-sdk/modules_libs/javafx.web/libjfxwebkit.so and build6/modular-sdk/modules_libs/javafx.web/libjfxwebkit.so differ
Files build5/publications/javafx.web-linux.jar and build6/publications/javafx.web-linux.jar differ
Files build5/sdk/lib/libjfxwebkit.so and build6/sdk/lib/libjfxwebkit.so differ

macOS

Only the file libjavafx_iio.dylib differs between builds, but I believe that was just by chance. Other test runs often show no differences at all or differences in other native libraries. Xcode version 14.3 may fix these differences, but my late 2014 Mac mini only supports up to Xcode 14.2.

john@macwifi:~/src/jfx$ for n in 1,2 3,4 5,6; do (set -x; diff -qr build[$n]); done
+ diff -qr build1 build2
Files build1/jmods/javafx.graphics.jmod and build2/jmods/javafx.graphics.jmod differ
Files build1/modular-sdk/modules_libs/javafx.graphics/libjavafx_iio.dylib and build2/modular-sdk/modules_libs/javafx.graphics/libjavafx_iio.dylib differ
Files build1/publications/javafx.graphics-mac.jar and build2/publications/javafx.graphics-mac.jar differ
Files build1/sdk/lib/libjavafx_iio.dylib and build2/sdk/lib/libjavafx_iio.dylib differ
+ diff -qr build3 build4
+ diff -qr build5 build6

Windows

Only the file jfxwebkit.dll differs between builds:

john@windows:~/src/jfx$ for n in 1,2 3,4 5,6; do (set -x; diff -qr build[$n]); done
+ diff -qr build1 build2
+ diff -qr build3 build4
+ diff -qr build5 build6
Files build5/jmods/javafx.web.jmod and build6/jmods/javafx.web.jmod differ
Files build5/modular-sdk/modules_libs/javafx.web/jfxwebkit.dll and build6/modular-sdk/modules_libs/javafx.web/jfxwebkit.dll differ
Files build5/publications/javafx.web-win.jar and build6/publications/javafx.web-win.jar differ
Files build5/sdk/bin/jfxwebkit.dll and build6/sdk/bin/jfxwebkit.dll differ

I found another build failure on Windows when running the latest Visual Studio 2022 Build Tools version 17.6.3, dated June 13, 2023. I opened the bug report, "JDK-8310161: Error C2666 compiling ICU 73.1 with VS 2022 17.6," as a heads-up for when we upgrade Visual Studio.

johanvos

This is a good step towards 100% reproducible builds.
I am slightly worried about the added complexity to the build.gradle but I don't see a real alternative.

kevinrushforth · 2023-06-20T11:09:05Z

/reviewers 2

openjdk · 2023-06-20T11:10:35Z

@kevinrushforth
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

openjdk · 2023-06-20T11:11:36Z

@jgneff This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8264449: Enable reproducible builds with SOURCE_DATE_EPOCH
8238650: Allow to override buildDate with SOURCE_DATE_EPOCH

Co-authored-by: Bernhard M. Wiedemann <javabmw@lsmod.de>
Reviewed-by: kcr, jvos

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 4 new commits pushed to the master branch:

4232183: 8307542: Call to FcConfigAppFontAddFile uses wrong prototype, arguments
7eb9a1c: 8306121: Scene not rendered initially when changing scenes after fix for JDK-8296621
8d13ba9: 8310024: Skip failing scene change tests on macOS
c20f6d0: 8304831: TextFlow.hitTest.insertionIndex incorrect with surrogate pairs

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

jgneff · 2023-06-20T16:31:53Z

Please see all three of the following pull requests for a complete history of these changes:

openjdk/jfx/99 - 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH (created 2020-01-29)
openjdk/jfx/422 - 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH (created 2021-03-09)
openjdk/jfx/446 - 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH (created 2021-03-30)

jgneff · 2023-06-20T16:40:27Z

/integrate

openjdk · 2023-06-20T16:42:00Z

Going to push as commit 0d9dcf3.
Since your change was applied there have been 4 commits pushed to the master branch:

4232183: 8307542: Call to FcConfigAppFontAddFile uses wrong prototype, arguments
7eb9a1c: 8306121: Scene not rendered initially when changing scenes after fix for JDK-8296621
8d13ba9: 8310024: Skip failing scene change tests on macOS
c20f6d0: 8304831: TextFlow.hitTest.insertionIndex incorrect with surrogate pairs

Your commit was automatically rebased without conflicts.

openjdk · 2023-06-20T16:42:13Z

@jgneff Pushed as commit 0d9dcf3.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

jgneff added 2 commits March 25, 2021 11:04

8238650: Allow to override buildDate with SOURCE_DATE_EPOCH

Verified

This commit was signed with the committer’s verified signature.

jgneff John Neffenger

GPG key ID: 7796E0517A0D2443

Verified
Learn about vigilant mode

7ef5031

Enable reproducible builds with SOURCE_DATE_EPOCH

Verified

This commit was signed with the committer’s verified signature.

jgneff John Neffenger

GPG key ID: 7796E0517A0D2443

Verified
Learn about vigilant mode

ad46dc3

openjdk bot added the rfr label Mar 30, 2021

jgneff mentioned this pull request Mar 30, 2021

8238650: Allow to override buildDate with SOURCE_DATE_EPOCH #422

Closed

3 tasks

kevinrushforth requested review from kevinrushforth, johanvos and tiainen March 31, 2021 13:26

sashamatveev approved these changes Apr 1, 2021

View reviewed changes

Revert format of timestamp in version OPT field

Verified

This commit was signed with the committer’s verified signature.

jgneff John Neffenger

GPG key ID: 7796E0517A0D2443

Verified
Learn about vigilant mode

46b1670

Merge branch 'master' into allow-reproducible-builds

Verified

This commit was signed with the committer’s verified signature.

jgneff John Neffenger

GPG key ID: 7796E0517A0D2443

Verified
Learn about vigilant mode

842470a

kevinrushforth approved these changes Jun 9, 2023

View reviewed changes

kevinrushforth requested a review from johanvos June 12, 2023 12:12

Merge branch 'master' into allow-reproducible-builds

Verified

This commit was signed with the committer’s verified signature.

jgneff John Neffenger

GPG key ID: 7796E0517A0D2443

Verified
Learn about vigilant mode

fd39e6d

kevinrushforth approved these changes Jun 13, 2023

View reviewed changes

johanvos approved these changes Jun 19, 2023

View reviewed changes

openjdk bot added the ready label Jun 20, 2023

openjdk bot added the integrated label Jun 20, 2023

openjdk bot closed this Jun 20, 2023

openjdk bot removed ready rfr labels Jun 20, 2023

jgneff deleted the allow-reproducible-builds branch June 20, 2023 18:32

jgneff mentioned this pull request Oct 1, 2023

JDK-8317308 JavaFX Developer build broken on Windows - NativeLibrary file contains invalid character ':' #1251

Closed

3 tasks

8264449: Enable reproducible builds with SOURCE_DATE_EPOCH #446

8264449: Enable reproducible builds with SOURCE_DATE_EPOCH #446

Conversation

jgneff commented Mar 30, 2021 • edited by openjdk bot Loading

Fixes

Progress

Issues

Reviewers

Contributors

Reviewing

Webrev

bridgekeeper bot commented Mar 30, 2021

mlbridge bot commented Mar 30, 2021 • edited Loading

Webrevs

jgneff commented Mar 30, 2021

jgneff commented Mar 30, 2021 • edited Loading

kevinrushforth commented Mar 31, 2021

openjdk bot commented Mar 31, 2021

kevinrushforth commented Mar 31, 2021

jgneff commented Mar 31, 2021

Linux

jgneff commented Apr 1, 2021

macOS

Windows

jgneff commented Apr 1, 2021

kevinrushforth commented Apr 1, 2021 • edited Loading

kevinrushforth commented Apr 1, 2021

arun-joseph commented Apr 1, 2021

kevinrushforth commented Apr 1, 2021

bmwiedemann commented Apr 1, 2021

kevinrushforth commented Apr 1, 2021

jgneff commented Apr 1, 2021

bmwiedemann commented Apr 1, 2021

openjdk bot commented Apr 1, 2021

kevinrushforth commented Apr 1, 2021

sashamatveev left a comment

Choose a reason for hiding this comment

mlbridge bot commented Apr 2, 2021

jgneff commented Apr 7, 2023

jgneff commented Apr 7, 2023

kevinrushforth commented Apr 7, 2023

jgneff commented Apr 26, 2023

Toolchains

Build Tools

jgneff commented Apr 28, 2023

bridgekeeper bot commented May 26, 2023

kevinrushforth commented Jun 2, 2023

kevinrushforth left a comment

Choose a reason for hiding this comment

macOS

Linux

Windows

jgneff commented Jun 13, 2023

jgneff commented Jun 15, 2023

Toolchains

Build Tools

Linux

macOS

Windows

johanvos left a comment

Choose a reason for hiding this comment

kevinrushforth commented Jun 20, 2023

openjdk bot commented Jun 20, 2023

openjdk bot commented Jun 20, 2023

jgneff commented Jun 20, 2023

jgneff commented Jun 20, 2023

openjdk bot commented Jun 20, 2023

openjdk bot commented Jun 20, 2023

jgneff commented Mar 30, 2021 •

edited by openjdk bot

Loading

mlbridge bot commented Mar 30, 2021 •

edited

Loading

jgneff commented Mar 30, 2021 •

edited

Loading

kevinrushforth commented Apr 1, 2021 •

edited

Loading