github-starred/barrier

Fork 0

mirror of https://github.com/debauchee/barrier.git synced 2026-05-15 14:16:02 -06:00

[GH-ISSUE #749] [Improvement] Parallelization for faster build time #587

New issue

Closed

opened 2026-05-05 06:43:36 -06:00 by gitea-mirror · 6 comments

gitea-mirror commented

2026-05-05 06:43:36 -06:00

Owner

Originally created by @HattDroid on GitHub (Jun 13, 2020).
Original GitHub issue: https://github.com/debauchee/barrier/issues/749

Hi,

Thanks for barrier, that is a great and really useful tool.

Not really an issue but a small tip that I think doesn't deserve a pull request.
I am building barrier quite often on my Linux system and found the process slow.
I just added the '-j' to 'make' in order to take into account parallelization on multi core machines.

Build time did not get divided by the number of cores on my machine (the code must have a lot of inter dependency I guess and the configure part is still serialized code), but it still got reduce from 2m49,402s to 0m50,226s.

Please find a diff with the latest commit below and attached if you want to merge it.

diff --git a/clean_build.sh b/clean_build.sh
index 3a92d8b9..43cbc583 100755
--- a/clean_build.sh
+++ b/clean_build.sh
@@ -28,5 +28,5 @@ mkdir build || exit 1
 cd build || exit 1
 echo Starting Barrier $B_BUILD_TYPE build...
 $B_CMAKE $B_CMAKE_FLAGS .. || exit 1
-make || exit 1
+make -j$(nproc) || exit 1
 echo "Build completed successfully"

parallel_barrier_build.txt

Originally created by @HattDroid on GitHub (Jun 13, 2020). Original GitHub issue: https://github.com/debauchee/barrier/issues/749 Hi, Thanks for barrier, that is a great and really useful tool. Not really an issue but a small tip that I think doesn't deserve a pull request. I am building barrier quite often on my Linux system and found the process slow. I just added the '-j' to 'make' in order to take into account parallelization on multi core machines. Build time did not get divided by the number of cores on my machine (the code must have a lot of inter dependency I guess and the configure part is still serialized code), but it still got reduce from 2m49,402s to 0m50,226s. Please find a diff with the latest commit below and attached if you want to merge it. ``` diff --git a/clean_build.sh b/clean_build.sh index 3a92d8b9..43cbc583 100755 --- a/clean_build.sh +++ b/clean_build.sh @@ -28,5 +28,5 @@ mkdir build || exit 1 cd build || exit 1 echo Starting Barrier $B_BUILD_TYPE build... $B_CMAKE $B_CMAKE_FLAGS .. || exit 1 -make || exit 1 +make -j$(nproc) || exit 1 echo "Build completed successfully" ``` [parallel_barrier_build.txt](https://github.com/debauchee/barrier/files/4773821/parallel_barrier_build.txt)

gitea-mirror closed this issue

2026-05-05 06:43:37 -06:00

gitea-mirror commented

2026-05-05 06:43:40 -06:00

Author

Owner

@shymega commented on GitHub (Jun 13, 2020):

This needs to be a PR. Please submit as a PR. Happy to let you know how, if you don't already know.

Issues are for bug reports and other issues really -- not code enhancements/additions.

Closing for the moment.

@shymega commented on GitHub (Jun 13, 2020): This needs to be a PR. Please submit as a PR. Happy to let you know how, if you don't already know. Issues are for bug reports and other issues really -- not code enhancements/additions. Closing for the moment.

gitea-mirror commented

2026-05-05 06:43:44 -06:00

Author

Owner

@p12tic commented on GitHub (Jun 13, 2020):

One thing to know is that there's no nproc on macOS and we use clean_build.sh there. So this change can't be applied as is.

@p12tic commented on GitHub (Jun 13, 2020): One thing to know is that there's no `nproc` on macOS and we use `clean_build.sh` there. So this change can't be applied as is.

gitea-mirror commented

2026-05-05 06:43:45 -06:00

Author

Owner

@shymega commented on GitHub (Jun 13, 2020):

Also this.

@shymega commented on GitHub (Jun 13, 2020): Also this.

gitea-mirror commented

2026-05-05 06:43:48 -06:00

Author

Owner

@shymega commented on GitHub (Jun 13, 2020):

Although, just thinking, could we get cmake to inject the -j argument based on the amount of CPU cores on the build host?

@shymega commented on GitHub (Jun 13, 2020): Although, just thinking, could we get cmake to inject the `-j` argument based on the amount of CPU cores on the build host?

gitea-mirror commented

2026-05-05 06:43:49 -06:00

Author

Owner

@HattDroid commented on GitHub (Jun 14, 2020):

Thanks for all the sensible details guys. Make sense not to adopt it out of the box.
Thanks shymega to be willing to help with how to PR.

Found this command to find the number of cores on a MacOS machine:

sysctl -n hw.ncpu

Anyone with a Mac could try it and let know? It seems to report a wrong value on my Mac VM.

About getting cmake to populate the number of available threads, I am not sure make or cmake are aware of the number of cores, or that it's part of their ambitions.

I know that running 'make -j' without specifying the number of cores will spawn the max number of compilation threads it can to build a project.
This could fit as an alternative working on all platforms.
At the end of the day the hardware will limit you in both cases and you end up around the same time as using nproc to use all of your cores/vcores.

@HattDroid commented on GitHub (Jun 14, 2020): Thanks for all the sensible details guys. Make sense not to adopt it out of the box. Thanks shymega to be willing to help with how to PR. Found this command to find the number of cores on a MacOS machine: `sysctl -n hw.ncpu` Anyone with a Mac could try it and let know? It seems to report a wrong value on my Mac VM. About getting cmake to populate the number of available threads, I am not sure make or cmake are aware of the number of cores, or that it's part of their ambitions. I know that running 'make -j' without specifying the number of cores will spawn the max number of compilation threads it can to build a project. This could fit as an alternative working on all platforms. At the end of the day the hardware will limit you in both cases and you end up around the same time as using nproc to use all of your cores/vcores.

gitea-mirror commented

2026-05-05 06:43:52 -06:00

Author

Owner

@simons-public commented on GitHub (Jun 14, 2020):

@HattDroid I can confirm sysctl -n hw.ncpu works on my physical mac.

@simons-public commented on GitHub (Jun 14, 2020): @HattDroid I can confirm `sysctl -n hw.ncpu` works on my physical mac.

No labels

Rows
Columns