diff options
| author | KiritoDv <kiritodev01@gmail.com> | 2024-03-22 09:22:43 -0600 |
|---|---|---|
| committer | KiritoDv <kiritodev01@gmail.com> | 2024-03-22 09:22:43 -0600 |
| commit | 0a28d35eb4d80a7d9d7737bd18cd0a71953404c8 (patch) | |
| tree | 201881458148150bd49bf180e4ddfef03f9a9bbf /lib/thread-pool | |
| parent | 95e7bc0d68f0f8687486b30f3a795f143e4456b0 (diff) | |
First conceptasync
Diffstat (limited to 'lib/thread-pool')
| -rw-r--r-- | lib/thread-pool/.github/ISSUE_TEMPLATE/bug_report.md | 37 | ||||
| -rw-r--r-- | lib/thread-pool/.github/ISSUE_TEMPLATE/failed-tests.md | 26 | ||||
| -rw-r--r-- | lib/thread-pool/.github/ISSUE_TEMPLATE/feature_request.md | 23 | ||||
| -rw-r--r-- | lib/thread-pool/.github/pull_request_template.md | 20 | ||||
| -rw-r--r-- | lib/thread-pool/BS_thread_pool.hpp | 1140 | ||||
| -rw-r--r-- | lib/thread-pool/BS_thread_pool_utils.hpp | 203 | ||||
| -rw-r--r-- | lib/thread-pool/CHANGELOG.md | 218 | ||||
| -rw-r--r-- | lib/thread-pool/CITATION.bib | 13 | ||||
| -rw-r--r-- | lib/thread-pool/CITATION.cff | 24 | ||||
| -rw-r--r-- | lib/thread-pool/LICENSE.txt | 21 | ||||
| -rw-r--r-- | lib/thread-pool/README.md | 1498 |
11 files changed, 3223 insertions, 0 deletions
diff --git a/lib/thread-pool/.github/ISSUE_TEMPLATE/bug_report.md b/lib/thread-pool/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..1f02ed3 --- /dev/null +++ b/lib/thread-pool/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,37 @@ +--- +name: Bug report +about: Found a bug? Report it here. +title: "[BUG]" +labels: bug +assignees: bshoshany + +--- + +**Describe the bug** + +A clear and concise description of what the bug is. + +**Minimal working example** + +A short but complete program that can be compiled to reproduce the error. Paste the program between the two code fences. If it's too long or requires multiple files, attach the file(s) instead. + +```cpp +``` + +**Behavior** + +What behavior did you expect to get? What actually happened? If the code failed to compile, please include the full output of the compiler. + +**System information** + +* CPU model, architecture, # of cores and threads: +* Operating system: +* Name and version of C++ compiler: +* Full command used for compiling, including all compiler flags: +* Thread pool library version: + +(Please note that only the latest version of the thread pool library is supported.) + +**Additional information** + +Include any additional information here. diff --git a/lib/thread-pool/.github/ISSUE_TEMPLATE/failed-tests.md b/lib/thread-pool/.github/ISSUE_TEMPLATE/failed-tests.md new file mode 100644 index 0000000..1d20f23 --- /dev/null +++ b/lib/thread-pool/.github/ISSUE_TEMPLATE/failed-tests.md @@ -0,0 +1,26 @@ +--- +name: Failed tests +about: The provided automated tests failed on your system? Report it here. +title: "[TEST]" +labels: bug +assignees: bshoshany + +--- + +**System information** + +* CPU model, architecture, # of cores and threads: +* Operating system: +* Name and version of C++ compiler: +* Full command used for compiling, including all compiler flags: +* Thread pool library version: + +(Please note that only the latest version of the thread pool library is supported.) + +**Log file** + +Please attach the log file generated by the automated test program to this issue. + +**Additional information** + +Include any additional information here. diff --git a/lib/thread-pool/.github/ISSUE_TEMPLATE/feature_request.md b/lib/thread-pool/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..7137cea --- /dev/null +++ b/lib/thread-pool/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,23 @@ +--- +name: Feature request +about: Want a new feature? Suggest it here. +title: "[REQ]" +labels: enhancement +assignees: bshoshany + +--- + +**Describe the new feature** + +A clear and concise description of the feature you want. + +**Code example** + +An example of code that utilizes the suggested feature. Paste or write it between the two code fences. + +```cpp +``` + +**Additional information** + +Include any additional information here. diff --git a/lib/thread-pool/.github/pull_request_template.md b/lib/thread-pool/.github/pull_request_template.md new file mode 100644 index 0000000..cabbe5a --- /dev/null +++ b/lib/thread-pool/.github/pull_request_template.md @@ -0,0 +1,20 @@ +**Pull request policy (please read)** + +> Contributions are always welcome. However, I release my projects in cumulative updates after editing and testing them locally on my system, so my policy is not to accept any pull requests. If you open a pull request, and I decide to incorporate your suggestion into the project, I will first modify your code to comply with the project's coding conventions (formatting, syntax, naming, comments, programming practices, etc.), and perform some tests to ensure that the change doesn't break anything. I will then merge it into the next release of the project, possibly together with some other changes. The new release will also include a note in `CHANGELOG.md` with a link to your pull request, and modifications to the documentation in `README.md` as needed. + +**Describe the changes** + +What does your pull request fix or add to the library? + +**Testing** + +Have you tested the new code using the provided automated test program and/or performed any other tests to ensure that it works correctly? If so, please provide information about the test system(s): + +* CPU model, architecture, # of cores and threads: +* Operating system: +* Name and version of C++ compiler: +* Full command used for compiling, including all compiler flags: + +**Additional information** + +Include any additional information here. diff --git a/lib/thread-pool/BS_thread_pool.hpp b/lib/thread-pool/BS_thread_pool.hpp new file mode 100644 index 0000000..dee006f --- /dev/null +++ b/lib/thread-pool/BS_thread_pool.hpp @@ -0,0 +1,1140 @@ +#pragma once + +/** + * @file BS_thread_pool.hpp + * @author Barak Shoshany (baraksh@gmail.com) (http://baraksh.com) + * @version 4.0.1 + * @date 2023-12-28 + * @copyright Copyright (c) 2023 Barak Shoshany. Licensed under the MIT license. If you found this project useful, please consider starring it on GitHub! If you use this library in software of any kind, please provide a link to the GitHub repository https://github.com/bshoshany/thread-pool in the source code and documentation. If you use this library in published research, please cite it as follows: Barak Shoshany, "A C++17 Thread Pool for High-Performance Scientific Computing", doi:10.5281/zenodo.4742687, arXiv:2105.00613 (May 2021) + * + * @brief BS::thread_pool: a fast, lightweight, and easy-to-use C++17 thread pool library. This header file contains the main thread pool class and some additional classes and definitions. No other files are needed in order to use the thread pool itself. + */ + +#include <chrono> // std::chrono +#include <condition_variable> // std::condition_variable +#include <cstddef> // std::size_t +#include <cstdint> // std::int_least16_t +#include <exception> // std::current_exception +#include <functional> // std::function +#include <future> // std::future, std::future_status, std::promise +#include <memory> // std::make_shared, std::make_unique, std::shared_ptr, std::unique_ptr +#include <mutex> // std::mutex, std::scoped_lock, std::unique_lock +#include <optional> // std::nullopt, std::optional +#include <queue> // std::priority_queue, std::queue +#include <stdexcept> // std::runtime_error +#include <thread> // std::thread +#include <type_traits> // std::conditional_t, std::decay_t, std::invoke_result_t, std::is_void_v, std::remove_const_t +#include <utility> // std::forward, std::move +#include <vector> // std::vector + +/** + * @brief A namespace used by Barak Shoshany's projects. + */ +namespace BS { +// Macros indicating the version of the thread pool library. +#define BS_THREAD_POOL_VERSION_MAJOR 4 +#define BS_THREAD_POOL_VERSION_MINOR 0 +#define BS_THREAD_POOL_VERSION_PATCH 1 + +class thread_pool; + +/** + * @brief A type to represent the size of things. + */ +using size_t = std::size_t; + +/** + * @brief A convenient shorthand for the type of `std::thread::hardware_concurrency()`. Should evaluate to unsigned int. + */ +using concurrency_t = std::invoke_result_t<decltype(std::thread::hardware_concurrency)>; + +#ifdef BS_THREAD_POOL_ENABLE_PRIORITY +/** + * @brief A type used to indicate the priority of a task. Defined to be an integer with a width of (at least) 16 bits. + */ +using priority_t = std::int_least16_t; + +/** + * @brief A namespace containing some pre-defined priorities for convenience. + */ +namespace pr { + constexpr priority_t highest = 32767; + constexpr priority_t high = 16383; + constexpr priority_t normal = 0; + constexpr priority_t low = -16384; + constexpr priority_t lowest = -32768; +} // namespace pr + +// Macros used internally to enable or disable the priority arguments in the relevant functions. +#define BS_THREAD_POOL_PRIORITY_INPUT , const priority_t priority = 0 +#define BS_THREAD_POOL_PRIORITY_OUTPUT , priority +#else +#define BS_THREAD_POOL_PRIORITY_INPUT +#define BS_THREAD_POOL_PRIORITY_OUTPUT +#endif + +/** + * @brief A namespace used to obtain information about the current thread. + */ +namespace this_thread { + /** + * @brief A type returned by `BS::this_thread::get_index()` which can optionally contain the index of a thread, if that thread belongs to a `BS::thread_pool`. Otherwise, it will contain no value. + */ + using optional_index = std::optional<size_t>; + + /** + * @brief A type returned by `BS::this_thread::get_pool()` which can optionally contain the pointer to the pool that owns a thread, if that thread belongs to a `BS::thread_pool`. Otherwise, it will contain no value. + */ + using optional_pool = std::optional<thread_pool*>; + + /** + * @brief A helper class to store information about the index of the current thread. + */ + class [[nodiscard]] thread_info_index + { + friend class BS::thread_pool; + + public: + /** + * @brief Get the index of the current thread. If this thread belongs to a `BS::thread_pool` object, it will have an index from 0 to `BS::thread_pool::get_thread_count() - 1`. Otherwise, for example if this thread is the main thread or an independent `std::thread`, `std::nullopt` will be returned. + * + * @return An `std::optional` object, optionally containing a thread index. Unless you are 100% sure this thread is in a pool, first use `std::optional::has_value()` to check if it contains a value, and if so, use `std::optional::value()` to obtain that value. + */ + [[nodiscard]] optional_index operator()() const + { + return index; + } + + private: + /** + * @brief The index of the current thread. + */ + optional_index index = std::nullopt; + }; // class thread_info_index + + /** + * @brief A helper class to store information about the thread pool that owns the current thread. + */ + class [[nodiscard]] thread_info_pool + { + friend class BS::thread_pool; + + public: + /** + * @brief Get the pointer to the thread pool that owns the current thread. If this thread belongs to a `BS::thread_pool` object, a pointer to that object will be returned. Otherwise, for example if this thread is the main thread or an independent `std::thread`, `std::nullopt` will be returned. + * + * @return An `std::optional` object, optionally containing a pointer to a thread pool. Unless you are 100% sure this thread is in a pool, first use `std::optional::has_value()` to check if it contains a value, and if so, use `std::optional::value()` to obtain that value. + */ + [[nodiscard]] optional_pool operator()() const + { + return pool; + } + + private: + /** + * @brief A pointer to the thread pool that owns the current thread. + */ + optional_pool pool = std::nullopt; + }; // class thread_info_pool + + /** + * @brief A `thread_local` object used to obtain information about the index of the current thread. + */ + inline thread_local thread_info_index get_index; + + /** + * @brief A `thread_local` object used to obtain information about the thread pool that owns the current thread. + */ + inline thread_local thread_info_pool get_pool; +} // namespace this_thread + +/** + * @brief A helper class to facilitate waiting for and/or getting the results of multiple futures at once. + * + * @tparam T The return type of the futures. + */ +template <typename T> +class [[nodiscard]] multi_future : public std::vector<std::future<T>> +{ +public: + // Inherit all constructors from the base class `std::vector`. + using std::vector<std::future<T>>::vector; + + // The copy constructor and copy assignment operator are deleted. The elements stored in a `multi_future` are futures, which cannot be copied. + multi_future(const multi_future&) = delete; + multi_future& operator=(const multi_future&) = delete; + + // The move constructor and move assignment operator are defaulted. + multi_future(multi_future&&) = default; + multi_future& operator=(multi_future&&) = default; + + /** + * @brief Get the results from all the futures stored in this `multi_future`, rethrowing any stored exceptions. + * + * @return If the futures return `void`, this function returns `void` as well. Otherwise, it returns a vector containing the results. + */ + [[nodiscard]] std::conditional_t<std::is_void_v<T>, void, std::vector<T>> get() + { + if constexpr (std::is_void_v<T>) + { + for (std::future<T>& future : *this) + future.get(); + return; + } + else + { + std::vector<T> results; + results.reserve(this->size()); + for (std::future<T>& future : *this) + results.push_back(future.get()); + return results; + } + } + + /** + * @brief Check how many of the futures stored in this `multi_future` are ready. + * + * @return The number of ready futures. + */ + [[nodiscard]] size_t ready_count() const + { + size_t count = 0; + for (const std::future<T>& future : *this) + { + if (future.wait_for(std::chrono::duration<double>::zero()) == std::future_status::ready) + ++count; + } + return count; + } + + /** + * @brief Check if all the futures stored in this `multi_future` are valid. + * + * @return `true` if all futures are valid, `false` if at least one of the futures is not valid. + */ + [[nodiscard]] bool valid() const + { + bool is_valid = true; + for (const std::future<T>& future : *this) + is_valid = is_valid && future.valid(); + return is_valid; + } + + /** + * @brief Wait for all the futures stored in this `multi_future`. + */ + void wait() const + { + for (const std::future<T>& future : *this) + future.wait(); + } + + /** + * @brief Wait for all the futures stored in this `multi_future`, but stop waiting after the specified duration has passed. This function first waits for the first future for the desired duration. If that future is ready before the duration expires, this function waits for the second future for whatever remains of the duration. It continues similarly until the duration expires. + * + * @tparam R An arithmetic type representing the number of ticks to wait. + * @tparam P An `std::ratio` representing the length of each tick in seconds. + * @param duration The amount of time to wait. + * @return `true` if all futures have been waited for before the duration expired, `false` otherwise. + */ + template <typename R, typename P> + bool wait_for(const std::chrono::duration<R, P>& duration) const + { + const std::chrono::time_point<std::chrono::steady_clock> start_time = std::chrono::steady_clock::now(); + for (const std::future<T>& future : *this) + { + future.wait_for(duration - (std::chrono::steady_clock::now() - start_time)); + if (duration < std::chrono::steady_clock::now() - start_time) + return false; + } + return true; + } + + /** + * @brief Wait for all the futures stored in this `multi_future`, but stop waiting after the specified time point has been reached. This function first waits for the first future until the desired time point. If that future is ready before the time point is reached, this function waits for the second future until the desired time point. It continues similarly until the time point is reached. + * + * @tparam C The type of the clock used to measure time. + * @tparam D An `std::chrono::duration` type used to indicate the time point. + * @param timeout_time The time point at which to stop waiting. + * @return `true` if all futures have been waited for before the time point was reached, `false` otherwise. + */ + template <typename C, typename D> + bool wait_until(const std::chrono::time_point<C, D>& timeout_time) const + { + for (const std::future<T>& future : *this) + { + future.wait_until(timeout_time); + if (timeout_time < std::chrono::steady_clock::now()) + return false; + } + return true; + } +}; // class multi_future + +/** + * @brief A fast, lightweight, and easy-to-use C++17 thread pool class. + */ +class [[nodiscard]] thread_pool +{ +public: + // ============================ + // Constructors and destructors + // ============================ + + /** + * @brief Construct a new thread pool. The number of threads will be the total number of hardware threads available, as reported by the implementation. This is usually determined by the number of cores in the CPU. If a core is hyperthreaded, it will count as two threads. + */ + thread_pool() : thread_pool(0, [] {}) {} + + /** + * @brief Construct a new thread pool with the specified number of threads. + * + * @param num_threads The number of threads to use. + */ + explicit thread_pool(const concurrency_t num_threads) : thread_pool(num_threads, [] {}) {} + + /** + * @brief Construct a new thread pool with the specified initialization function. + * + * @param init_task An initialization function to run in each thread before it starts to execute any submitted tasks. The function must take no arguments and have no return value. It will only be executed exactly once, when the thread is first constructed. + */ + explicit thread_pool(const std::function<void()>& init_task) : thread_pool(0, init_task) {} + + /** + * @brief Construct a new thread pool with the specified number of threads and initialization function. + * + * @param num_threads The number of threads to use. + * @param init_task An initialization function to run in each thread before it starts to execute any submitted tasks. The function must take no arguments and have no return value. It will only be executed exactly once, when the thread is first constructed. + */ + thread_pool(const concurrency_t num_threads, const std::function<void()>& init_task) : thread_count(determine_thread_count(num_threads)), threads(std::make_unique<std::thread[]>(determine_thread_count(num_threads))) + { + create_threads(init_task); + } + + // The copy and move constructors and assignment operators are deleted. The thread pool uses a mutex, which cannot be copied or moved. + thread_pool(const thread_pool&) = delete; + thread_pool(thread_pool&&) = delete; + thread_pool& operator=(const thread_pool&) = delete; + thread_pool& operator=(thread_pool&&) = delete; + + /** + * @brief Destruct the thread pool. Waits for all tasks to complete, then destroys all threads. Note that if the pool is paused, then any tasks still in the queue will never be executed. + */ + ~thread_pool() + { + wait(); + destroy_threads(); + } + + // ======================= + // Public member functions + // ======================= + +#ifdef BS_THREAD_POOL_ENABLE_NATIVE_HANDLES + /** + * @brief Get a vector containing the underlying implementation-defined thread handles for each of the pool's threads, as obtained by `std::thread::native_handle()`. Only enabled if `BS_THREAD_POOL_ENABLE_NATIVE_HANDLES` is defined. + * + * @return The native thread handles. + */ + [[nodiscard]] std::vector<std::thread::native_handle_type> get_native_handles() const + { + std::vector<std::thread::native_handle_type> native_handles(thread_count); + for (concurrency_t i = 0; i < thread_count; ++i) + { + native_handles[i] = threads[i].native_handle(); + } + return native_handles; + } +#endif + + /** + * @brief Get the number of tasks currently waiting in the queue to be executed by the threads. + * + * @return The number of queued tasks. + */ + [[nodiscard]] size_t get_tasks_queued() const + { + const std::scoped_lock tasks_lock(tasks_mutex); + return tasks.size(); + } + + /** + * @brief Get the number of tasks currently being executed by the threads. + * + * @return The number of running tasks. + */ + [[nodiscard]] size_t get_tasks_running() const + { + const std::scoped_lock tasks_lock(tasks_mutex); + return tasks_running; + } + + /** + * @brief Get the total number of unfinished tasks: either still waiting in the queue, or running in a thread. Note that `get_tasks_total() == get_tasks_queued() + get_tasks_running()`. + * + * @return The total number of tasks. + */ + [[nodiscard]] size_t get_tasks_total() const + { + const std::scoped_lock tasks_lock(tasks_mutex); + return tasks_running + tasks.size(); + } + + /** + * @brief Get the number of threads in the pool. + * + * @return The number of threads. + */ + [[nodiscard]] concurrency_t get_thread_count() const + { + return thread_count; + } + + /** + * @brief Get a vector containing the unique identifiers for each of the pool's threads, as obtained by `std::thread::get_id()`. + * + * @return The unique thread identifiers. + */ + [[nodiscard]] std::vector<std::thread::id> get_thread_ids() const + { + std::vector<std::thread::id> thread_ids(thread_count); + for (concurrency_t i = 0; i < thread_count; ++i) + { + thread_ids[i] = threads[i].get_id(); + } + return thread_ids; + } + +#ifdef BS_THREAD_POOL_ENABLE_PAUSE + /** + * @brief Check whether the pool is currently paused. Only enabled if `BS_THREAD_POOL_ENABLE_PAUSE` is defined. + * + * @return `true` if the pool is paused, `false` if it is not paused. + */ + [[nodiscard]] bool is_paused() const + { + const std::scoped_lock tasks_lock(tasks_mutex); + return paused; + } + + /** + * @brief Pause the pool. The workers will temporarily stop retrieving new tasks out of the queue, although any tasks already executed will keep running until they are finished. Only enabled if `BS_THREAD_POOL_ENABLE_PAUSE` is defined. + */ + void pause() + { + const std::scoped_lock tasks_lock(tasks_mutex); + paused = true; + } +#endif + + /** + * @brief Purge all the tasks waiting in the queue. Tasks that are currently running will not be affected, but any tasks still waiting in the queue will be discarded, and will never be executed by the threads. Please note that there is no way to restore the purged tasks. + */ + void purge() + { + const std::scoped_lock tasks_lock(tasks_mutex); + while (!tasks.empty()) + tasks.pop(); + } + + /** + * @brief Submit a function with no arguments and no return value into the task queue, with the specified priority. To push a function with arguments, enclose it in a lambda expression. Does not return a future, so the user must use `wait()` or some other method to ensure that the task finishes executing, otherwise bad things will happen. + * + * @tparam F The type of the function. + * @param task The function to push. + * @param priority The priority of the task. Should be between -32,768 and 32,767 (a signed 16-bit integer). The default is 0. Only enabled if `BS_THREAD_POOL_ENABLE_PRIORITY` is defined. + */ + template <typename F> + void detach_task(F&& task BS_THREAD_POOL_PRIORITY_INPUT) + { + { + const std::scoped_lock tasks_lock(tasks_mutex); + tasks.emplace(std::forward<F>(task) BS_THREAD_POOL_PRIORITY_OUTPUT); + } + task_available_cv.notify_one(); + } + + /** + * @brief Parallelize a loop by automatically splitting it into blocks and submitting each block separately to the queue, with the specified priority. The block function takes two arguments, the start and end of the block, so that it is only called only once per block, but it is up to the user make sure the block function correctly deals with all the indices in each block. Does not return a `multi_future`, so the user must use `wait()` or some other method to ensure that the loop finishes executing, otherwise bad things will happen. + * + * @tparam T The type of the indices. Should be a signed or unsigned integer. + * @tparam F The type of the function to loop through. + * @param first_index The first index in the loop. + * @param index_after_last The index after the last index in the loop. The loop will iterate from `first_index` to `(index_after_last - 1)` inclusive. In other words, it will be equivalent to `for (T i = first_index; i < index_after_last; ++i)`. Note that if `index_after_last <= first_index`, no blocks will be submitted. + * @param block A function that will be called once per block. Should take exactly two arguments: the first index in the block and the index after the last index in the block. `block(start, end)` should typically involve a loop of the form `for (T i = start; i < end; ++i)`. + * @param num_blocks The maximum number of blocks to split the loop into. The default is 0, which means the number of blocks will be equal to the number of threads in the pool. + * @param priority The priority of the tasks. Should be between -32,768 and 32,767 (a signed 16-bit integer). The default is 0. Only enabled if `BS_THREAD_POOL_ENABLE_PRIORITY` is defined. + */ + template <typename T, typename F> + void detach_blocks(const T first_index, const T index_after_last, F&& block, const size_t num_blocks = 0 BS_THREAD_POOL_PRIORITY_INPUT) + { + if (index_after_last > first_index) + { + const blocks blks(first_index, index_after_last, num_blocks ? num_blocks : thread_count); + for (size_t blk = 0; blk < blks.get_num_blocks(); ++blk) + detach_task( + [block = std::forward<F>(block), start = blks.start(blk), end = blks.end(blk)] + { + block(start, end); + } BS_THREAD_POOL_PRIORITY_OUTPUT); + } + } + + /** + * @brief Parallelize a loop by automatically splitting it into blocks and submitting each block separately to the queue, with the specified priority. The loop function takes one argument, the loop index, so that it is called many times per block. Does not return a `multi_future`, so the user must use `wait()` or some other method to ensure that the loop finishes executing, otherwise bad things will happen. + * + * @tparam T The type of the indices. Should be a signed or unsigned integer. + * @tparam F The type of the function to loop through. + * @param first_index The first index in the loop. + * @param index_after_last The index after the last index in the loop. The loop will iterate from `first_index` to `(index_after_last - 1)` inclusive. In other words, it will be equivalent to `for (T i = first_index; i < index_after_last; ++i)`. Note that if `index_after_last <= first_index`, no blocks will be submitted. + * @param loop The function to loop through. Will be called once per index, many times per block. Should take exactly one argument: the loop index. + * @param num_blocks The maximum number of blocks to split the loop into. The default is 0, which means the number of blocks will be equal to the number of threads in the pool. + * @param priority The priority of the tasks. Should be between -32,768 and 32,767 (a signed 16-bit integer). The default is 0. Only enabled if `BS_THREAD_POOL_ENABLE_PRIORITY` is defined. + */ + template <typename T, typename F> + void detach_loop(const T first_index, const T index_after_last, F&& loop, const size_t num_blocks = 0 BS_THREAD_POOL_PRIORITY_INPUT) + { + if (index_after_last > first_index) + { + const blocks blks(first_index, index_after_last, num_blocks ? num_blocks : thread_count); + for (size_t blk = 0; blk < blks.get_num_blocks(); ++blk) + detach_task( + [loop = std::forward<F>(loop), start = blks.start(blk), end = blks.end(blk)] + { + for (T i = start; i < end; ++i) + loop(i); + } BS_THREAD_POOL_PRIORITY_OUTPUT); + } + } + + /** + * @brief Submit a sequence of tasks enumerated by indices to the queue, with the specified priority. Does not return a `multi_future`, so the user must use `wait()` or some other method to ensure that the sequence finishes executing, otherwise bad things will happen. + * + * @tparam T The type of the indices. Should be a signed or unsigned integer. + * @tparam F The type of the function used to define the sequence. + * @param first_index The first index in the sequence. + * @param index_after_last The index after the last index in the sequence. The sequence will iterate from `first_index` to `(index_after_last - 1)` inclusive. In other words, it will be equivalent to `for (T i = first_index; i < index_after_last; ++i)`. Note that if `index_after_last <= first_index`, no tasks will be submitted. + * @param sequence The function used to define the sequence. Will be called once per index. Should take exactly one argument, the index. + * @param priority The priority of the tasks. Should be between -32,768 and 32,767 (a signed 16-bit integer). The default is 0. Only enabled if `BS_THREAD_POOL_ENABLE_PRIORITY` is defined. + */ + template <typename T, typename F> + void detach_sequence(const T first_index, const T index_after_last, F&& sequence BS_THREAD_POOL_PRIORITY_INPUT) + { + for (T i = first_index; i < index_after_last; ++i) + detach_task( + [sequence = std::forward<F>(sequence), i] + { + sequence(i); + } BS_THREAD_POOL_PRIORITY_OUTPUT); + } + + /** + * @brief Reset the pool with the total number of hardware threads available, as reported by the implementation. Waits for all currently running tasks to be completed, then destroys all threads in the pool and creates a new thread pool with the new number of threads. Any tasks that were waiting in the queue before the pool was reset will then be executed by the new threads. If the pool was paused before resetting it, the new pool will be paused as well. + */ + void reset() + { + reset(0, [] {}); + } + + /** + * @brief Reset the pool with a new number of threads. Waits for all currently running tasks to be completed, then destroys all threads in the pool and creates a new thread pool with the new number of threads. Any tasks that were waiting in the queue before the pool was reset will then be executed by the new threads. If the pool was paused before resetting it, the new pool will be paused as well. + * + * @param num_threads The number of threads to use. + */ + void reset(const concurrency_t num_threads) + { + reset(num_threads, [] {}); + } + + /** + * @brief Reset the pool with the total number of hardware threads available, as reported by the implementation, and a new initialization function. Waits for all currently running tasks to be completed, then destroys all threads in the pool and creates a new thread pool with the new number of threads and initialization function. Any tasks that were waiting in the queue before the pool was reset will then be executed by the new threads. If the pool was paused before resetting it, the new pool will be paused as well. + * + * @param init_task An initialization function to run in each thread before it starts to execute any submitted tasks. The function must take no arguments and have no return value. It will only be executed exactly once, when the thread is first constructed. + */ + void reset(const std::function<void()>& init_task) + { + reset(0, init_task); + } + + /** + * @brief Reset the pool with a new number of threads and a new initialization function. Waits for all currently running tasks to be completed, then destroys all threads in the pool and creates a new thread pool with the new number of threads and initialization function. Any tasks that were waiting in the queue before the pool was reset will then be executed by the new threads. If the pool was paused before resetting it, the new pool will be paused as well. + * + * @param num_threads The number of threads to use. + * @param init_task An initialization function to run in each thread before it starts to execute any submitted tasks. The function must take no arguments and have no return value. It will only be executed exactly once, when the thread is first constructed. + */ + void reset(const concurrency_t num_threads, const std::function<void()>& init_task) + { + std::unique_lock tasks_lock(tasks_mutex); +#ifdef BS_THREAD_POOL_ENABLE_PAUSE + const bool was_paused = paused; + paused = true; +#endif + tasks_lock.unlock(); + wait(); + destroy_threads(); + thread_count = determine_thread_count(num_threads); + threads = std::make_unique<std::thread[]>(thread_count); + create_threads(init_task); + tasks_lock.lock(); +#ifdef BS_THREAD_POOL_ENABLE_PAUSE + paused = was_paused; +#endif + } + + /** + * @brief Submit a function with no arguments into the task queue, with the specified priority. To submit a function with arguments, enclose it in a lambda expression. If the function has a return value, get a future for the eventual returned value. If the function has no return value, get an `std::future<void>` which can be used to wait until the task finishes. + * + * @tparam F The type of the function. + * @tparam R The return type of the function (can be `void`). + * @param task The function to submit. + * @param priority The priority of the task. Should be between -32,768 and 32,767 (a signed 16-bit integer). The default is 0. Only enabled if `BS_THREAD_POOL_ENABLE_PRIORITY` is defined. + * @return A future to be used later to wait for the function to finish executing and/or obtain its returned value if it has one. + */ + template <typename F, typename R = std::invoke_result_t<std::decay_t<F>>> + [[nodiscard]] std::future<R> submit_task(F&& task BS_THREAD_POOL_PRIORITY_INPUT) + { + const std::shared_ptr<std::promise<R>> task_promise = std::make_shared<std::promise<R>>(); + detach_task( + [task = std::forward<F>(task), task_promise] + { + try + { + if constexpr (std::is_void_v<R>) + { + task(); + task_promise->set_value(); + } + else + { + task_promise->set_value(task()); + } + } + catch (...) + { + try + { + task_promise->set_exception(std::current_exception()); + } + catch (...) + { + } + } + } BS_THREAD_POOL_PRIORITY_OUTPUT); + return task_promise->get_future(); + } + + /** + * @brief Parallelize a loop by automatically splitting it into blocks and submitting each block separately to the queue, with the specified priority. The block function takes two arguments, the start and end of the block, so that it is only called only once per block, but it is up to the user make sure the block function correctly deals with all the indices in each block. Returns a `multi_future` that contains the futures for all of the blocks. + * + * @tparam T The type of the indices. Should be a signed or unsigned integer. + * @tparam F The type of the function to loop through. + * @tparam R The return type of the function to loop through (can be `void`). + * @param first_index The first index in the loop. + * @param index_after_last The index after the last index in the loop. The loop will iterate from `first_index` to `(index_after_last - 1)` inclusive. In other words, it will be equivalent to `for (T i = first_index; i < index_after_last; ++i)`. Note that if `index_after_last <= first_index`, no blocks will be submitted, and an empty `multi_future` will be returned. + * @param block A function that will be called once per block. Should take exactly two arguments: the first index in the block and the index after the last index in the block. `block(start, end)` should typically involve a loop of the form `for (T i = start; i < end; ++i)`. + * @param num_blocks The maximum number of blocks to split the loop into. The default is 0, which means the number of blocks will be equal to the number of threads in the pool. + * @param priority The priority of the tasks. Should be between -32,768 and 32,767 (a signed 16-bit integer). The default is 0. Only enabled if `BS_THREAD_POOL_ENABLE_PRIORITY` is defined. + * @return A `multi_future` that can be used to wait for all the blocks to finish. If the block function returns a value, the `multi_future` can also be used to obtain the values returned by each block. + */ + template <typename T, typename F, typename R = std::invoke_result_t<std::decay_t<F>, T, T>> + [[nodiscard]] multi_future<R> submit_blocks(const T first_index, const T index_after_last, F&& block, const size_t num_blocks = 0 BS_THREAD_POOL_PRIORITY_INPUT) + { + if (index_after_last > first_index) + { + const blocks blks(first_index, index_after_last, num_blocks ? num_blocks : thread_count); + multi_future<R> future; + future.reserve(blks.get_num_blocks()); + for (size_t blk = 0; blk < blks.get_num_blocks(); ++blk) + future.push_back(submit_task( + [block = std::forward<F>(block), start = blks.start(blk), end = blks.end(blk)] + { + return block(start, end); + } BS_THREAD_POOL_PRIORITY_OUTPUT)); + return future; + } + return {}; + } + + /** + * @brief Parallelize a loop by automatically splitting it into blocks and submitting each block separately to the queue, with the specified priority. The loop function takes one argument, the loop index, so that it is called many times per block. It must have no return value. Returns a `multi_future` that contains the futures for all of the blocks. + * + * @tparam T The type of the indices. Should be a signed or unsigned integer. + * @tparam F The type of the function to loop through. + * @param first_index The first index in the loop. + * @param index_after_last The index after the last index in the loop. The loop will iterate from `first_index` to `(index_after_last - 1)` inclusive. In other words, it will be equivalent to `for (T i = first_index; i < index_after_last; ++i)`. Note that if `index_after_last <= first_index`, no tasks will be submitted, and an empty `multi_future` will be returned. + * @param loop The function to loop through. Will be called once per index, many times per block. Should take exactly one argument: the loop index. It cannot have a return value. + * @param num_blocks The maximum number of blocks to split the loop into. The default is 0, which means the number of blocks will be equal to the number of threads in the pool. + * @param priority The priority of the tasks. Should be between -32,768 and 32,767 (a signed 16-bit integer). The default is 0. Only enabled if `BS_THREAD_POOL_ENABLE_PRIORITY` is defined. + * @return A `multi_future` that can be used to wait for all the blocks to finish. + */ + template <typename T, typename F> + [[nodiscard]] multi_future<void> submit_loop(const T first_index, const T index_after_last, F&& loop, const size_t num_blocks = 0 BS_THREAD_POOL_PRIORITY_INPUT) + { + if (index_after_last > first_index) + { + const blocks blks(first_index, index_after_last, num_blocks ? num_blocks : thread_count); + multi_future<void> future; + future.reserve(blks.get_num_blocks()); + for (size_t blk = 0; blk < blks.get_num_blocks(); ++blk) + future.push_back(submit_task( + [loop = std::forward<F>(loop), start = blks.start(blk), end = blks.end(blk)] + { + for (T i = start; i < end; ++i) + loop(i); + } BS_THREAD_POOL_PRIORITY_OUTPUT)); + return future; + } + return {}; + } + + /** + * @brief Submit a sequence of tasks enumerated by indices to the queue, with the specified priority. Returns a `multi_future` that contains the futures for all of the tasks. + * + * @tparam T The type of the indices. Should be a signed or unsigned integer. + * @tparam F The type of the function used to define the sequence. + * @tparam R The return type of the function used to define the sequence (can be `void`). + * @param first_index The first index in the sequence. + * @param index_after_last The index after the last index in the sequence. The sequence will iterate from `first_index` to `(index_after_last - 1)` inclusive. In other words, it will be equivalent to `for (T i = first_index; i < index_after_last; ++i)`. Note that if `index_after_last <= first_index`, no tasks will be submitted, and an empty `multi_future` will be returned. + * @param sequence The function used to define the sequence. Will be called once per index. Should take exactly one argument, the index. + * @param priority The priority of the tasks. Should be between -32,768 and 32,767 (a signed 16-bit integer). The default is 0. Only enabled if `BS_THREAD_POOL_ENABLE_PRIORITY` is defined. + * @return A `multi_future` that can be used to wait for all the tasks to finish. If the sequence function returns a value, the `multi_future` can also be used to obtain the values returned by each task. + */ + template <typename T, typename F, typename R = std::invoke_result_t<std::decay_t<F>, T>> + [[nodiscard]] multi_future<R> submit_sequence(const T first_index, const T index_after_last, F&& sequence BS_THREAD_POOL_PRIORITY_INPUT) + { + if (index_after_last > first_index) + { + multi_future<R> future; + future.reserve(static_cast<size_t>(index_after_last - first_index)); + for (T i = first_index; i < index_after_last; ++i) + future.push_back(submit_task( + [sequence = std::forward<F>(sequence), i] + { + return sequence(i); + } BS_THREAD_POOL_PRIORITY_OUTPUT)); + return future; + } + return {}; + } + +#ifdef BS_THREAD_POOL_ENABLE_PAUSE + /** + * @brief Unpause the pool. The workers will resume retrieving new tasks out of the queue. Only enabled if `BS_THREAD_POOL_ENABLE_PAUSE` is defined. + */ + void unpause() + { + { + const std::scoped_lock tasks_lock(tasks_mutex); + paused = false; + } + task_available_cv.notify_all(); + } +#endif + +// Macros used internally to enable or disable pausing in the waiting and worker functions. +#ifdef BS_THREAD_POOL_ENABLE_PAUSE +#define BS_THREAD_POOL_PAUSED_OR_EMPTY (paused || tasks.empty()) +#else +#define BS_THREAD_POOL_PAUSED_OR_EMPTY tasks.empty() +#endif + + /** + * @brief Wait for tasks to be completed. Normally, this function waits for all tasks, both those that are currently running in the threads and those that are still waiting in the queue. However, if the pool is paused, this function only waits for the currently running tasks (otherwise it would wait forever). Note: To wait for just one specific task, use `submit_task()` instead, and call the `wait()` member function of the generated future. + * + * @throws `wait_deadlock` if called from within a thread of the same pool, which would result in a deadlock. Only enabled if `BS_THREAD_POOL_ENABLE_WAIT_DEADLOCK_CHECK` is defined. + */ + void wait() + { +#ifdef BS_THREAD_POOL_ENABLE_WAIT_DEADLOCK_CHECK + if (this_thread::get_pool() == this) + throw wait_deadlock(); +#endif + std::unique_lock tasks_lock(tasks_mutex); + waiting = true; + tasks_done_cv.wait(tasks_lock, + [this] + { + return (tasks_running == 0) && BS_THREAD_POOL_PAUSED_OR_EMPTY; + }); + waiting = false; + } + + /** + * @brief Wait for tasks to be completed, but stop waiting after the specified duration has passed. + * + * @tparam R An arithmetic type representing the number of ticks to wait. + * @tparam P An `std::ratio` representing the length of each tick in seconds. + * @param duration The amount of time to wait. + * @return `true` if all tasks finished running, `false` if the duration expired but some tasks are still running. + * + * @throws `wait_deadlock` if called from within a thread of the same pool, which would result in a deadlock. Only enabled if `BS_THREAD_POOL_ENABLE_WAIT_DEADLOCK_CHECK` is defined. + */ + template <typename R, typename P> + bool wait_for(const std::chrono::duration<R, P>& duration) + { +#ifdef BS_THREAD_POOL_ENABLE_WAIT_DEADLOCK_CHECK + if (this_thread::get_pool() == this) + throw wait_deadlock(); +#endif + std::unique_lock tasks_lock(tasks_mutex); + waiting = true; + const bool status = tasks_done_cv.wait_for(tasks_lock, duration, + [this] + { + return (tasks_running == 0) && BS_THREAD_POOL_PAUSED_OR_EMPTY; + }); + waiting = false; + return status; + } + + /** + * @brief Wait for tasks to be completed, but stop waiting after the specified time point has been reached. + * + * @tparam C The type of the clock used to measure time. + * @tparam D An `std::chrono::duration` type used to indicate the time point. + * @param timeout_time The time point at which to stop waiting. + * @return `true` if all tasks finished running, `false` if the time point was reached but some tasks are still running. + * + * @throws `wait_deadlock` if called from within a thread of the same pool, which would result in a deadlock. Only enabled if `BS_THREAD_POOL_ENABLE_WAIT_DEADLOCK_CHECK` is defined. + */ + template <typename C, typename D> + bool wait_until(const std::chrono::time_point<C, D>& timeout_time) + { +#ifdef BS_THREAD_POOL_ENABLE_WAIT_DEADLOCK_CHECK + if (this_thread::get_pool() == this) + throw wait_deadlock(); +#endif + std::unique_lock tasks_lock(tasks_mutex); + waiting = true; + const bool status = tasks_done_cv.wait_until(tasks_lock, timeout_time, + [this] + { + return (tasks_running == 0) && BS_THREAD_POOL_PAUSED_OR_EMPTY; + }); + waiting = false; + return status; + } + +#ifdef BS_THREAD_POOL_ENABLE_WAIT_DEADLOCK_CHECK + // ============== + // Public classes + // ============== + + /** + * @brief An exception that will be thrown by `wait()`, `wait_for()`, and `wait_until()` if the user tries to call them from within a thread of the same pool, which would result in a deadlock. + */ + struct wait_deadlock : public std::runtime_error + { + wait_deadlock() : std::runtime_error("BS::thread_pool::wait_deadlock"){}; + }; +#endif + +private: + // ======================== + // Private member functions + // ======================== + + /** + * @brief Create the threads in the pool and assign a worker to each thread. + * + * @param init_task An initialization function to run in each thread before it starts to execute any submitted tasks. + */ + void create_threads(const std::function<void()>& init_task) + { + { + const std::scoped_lock tasks_lock(tasks_mutex); + tasks_running = thread_count; + workers_running = true; + } + for (concurrency_t i = 0; i < thread_count; ++i) + { + threads[i] = std::thread(&thread_pool::worker, this, i, init_task); + } + } + + /** + * @brief Destroy the threads in the pool. + */ + void destroy_threads() + { + { + const std::scoped_lock tasks_lock(tasks_mutex); + workers_running = false; + } + task_available_cv.notify_all(); + for (concurrency_t i = 0; i < thread_count; ++i) + { + threads[i].join(); + } + } + + /** + * @brief Determine how many threads the pool should have, based on the parameter passed to the constructor or reset(). + * + * @param num_threads The parameter passed to the constructor or `reset()`. If the parameter is a positive number, then the pool will be created with this number of threads. If the parameter is non-positive, or a parameter was not supplied (in which case it will have the default value of 0), then the pool will be created with the total number of hardware threads available, as obtained from `std::thread::hardware_concurrency()`. If the latter returns zero for some reason, then the pool will be created with just one thread. + * @return The number of threads to use for constructing the pool. + */ + [[nodiscard]] static concurrency_t determine_thread_count(const concurrency_t num_threads) + { + if (num_threads > 0) + return num_threads; + if (std::thread::hardware_concurrency() > 0) + return std::thread::hardware_concurrency(); + return 1; + } + + /** + * @brief A worker function to be assigned to each thread in the pool. Waits until it is notified by `detach_task()` that a task is available, and then retrieves the task from the queue and executes it. Once the task finishes, the worker notifies `wait()` in case it is waiting. + * + * @param idx The index of this thread. + * @param init_task An initialization function to run in this thread before it starts to execute any submitted tasks. + */ + void worker(const concurrency_t idx, const std::function<void()>& init_task) + { + this_thread::get_index.index = idx; + this_thread::get_pool.pool = this; + init_task(); + std::unique_lock tasks_lock(tasks_mutex); + while (true) + { + --tasks_running; + tasks_lock.unlock(); + if (waiting && (tasks_running == 0) && BS_THREAD_POOL_PAUSED_OR_EMPTY) + tasks_done_cv.notify_all(); + tasks_lock.lock(); + task_available_cv.wait(tasks_lock, + [this] + { + return !BS_THREAD_POOL_PAUSED_OR_EMPTY || !workers_running; + }); + if (!workers_running) + break; + { +#ifdef BS_THREAD_POOL_ENABLE_PRIORITY + const std::function<void()> task = std::move(std::remove_const_t<pr_task&>(tasks.top()).task); + tasks.pop(); +#else + const std::function<void()> task = std::move(tasks.front()); + tasks.pop(); +#endif + ++tasks_running; + tasks_lock.unlock(); + task(); + } + tasks_lock.lock(); + } + this_thread::get_index.index = std::nullopt; + this_thread::get_pool.pool = std::nullopt; + } + + // =============== + // Private classes + // =============== + + /** + * @brief A helper class to divide a range into blocks. Used by `detach_blocks()`, `submit_blocks()`, `detach_loop()`, and `submit_loop()`. + * + * @tparam T The type of the indices. Should be a signed or unsigned integer. + */ + template <typename T> + class [[nodiscard]] blocks + { + public: + /** + * @brief Construct a `blocks` object with the given specifications. + * + * @param first_index_ The first index in the range. + * @param index_after_last_ The index after the last index in the range. + * @param num_blocks_ The desired number of blocks to divide the range into. + */ + blocks(const T first_index_, const T index_after_last_, const size_t num_blocks_) : first_index(first_index_), index_after_last(index_after_last_), num_blocks(num_blocks_) + { + if (index_after_last > first_index) + { + const size_t total_size = static_cast<size_t>(index_after_last - first_index); + if (num_blocks > total_size) + num_blocks = total_size; + block_size = total_size / num_blocks; + remainder = total_size % num_blocks; + if (block_size == 0) + { + block_size = 1; + num_blocks = (total_size > 1) ? total_size : 1; + } + } + else + { + num_blocks = 0; + } + } + + /** + * @brief Get the first index of a block. + * + * @param block The block number. + * @return The first index. + */ + [[nodiscard]] T start(const size_t block) const + { + return first_index + static_cast<T>(block * block_size) + static_cast<T>(block < remainder ? block : remainder); + } + + /** + * @brief Get the index after the last index of a block. + * + * @param block The block number. + * @return The index after the last index. + */ + [[nodiscard]] T end(const size_t block) const + { + return (block == num_blocks - 1) ? index_after_last : start(block + 1); + } + + /** + * @brief Get the number of blocks. Note that this may be different than the desired number of blocks that was passed to the constructor. + * + * @return The number of blocks. + */ + [[nodiscard]] size_t get_num_blocks() const + { + return num_blocks; + } + + private: + /** + * @brief The size of each block (except possibly the last block). + */ + size_t block_size = 0; + + /** + * @brief The first index in the range. + */ + T first_index = 0; + + /** + * @brief The index after the last index in the range. + */ + T index_after_last = 0; + + /** + * @brief The number of blocks. + */ + size_t num_blocks = 0; + + /** + * @brief The remainder obtained after dividing the total size by the number of blocks. + */ + size_t remainder = 0; + }; // class blocks + +#ifdef BS_THREAD_POOL_ENABLE_PRIORITY + /** + * @brief A helper class to store a task with an assigned priority. + */ + class [[nodiscard]] pr_task + { + friend class thread_pool; + + public: + /** + * @brief Construct a new task with an assigned priority by copying the task. + * + * @param task_ The task. + * @param priority_ The desired priority. + */ + explicit pr_task(const std::function<void()>& task_, const priority_t priority_ = 0) : task(task_), priority(priority_) {} + + /** + * @brief Construct a new task with an assigned priority by moving the task. + * + * @param task_ The task. + * @param priority_ The desired priority. + */ + explicit pr_task(std::function<void()>&& task_, const priority_t priority_ = 0) : task(std::move(task_)), priority(priority_) {} + + /** + * @brief Compare the priority of two tasks. + * + * @param lhs The first task. + * @param rhs The second task. + * @return `true` if the first task has a lower priority than the second task, `false` otherwise. + */ + [[nodiscard]] friend bool operator<(const pr_task& lhs, const pr_task& rhs) + { + return lhs.priority < rhs.priority; + } + + private: + /** + * @brief The task. + */ + std::function<void()> task = {}; + + /** + * @brief The priority of the task. + */ + priority_t priority = 0; + }; // class pr_task +#endif + + // ============ + // Private data + // ============ + +#ifdef BS_THREAD_POOL_ENABLE_PAUSE + /** + * @brief A flag indicating whether the workers should pause. When set to `true`, the workers temporarily stop retrieving new tasks out of the queue, although any tasks already executed will keep running until they are finished. When set to `false` again, the workers resume retrieving tasks. + */ + bool paused = false; +#endif + + /** + * @brief A condition variable to notify `worker()` that a new task has become available. + */ + std::condition_variable task_available_cv = {}; + + /** + * @brief A condition variable to notify `wait()` that the tasks are done. + */ + std::condition_variable tasks_done_cv = {}; + + /** + * @brief A queue of tasks to be executed by the threads. + */ +#ifdef BS_THREAD_POOL_ENABLE_PRIORITY + std::priority_queue<pr_task> tasks = {}; +#else + std::queue<std::function<void()>> tasks = {}; +#endif + + /** + * @brief A counter for the total number of currently running tasks. + */ + size_t tasks_running = 0; + + /** + * @brief A mutex to synchronize access to the task queue by different threads. + */ + mutable std::mutex tasks_mutex = {}; + + /** + * @brief The number of threads in the pool. + */ + concurrency_t thread_count = 0; + + /** + * @brief A smart pointer to manage the memory allocated for the threads. + */ + std::unique_ptr<std::thread[]> threads = nullptr; + + /** + * @brief A flag indicating that `wait()` is active and expects to be notified whenever a task is done. + */ + bool waiting = false; + + /** + * @brief A flag indicating to the workers to keep running. When set to `false`, the workers terminate permanently. + */ + bool workers_running = false; +}; // class thread_pool +} // namespace BS
\ No newline at end of file diff --git a/lib/thread-pool/BS_thread_pool_utils.hpp b/lib/thread-pool/BS_thread_pool_utils.hpp new file mode 100644 index 0000000..a09630f --- /dev/null +++ b/lib/thread-pool/BS_thread_pool_utils.hpp @@ -0,0 +1,203 @@ +#pragma once + +/** + * @file BS_thread_pool_utils.hpp + * @author Barak Shoshany (baraksh@gmail.com) (http://baraksh.com) + * @version 4.0.1 + * @date 2023-12-28 + * @copyright Copyright (c) 2023 Barak Shoshany. Licensed under the MIT license. If you found this project useful, please consider starring it on GitHub! If you use this library in software of any kind, please provide a link to the GitHub repository https://github.com/bshoshany/thread-pool in the source code and documentation. If you use this library in published research, please cite it as follows: Barak Shoshany, "A C++17 Thread Pool for High-Performance Scientific Computing", doi:10.5281/zenodo.4742687, arXiv:2105.00613 (May 2021) + * + * @brief BS::thread_pool: a fast, lightweight, and easy-to-use C++17 thread pool library. This header file contains independent utility classes that are part of the library, but are not needed to use the thread pool itself. + */ + +#include <atomic> // std::atomic +#include <chrono> // std::chrono +#include <future> // std::promise, std::shared_future +#include <initializer_list> // std::initializer_list +#include <iostream> // std::cout +#include <memory> // std::make_unique, std::unique_ptr +#include <mutex> // std::mutex, std::scoped_lock +#include <ostream> // std::endl, std::flush, std::ostream +#include <utility> // std::forward + +/** + * @brief A namespace used by Barak Shoshany's projects. + */ +namespace BS { +// Macros indicating the version of the thread pool utilities library. +#define BS_THREAD_POOL_UTILS_VERSION_MAJOR 4 +#define BS_THREAD_POOL_UTILS_VERSION_MINOR 0 +#define BS_THREAD_POOL_UTILS_VERSION_PATCH 1 + +/** + * @brief A utility class to allow simple signalling between threads. + */ +class [[nodiscard]] signaller +{ +public: + /** + * @brief Construct a new signaller. + */ + signaller() : promise(), future(promise.get_future()) {} + + // The copy constructor and copy assignment operator are deleted. The signaller works using a promise, which cannot be copied. + signaller(const signaller&) = delete; + signaller& operator=(const signaller&) = delete; + + // The move constructor and move assignment operator are defaulted. + signaller(signaller&&) = default; + signaller& operator=(signaller&&) = default; + + /** + * @brief Inform any waiting threads that the signaller is ready. + */ + void ready() + { + promise.set_value(); + } + + /** + * @brief Wait until the signaller is ready. + */ + void wait() + { + future.wait(); + } + +private: + /** + * @brief A promise used to set the state of the signaller. + */ + std::promise<void> promise; + + /** + * @brief A future used to wait for the signaller. + */ + std::shared_future<void> future; +}; // class signaller + +/** + * @brief A utility class to synchronize printing to an output stream by different threads. + */ +class [[nodiscard]] synced_stream +{ +public: + /** + * @brief Construct a new synced stream. + * + * @param stream The output stream to print to. The default value is `std::cout`. + */ + explicit synced_stream(std::ostream& stream = std::cout) : out_stream(stream) {} + + // The copy and move constructors and assignment operators are deleted. The synced stream uses a mutex, which cannot be copied or moved. + synced_stream(const synced_stream&) = delete; + synced_stream(synced_stream&&) = delete; + synced_stream& operator=(const synced_stream&) = delete; + synced_stream& operator=(synced_stream&&) = delete; + + /** + * @brief Print any number of items into the output stream. Ensures that no other threads print to this stream simultaneously, as long as they all exclusively use the same `synced_stream` object to print. + * + * @tparam T The types of the items. + * @param items The items to print. + */ + template <typename... T> + void print(T&&... items) + { + const std::scoped_lock stream_lock(stream_mutex); + (out_stream << ... << std::forward<T>(items)); + } + + /** + * @brief Print any number of items into the output stream, followed by a newline character. Ensures that no other threads print to this stream simultaneously, as long as they all exclusively use the same `synced_stream` object to print. + * + * @tparam T The types of the items. + * @param items The items to print. + */ + template <typename... T> + void println(T&&... items) + { + print(std::forward<T>(items)..., '\n'); + } + + /** + * @brief A stream manipulator to pass to a `synced_stream` (an explicit cast of `std::endl`). Prints a newline character to the stream, and then flushes it. Should only be used if flushing is desired, otherwise a newline character should be used instead. + */ + inline static std::ostream& (&endl)(std::ostream&) = static_cast<std::ostream& (&)(std::ostream&)>(std::endl); + + /** + * @brief A stream manipulator to pass to a `synced_stream` (an explicit cast of `std::flush`). Used to flush the stream. + */ + inline static std::ostream& (&flush)(std::ostream&) = static_cast<std::ostream& (&)(std::ostream&)>(std::flush); + +private: + /** + * @brief The output stream to print to. + */ + std::ostream& out_stream; + + /** + * @brief A mutex to synchronize printing. + */ + mutable std::mutex stream_mutex = {}; +}; // class synced_stream + +/** + * @brief A utility class to measure execution time for benchmarking purposes. + */ +class [[nodiscard]] timer +{ +public: + /** + * @brief Construct a new timer and immediately start measuring time. + */ + timer() = default; + + /** + * @brief Get the number of milliseconds that have elapsed since the object was constructed or since `start()` was last called, but keep the timer ticking. + * + * @return The number of milliseconds. + */ + [[nodiscard]] std::chrono::milliseconds::rep current_ms() const + { + return (std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - start_time)).count(); + } + + /** + * @brief Start (or restart) measuring time. Note that the timer starts ticking as soon as the object is created, so this is only necessary if we want to restart the clock later. + */ + void start() + { + start_time = std::chrono::steady_clock::now(); + } + + /** + * @brief Stop measuring time and store the elapsed time since the object was constructed or since `start()` was last called. + */ + void stop() + { + elapsed_time = std::chrono::steady_clock::now() - start_time; + } + + /** + * @brief Get the number of milliseconds stored when `stop()` was last called. + * + * @return The number of milliseconds. + */ + [[nodiscard]] std::chrono::milliseconds::rep ms() const + { + return (std::chrono::duration_cast<std::chrono::milliseconds>(elapsed_time)).count(); + } + +private: + /** + * @brief The time point when measuring started. + */ + std::chrono::time_point<std::chrono::steady_clock> start_time = std::chrono::steady_clock::now(); + + /** + * @brief The duration that has elapsed between `start()` and `stop()`. + */ + std::chrono::duration<double> elapsed_time = std::chrono::duration<double>::zero(); +}; // class timer +} // namespace BS
\ No newline at end of file diff --git a/lib/thread-pool/CHANGELOG.md b/lib/thread-pool/CHANGELOG.md new file mode 100644 index 0000000..0fc03d2 --- /dev/null +++ b/lib/thread-pool/CHANGELOG.md @@ -0,0 +1,218 @@ +[](https://doi.org/10.5281/zenodo.4742687) +[](https://arxiv.org/abs/2105.00613) +[](https://github.com/bshoshany/thread-pool/blob/master/LICENSE.txt) + + + +[](https://github.com/bshoshany/thread-pool) +[](https://twitter.com/BarakShoshany) +[](https://vscode.dev/github/bshoshany/thread-pool) + +# `BS::thread_pool`: a fast, lightweight, and easy-to-use C++17 thread pool library + +By Barak Shoshany<br /> +Email: [baraksh@gmail.com](mailto:baraksh@gmail.com)<br /> +Website: [https://baraksh.com/](https://baraksh.com/)<br /> +GitHub: [https://github.com/bshoshany](https://github.com/bshoshany)<br /> + +* [Version history](#version-history) + * [v3.3.0 (2022-08-03)](#v330-2022-08-03) + * [v3.2.0 (2022-07-28)](#v320-2022-07-28) + * [v3.1.0 (2022-07-13)](#v310-2022-07-13) + * [v3.0.0 (2022-05-30)](#v300-2022-05-30) + * [v2.0.0 (2021-08-14)](#v200-2021-08-14) + * [v1.9 (2021-07-29)](#v19-2021-07-29) + * [v1.8 (2021-07-28)](#v18-2021-07-28) + * [v1.7 (2021-06-02)](#v17-2021-06-02) + * [v1.6 (2021-05-26)](#v16-2021-05-26) + * [v1.5 (2021-05-07)](#v15-2021-05-07) + * [v1.4 (2021-05-05)](#v14-2021-05-05) + * [v1.3 (2021-05-03)](#v13-2021-05-03) + * [v1.2 (2021-04-29)](#v12-2021-04-29) + * [v1.1 (2021-04-24)](#v11-2021-04-24) + * [v1.0 (2021-01-15)](#v10-2021-01-15) + +## Version history + +### v3.3.0 (2022-08-03) + +* `BS_thread_pool.hpp`: + * The public member variable `paused` of `BS::thread_pool` has been made private for future-proofing (in case future versions implement a more involved pausing mechanism) and better encapsulation. It is now accessible only via the `pause()`, `unpause()`, and `is_paused()` member functions. In other words: + * Replace `pool.paused = true` with `pool.pause()`. + * Replace `pool.paused = false` with `pool.unpause()`. + * Replace `if (pool.paused)` (or similar) with `if (pool.is_paused())`. + * The public member variable `f` of `BS::multi_future` has been renamed to `futures` for clarity, and has been made private for encapsulation and simplification purposes. Instead of operating on the vector `futures` itself, you can now use the `[]` operator of the `BS::multi_future` to access the future at a specific index directly, or the `push_back()` member function to append a new future to the list. The `size()` member function tells you how many futures are currently stored in the object. + * The explicit casts of `std::endl` and `std::flush`, added in v3.2.0 to enable flushing a `BS::synced_stream`, caused ODR (One Definition Rule) violations if `BS_thread_pool.hpp` was included in two different translation units, since they were mistakenly not defined as `inline`. To fix this, I decided to make them static members of `BS::synced_stream` instead of global variables, which also makes the code better organized in my opinion. These objects can now be accessed as `BS::synced_stream::endl` and `BS::synced_stream::flush`. I also added an example for how to use them in `README.md`. See [#64](https://github.com/bshoshany/thread-pool/issues/64). +* `BS_thread_pool_light.hpp`: + * This package started out as a very lightweight thread pool, but over time has expanded to include many additional features, and at the time of writing it has a total of 340 lines of code, including all the helper classes. Therefore, I have decided to bundle a light version of the thread pool in a separate and stand-alone header file, `BS_thread_pool_light.hpp`, with only 170 lines of code (half the size of the full package). This file does not contain any of the helper classes, only a new `BS::thread_pool_light` class, which is a minimal thread pool with only the 5 most basic member functions: + * `get_thread_count()` + * `push_loop()` + * `push_task()` + * `submit()` + * `wait_for_tasks()` + * A separate test program `BS_thread_pool_light_test.cpp` tests only the features of the lightweight `BS::thread_pool_light` class. In the spirit of minimalism, it does not generate a log file and does not do any benchmarks. + * To be perfectly clear, each header file is 100% stand-alone. If you wish to use the full package, you only need `BS_thread_pool.hpp`, and if you wish to use the light version, you only need `BS_thread_pool_light.hpp`. Only a single header file needs to be included in your project. + +### v3.2.0 (2022-07-28) + +* `BS_thread_pool.hpp`: + * Main `BS::thread_pool` class: + * Added a new member function, `push_loop()`, which does the same thing as `parallelize_loop()`, except that it does not return a `BS::multi_future` with the futures for each block. Just like `push_task()` vs. `submit()`, this avoids the overhead of creating the futures, but the user must use `wait_for_tasks()` or some other method to ensure that the loop finishes executing, otherwise bad things will happen. + * `push_task()` and `submit()` now utilize perfect forwarding in order to support more types of tasks - in particular member functions, which in previous versions could not be submitted unless wrapped in a lambda. To submit a member function, use the syntax `submit(&class::function, &object, args)`. More information can be found in `README.md`. See [#9](https://github.com/bshoshany/thread-pool/issues/9). + * `push_loop()` and `parallelize_loop()` now have overloads where the first argument (the first index in the loop) is omitted, in which case it is assumed to be 0. This is for convenience, as the case where the first index is 0 is very common. + * Helper classes: + * `BS::synced_stream` now utilizes perfect forwarding in the member functions `print()` and `println()`. + * Previously, it was impossible to pass the flushing manipulators `std::endl` and `std::flush` to `print()` and `println()`, since the compiler could not figure out which template specializations to use. The new objects `BS::endl` and `BS::flush` are explicit casts of these manipulators, whose sole purpose is to enable passing them to `print()` and `println()`. + * `BS::multi_future::get()` now rethrows exceptions generated by the futures, even if the futures return `void`. See [#62](https://github.com/bshoshany/thread-pool/pull/62). + * Added a new helper class, `BS::blocks`, which is used by `parallelize_loop()` and `push_loop()` to divide a range into blocks. This class is not documented in `README.md`, as it most likely will not be of interest to most users, but it is still publicly available, in case you want to parallelize something manually but still benefit from the built-in algorithm for splitting a range into blocks. +* `BS_thread_pool_test.cpp`: + * Added plenty of new tests for the new features described above. + * Fixed a bug in `count_unique_threads()` that caused it to get stuck on certain systems. + * `dual_println()` now also flushes the stream using `BS::endl`, so that if the test gets stuck, the log file will still contain everything up to that point. (Note: It is a common misconception that `std::endl` and `'\n'` are interchangeable. `std::endl` not only prints a newline character, it also flushes the stream, which is not always desirable, as it may reduce performance.) + * The performance test has been modified as follows: + * Instead of generating random vectors using `std::mersenne_twister_engine`, which proved to be inconsistent across different compilers and systems, the test now generates each element via an arbitrarily-chosen numerical operation. In my testing, this provided much more consistent results. + * Instead of using a hard-coded vector size, a suitable vector size is now determined dynamically at runtime. + * Instead of using `parallelize_loop()`, the test now uses the new `push_loop()` function to squeeze out a bit more performance. + * Instead of setting the test parameters to achieve a fixed single-threaded mean execution time of 300 ms, the test now aims to achieve a fixed multi-threaded mean execution time of 50 ms when the number of blocks is equal to the number of threads. This allows for more reliable results on very fast CPUs with a very large number of threads, where the mean execution time when using all the threads could previously be below a statistically significant value. + * The number of vectors is now restricted to be a multiple of the number of threads, so that the blocks are always all of the same size. +* `README.md`: + * Added instructions and examples for the new features described above. + * Rewrote the documentation for `parallelize_loop()` to make it clearer. + +### v3.1.0 (2022-07-13) + +* `BS_thread_pool.hpp`: + * Fixed an issue where `wait_for_tasks()` would sometimes get stuck if `push_task()` was executed immediately before `wait_for_tasks()`. + * Both the thread pool constructor and the `reset()` member function now determine the number of threads to use in the pool as follows. If the parameter is a positive number, then the pool will be created with this number of threads. If the parameter is non-positive, or a parameter was not supplied, then the pool will be created with the total number of hardware threads available, as obtained from `std::thread::hardware_concurrency()`. If the latter returns a non-positive number for some reason, then the pool will be created with just one thread. See [#51](https://github.com/bshoshany/thread-pool/issues/51) and [#52](https://github.com/bshoshany/thread-pool/issues/52). + * Added the `[[nodiscard]]` attribute to classes and class members, in order to warn the user when accidentally discarding an important return value, such as a future or the return value of a function with no useful side-effects. For example, if you use `submit()` and don't save the future it returns, the compiler will now generate a warning. (If a future is not needed, then you should use `push_task()` instead.) + * Removed the `explicit` specifier from all constructors, as it prevented the default constructor from being used with static class members. See [#48](https://github.com/bshoshany/thread-pool/issues/48>). +* `BS_thread_pool_test.cpp`: + * Improved `count_unique_threads()` using condition variables, to ensure that each thread in the pool runs at least one task regardless of how fast it takes to run the tasks. + * When appropriate, `check()` now explicitly reports what the obtained result was and what it was expected to be. + * `check_task_monitoring()` and `check_pausing()` now explicitly report the results of the monitoring at each step. + * Changed all instances of `std::vector<std::atomic<bool>>` to `std::unique_ptr<std::atomic<bool>[]>`. See [#44](https://github.com/bshoshany/thread-pool/issues/44). + * Converted a few more C-style casts to C++ cast expressions. +* `README.md`: + * Added instructions for using this package with the [Conan](https://conan.io/) C/C++ package manager. Please refer to [this package's page on ConanCenter](https://conan.io/center/bshoshany-thread-pool) to learn how to use Conan to include this package in your project with various build systems. +* If you found this project useful, please consider [starring it on GitHub](https://github.com/bshoshany/thread-pool/stargazers)! This allows me to see how many people are using my code, and motivates me to keep working to improve it. + +### v3.0.0 (2022-05-30) + +* This is a major new release with many changes and improvements! Please note that code written using previous releases will need to be slightly modified to work with the new release. The changes needed to migrate to the new API are explicitly indicated below for your convenience. +* Breaking changes to the library header file: + * The header file has been renamed to `BS_thread_pool.hpp` to avoid potential conflict with other thread pool libraries. + * **API migration:** The library must now be included by invoking `#include "BS_thread_pool.hpp"`. + * All the definitions in the library, including the `thread_pool` class and the helper classes, are now located in the namespace `BS`. This namespace will also be used for my other C++ projects, and is intended to ensure consistency between my projects while avoiding potential name conflicts with other libraries. + * **API migration:** The thread pool class should now be invoked as `BS::thread_pool`. Alternatively, it is possible to employ `using BS::thread_pool` or even `using namespace BS` and then invoke `thread_pool` directly. Same for the `BS::synced_stream` and `BS::timer` helper classes. + * The macro `THREAD_POOL_VERSION`, which contains the version number and release date of the library, has been renamed to `BS_THREAD_POOL_VERSION` to avoid potential conflicts. + * **API migration:** The version must now be read from the macro `BS_THREAD_POOL_VERSION`. + * The public member `sleep_duration` has been removed. The thread pool now uses condition variables instead of sleep to facilitate waiting. This significantly improves performance (by 10%-50% in my testing), drastically decreases idle CPU utilization, and eliminates the need to set an optimal sleep time. This was a highly-requested change; see [issue #1](https://github.com/bshoshany/thread-pool/issues/1), [issue #12](https://github.com/bshoshany/thread-pool/issues/12), and [pull request #23](https://github.com/bshoshany/thread-pool/pull/23). + * **API migration:** Remove any code that relates to the public member `sleep_duration`. + * The template specializations for `submit()` have been merged. Now instead of two versions, one for functions with a return value and one for functions without a return value, there is just one version, which can accept any function. This makes the code more compact (and elegant). If a function with no return value is submitted, an `std::future<void>` is returned (the previous version returned an `std::future<bool>`) + * **API migration:** To wait for a task with no return value, simply call `wait()` or `get()` on the corresponding `std::future<void>`. + * `parallelize_loop()` now returns a future in the form of a new `BS::multi_future` helper class template. The member function `wait()` of this future allows waiting until all of the loop's blocks finish executing. In previous versions, calling `parallelize_loop()` both parallelized the loop and waited for the blocks to finish; now it is possible to do other stuff while the loop executes. + * **API migration:** Since `parallelize_loop()` no longer automatically blocks, you should either store the result in a `BS::multi_future` object and call its `wait()` member function, or simply call `parallelize_loop().wait()` to reproduce the old behavior. +* Non-breaking changes to the library header file: + * It is now possible to use `parallelize_loop()` with functions that have return values and get these values from all blocks at once through the `get()` member function of the `BS::multi_future`. + * The template specializations for `push_task()` have been merged. Now instead of two versions, one for functions with arguments and one for functions without arguments, there is just one version, which can accept any function. + * Constructors have been made `explicit`. See [issue #28](https://github.com/bshoshany/thread-pool/issues/28). + * `submit()` now uses `std::make_shared` instead of `new` to create the shared pointer. This means only one memory allocation is performed instead of two, which should improve performance. In addition, all unique pointers are now created using `std::make_unique`. + * A new helper class template, `BS::multi_future`, has been added. It's basically just a wrapper around `std::vector<std::future<T>>`. This class is used by the new implementation of `parallelize_loop()` to allow waiting for the entire loop, consisting of multiple tasks with their corresponding futures, to finish executing. + * `BS::multi_future` can also be used independently to handle multiple futures at once. For example, you can now keep track of several groups of tasks by storing their futures inside separate `BS::multi_future` objects and use either `wait()` to wait for all tasks in a specific group to finish or `get()` to get an `std::vector` with the return values of every task in the group. + * Integer types are now chosen in a smarter way to improve portability, allow for better compatibility with 32-bit systems, and prevent potential conversion errors. + * Added a new type, `BS::concurrency_t`, equal to the return type of `std::thread::hardware_concurrency()`. This is probably pointless, since the C++ standard requires this to be `unsigned int`, but it seems to me to make the code slightly more portable, in case some non-conforming compiler chooses to use a different integer type. + * C-style casts have been converted to C++ cast expressions for added clarity. + * Miscellaneous minor optimizations and style improvements. +* Changes to the test program: + * The program has been renamed to `BS_thread_pool_test.cpp` to avoid potential conflict with other thread pool libraries. + * The program now returns `EXIT_FAILURE` if any of the tests failed, for automation purposes. See [pull request #42](https://github.com/bshoshany/thread-pool/pull/42). + * Fixed incorrect check order in `check_task_monitoring()`. See [pull request #43](https://github.com/bshoshany/thread-pool/pull/43). + * Added a new test for `parallelize_loop()` with a return value. + * Improved some of the tests to make them more reliable. For example, `count_unique_threads()` now uses futures (stored in a `BS::multi_future<void>` object). + * The program now uses `std::vector` instead of matrices, for both consistency checks and benchmarks, in order to simplify the code and considerably reduce its length. + * The benchmarks have been simplified. There's now only one test: filling a specific number of vectors of fixed size with random values. This may be replaced with something more practical in a future released, but at least on the systems I've tested on, it does demonstrate a very significant multi-threading speedup. + * In addition to multi-threaded tests with different numbers of tasks, the benchmark now also includes a single-threaded test. This allows for more accurate benchmarks compared to previous versions, as the (slight) parallelization overhead is now taken into account when calculating the maximum speedup. + * The program decides how many vectors to use for benchmarking by testing how many are needed to reach a target duration in the single-threaded test. This ensures that the test takes approximately the same amount of time on different systems, and is thus more consistent and portable. + * Miscellaneous minor optimizations and style improvements. +* Changes to `README.md`: + * Many sections have been rewritten and/or polished. + * Explanations and examples of all the new features have been added. + * Added an acknowledgements section. +* Miscellaneous changes: + * Added a `CITATION.bib` file (in BibTeX format) to the GitHub repository. You can use it to easily cite this package if you use it in any research papers. + * Added a `CITATION.cff` file (in YAML format) to the GitHub repository. This should add [an option to get a citation in different formats](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-repository-on-github/about-citation-files) directly from GitHub repository by clicking on "cite this repository" on the sidebar to the right. + * Added templates for GitHub issues and pull requests. + +### v2.0.0 (2021-08-14) + +* From now on, version numbers will adhere to the [Semantic Versioning](https://semver.org/) specification in the format **major.minor.patch**. +* A file named `thread_pool_test.cpp` has been added to the package. It will perform automated tests of all aspects of the package, and benchmark some multithreaded matrix operations. Please run it on your system and [submit a bug report](https://github.com/bshoshany/thread-pool/issues) if any of the tests fail. In addition, the code is thoroughly documented, and is meant to serve as an extensive example of how to properly use the package. +* The package is now available through [vcpkg](https://github.com/microsoft/vcpkg). Instructions for how to install it have been added to `README.md`. See [this pull request](https://github.com/bshoshany/thread-pool/pull/18). +* The package now defines a macro `THREAD_POOL_VERSION`, which returns the version number and release date of the thread pool library as a string. +* `parallelize_loop()` has undergone some major changes (and is now incompatible with v1.x): + * The second argument is now the index **after** the last index, instead of the last index itself. This is more consistent with C++ conventions (e.g. standard library algorithms) where the range is always `[first, last)`. For example, for an array with `n` indices, instead of `parallelize_loop(0, n - 1, ...)` you should now write `parallelize_loop(0, n, ...)`. + * The `loop` function is now only called once per block, instead of once per index, as was the case before. This should provide a performance boost due to significantly reducing the number of function calls, and it also allows you to conserve resources by using them only once per block instead of once per index (an example can be found in the `random_matrix_generator` class in `thread_pool_test.cpp`). It also means that `loop` now takes two arguments: the first index in the block and the index after the last index in the block. Thus, `loop(start, end)` should typically involve a loop of the form `for (T i = start; i < end; i++)`. + * The first and last indices can now be of two different integer types. Previously, `parallelize_loop(0, i, ...)` did not work if `i` was not an `int`, because `0` was interpreted as `int`, and the two arguments had to be of the same type. Therefore, one had to use casting, e.g. `parallelize_loop((size_t)0, i)`, to make it work. Now this is no longer necessary; the common type is inferred automatically using `std::common_type_t`. + +### v1.9 (2021-07-29) + +* Fixed a bug in `reset()` which caused it to create the wrong number of threads. + +### v1.8 (2021-07-28) + +* The version history has become too long to be included in `README.md`, so I moved it to a separate file, `CHANGELOG.md`. +* A button to open this repository directly in Visual Studio Code has been added to the badges in `README.md`. +* An internal variable named `promise` has been renamed to `task_promise` to avoid any potential errors in case the user invokes `using namespace std`. +* `submit()` now catches exceptions thrown by the submitted task and forwards them to the future. See [this issue](https://github.com/bshoshany/thread-pool/issues/14). +* Eliminated compiler warnings that appeared when using the `-Weffc++` flag in GCC. See [this pull request](https://github.com/bshoshany/thread-pool/pull/17). + +### v1.7 (2021-06-02) + +* Fixed a bug in `parallelize_loop()` which prevented it from actually running loops in parallel, see [this issue](https://github.com/bshoshany/thread-pool/issues/11). + +### v1.6 (2021-05-26) + +* Since MSVC does not interpret `and` as `&&` by default, the previous release did not compile with MSVC unless the `/permissive-` or `/Za` compiler flags were used. This has been fixed in this version, and the code now successfully compiles with GCC, Clang, and MSVC. See [this pull request](https://github.com/bshoshany/thread-pool/pull/10). + +### v1.5 (2021-05-07) + +* This library now has a DOI for citation purposes. Information on how to cite it in publications has been added to the source code and to `README.md`. +* Added GitHub badges to `README.md`. + +### v1.4 (2021-05-05) + +* Added three new public member functions to monitor the tasks submitted to the pool: + * `get_tasks_queued()` gets the number of tasks currently waiting in the queue to be executed by the threads. + * `get_tasks_running()` gets the number of tasks currently being executed by the threads. + * `get_tasks_total()` gets the total number of unfinished tasks - either still in the queue, or running in a thread. + * Note that `get_tasks_running() == get_tasks_total() - get_tasks_queued()`. + * Renamed the private member variable `tasks_waiting` to `tasks_total` to make its purpose clearer. +* Added an option to temporarily pause the workers: + * When public member variable `paused` is set to `true`, the workers temporarily stop popping new tasks out of the queue, although any tasks already executed will keep running until they are done. Set to `false` again to resume popping tasks. + * While the workers are paused, `wait_for_tasks()` will wait for the running tasks instead of all tasks (otherwise it would wait forever). + * By utilizing the new pausing mechanism, `reset()` can now change the number of threads on-the-fly while there are still tasks waiting in the queue. The new thread pool will resume executing tasks from the queue once it is created. +* `parallelize_loop()` and `wait_for_tasks()` now have the same behavior as the worker function with regards to waiting for tasks to complete. If the relevant tasks are not yet complete, then before checking again, they will sleep for `sleep_duration` microseconds, unless that variable is set to zero, in which case they will call `std::this_thread::yield()`. This should improve performance and reduce CPU usage. +* Merged [this commit](https://github.com/bshoshany/thread-pool/pull/8): Fixed weird error when using MSVC and including `windows.h`. +* The `README.md` file has been reorganized and expanded. + +### v1.3 (2021-05-03) + +* Fixed [this issue](https://github.com/bshoshany/thread-pool/issues/3): Removed `std::move` from the `return` statement in `push_task()`. This previously generated a `-Wpessimizing-move` warning in Clang. The assembly code generated by the compiler seems to be the same before and after this change, presumably because the compiler eliminates the `std::move` automatically, but this change gets rid of the Clang warning. +* Fixed [this issue](https://github.com/bshoshany/thread-pool/issues/5): Removed a debugging message printed to `std::cout`, which was left in the code by mistake. +* Fixed [this issue](https://github.com/bshoshany/thread-pool/issues/6): `parallelize_loop()` no longer sends references for the variables `start` and `stop` when calling `push_task()`, which may lead to undefined behavior. +* A companion paper is now published at <a href="https://arxiv.org/abs/2105.00613">arXiv:2105.00613</a>, including additional information such as performance tests on systems with up to 80 hardware threads. The `README.md` has been updated, and it is now roughly identical in content to the paper. + +### v1.2 (2021-04-29) + +* The worker function, which controls the execution of tasks by each thread, now sleeps by default instead of yielding. Previously, when the worker could not find any tasks in the queue, it called `std::this_thread::yield()` and then tried again. However, this caused the workers to have high CPU usage when idle, [as reported by some users](https://github.com/bshoshany/thread-pool/issues/1). Now, when the worker function cannot find a task to run, it instead sleeps for a duration given by the public member variable `sleep_duration` (in microseconds) before checking the queue again. The default value is `1000` microseconds, which I found to be optimal in terms of both CPU usage and performance, but your own optimal value may be different. +* If the constructor is called with an argument of zero for the number of threads, then the default value, `std::thread::hardware_concurrency()`, is used instead. +* Added a simple helper class, `timer`, which can be used to measure execution time for benchmarking purposes. +* Improved and expanded the documentation. + +### v1.1 (2021-04-24) + +* Cosmetic changes only. Fixed a typo in the Doxygen comments and added a link to the GitHub repository. + +### v1.0 (2021-01-15) + +* Initial release. diff --git a/lib/thread-pool/CITATION.bib b/lib/thread-pool/CITATION.bib new file mode 100644 index 0000000..51406fd --- /dev/null +++ b/lib/thread-pool/CITATION.bib @@ -0,0 +1,13 @@ +@article{Shoshany2021_ThreadPool, + archiveprefix = {arXiv}, + author = {Barak Shoshany}, + doi = {10.5281/zenodo.4742687}, + eid = {arXiv:2105.00613}, + eprint = {2105.00613}, + journal = {arXiv e-prints}, + keywords = {Computer Science - Distributed, Parallel, and Cluster Computing, D.1.3, D.1.5}, + month = {May}, + primaryclass = {cs.DC}, + title = {{A C++17 Thread Pool for High-Performance Scientific Computing}}, + year = {2021} +} diff --git a/lib/thread-pool/CITATION.cff b/lib/thread-pool/CITATION.cff new file mode 100644 index 0000000..99bb729 --- /dev/null +++ b/lib/thread-pool/CITATION.cff @@ -0,0 +1,24 @@ +--- +authors: + - family-names: "Shoshany" + given-names: "Barak" + orcid: "https://orcid.org/0000-0003-2222-127X" +cff-version: "1.2.0" +date-released: "2021-05-03" +doi: "10.5281/zenodo.4742687" +license: "MIT" +message: "If you use this package in published research, please cite it as follows." +repository-code: "https://github.com/bshoshany/thread-pool" +title: "A C++17 Thread Pool for High-Performance Scientific Computing" +preferred-citation: + type: "article" + authors: + - family-names: "Shoshany" + given-names: "Barak" + orcid: "https://orcid.org/0000-0003-2222-127X" + doi: "10.5281/zenodo.4742687" + journal: "arXiv" + month: 5 + title: "A C++17 Thread Pool for High-Performance Scientific Computing" + url: "https://arxiv.org/abs/2105.00613" + year: 2021 diff --git a/lib/thread-pool/LICENSE.txt b/lib/thread-pool/LICENSE.txt new file mode 100644 index 0000000..63a12c8 --- /dev/null +++ b/lib/thread-pool/LICENSE.txt @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2022 Barak Shoshany + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/lib/thread-pool/README.md b/lib/thread-pool/README.md new file mode 100644 index 0000000..ff92364 --- /dev/null +++ b/lib/thread-pool/README.md @@ -0,0 +1,1498 @@ +[](https://doi.org/10.5281/zenodo.4742687) +[](https://arxiv.org/abs/2105.00613) +[](https://github.com/bshoshany/thread-pool/blob/master/LICENSE.txt) + + + +[](https://github.com/bshoshany/thread-pool) +[](https://twitter.com/BarakShoshany) +[](https://vscode.dev/github/bshoshany/thread-pool) + +# `BS::thread_pool`: a fast, lightweight, and easy-to-use C++17 thread pool library + +By Barak Shoshany<br /> +Email: [baraksh@gmail.com](mailto:baraksh@gmail.com)<br /> +Website: [https://baraksh.com/](https://baraksh.com/)<br /> +GitHub: [https://github.com/bshoshany](https://github.com/bshoshany)<br /> + +This is the complete documentation for v3.3.0 of the library, released on 2022-08-03. + +* [Introduction](#introduction) + * [Motivation](#motivation) + * [Overview of features](#overview-of-features) + * [Compiling and compatibility](#compiling-and-compatibility) + * [Installing using vcpkg](#installing-using-vcpkg) + * [Installing using Conan](#installing-using-conan) +* [Getting started](#getting-started) + * [Including the library](#including-the-library) + * [Constructors](#constructors) + * [Getting and resetting the number of threads in the pool](#getting-and-resetting-the-number-of-threads-in-the-pool) + * [Finding the version of the package](#finding-the-version-of-the-package) +* [Submitting and waiting for tasks](#submitting-and-waiting-for-tasks) + * [Submitting tasks to the queue with futures](#submitting-tasks-to-the-queue-with-futures) + * [Submitting tasks to the queue without futures](#submitting-tasks-to-the-queue-without-futures) + * [Manually waiting for all tasks to complete](#manually-waiting-for-all-tasks-to-complete) + * [Submitting class member functions to the queue](#submitting-class-member-functions-to-the-queue) + * [Parallelizing loops](#parallelizing-loops) + * [Loops with return values](#loops-with-return-values) + * [Parallelizing loops without futures](#parallelizing-loops-without-futures) +* [Helper classes](#helper-classes) + * [Synchronizing printing to an output stream](#synchronizing-printing-to-an-output-stream) + * [Handling multiple futures at once](#handling-multiple-futures-at-once) + * [Measuring execution time](#measuring-execution-time) +* [Other features](#other-features) + * [Monitoring the tasks](#monitoring-the-tasks) + * [Pausing the workers](#pausing-the-workers) + * [Exception handling](#exception-handling) +* [Testing the package](#testing-the-package) + * [Automated tests](#automated-tests) + * [Performance tests](#performance-tests) +* [The light version of the package](#the-light-version-of-the-package) +* [About the project](#about-the-project) + * [Issue and pull request policy](#issue-and-pull-request-policy) + * [Acknowledgements](#acknowledgements) + * [Starring the repository](#starring-the-repository) + * [Copyright and citing](#copyright-and-citing) + +## Introduction + +### Motivation + +Multithreading is essential for modern high-performance computing. Since C++11, the C++ standard library has included built-in low-level multithreading support using constructs such as `std::thread`. However, `std::thread` creates a new thread each time it is called, which can have a significant performance overhead. Furthermore, it is possible to create more threads than the hardware can handle simultaneously, potentially resulting in a substantial slowdown. + +The library presented here contains a thread pool class, `BS::thread_pool`, which avoids these issues by creating a fixed pool of threads once and for all, and then continuously reusing the same threads to perform different tasks throughout the lifetime of the program. By default, the number of threads in the pool is equal to the maximum number of threads that the hardware can run in parallel. + +The user submits tasks to be executed into a queue. Whenever a thread becomes available, it retrieves the next task from the queue and executes it. The pool automatically produces an `std::future` for each task, which allows the user to wait for the task to finish executing and/or obtain its eventual return value, if applicable. Threads and tasks are autonomously managed by the pool in the background, without requiring any input from the user aside from submitting the desired tasks. + +The design of this package was guided by four important principles. First, *compactness*: the entire library consists of just one small self-contained header file, with no other components or dependencies. Second, *portability*: the package only utilizes the C++17 standard library, without relying on any compiler extensions or 3rd-party libraries, and is therefore compatible with any modern standards-conforming C++17 compiler on any platform. Third, *ease of use*: the package is extensively documented, and programmers of any level should be able to use it right out of the box. + +The fourth and final guiding principle is *performance*: each and every line of code in this library was carefully designed with maximum performance in mind, and performance was tested and verified on a variety of compilers and platforms. Indeed, the library was originally designed for use in the author's own computationally-intensive scientific computing projects, running both on high-end desktop/laptop computers and high-performance computing nodes. + +Other, more advanced multithreading libraries may offer more features and/or higher performance. However, they typically consist of a vast codebase with multiple components and dependencies, and involve complex APIs that require a substantial time investment to learn. This library is not intended to replace these more advanced libraries; instead, it was designed for users who don't require very advanced features, and prefer a simple and lightweight package that is easy to learn and use and can be readily incorporated into existing or new projects. + +### Overview of features + +* **Fast:** + * Built from scratch with maximum performance in mind. + * Suitable for use in high-performance computing nodes with a very large number of CPU cores. + * Compact code, to reduce both compilation time and binary size. + * Reusing threads avoids the overhead of creating and destroying them for individual tasks. + * A task queue ensures that there are never more threads running in parallel than allowed by the hardware. +* **Lightweight:** + * Single header file: simply `#include "BS_thread_pool.hpp"` and you're all set! + * Header-only: no need to install or build the library. + * Self-contained: no external requirements or dependencies. + * Portable: uses only the C++ standard library, and works with any C++17-compliant compiler. + * Only ~340 lines of code, excluding comments and blank lines. + * A stand-alone "light version" of the thread pool is also available in the `BS_thread_pool_light.hpp` header file, with only ~170 lines of code. +* **Easy to use:** + * Very simple operation, using a handful of member functions. + * Every task submitted to the queue using the `submit()` member function automatically generates an `std::future`, which can be used to wait for the task to finish executing and/or obtain its eventual return value. + * Optionally, tasks may also be submitted using the `push_task()` member function without generating a future, sacrificing convenience for even greater performance. + * The code is thoroughly documented using Doxygen comments - not only the interface, but also the implementation, in case the user would like to make modifications. + * The included test program `BS_thread_pool_test.cpp` can be used to perform exhaustive automated tests and benchmarks, and also serves as a comprehensive example of how to properly use the package. +* **Helper classes:** + * Automatically parallelize a loop into any number of parallel tasks using the `parallelize_loop()` member function, and track its execution using the `BS::multi_future` helper class. + * Synchronize output to a stream from multiple threads in parallel using the `BS::synced_stream` helper class. + * Easily measure execution time for benchmarking purposes using the `BS::timer` helper class. +* **Additional features:** + * Easily wait for all tasks in the queue to complete using the `wait_for_tasks()` member function. + * Change the number of threads in the pool safely and on-the-fly as needed using the `reset()` member function. + * Monitor the number of queued and/or running tasks using the `get_tasks_queued()`, `get_tasks_running()`, and `get_tasks_total()` member functions. + * Freely pause and resume the pool using the `pause()`, `unpause()`, and `is_paused()` member functions. When paused, threads do not retrieve new tasks out of the queue. + * Catch exceptions thrown by the submitted tasks. + * Submit class member functions to the pool, either applied to a specific object or from within the object itself. + * Under continuous and active development. Bug reports and feature requests are welcome, and should be made via [GitHub issues](https://github.com/bshoshany/thread-pool/issues). + +### Compiling and compatibility + +This library should successfully compile on any C++17 standard-compliant compiler, on all operating systems and architectures for which such a compiler is available. Compatibility was verified with a 12-core / 24-thread AMD Ryzen 9 3900X CPU using the following compilers and platforms: + +* Windows 11 build 22000.795: + * [Clang](https://clang.llvm.org/) v14.0.6 + * [GCC](https://gcc.gnu.org/) v12.1.0 ([WinLibs build](https://winlibs.com/)) + * [MSVC](https://docs.microsoft.com/en-us/cpp/) v19.32.31332 +* Ubuntu 22.04 LTS: + * [Clang](https://clang.llvm.org/) v14.0.0 + * [GCC](https://gcc.gnu.org/) v12.0.1 + +In addition, this library was tested on a [Compute Canada](https://www.computecanada.ca/) node equipped with two 20-core / 40-thread Intel Xeon Gold 6148 CPUs (for a total of 40 cores and 80 threads), running CentOS Linux 7.9.2009, using [GCC](https://gcc.gnu.org/) v12.1.1. + +The test program `BS_thread_pool_test.cpp` was compiled without warnings (with the warning flags `-Wall -Wextra -Wconversion -Wsign-conversion -Wpedantic -Weffc++ -Wshadow` in GCC/Clang and `/W4` in MSVC), executed, and successfully completed all [automated tests](#testing-the-package) and benchmarks using all of the compilers and systems mentioned above. + +As this library requires C++17 features, the code must be compiled with C++17 support: + +* For Clang or GCC, use the `-std=c++17` flag. On Linux, you will also need to use the `-pthread` flag to enable the POSIX threads library. +* For MSVC, use `/std:c++17`, and preferably also `/permissive-` to ensure standards conformance. + +For maximum performance, it is recommended to compile with all available compiler optimizations: + +* For Clang or GCC, use the `-O3` flag. +* For MSVC, use `/O2`. + +As an example, to compile the test program `BS_thread_pool_test.cpp` with warnings and optimizations, it is recommended to use the following commands: + +* On Linux with GCC: `g++ BS_thread_pool_test.cpp -std=c++17 -O3 -Wall -Wextra -Wconversion -Wsign-conversion -Wpedantic -Weffc++ -Wshadow -pthread -o BS_thread_pool_test` +* On Linux with Clang: replace `g++` with `clang++`. +* On Windows with GCC or Clang: replace `-o BS_thread_pool_test` with `-o BS_thread_pool_test.exe` and remove `-pthread`. +* On Windows with MSVC: `cl BS_thread_pool_test.cpp /std:c++17 /permissive- /O2 /W4 /EHsc /Fe:BS_thread_pool_test.exe` + +### Installing using vcpkg + +If you are using the [vcpkg](https://github.com/microsoft/vcpkg) C/C++ library manager, you can easily download and install this package with the following commands. + +On Linux/macOS: + +```none +./vcpkg install bshoshany-thread-pool +``` + +On Windows: + +```none +.\vcpkg install bshoshany-thread-pool:x86-windows bshoshany-thread-pool:x64-windows +``` + +The thread pool will then be available automatically in the build system you integrated vcpkg with (e.g. MSBuild or CMake). Simply write `#include "BS_thread_pool.hpp"` in any project to use the thread pool, without having to copy to file into the project first. I will update the vcpkg port with each new release, so it will be updated automatically when you run `vcpkg upgrade`. + +Please see the [vcpkg repository](https://github.com/microsoft/vcpkg) for more information on how to use vcpkg. + +### Installing using Conan + +If you are using the [Conan](https://conan.io/) C/C++ package manager, please refer to [this package's page on ConanCenter](https://conan.io/center/bshoshany-thread-pool) to learn how to use Conan to include this package in your project with various build systems. + +## Getting started + +### Including the library + +If you are not using a C++ library manager (such as vcpkg), simply download the [latest release](https://github.com/bshoshany/thread-pool/releases) from the GitHub repository, place the single header file `BS_thread_pool.hpp` in the desired folder, and include it in your program: + +```cpp +#include "BS_thread_pool.hpp" +``` + +The thread pool will now be accessible via the `BS::thread_pool` class. + +### Constructors + +The default constructor creates a thread pool with as many threads as the hardware can handle concurrently, as reported by the implementation via `std::thread::hardware_concurrency()`. This is usually determined by the number of cores in the CPU. If a core is hyperthreaded, it will count as two threads. For example: + +```cpp +// Constructs a thread pool with as many threads as available in the hardware. +BS::thread_pool pool; +``` + +Optionally, a number of threads different from the hardware concurrency can be specified as an argument to the constructor. However, note that adding more threads than the hardware can handle will **not** improve performance, and in fact will most likely hinder it. This option exists in order to allow using **less** threads than the hardware concurrency, in cases where you wish to leave some threads available for other processes. For example: + +```cpp +// Constructs a thread pool with only 12 threads. +BS::thread_pool pool(12); +``` + +If your program's main thread only submits tasks to the thread pool and waits for them to finish, and does not perform any computationally intensive tasks on its own, then it is recommended to use the default value for the number of threads. This ensures that all of the threads available in the hardware will be put to work while the main thread waits. + +However, if your main thread does perform computationally intensive tasks on its own, then it is recommended to use the value `std::thread::hardware_concurrency() - 1` for the number of threads. In this case, the main thread plus the thread pool will together take up exactly all the threads available in the hardware. + +### Getting and resetting the number of threads in the pool + +The member function `get_thread_count()` returns the number of threads in the pool. This will be equal to `std::thread::hardware_concurrency()` if the default constructor was used. + +It is generally unnecessary to change the number of threads in the pool after it has been created, since the whole point of a thread pool is that you only create the threads once. However, if needed, this can be done, safely and on-the-fly, using the `reset()` member function. + +`reset()` will wait for all currently running tasks to be completed, but will leave the rest of the tasks in the queue. Then it will destroy the thread pool and create a new one with the desired new number of threads, as specified in the function's argument (or the hardware concurrency if no argument is given). The new thread pool will then resume executing the tasks that remained in the queue and any new submitted tasks. + +### Finding the version of the package + +If desired, the version of this package may be read during compilation time from the macro `BS_THREAD_POOL_VERSION`. The value will be a string containing the version number and release date. For example: + +```cpp +std::cout << "Thread pool library version is " << BS_THREAD_POOL_VERSION << ".\n"; +``` + +Sample output: + +```none +Thread pool library version is v3.1.0 (2022-07-13). +``` + +This can be used, for example, to allow the same code to work with several incompatible versions of the library. + +## Submitting and waiting for tasks + +### Submitting tasks to the queue with futures + +A task can be any function, with zero or more arguments, and with or without a return value. Once a task has been submitted to the queue, it will be executed as soon as a thread becomes available. Tasks are executed in the order that they were submitted (first-in, first-out). + +The member function `submit()` is used to submit tasks to the queue. The first argument is the function to execute, and the rest of the arguments are the arguments to pass to the function, if any. The return value is an `std::future` associated to the task. For example: + +```cpp +// Submit a task without arguments to the queue, and get a future for it. +auto my_future = pool.submit(task); +// Submit a task with one argument to the queue, and get a future for it. +auto my_future = pool.submit(task, arg); +// Submit a task with two arguments to the queue, and get a future for it. +auto my_future = pool.submit(task, arg1, arg2); +``` + +If the submitted function has a return value of type `T`, then the future will be of type `std::future<T>`, and will be set to the return value when the function finishes its execution. If the submitted function does not have a return value, then the future will be an `std::future<void>`, which will not return any value but may still be used to wait for the function to finish. + +Using `auto` for the return value of `submit()` means the compiler will automatically detect which instance of the template `std::future` to use. However, specifying the particular type `std::future<T>`, as in the examples below, is recommended for increased readability. + +To wait until the task finishes, use the member function `wait()` of the future. To obtain the return value, use the member function `get()`, which will also automatically wait for the task to finish if it hasn't yet. For example: + +```cpp +// Submit a task and get a future. +auto my_future = pool.submit(task); +// Do some other stuff while the task is executing. +do_stuff(); +// Get the task's return value from the future, waiting for it to finish running if needed. +auto my_return_value = my_future.get(); +``` + +Here are some more concrete examples. The following program will print out `42`: + +```cpp +#include "BS_thread_pool.hpp" + +int main() +{ + BS::thread_pool pool; + std::future<int> my_future = pool.submit([] { return 42; }); + std::cout << my_future.get(); +} +``` + +Here we used a [lambda expression](https://en.cppreference.com/w/cpp/language/lambda) to quickly define the function on-the-fly. However, we can also use a previously-defined function: + +```cpp +#include "BS_thread_pool.hpp" + +int the_answer() +{ + return 42; +} + +int main() +{ + BS::thread_pool pool; + std::future<int> my_future = pool.submit(the_answer); + std::cout << my_future.get(); +} +``` + +The following is an example of submitting a function with arguments: + +```cpp +#include "BS_thread_pool.hpp" + +int multiply(const int a, const int b) +{ + return a * b; +} + +int main() +{ + BS::thread_pool pool; + std::future<int> my_future = pool.submit(multiply, 6, 7); + std::cout << my_future.get(); +} +``` + +Finally, here is an example of submitting a function with no return value and then using the future to wait for it to finish executing: + +```cpp +#include "BS_thread_pool.hpp" + +void sleep() +{ + std::this_thread::sleep_for(std::chrono::milliseconds(1000)); +} + +int main() +{ + BS::thread_pool pool; + std::future<void> my_future = pool.submit(sleep); + std::cout << "Waiting... "; + my_future.wait(); + std::cout << "Done."; +} +``` + +Here, the command `std::this_thread::sleep_for(std::chrono::milliseconds(1000))` instructs the thread to sleep for 1 second. + +### Submitting tasks to the queue without futures + +Usually, it is best to submit a task to the queue using `submit()`. This allows you to wait for the task to finish and/or get its return value later. However, sometimes a future is not needed, for example when you just want to "set and forget" a certain task, or if the task already communicates with the main thread or with other tasks without using futures, such as via condition variables. In such cases, you may wish to avoid the overhead involved in assigning a future to the task in order to increase performance. + +The member function `push_task()` allows you to submit a task to the queue without generating a future for it. The task can have any number of arguments, but it cannot have a return value. For example: + +```cpp +// Submit a task without arguments or return value to the queue. +pool.push_task(task); +// Submit a task with one argument and no return value to the queue. +pool.push_task(task, arg); +// Submit a task with two arguments and no return value to the queue. +pool.push_task(task, arg1, arg2); +``` + +**Warning!** Since `push_task()` does not return a future, there is no built-in way for the user to know when the task finishes executing. You must use either `wait_for_tasks()` (see below), or some other method such as condition variables, to ensure that the task finishes executing before trying to use anything that depends on its output. Otherwise, bad things will happen! + +### Manually waiting for all tasks to complete + +To wait for a **single** submitted task to complete, use `submit()` and then use the `wait()` or `get()` member functions of the obtained future. However, in cases where you need to wait until **all** submitted tasks finish their execution, or if the tasks have been submitted without futures using `push_task()`, you can use the member function `wait_for_tasks()`. + +Consider, for example, the following code: + +```cpp +#include "BS_thread_pool.hpp" + +int main() +{ + BS::thread_pool pool(5); + int squares[100]; + for (int i = 0; i < 100; ++i) + pool.push_task( + [&squares, i] + { + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + squares[i] = i * i; + }); + std::cout << squares[50]; +} +``` + +The output will most likely be garbage, since the task that modifies `squares[50]` has not yet finished executing by the time we try to access that element - it's still waiting in the queue. One solution would be to use `submit()` instead of `push_task()`, but perhaps we don't want the overhead of generating 100 different futures. Instead, simply adding the line + +```cpp +pool.wait_for_tasks(); +``` + +after the `for` loop will ensure - as efficiently as possible - that all tasks have finished running before we attempt to access any elements of the array `squares`, and the code will print out the value `2500` as expected. + +Note, however, that `wait_for_tasks()` will wait for **all** the tasks in the queue, including those that are unrelated to the `for` loop. Using [`parallelize_loop()`](#parallelizing-loops) would make much more sense in this particular case, as it will allow waiting only for the tasks related to the loop. + +### Submitting class member functions to the queue + +Consider the following program: + +```cpp +#include "BS_thread_pool.hpp" + +BS::thread_pool pool; + +class flag_class +{ +public: + bool get_flag() const + { + return flag; + } + + void set_flag(const bool arg) + { + flag = arg; + } + +private: + bool flag = false; +}; + +int main() +{ + flag_class flag_object; + flag_object.set_flag(true); + std::cout << std::boolalpha << flag_object.get_flag() << '\n'; +} +``` + +This program creates a new object `flag_object` of the class `flag_class`, sets the flag to `true` using the member function `set_flag()`, and then prints out the flag's value. But what if you want to submit the member function `set_flag()` as a task to the thread pool? + +To submit member functions to the pool, use the following general syntax: + +```cpp +pool.push_task(&class::function, &object, args); +``` + +The same syntax also works with `submit()`. Note that, in the second argument, you must specify the object on which the member function will be executed (unless it's a static member function, in which case you just submit it like any other function). Also note that both the first and second arguments must be **pointers**, so they must be preceded by the `&` operator. + +If you remove the `&` from the first argument, the code won't work with most compilers, and if you remove the `&` from the second argument, the function will act on a **copy** of the object, rather than on the object itself, so any changes made to the object will not be saved. Therefore, it's important to ensure that both arguments are pointers. + +To make the above program submit the member function `set_flag()` to the thread pool, simply replace the line: + +```cpp +flag_object.set_flag(true); +``` + +with: + +```cpp +pool.push_task(&flag_class::set_flag, &flag_object, true); +pool.wait_for_tasks(); +``` + +Here the class is `flag_class`, the name of the function is `set_flag`, the object we want the function to act on is `flag_object`, and the argument to pass to the function is `true`. + +Another thing you might want to do is call a member function from within the object itself, that is, from another member function. This follows a similar syntax, except that you don't need to specify the class, and you use `this` to get a pointer to the current object (no `&` necessary, since `this` is already a pointer). Here is an example, this time using `submit()`: + +```cpp +#include "BS_thread_pool.hpp" + +BS::thread_pool pool; + +class flag_class +{ +public: + bool get_flag() const + { + return flag; + } + + void set_flag(const bool arg) + { + flag = arg; + } + + void set_flag_to_true() + { + pool.submit(&flag_class::set_flag, this, true).wait(); + } + +private: + bool flag = false; +}; + +int main() +{ + flag_class flag_object; + flag_object.set_flag_to_true(); + std::cout << std::boolalpha << flag_object.get_flag() << '\n'; +} +``` + +### Parallelizing loops + +One of the most common and effective methods of parallelization is splitting a loop into smaller loops and running them in parallel. It is most effective in "embarrassingly parallel" computations, such as vector or matrix operations, where each iteration of the loop is completely independent of every other iteration. For example, if we are summing up two vectors of 1000 elements each, and we have 10 threads, we could split the summation into 10 blocks of 100 elements each, and run all the blocks in parallel, potentially increasing performance by up to a factor of 10. + +`BS::thread_pool` can automatically parallelize loops. To see how this works, consider the following generic loop: + +```cpp +for (T i = start; i < end; ++i) + do_something(i); +``` + +where: + +* `T` is any signed or unsigned integer type. +* The loop is over the range `[start, end)`, i.e. inclusive of `start` but exclusive of `end`. +* `do_something()` is an operation performed for each loop index `i`, such as modifying an array with `end - start` elements. + +This loop may be automatically parallelized and submitted to the thread pool's queue using the member function `parallelize_loop()`, which has the follows syntax: + +```cpp +pool.parallelize_loop(start, end, loop, num_blocks); +``` + +where: + +* `start` is the first index in the range. + * This argument can be omitted, in which case it is assumed that the loop starts at 0. That is, `parallelize_loop(end, loop, num_blocks)` is equivalent to `parallelize_loop(0, end, loop, num_blocks)`. +* `end` is the index after the last index in the range, such that the full range is `[start, end)`. In other words, the loop will be equivalent to the one above if `start` and `end` are the same. + * `start` and `end` should both be integers, but they need not be of the same integer type. `parallelize_loop()` will automatically determine the best type to use for the loop indices. +* `loop()` is any function that takes two indices, `a`, and `b`, and executes only the portion of the loop in the range `[a, b)`. Typically, `loop()` will include a `for` loop of the form `for (T i = a; i < b; ++i)`. +* `num_blocks` is the number of blocks of the form `[a, b)` to split the loop into. For example, if the range is `[0, 9)` and there are 3 blocks, then the blocks will be the ranges `[0, 3)`, `[3, 6)`, and `[6, 9)`. If possible, the blocks will be equal in size; otherwise, the last block may be a bit longer. + * This argument can be omitted, in which case the number of blocks will be the number of threads in the pool. + +Each block will be submitted to the thread pool's queue as a separate task. Therefore, a loop that is split into 3 blocks will be split into 3 individual tasks, which may run in parallel. If there is only one block, then the entire loop will run as one task, and no parallelization will take place. + +To parallelize the generic loop above, we use the following code: + +```cpp +auto loop = [](const T a, const T b) +{ + for (T i = a; i < b; ++i) + do_something(i); +}; +BS::multi_future<void> loop_future = pool.parallelize_loop(start, end, loop, num_blocks); +loop_future.wait(); +``` + +Here we defined `loop()` as a lambda function. Of course, `loop()` could also be defined as a lambda within the call to `parallelize_loop()` itself, as in the examples below; or it could be any ordinary function, but a lambda is preferred since one typically would like to capture some of the surrounding variables. + +`parallelize_loop()` returns an object of the helper class template `BS::multi_future`. Each of the `num_blocks` blocks will have an `std::future` assigned to it, and all these futures will be stored inside the returned `BS::multi_future` object. When `loop_future.wait()` is called, the main thread will wait until all tasks generated by `parallelize_loop()` finish executing, and only those tasks - not any other tasks that also happen to be in the queue. This is essentially the role of the `BS::multi_future` class: to wait for a specific **group of tasks**, in this case the tasks running the loop blocks. + +What value should you use for `num_blocks`? Omitting this argument, so that the number of blocks will be equal to the number of threads in the pool, is typically a good choice. For best performance, it is recommended to do your own benchmarks to find the optimal number of blocks for each loop (you can use the `BS::timer` helper class - see [below](#measuring-execution-time)). Using less tasks than there are threads may be preferred if you are also running other tasks in parallel. Using more tasks than there are threads may improve performance in some cases. + +As a simple example, the following code calculates and prints the squares of all integers from 0 to 99: + +```cpp +#include "BS_thread_pool.hpp" + +int main() +{ + int squares[100]; + for (int i = 0; i < 100; ++i) + { + squares[i] = i * i; + std::cout << i << "^2 = " << squares[i] << " "; + } +} +``` + +We can parallelize it as follows: + +```cpp +#include "BS_thread_pool.hpp" + +int main() +{ + BS::thread_pool pool(10); + int squares[100]; + pool.parallelize_loop(100, + [&squares](const int a, const int b) + { + for (int i = a; i < b; ++i) + squares[i] = i * i; + }) + .wait(); + for (int i = 0; i < 100; ++i) + std::cout << i << "^2 = " << squares[i] << " "; +} +``` + +Since there are 10 threads, and we omitted the `num_blocks` argument, the loop will be divided into 10 blocks, each calculating 10 squares. Also, since the loop starts from 0, we did not need to specify the first index. + +In this example, instead of storing the `BS::multi_future` object and then using it to wait, we simply called the `wait()` member function directly on the temporary object returned by `parallelize_loop()` without storing it anywhere. This is a convenient shortcut when we have nothing else to do while waiting. + +Notice that here we parallelized the calculation of the squares, but we did not parallelize printing the results. This is for two reasons: + +1. We want to print out the squares in ascending order, and we have no guarantee that the blocks will be executed in the correct order. This is very important; you must never expect that the parallelized loop will execute at the same order as the non-parallelized loop. +2. If we did print out the squares from within the parallel tasks, we would get a huge mess, since all 10 blocks would print to the standard output at once. [Later](#synchronizing-printing-to-an-output-stream) we will see how to synchronize printing to a stream from multiple tasks at the same time. + +### Loops with return values + +Usually, `parallelize_loop()` should take functions with no return values. This is because the function will be executed once for each block, but the blocks are managed by the thread pool, so there's limited usability in returning one value per block. However, for the case where this is desired, such as for summation or some sorting algorithms, `parallelize_loop()` does accept functions with return values, in which case it returns a `BS::multi_future<T>` object where `T` is the type of the return values. + +Here's an example of summing all the numbers from 1 to 100: + +```cpp +#include "BS_thread_pool.hpp" + +int main() +{ + BS::thread_pool pool; + BS::multi_future<int> mf = pool.parallelize_loop(1, 101, + [](const int a, const int b) + { + int block_total = 0; + for (int i = a; i < b; ++i) + block_total += i; + return block_total; + }); + std::vector<int> totals = mf.get(); + int sum = 0; + for (const int t : totals) + sum += t; + std::cout << sum; +} +``` + +Calling `get()` on a `BS::multi_future<T>` object returns an `std::vector<T>` with the values obtained from each future. In this case, the values will be the partial sums from each block, so when we add them up, we will get the total sum. + +### Parallelizing loops without futures + +Just as in the case of [`push_task()`](#submitting-tasks-to-the-queue-without-futures) vs. [`submit()`](#submitting-tasks-to-the-queue-with-futures), sometimes you may want to parallelize a loop, but you don't need it to return a `BS::multi_future`. In this case, you can save the overhead of generating the futures (which can be significant, depending on the number of blocks) by using `push_loop()` instead of `parallelize_loop()`, with the same arguments. + +For example, you could also run the loop of squares example above as follows: + +```cpp +#include "BS_thread_pool.hpp" + +int main() +{ + BS::thread_pool pool(10); + int squares[100]; + pool.push_loop(100, + [&squares](const int a, const int b) + { + for (int i = a; i < b; ++i) + squares[i] = i * i; + }); + pool.wait_for_tasks(); + for (int i = 0; i < 100; ++i) + std::cout << i << "^2 = " << squares[i] << " "; +} +``` + +As with `parallelize_loop()`, the first argument can be omitted if the start index is 0, and the last argument can be omitted if the number of blocks should be equal to the number of threads. + +**Warning!** Since `push_loop()` does not return a `BS::multi_future`, there is no built-in way for the user to know when the loop finishes executing. You must use either [`wait_for_tasks()`](#manually-waiting-for-all-tasks-to-complete), or some other method such as condition variables, to ensure that the loop finishes executing before trying to use anything that depends on its output. Otherwise, bad things will happen! + +## Helper classes + +### Synchronizing printing to an output stream + +When printing to an output stream from multiple threads in parallel, the output may become garbled. For example, consider this code: + +```cpp +#include "BS_thread_pool.hpp" + +int main() +{ + BS::thread_pool pool; + for (size_t i = 1; i <= 5; ++i) + pool.push_task([i] { std::cout << "Task no. " << i << " executing.\n"; }); +} +``` + +The output may look as follows: + +```none +Task no. Task no. 2Task no. 5 executing. +Task no. executing. +Task no. 1 executing. +4 executing. +3 executing. +``` + +The reason is that, although each **individual** insertion to `std::cout` is thread-safe, there is no mechanism in place to ensure subsequent insertions from the same thread are printed contiguously. + +The helper class `BS::synced_stream` is designed to eliminate such synchronization issues. The constructor takes one optional argument, specifying the output stream to print to. If no argument is supplied, `std::cout` will be used: + +```cpp +// Construct a synced stream that will print to std::cout. +BS::synced_stream sync_out; +// Construct a synced stream that will print to the output stream my_stream. +BS::synced_stream sync_out(my_stream); +``` + +The member function `print()` takes an arbitrary number of arguments, which are inserted into the stream one by one, in the order they were given. `println()` does the same, but also prints a newline character `\n` at the end, for convenience. A mutex is used to synchronize this process, so that any other calls to `print()` or `println()` using the same `BS::synced_stream` object must wait until the previous call has finished. + +As an example, this code: + +```cpp +#include "BS_thread_pool.hpp" + +int main() +{ + BS::synced_stream sync_out; + BS::thread_pool pool; + for (size_t i = 1; i <= 5; ++i) + pool.push_task([i, &sync_out] { sync_out.println("Task no. ", i, " executing."); }); +} +``` + +Will print out: + +```none +Task no. 1 executing. +Task no. 2 executing. +Task no. 3 executing. +Task no. 4 executing. +Task no. 5 executing. +``` + +**Warning:** Always create the `BS::synced_stream` object **before** the `BS::thread_pool` object, as we did in this example. When the `BS::thread_pool` object goes out of scope, it waits for the remaining tasks to be executed. If the `BS::synced_stream` object goes out of scope before the `BS::thread_pool` object, then any tasks using the `BS::synced_stream` will crash. Since objects are destructed in the opposite order of construction, creating the `BS::synced_stream` object before the `BS::thread_pool` object ensures that the `BS::synced_stream` is always available to the tasks, even while the pool is destructing. + +Most stream manipulators defined in the headers `<ios>` and `<iomanip>`, such as `std::setw` (set the character width of the next output), `std::setprecision` (set the precision of floating point numbers), and `std::fixed` (display floating point numbers with a fixed number of digits), can be passed to `print()` and `println()` just as you would pass them to a stream. + +The only exceptions are the flushing manipulators `std::endl` and `std::flush`, which will not work because the compiler will not be able to figure out which template specializations to use. Instead, use `BS::synced_stream::endl` and `BS::synced_stream::flush`. Here is an example: + +```cpp +#include "BS_thread_pool.hpp" +#include <cmath> +#include <iomanip> + +int main() +{ + BS::synced_stream sync_out; + BS::thread_pool pool; + sync_out.print(std::setprecision(10), std::fixed); + for (size_t i = 1; i <= 10; ++i) + pool.push_task([i, &sync_out] { sync_out.print("The square root of ", std::setw(2), i, " is ", std::sqrt(i), ".", BS::synced_stream::endl); }); +} +``` + +### Handling multiple futures at once + +The helper class template `BS::multi_future<T>`, already introduced in the context of `parallelize_loop()`, provides a convenient way to collect and access groups of futures. This class works similarly to STL containers such as `std::vector`: + +* When you create a new object, either use the default constructor to create an empty object and add futures to it later, or pass the desired number of futures to the constructor in advance. +* Use the `[]` operator to access the future at a specific index, or the `push_back()` member function to append a new future to the list. +* The `size()` member function tells you how many futures are currently stored in the object. +* Once all the futures are stored, you can use `wait()` to wait for all of them at once or `get()` to get an `std::vector<T>` with the results from all of them. + +Aside from using `BS::multi_future` to track the execution of parallelized loops, it can also be used whenever you have several different groups of tasks and you want to track the execution of each group individually. Here's a simple example: + +```cpp +#include "BS_thread_pool.hpp" +#include <cmath> + +BS::synced_stream sync_out; +BS::thread_pool pool; + +double power(const double i, const double j) +{ + std::this_thread::sleep_for(std::chrono::milliseconds(10 * pool.get_thread_count())); + return std::pow(i, j); +} + +void print_vector(const std::vector<double>& vec) +{ + for (const double i : vec) + sync_out.print(i, ' '); + sync_out.println(); +} + +int main() +{ + constexpr size_t n = 100; + + // First group of tasks: calculate n squares. + // Here we create an empty BS::multi_future object, and append futures to it via push_back(). + BS::multi_future<double> mf_squares; + for (int i = 0; i < n; ++i) + mf_squares.push_back(pool.submit(power, i, 2)); + + // Second group of tasks: calculate n cubes. + // In this case, we create a BS::multi_future object of the desired size in advance, and store the futures via the [] operator. This is faster since there will be no memory reallocations, but also more prone to errors. + BS::multi_future<double> mf_cubes(n); + for (int i = 0; i < n; ++i) + mf_cubes[i] = pool.submit(power, i, 3); + + // Both groups are now queued, but it will take some time until they all execute. + + /// ... + /// Do some stuff while the first group of tasks executes... + /// ... + + // Get and print the results from the first group. + sync_out.println("Squares:"); + print_vector(mf_squares.get()); + + /// ... + /// Do other stuff while the second group of tasks executes... + /// ... + + // Get and print the results from the second group. + sync_out.println("Cubes:"); + print_vector(mf_cubes.get()); +} +``` + +In this example, we simulate complicated tasks by having each task wait for a bit before returning its result. We collect the futures of the tasks submitted within each group into two separate `BS::multi_future<double>` objects. `mf_squares` holds the results from the first group, and `mf_cubes` holds the results from the second group. Now we can wait for and/or get the results from `mf_squares` whenever is convenient, and separately wait for and/or get the results from `mf_cubes` at another time. + +### Measuring execution time + +If you are using a thread pool, then your code is most likely performance-critical. Achieving maximum performance requires performing a considerable amount of benchmarking to determine the optimal settings and algorithms. Therefore, it is important to be able to measure the execution time of various computations and operations under different conditions. + +The helper class `BS::timer` provides a simple way to measure execution time. It is very straightforward to use: + +1. Create a new `BS::timer` object. +2. Immediately before you execute the computation that you want to time, call the `start()` member function. +3. Immediately after the computation ends, call the `stop()` member function. +4. Use the member function `ms()` to obtain the elapsed time for the computation in milliseconds. + +For example: + +```cpp +BS::timer tmr; +tmr.start(); +do_something(); +tmr.stop(); +std::cout << "The elapsed time was " << tmr.ms() << " ms.\n"; +``` + +A practical application of the `BS::timer` class can be found in the benchmark portion of the test program `BS_thread_pool_test.cpp`. + +## Other features + +### Monitoring the tasks + +Sometimes you may wish to monitor what is happening with the tasks you submitted to the pool. This may be done using three member functions: + +* `get_tasks_queued()` gets the number of tasks currently waiting in the queue to be executed by the threads. +* `get_tasks_running()` gets the number of tasks currently being executed by the threads. +* `get_tasks_total()` gets the total number of unfinished tasks: either still in the queue, or running in a thread. +* Note that `get_tasks_total() == get_tasks_queued() + get_tasks_running()`. + +These functions are demonstrated in the following program: + +```cpp +#include "BS_thread_pool.hpp" + +BS::synced_stream sync_out; +BS::thread_pool pool(4); + +void sleep_half_second(const size_t i) +{ + std::this_thread::sleep_for(std::chrono::milliseconds(500)); + sync_out.println("Task ", i, " done."); +} + +void monitor_tasks() +{ + sync_out.println(pool.get_tasks_total(), " tasks total, ", pool.get_tasks_running(), " tasks running, ", pool.get_tasks_queued(), " tasks queued."); +} + +int main() +{ + for (size_t i = 0; i < 12; ++i) + pool.push_task(sleep_half_second, i); + monitor_tasks(); + std::this_thread::sleep_for(std::chrono::milliseconds(750)); + monitor_tasks(); + std::this_thread::sleep_for(std::chrono::milliseconds(500)); + monitor_tasks(); + std::this_thread::sleep_for(std::chrono::milliseconds(500)); + monitor_tasks(); +} +``` + +Assuming you have at least 4 hardware threads (so that 4 tasks can run concurrently), the output should be similar to: + +```none +12 tasks total, 0 tasks running, 12 tasks queued. +Task 0 done. +Task 1 done. +Task 2 done. +Task 3 done. +8 tasks total, 4 tasks running, 4 tasks queued. +Task 4 done. +Task 5 done. +Task 6 done. +Task 7 done. +4 tasks total, 4 tasks running, 0 tasks queued. +Task 8 done. +Task 9 done. +Task 10 done. +Task 11 done. +0 tasks total, 0 tasks running, 0 tasks queued. +``` + +### Pausing the workers + +Sometimes you may wish to temporarily pause the execution of tasks, or perhaps you want to submit tasks to the queue in advance and only start executing them at a later time. You can do this using the member functions `pause()`, `unpause()`, and `is_paused()`. + +When you call `pause()`, the workers will temporarily stop retrieving new tasks out of the queue. However, any tasks already executed will keep running until they are done, since the thread pool has no control over the internal code of your tasks. If you need to pause a task in the middle of its execution, you must do that manually by programming your own pause mechanism into the task itself. To resume retrieving tasks, call `unpause()`. To check whether the pool is currently paused, call `is_paused()`. + +Here is an example: + +```cpp +#include "BS_thread_pool.hpp" + +BS::synced_stream sync_out; +BS::thread_pool pool(4); + +void sleep_half_second(const size_t i) +{ + std::this_thread::sleep_for(std::chrono::milliseconds(500)); + sync_out.println("Task ", i, " done."); +} + +void check_if_paused() +{ + if (pool.is_paused()) + sync_out.println("Pool paused."); + else + sync_out.println("Pool unpaused."); +} + +int main() +{ + for (size_t i = 0; i < 8; ++i) + pool.push_task(sleep_half_second, i); + sync_out.println("Submitted 8 tasks."); + std::this_thread::sleep_for(std::chrono::milliseconds(250)); + pool.pause(); + check_if_paused(); + std::this_thread::sleep_for(std::chrono::milliseconds(1000)); + sync_out.println("Still paused..."); + std::this_thread::sleep_for(std::chrono::milliseconds(1000)); + for (size_t i = 8; i < 12; ++i) + pool.push_task(sleep_half_second, i); + sync_out.println("Submitted 4 more tasks."); + sync_out.println("Still paused..."); + std::this_thread::sleep_for(std::chrono::milliseconds(1000)); + pool.unpause(); + check_if_paused(); +} +``` + +Assuming you have at least 4 hardware threads, the output should be similar to: + +```none +Submitted 8 tasks. +Pool paused. +Task 0 done. +Task 1 done. +Task 2 done. +Task 3 done. +Still paused... +Submitted 4 more tasks. +Still paused... +Pool unpaused. +Task 4 done. +Task 5 done. +Task 6 done. +Task 7 done. +Task 8 done. +Task 9 done. +Task 10 done. +Task 11 done. +``` + +Here is what happened. We initially submitted a total of 8 tasks to the queue. Since we waited for 250ms before pausing, the first 4 tasks have already started running, so they kept running until they finished. While the pool was paused, we submitted 4 more tasks to the queue, but they just waited at the end of the queue. When we unpaused, the remaining 4 initial tasks were executed, followed by the 4 new tasks. + +While the workers are paused, `wait_for_tasks()` will wait for the running tasks instead of all tasks (otherwise it would wait forever). This is demonstrated by the following program: + +```cpp +#include "BS_thread_pool.hpp" + +BS::synced_stream sync_out; +BS::thread_pool pool(4); + +void sleep_half_second(const size_t i) +{ + std::this_thread::sleep_for(std::chrono::milliseconds(500)); + sync_out.println("Task ", i, " done."); +} + +void check_if_paused() +{ + if (pool.is_paused()) + sync_out.println("Pool paused."); + else + sync_out.println("Pool unpaused."); +} + +int main() +{ + for (size_t i = 0; i < 8; ++i) + pool.push_task(sleep_half_second, i); + sync_out.println("Submitted 8 tasks. Waiting for them to complete."); + pool.wait_for_tasks(); + for (size_t i = 8; i < 20; ++i) + pool.push_task(sleep_half_second, i); + sync_out.println("Submitted 12 more tasks."); + std::this_thread::sleep_for(std::chrono::milliseconds(250)); + pool.pause(); + check_if_paused(); + sync_out.println("Waiting for the ", pool.get_tasks_running(), " running tasks to complete."); + pool.wait_for_tasks(); + sync_out.println("All running tasks completed. ", pool.get_tasks_queued(), " tasks still queued."); + std::this_thread::sleep_for(std::chrono::milliseconds(1000)); + sync_out.println("Still paused..."); + std::this_thread::sleep_for(std::chrono::milliseconds(1000)); + sync_out.println("Still paused..."); + std::this_thread::sleep_for(std::chrono::milliseconds(1000)); + pool.unpause(); + check_if_paused(); + std::this_thread::sleep_for(std::chrono::milliseconds(250)); + sync_out.println("Waiting for the remaining ", pool.get_tasks_total(), " tasks (", pool.get_tasks_running(), " running and ", pool.get_tasks_queued(), " queued) to complete."); + pool.wait_for_tasks(); + sync_out.println("All tasks completed."); +} +``` + +The output should be similar to: + +```none +Submitted 8 tasks. Waiting for them to complete. +Task 0 done. +Task 1 done. +Task 2 done. +Task 3 done. +Task 4 done. +Task 5 done. +Task 6 done. +Task 7 done. +Submitted 12 more tasks. +Pool paused. +Waiting for the 4 running tasks to complete. +Task 8 done. +Task 9 done. +Task 10 done. +Task 11 done. +All running tasks completed. 8 tasks still queued. +Still paused... +Still paused... +Pool unpaused. +Waiting for the remaining 8 tasks (4 running and 4 queued) to complete. +Task 12 done. +Task 13 done. +Task 14 done. +Task 15 done. +Task 16 done. +Task 17 done. +Task 18 done. +Task 19 done. +All tasks completed. +``` + +The first `wait_for_tasks()`, which was called while the pool was not paused, waited for all 8 tasks, both running and queued. The second `wait_for_tasks()`, which was called after pausing the pool, only waited for the 4 running tasks, while the other 8 tasks remained queued, and were not executed since the pool was paused. Finally, the third `wait_for_tasks()`, which was called after unpausing the pool, waited for the remaining 8 tasks, both running and queued. + +**Warning**: If the thread pool is destroyed while paused, any tasks still in the queue will never be executed! + +### Exception handling + +`submit()` catches any exceptions thrown by the submitted task and forwards them to the corresponding future. They can then be caught when invoking the `get()` member function of the future. For example: + +```cpp +#include "BS_thread_pool.hpp" + +BS::synced_stream sync_out; +BS::thread_pool pool; + +double inverse(const double x) +{ + if (x == 0) + throw std::runtime_error("Division by zero!"); + else + return 1 / x; +} + +int main() +{ + constexpr double num = 0; + std::future<double> my_future = pool.submit(inverse, num); + try + { + const double result = my_future.get(); + sync_out.println("The inverse of ", num, " is ", result, "."); + } + catch (const std::exception& e) + { + sync_out.println("Caught exception: ", e.what()); + } +} +``` + +The output will be: + +```none +Caught exception: Division by zero! +``` + +However, if you change `num` to any non-zero number, no exceptions will be thrown and the inverse will be printed. + +It is important to note that `wait()` does not throw any exceptions; only `get()` does. Therefore, even if your task does not return anything, i.e. your future is an `std::future<void>`, you must still use `get()` on the future obtained from it if you want to catch exceptions thrown by it. Here is an example: + +```cpp +#include "BS_thread_pool.hpp" + +BS::synced_stream sync_out; +BS::thread_pool pool; + +void print_inverse(const double x) +{ + if (x == 0) + throw std::runtime_error("Division by zero!"); + else + sync_out.println("The inverse of ", x, " is ", 1 / x, "."); +} + +int main() +{ + constexpr double num = 0; + std::future<void> my_future = pool.submit(print_inverse, num); + try + { + my_future.get(); + } + catch (const std::exception& e) + { + sync_out.println("Caught exception: ", e.what()); + } +} +``` + +When using `BS::multi_future` to handle multiple futures at once, exception handling works the same way: if any of the futures may throw exceptions, you may catch these exceptions when calling `get()`, even in the case of `BS::multi_future<void>`. + +## Testing the package + +The included file `BS_thread_pool_test.cpp` will perform automated tests of all aspects of the package, and perform simple benchmarks. The output will be printed both to `std::cout` and to a file named `BS_thread_pool_test-yyyy-mm-dd_hh.mm.ss.log` based on the current date and time. In addition, the code is thoroughly documented, and is meant to serve as an extensive example of how to properly use the package. + +Please make sure to: + +1. [Compile](#compiling-and-compatibility) `BS_thread_pool_test.cpp` with optimization flags enabled (e.g. `-O3` on GCC / Clang or `/O2` on MSVC). +2. Run the test without any other applications, especially multithreaded applications, running in parallel. + +If any of the tests fail, please [submit a bug report](https://github.com/bshoshany/thread-pool/issues) including the exact specifications of your system (OS, CPU, compiler, etc.) and the generated log file. + +### Automated tests + +A sample output of a successful run of the automated tests is as follows: + +```none +BS::thread_pool: a fast, lightweight, and easy-to-use C++17 thread pool library +(c) 2022 Barak Shoshany (baraksh@gmail.com) (http://baraksh.com) +GitHub: https://github.com/bshoshany/thread-pool + +Thread pool library version is v3.3.0 (2022-08-03). +Hardware concurrency is 24. +Generating log file: BS_thread_pool_test-2022-08-03_12.32.04.log. + +Important: Please do not run any other applications, especially multithreaded applications, in parallel with this test! + +==================================== +Checking that the constructor works: +==================================== +Checking that the thread pool reports a number of threads equal to the hardware concurrency... +Expected: 24, obtained: 24 -> PASSED! +Checking that the manually counted number of unique thread IDs is equal to the reported number of threads... +Expected: 24, obtained: 24 -> PASSED! + +============================ +Checking that reset() works: +============================ +Checking that after reset() the thread pool reports a number of threads equal to half the hardware concurrency... +Expected: 12, obtained: 12 -> PASSED! +Checking that after reset() the manually counted number of unique thread IDs is equal to the reported number of threads... +Expected: 12, obtained: 12 -> PASSED! +Checking that after a second reset() the thread pool reports a number of threads equal to the hardware concurrency... +Expected: 24, obtained: 24 -> PASSED! +Checking that after a second reset() the manually counted number of unique thread IDs is equal to the reported number of threads... +Expected: 24, obtained: 24 -> PASSED! + +================================ +Checking that push_task() works: +================================ +Checking that push_task() works for a function with no arguments or return value... +-> PASSED! +Checking that push_task() works for a function with one argument and no return value... +-> PASSED! +Checking that push_task() works for a function with two arguments and no return value... +-> PASSED! + +============================= +Checking that submit() works: +============================= +Checking that submit() works for a function with no arguments or return value... +-> PASSED! +Checking that submit() works for a function with one argument and no return value... +-> PASSED! +Checking that submit() works for a function with two arguments and no return value... +-> PASSED! +Checking that submit() works for a function with no arguments and a return value... +-> PASSED! +Checking that submit() works for a function with one argument and a return value... +-> PASSED! +Checking that submit() works for a function with two arguments and a return value... +-> PASSED! + +================================================ +Checking that submitting member functions works: +================================================ +Checking that push_task() works for a member function with no arguments or return value... +-> PASSED! +Checking that push_task() works for a member function with one argument and no return value... +-> PASSED! +Checking that submit() works for a member function with no arguments or return value... +-> PASSED! +Checking that submit() works for a member function with one argument and no return value... +-> PASSED! +Checking that submit() works for a member function with no arguments and a return value... +-> PASSED! +Checking that submit() works for a member function with one argument and a return value... +-> PASSED! + +====================================================================== +Checking that submitting member functions from within an object works: +====================================================================== +Checking that push_task() works within an object for a member function with no arguments or return value... +-> PASSED! +Checking that push_task() works within an object for a member function with one argument and no return value... +-> PASSED! +Checking that submit() works within an object for a member function with no arguments or return value... +-> PASSED! +Checking that submit() works within an object for a member function with one argument and no return value... +-> PASSED! +Checking that submit() works within an object for a member function with no arguments and a return value... +-> PASSED! +Checking that submit() works within an object for a member function with one argument and a return value... +-> PASSED! + +======================================= +Checking that wait_for_tasks() works... +======================================= +Waiting for tasks... +-> PASSED! + +====================================================== +Checking that push_loop() and parallelize_loop() work: +====================================================== +Verifying that push_loop() from 917499 to 884861 with 19 tasks modifies all indices... +-> PASSED! +Verifying that push_loop() from 235488 to 296304 with 11 tasks modifies all indices... +-> PASSED! +Verifying that push_loop() from -790296 to -152228 with 21 tasks modifies all indices... +-> PASSED! +Verifying that push_loop() from -937055 to -135942 with 10 tasks modifies all indices... +-> PASSED! +Verifying that push_loop() from 372276 to 486867 with 3 tasks modifies all indices... +-> PASSED! +Verifying that push_loop() from 890415 to -163491 with 5 tasks modifies all indices... +-> PASSED! +Verifying that push_loop() from 637645 to -894687 with 7 tasks modifies all indices... +-> PASSED! +Verifying that push_loop() from 308032 to -254915 with 20 tasks modifies all indices... +-> PASSED! +Verifying that push_loop() from 499518 to 104936 with 17 tasks modifies all indices... +-> PASSED! +Verifying that push_loop() from 19080 to -378567 with 5 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from -298981 to -724834 with 11 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from 232695 to 767243 with 3 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from 177768 to 966097 with 10 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from 474617 to -155690 with 15 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from -733576 to 547977 with 9 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from -723922 to 992233 with 1 task modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from 957397 to 364478 with 5 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from 776948 to 895847 with 3 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from 696779 to 400637 with 17 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from -5265 to 746418 with 23 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from -229724 to -883103 with 9 tasks correctly sums all indices... +Expected: -727098445812, obtained: -727098445812 -> PASSED! +Verifying that parallelize_loop() from 730130 to -370499 with 3 tasks correctly sums all indices... +Expected: 395819207270, obtained: 395819207270 -> PASSED! +Verifying that parallelize_loop() from -493633 to 957239 with 14 tasks correctly sums all indices... +Expected: 672631513560, obtained: 672631513560 -> PASSED! +Verifying that parallelize_loop() from -455240 to -60850 with 14 tasks correctly sums all indices... +Expected: -203541129490, obtained: -203541129490 -> PASSED! +Verifying that parallelize_loop() from -287333 to 298991 with 9 tasks correctly sums all indices... +Expected: 6834778868, obtained: 6834778868 -> PASSED! +Verifying that parallelize_loop() from 62326 to -392718 with 18 tasks correctly sums all indices... +Expected: -150343352292, obtained: -150343352292 -> PASSED! +Verifying that parallelize_loop() from 663186 to 380865 with 23 tasks correctly sums all indices... +Expected: 294757240050, obtained: 294757240050 -> PASSED! +Verifying that parallelize_loop() from 609125 to -43020 with 1 task correctly sums all indices... +Expected: 369181893080, obtained: 369181893080 -> PASSED! +Verifying that parallelize_loop() from 465469 to 112037 with 4 tasks correctly sums all indices... +Expected: 204108747160, obtained: 204108747160 -> PASSED! +Verifying that parallelize_loop() from 690574 to 113023 with 15 tasks correctly sums all indices... +Expected: 464117673396, obtained: 464117673396 -> PASSED! +Verifying that parallelize_loop() with identical start and end indices does nothing... +-> PASSED! +Trying parallelize_loop() with start and end indices of different types: +Verifying that parallelize_loop() from 894645 to 908567 with 9 tasks modifies all indices... +-> PASSED! +Trying the overloads for push_loop() and parallelize_loop() for the case where the first index is equal to 0: +Verifying that push_loop() from 0 to 949967 with 10 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from 0 to 241018 with 3 tasks modifies all indices... +-> PASSED! +Verifying that parallelize_loop() from 0 to 574984 with 19 tasks correctly sums all indices... +Expected: 330606025272, obtained: 330606025272 -> PASSED! + +==================================== +Checking that task monitoring works: +==================================== +Resetting pool to 4 threads. +Submitting 12 tasks. +After submission, should have: 12 tasks total, 4 tasks running, 8 tasks queued... +Result: 12 tasks total, 4 tasks running, 8 tasks queued -> PASSED! +Task 2 released. +Task 0 released. +Task 1 released. +Task 3 released. +After releasing 4 tasks, should have: 8 tasks total, 4 tasks running, 4 tasks queued... +Result: 8 tasks total, 4 tasks running, 4 tasks queued -> PASSED! +Task 4 released. +Task 7 released. +Task 6 released. +Task 5 released. +After releasing 4 more tasks, should have: 4 tasks total, 4 tasks running, 0 tasks queued... +Result: 4 tasks total, 4 tasks running, 0 tasks queued -> PASSED! +Task 11 released. +Task 10 released. +Task 9 released. +Task 8 released. +After releasing the final 4 tasks, should have: 0 tasks total, 0 tasks running, 0 tasks queued... +Result: 0 tasks total, 0 tasks running, 0 tasks queued -> PASSED! +Resetting pool to 24 threads. + +============================ +Checking that pausing works: +============================ +Resetting pool to 4 threads. +Checking that the pool correctly reports that it is not paused. +-> PASSED! +Pausing pool. +Checking that the pool correctly reports that it is paused. +-> PASSED! +Submitting 12 tasks, each one waiting for 200ms. +Immediately after submission, should have: 12 tasks total, 0 tasks running, 12 tasks queued... +Result: 12 tasks total, 0 tasks running, 12 tasks queued -> PASSED! +300ms later, should still have: 12 tasks total, 0 tasks running, 12 tasks queued... +Result: 12 tasks total, 0 tasks running, 12 tasks queued -> PASSED! +Unpausing pool. +Checking that the pool correctly reports that it is not paused. +-> PASSED! +Task 1 done. +Task 2 done. +Task 0 done. +Task 3 done. +300ms later, should have: 8 tasks total, 4 tasks running, 4 tasks queued... +Result: 8 tasks total, 4 tasks running, 4 tasks queued -> PASSED! +Pausing pool and using wait_for_tasks() to wait for the running tasks. +Task 7 done. +Task 5 done. +Task 6 done. +Task 4 done. +After waiting, should have: 4 tasks total, 0 tasks running, 4 tasks queued... +Result: 4 tasks total, 0 tasks running, 4 tasks queued -> PASSED! +200ms later, should still have: 4 tasks total, 0 tasks running, 4 tasks queued... +Result: 4 tasks total, 0 tasks running, 4 tasks queued -> PASSED! +Unpausing pool and using wait_for_tasks() to wait for all tasks. +Task 8 done. +Task 9 done. +Task 10 done. +Task 11 done. +After waiting, should have: 0 tasks total, 0 tasks running, 0 tasks queued... +Result: 0 tasks total, 0 tasks running, 0 tasks queued -> PASSED! +Resetting pool to 24 threads. + +======================================= +Checking that exception handling works: +======================================= +Checking that exceptions are forwarded correctly by submit()... +Throwing exception... +-> PASSED! +Checking that exceptions are forwarded correctly by BS::multi_future... +Throwing exception... +Throwing exception... +-> PASSED! + +============================================================ +Testing that vector operations produce the expected results: +============================================================ +Adding two vectors with 77579 elements using 9 tasks... +-> PASSED! +Adding two vectors with 925926 elements using 2 tasks... +-> PASSED! +Adding two vectors with 367682 elements using 22 tasks... +-> PASSED! +Adding two vectors with 28482 elements using 2 tasks... +-> PASSED! +Adding two vectors with 486607 elements using 19 tasks... +-> PASSED! +Adding two vectors with 688249 elements using 10 tasks... +-> PASSED! +Adding two vectors with 473738 elements using 18 tasks... +-> PASSED! +Adding two vectors with 743021 elements using 12 tasks... +-> PASSED! +Adding two vectors with 209804 elements using 11 tasks... +-> PASSED! +Adding two vectors with 635671 elements using 23 tasks... +-> PASSED! + +++++++++++++++++++++++++++++++ +SUCCESS: Passed all 88 checks! +++++++++++++++++++++++++++++++ +``` + +### Performance tests + +If all checks passed, `BS_thread_pool_test.cpp` will perform simple benchmarks by filling a specific number of vectors of fixed size with random values. The program decides how many vectors to use by testing how many are needed to reach a target duration in the single-threaded test. This ensures that the test takes approximately the same amount of time on different systems, and is thus more consistent and portable. + +Once the required number of vectors has been determined, the program will test the performance of several multi-threaded tests, dividing the total number of vectors into different numbers of tasks, compare them to the performance of the single-threaded test, and indicate the maximum speedup obtained. + +Please note that these benchmarks are only intended to demonstrate that the package can provide a significant speedup, and it is highly recommended to perform your own benchmarks with your specific system, compiler, and code. + +Here we will present the results of the performance test running on a high-end desktop computer equipped with a 12-core / 24-thread AMD Ryzen 9 3900X CPU at 3.8 GHz and 32 GB of DDR4 RAM at 3600 MHz, compiled using [MSVC](https://docs.microsoft.com/en-us/cpp/) v19.32.31332 on Windows 11 build 22000.795 with the `/O2` compiler flag. The output was as follows: + +```none +====================== +Performing benchmarks: +====================== +Using 24 threads. +Each test will be repeated 20 times to collect reliable statistics. +Determining the number and size of vectors to generate in order to achieve an approximate mean execution time of 50 ms with 24 tasks... +Generating 3312 vectors with 4096 elements each: +Single-threaded, mean execution time was 542.2 ms with standard deviation 5.8 ms. +With 6 tasks, mean execution time was 95.2 ms with standard deviation 1.7 ms. +With 12 tasks, mean execution time was 49.6 ms with standard deviation 0.7 ms. +With 24 tasks, mean execution time was 29.0 ms with standard deviation 2.9 ms. +With 48 tasks, mean execution time was 33.2 ms with standard deviation 4.3 ms. +With 96 tasks, mean execution time was 35.5 ms with standard deviation 1.9 ms. +Maximum speedup obtained by multithreading vs. single-threading: 18.7x, using 24 tasks. + ++++++++++++++++++++++++++++++++++++++++ +Thread pool performance test completed! ++++++++++++++++++++++++++++++++++++++++ +``` + +This CPU has 12 physical cores, with each core providing two separate logical cores via hyperthreading, for a total of 24 threads. Without hyperthreading, we would expect a maximum theoretical speedup of 12x. With hyperthreading, one might naively expect to achieve up to a 24x speedup, but this is in fact impossible, as both logical cores share the same physical core's resources. However, generally we would expect [an estimated 30% additional speedup](https://software.intel.com/content/www/us/en/develop/articles/how-to-determine-the-effectiveness-of-hyper-threading-technology-with-an-application.html) from hyperthreading, which amounts to around 15.6x in this case. In our performance test, we see a speedup of 18.7x, saturating and even surpassing this estimated theoretical upper bound. + +## The light version of the package + +This package started out as a very lightweight thread pool, but over time has expanded to include many additional features and helper classes. Therefore, I have decided to bundle a light version of the thread pool in a separate and stand-alone header file, `BS_thread_pool_light.hpp`, which is about half the size of the full package. + +This file does not contain any of the helper classes, only a new `BS::thread_pool_light` class, which is a minimal thread pool with only the 5 most basic member functions: + +* `get_thread_count()` +* `push_loop()` +* `push_task()` +* `submit()` +* `wait_for_tasks()` + +A separate test program `BS_thread_pool_light_test.cpp` tests only the features of the lightweight `BS::thread_pool_light` class. In the spirit of minimalism, it does not generate a log file and does not do any benchmarks. + +To be perfectly clear, each header file is 100% stand-alone. If you wish to use the full package, you only need `BS_thread_pool.hpp`, and if you wish to use the light version, you only need `BS_thread_pool_light.hpp`. Only a single header file needs to be included in your project. + +## About the project + +### Issue and pull request policy + +This package is under continuous and active development. If you encounter any bugs, or if you would like to request any additional features, please feel free to [open a new issue on GitHub](https://github.com/bshoshany/thread-pool/issues) and I will look into it as soon as I can. + +Contributions are always welcome. However, I release my projects in cumulative updates after editing and testing them locally on my system, so my policy is not to accept any pull requests. If you open a pull request, and I decide to incorporate your suggestion into the project, I will first modify your code to comply with the project's coding conventions (formatting, syntax, naming, comments, programming practices, etc.), and perform some tests to ensure that the change doesn't break anything. I will then merge it into the next release of the project, possibly together with some other changes. The new release will also include a note in `CHANGELOG.md` with a link to your pull request, and modifications to the documentation in `README.md` as needed. + +### Acknowledgements + +Many GitHub users have helped improve this project, directly or indirectly, via issues, pull requests, comments, and/or personal correspondence. Please see `CHANGELOG.md` for links to specific issues and pull requests that have been the most helpful. Thank you all for your contribution! :) + +### Starring the repository + +If you found this project useful, please consider [starring it on GitHub](https://github.com/bshoshany/thread-pool/stargazers)! This allows me to see how many people are using my code, and motivates me to keep working to improve it. + +### Copyright and citing + +Copyright (c) 2022 [Barak Shoshany](http://baraksh.com). Licensed under the [MIT license](LICENSE.txt). + +If you use the library in software of any kind, please provide a link to [the GitHub repository](https://github.com/bshoshany/thread-pool) in the source code and documentation. + +If you use this library in published research, please cite it as follows: + +* Barak Shoshany, *"A C++17 Thread Pool for High-Performance Scientific Computing"*, [doi:10.5281/zenodo.4742687](https://doi.org/10.5281/zenodo.4742687), [arXiv:2105.00613](https://arxiv.org/abs/2105.00613) (May 2021) + +You can use the following BibTeX entry: + +```none +@article{Shoshany2021_ThreadPool, + archiveprefix = {arXiv}, + author = {Barak Shoshany}, + doi = {10.5281/zenodo.4742687}, + eid = {arXiv:2105.00613}, + eprint = {2105.00613}, + journal = {arXiv e-prints}, + keywords = {Computer Science - Distributed, Parallel, and Cluster Computing, D.1.3, D.1.5}, + month = {May}, + primaryclass = {cs.DC}, + title = {{A C++17 Thread Pool for High-Performance Scientific Computing}}, + year = {2021} +} +``` + +Please note that the [companion paper on arXiv](https://arxiv.org/abs/2105.00613) is updated infrequently. The paper is intended to facilitate discovery of the package by scientists who may find it useful for scientific computing purposes and to allow citing the package in scientific research, but most users should read the `README.md` file on [the GitHub repository](https://github.com/bshoshany/thread-pool) instead, as it is guaranteed to always be up to date. |
