SourceXtractorPlusPlus
0.16
Please provide a description of the project.
|
#include <Prefetcher.h>
Classes | |
struct | EventType |
Public Member Functions | |
Prefetcher (const std::shared_ptr< Euclid::ThreadPool > &thread_pool, unsigned max_queue_size) | |
virtual | ~Prefetcher () |
void | handleMessage (const std::shared_ptr< SourceInterface > &message) override |
void | handleMessage (const ProcessSourcesEvent &message) override |
template<typename Container > | |
void | requestProperties (Container &&properties) |
void | wait () |
![]() | |
virtual | ~Observer ()=default |
![]() | |
virtual | ~Observable ()=default |
Destructor. More... | |
virtual void | addObserver (std::shared_ptr< Observer< std::shared_ptr< SourceInterface > >> observer) |
Adds an Observer that will be notified when notify Observers is called. More... | |
virtual void | removeObserver (std::shared_ptr< Observer< std::shared_ptr< SourceInterface > >> observer) |
Removes a previously added Observer from the list of Observers to notify. More... | |
![]() | |
virtual | ~Observer ()=default |
![]() | |
virtual | ~Observable ()=default |
Destructor. More... | |
virtual void | addObserver (std::shared_ptr< Observer< ProcessSourcesEvent >> observer) |
Adds an Observer that will be notified when notify Observers is called. More... | |
virtual void | removeObserver (std::shared_ptr< Observer< ProcessSourcesEvent >> observer) |
Removes a previously added Observer from the list of Observers to notify. More... | |
Private Member Functions | |
void | requestProperty (const PropertyId &property_id) |
void | outputLoop () |
Private Attributes | |
std::shared_ptr< Euclid::ThreadPool > | m_thread_pool |
Pointer to the pool of worker threads. More... | |
std::set< PropertyId > | m_prefetch_set |
Properties to prefetch. More... | |
std::unique_ptr< std::thread > | m_output_thread |
Orchestration thread. More... | |
std::condition_variable | m_new_output |
Notifies there is a new source done processing. More... | |
std::map< intptr_t, std::shared_ptr< SourceInterface > > | m_finished_sources |
Finished sources. More... | |
std::deque< ProcessSourcesEvent > | m_event_queue |
Queue of received ProcessSourceEvent, order preserved. More... | |
std::deque< EventType > | m_received |
Queue of type of received events. Used to pass downstream events respecting the received order. More... | |
std::mutex | m_queue_mutex |
std::atomic_bool | m_stop |
Termination condition for the output loop. More... | |
Euclid::Semaphore | m_semaphore |
Keep the queue under control. More... | |
Additional Inherited Members | |
![]() | |
void | notifyObservers (const std::shared_ptr< SourceInterface > &message) const |
![]() | |
void | notifyObservers (const ProcessSourcesEvent &message) const |
The pre-fetcher allows later stages, as the grouping or the cleaning, to ask in advance for some compute intensive properties, so they can be done multi-threaded before it reaches them.
The pre-fetcher must handle also ProcessSourcesEvent, as they are synchronization points. When one is received, only sources detected before the event will be passed along. Everyone else will have to wait until there are no more soures prior to the event being processed. Then, they will be released and sent along.
Definition at line 41 of file Prefetcher.h.
SourceXtractor::Prefetcher::Prefetcher | ( | const std::shared_ptr< Euclid::ThreadPool > & | thread_pool, |
unsigned | max_queue_size | ||
) |
Constructor
thread_pool | Alexandria thread pool |
Definition at line 44 of file Prefetcher.cpp.
References m_output_thread, and outputLoop().
|
virtual |
Destructor
Definition at line 49 of file Prefetcher.cpp.
References std::thread::joinable(), m_output_thread, and wait().
|
overridevirtual |
Handle ProcessSourcesEvent. All sources received prior to this message need to be processed before sources coming after are passed along.
message |
Implements SourceXtractor::Observer< ProcessSourcesEvent >.
Definition at line 132 of file Prefetcher.cpp.
References SourceXtractor::logger, m_event_queue, m_new_output, m_queue_mutex, m_received, std::condition_variable::notify_one(), and SourceXtractor::Prefetcher::EventType::PROCESS_SOURCE.
|
overridevirtual |
Trigger multi-threaded measurements on the source interface. Once they are done, the message will be passed along.
message |
Implements SourceXtractor::Observer< std::shared_ptr< SourceInterface > >.
Definition at line 54 of file Prefetcher.cpp.
References Euclid::Semaphore::acquire(), std::shared_ptr< T >::get(), std::lock(), m_finished_sources, m_new_output, m_prefetch_set, m_queue_mutex, m_received, m_semaphore, m_thread_pool, std::condition_variable::notify_one(), SourceXtractor::Prefetcher::EventType::SOURCE, and Euclid::ThreadPool::submit().
|
private |
Definition at line 81 of file Prefetcher.cpp.
References Euclid::ThreadPool::activeThreads(), SourceXtractor::logger, m_event_queue, m_finished_sources, m_new_output, m_queue_mutex, m_received, m_semaphore, m_stop, m_thread_pool, std::next(), SourceXtractor::Observable< ProcessSourcesEvent >::notifyObservers(), SourceXtractor::Observable< T >::notifyObservers(), SourceXtractor::Prefetcher::EventType::PROCESS_SOURCE, Euclid::Semaphore::release(), and std::condition_variable::wait_for().
Referenced by Prefetcher().
|
inline |
Tell the prefetcher to compute this property
Container | Any iterable container with a set/list of properties |
properties | PropertyId instances |
Definition at line 81 of file Prefetcher.h.
References requestProperty().
|
private |
Definition at line 76 of file Prefetcher.cpp.
References SourceXtractor::PropertyId::getString(), SourceXtractor::logger, and m_prefetch_set.
Referenced by requestProperties().
void SourceXtractor::Prefetcher::wait | ( | ) |
Wait for the multi-threaded computation to finish. This must be done as the segmentation may be completely finished, and the measurement queue empty, but some sources may still be here due to some compute-heavy property
Definition at line 142 of file Prefetcher.cpp.
References std::thread::join(), m_output_thread, and m_stop.
Referenced by ~Prefetcher().
|
private |
Queue of received ProcessSourceEvent, order preserved.
Definition at line 116 of file Prefetcher.h.
Referenced by handleMessage(), and outputLoop().
|
private |
Finished sources.
Definition at line 114 of file Prefetcher.h.
Referenced by handleMessage(), and outputLoop().
|
private |
Notifies there is a new source done processing.
Definition at line 112 of file Prefetcher.h.
Referenced by handleMessage(), and outputLoop().
|
private |
Orchestration thread.
Definition at line 110 of file Prefetcher.h.
Referenced by Prefetcher(), wait(), and ~Prefetcher().
|
private |
Properties to prefetch.
Definition at line 108 of file Prefetcher.h.
Referenced by handleMessage(), and requestProperty().
|
private |
Definition at line 120 of file Prefetcher.h.
Referenced by handleMessage(), and outputLoop().
|
private |
Queue of type of received events. Used to pass downstream events respecting the received order.
Definition at line 118 of file Prefetcher.h.
Referenced by handleMessage(), and outputLoop().
|
private |
Keep the queue under control.
Definition at line 126 of file Prefetcher.h.
Referenced by handleMessage(), and outputLoop().
|
private |
Termination condition for the output loop.
Definition at line 123 of file Prefetcher.h.
Referenced by outputLoop(), and wait().
|
private |
Pointer to the pool of worker threads.
Definition at line 106 of file Prefetcher.h.
Referenced by handleMessage(), and outputLoop().