a MainThread may be a "liquid" concept. A dedicated worker thread
may be considered a "MainThread" from the point of view of EntropySQLRepository.
_start_cleanup_monitor checks whether the calling thread is the MainThread
in order to determine if a cleanup thread should be started joining
the current thread. The problem with this approach is that if our
worker thread is a long lasting thread, we will end up with hundreds
of cleanup monitor threads waiting for the worker thread to terminate.
This will never happen though.
We can now reimplement isMainThread in order to tell the internal
function not to start the cleanup monitor thread also for arbitrary
threads (like the worker thread in RigoDaemon).
The old "dependencies" metadata is deprecated. It was found that
the generated metadata might get corrupted by colliding atom strings.
The new implementation avoids collisions completely and is more
efficient.
etpConst['clientdbid'] is kept for backward compatibility, but will be removed soon
While etpConst['serverdbid'] and etpConst['clientserverrepoid'] are gone.
The original idea was to avoid doing cursor and connection resources
cleanup (left by old and dead threads) synchronously every time
_connection() and/or _cursor() is accessed. This strategy also had
a huge drawback: with no activity on the object, resources were
left hanging there forever.
This commit introduces a better strategy for transparent and automatic
cleanup of resources belonging to terminated threads: every time a new
thread_id arrives at _cursor() or _connection(), a new daemon thread
starts and synchronizes with the caller through a simple Thread.join()
(because it's a daemon thread, we can join() daemon threads as well,
even if this is not really compliant with the specs, but it seems to work
just fine in Python).
When the caller thread is joined, it is possible to start the resources
cleanup procedure, carefully taking into account that thread_ids are
recycled and thus there might be clashing with newly created threads.
This helped a design issue to emerge from the sand (like a zombie
at the seaside): it is impossible to cleanup resources left by the
MainThread because this thread never ends living, and if it dies,
everything dies, obviously. So, the first implementation of this new
strategy was NOT touching the MainThread resources but then, the old
behaviour was to kill them as well on EntropyRepository.close().
So, the final version of this patch kept the old buggy behaviour of
touching MainThread stuff (nein, nein, nein, nein would Hitler say).
However, a new keyword argument "safe" has been added to the close()
method so it is possible to start migrating code to the dark side of the
power.
This means nothing really changed for API consumers yet, just entropy.db.sql
code being more efficient (no weird for loops and synchronous crap)
and actually faster (multi-threading ftw).
The licensename column is declared as UNIQUE, multiple threads inserting
rows can cause unique constraint violations. Considering the nature of
the data, using "INSERT OR REPLACE" can be considered safe and actually
wanted.
entropy.db.mysql and entropy.db.sqlite are now subclassing
EntropySQLRepository. Methods will be moved there during the
next forthcoming overhaul. Implement ModuleProxy support and
alleviate the exception class objects issue (sqlite3 based
exceptions are thrown by entropy.db.sqlite and oursql based
exceptions are thrown by entropydb.mysql, and there is no
easy/quick fix for this apparently, besides wrapping all the
cursor calls).
Ettore Di Giacinto
Geaaru
Sławomir Nizio
Fabio Erculiani