This commit partially reverts commit d6b7a21314.
Package revision is no longer incremented across package moves.
This has two advantages:
- much less traffic generated on the mirror infrastructure
- less updates for sabayon-limbo users when packages are moved to main
Bumping the revision was required by sabayon-weekly, which had the problem
that some package files were replaced during normal activity on
sabayonlinux.org and sabayon-limbo on the mirror. This generated checksum
errors, thus adding the checksum in the package file name ensures that
Entropy Server will never overwrite package file names unless the checksum
also matches.
Having a SHA1 checksum in the file name is also good for security, and
we may even want to create a SHA1 from the GPG signature in future.
Entropy Server now supports repositories defined inside
/etc/entropy/repositories.conf.d/ files, written using the
syntax detailed below. This improves the ability to enable, disable,
add and remove repositories programmatically. Furthermore, it
makes possible to extend the supported parameters without breaking
backward compatibility.
In order to differentiate Entropy Client repository definitions between
Entropy Server ones, each repository section must start with "[server=".
This is an example of the syntax (with a complete listing
of the supported arguments):
[server=sabayon-limbo]
desc = Sabayon Linux Official Testing Repository
repo = ssh://username@full.host:~username/sabayon-limbo
enabled = <true/false>
[server=sabayon-limbo]
desc = This statement will be ignored.
repo-only = ssh://username@repo.host:~username/sabayon-limbo
pkg-only = ssh://username@pkg.host:~username/sabayon-limbo
[server=sabayon-base]
desc = This is the base repository.
repo-only = ssh://username@repo.host:~username/sabayon-base
pkg-only = ssh://username@pkg.host:~username/sabayon-base
base = <true/false>
As you can see, multiple statements for the same repository
are allowed. However, only the first desc = statement will be
considered, while there can be as many {pkg,repo}* = as you want.
The repository order is important, but this is guaranteed by the
fact that configuration files are parsed in lexical order.
Statements description:
- "desc": stands for description, the repository name description.
- "repo": the push & pull URI, for both packages and repository database.
- "repo-only": same as repo, but only for the repository database
push & pull.
- "pkg-only": same as repo, but only for the packages push & pull.
The supported protocols are those supported by entropy.fetchers.
- "enabled": if set, its value can be either "true" or "false". The default
value is "true". It indicates if a repository is configured
but currently disabled or enabled. Please take into account
that config files in /etc/entropy/repositories.conf.d/ starting
with "_" are considered to contain disabled repositories. This
is just provided for convienence.
- "base": if set, its value can be either "true" or "false". The default
value is "false". If no repository has the flag set, the first
listed repository will be the base one. Only the first repository
with "base = true" will be considered. The base repository is the
repository that is considered base for all the others
(the main one).
etpConst['clientdbid'] is kept for backward compatibility, but will be removed soon
While etpConst['serverdbid'] and etpConst['clientserverrepoid'] are gone.
The original idea was to avoid doing cursor and connection resources
cleanup (left by old and dead threads) synchronously every time
_connection() and/or _cursor() is accessed. This strategy also had
a huge drawback: with no activity on the object, resources were
left hanging there forever.
This commit introduces a better strategy for transparent and automatic
cleanup of resources belonging to terminated threads: every time a new
thread_id arrives at _cursor() or _connection(), a new daemon thread
starts and synchronizes with the caller through a simple Thread.join()
(because it's a daemon thread, we can join() daemon threads as well,
even if this is not really compliant with the specs, but it seems to work
just fine in Python).
When the caller thread is joined, it is possible to start the resources
cleanup procedure, carefully taking into account that thread_ids are
recycled and thus there might be clashing with newly created threads.
This helped a design issue to emerge from the sand (like a zombie
at the seaside): it is impossible to cleanup resources left by the
MainThread because this thread never ends living, and if it dies,
everything dies, obviously. So, the first implementation of this new
strategy was NOT touching the MainThread resources but then, the old
behaviour was to kill them as well on EntropyRepository.close().
So, the final version of this patch kept the old buggy behaviour of
touching MainThread stuff (nein, nein, nein, nein would Hitler say).
However, a new keyword argument "safe" has been added to the close()
method so it is possible to start migrating code to the dark side of the
power.
This means nothing really changed for API consumers yet, just entropy.db.sql
code being more efficient (no weird for loops and synchronous crap)
and actually faster (multi-threading ftw).
The new WebService get_documents() is more efficient in terms of
server resources consumption since it doesn't force the WS engine
to calculate the full result set length, which had little use anyway
in Rigo.
This commit switches entropy.client.services' DocumentList to and
reverse dependencies to use the new WS API. The older WS API will be
kept alive for a while (6 months, roughly).
Introduce Entropy conditional dependencies supoort if
ETP_PORTAGE_CONDITIONAL_DEPS_ENABLE is set in the environment.
This feature is disabled by default because it braks backward
compatibility and older Entropy Clients are broken wrt this.
