Commit Graph

240 Commits

Author SHA1 Message Date
Fabio Erculiani 92e42ca3aa [entropy.db*] EntropyRepository, do not close() on object destruction
Closing the underlying sqlite3 db on object destruction (__del__())
causes funny race conditions when concurrently accessing the object
itself.

When the Garbage Collector tries to free memory, which can happen
when no more references pointing to self are used, by calling __del__()
(which called close()) it is possible to run into troubles if another
thread is inside a method of the same object holding a valid sqlite cursor.

Moreover, no external arbitration is possible if the garbage collection
gets in the middle and calls close() through the destructor on behalf
of a poor random innocent thread.

Solution is simple, destructor is evil and resource leaks have to be
handled where they actually are. Bye bye __del__().
2012-03-22 19:09:24 +01:00
Fabio Erculiani 5557bf8b3f [entropy.misc] add non-blocking methods to ReadersWritersSemaphore 2012-03-20 14:32:37 +01:00
Fabio Erculiani 5839d81819 [entropy.const] const_debug_write: add force argument support 2012-03-18 18:29:09 +01:00
Fabio Erculiani c62790b933 [entropy.db] add searchKeySlotTag()
Implement searchKeySlotTag(), it is the same as searchKeySlot() but
it also allows to specify package tag.
2012-03-17 17:05:39 +01:00
Fabio Erculiani 361c4166da [entropy.client] introduce Client.unavailable_repositories()
The method returns a sorted list of repositories identifiers
belonging to repositories that are unavaiable or invalid.
2012-03-16 16:04:15 +01:00
Fabio Erculiani b7d198c776 [entropy.client] Client, more locking cleanup
Never unlink() a lock when releasing it. This is quite bad when
used with shared locks.
At the same time, don't write any pid information in it, because
it's not always reliable (see previous commit).
2012-03-14 16:21:20 +01:00
Fabio Erculiani 7d2633f2a0 [entropy.client] _lock_resource: avoid to deadlock when _unlock_resource is called 2012-03-13 16:05:40 +01:00
Fabio Erculiani 63e9764bfa [entropy.db] EntropyRepository: mtime() can return OSError, need to cope with it 2012-03-13 16:05:40 +01:00
Fabio Erculiani 6bb4f6b28c [entropy.db] EntropyRepository: validate Live cache against mtime()
Scenario: Process A is writing to EntropyRepository, adding new
package entries. Process B is reading from EntropyRepository,
querying for the same package entries, for example by using
retrieveKeySlot(). This method uses a dict-based cache to speed
up things, but this should be invalidated also when the mtime()
value changes.
2012-03-13 09:01:49 +01:00
Fabio Erculiani 9584c0ae71 [entropy.cache] better handle sleeping in EntropyCacheWriter thread
Make sure we never block on stop() but also we never run for nothing.
Now calling release() as many times as calling acquire()
2012-03-12 16:50:40 +01:00
Fabio Erculiani 5649c1dc80 [entropy.cache] trivial tweaks to cacher thread 2012-03-12 16:26:15 +01:00
Fabio Erculiani caf2e2192d [entropy.db] atomMatch: versions can be an empty set
When matching >=dev-lang/python-2.6 having some weird mask, it
can happen to end up having no matches left after the last
checkpoint. Make sure to handle the case even on the very last mile
2012-03-11 19:36:41 +01:00
Fabio Erculiani 1f334138fb [entropy.client] reorder_mirrors: hostname property can be None
If mirror string (URL string) is malformed, the hostname object
property can be None.
2012-03-11 18:53:42 +01:00
Fabio Erculiani 0e9491082b [entropy.misc] introduce ReadersWritersSemaphore object 2012-03-11 12:33:29 +01:00
Fabio Erculiani 62a67600d8 [entropy.misc] drop MasterSlaveLock, no longer used 2012-03-11 12:14:52 +01:00
Fabio Erculiani 37133e2ab2 [entropy.client] add Client.promote_resources()
Promoting a previously acquired shared Entropy Resources Lock to
an exclusive one is now possible via Client.promote_resources().
2012-03-09 09:42:27 +01:00
Fabio Erculiani 9b52c0f04a [entropy.client.interfaces.repository] provide unlocked_sync() method
As done with entropy.security, unlocked_sync() will make possible
for callers to handle the Entropy Resources Lock on their own.
2012-03-09 09:32:06 +01:00
Fabio Erculiani 922fc34d7e [entropy.client.interfaces.db] do not touch lock files permissions
Entropy Resources Lock file is in the etpConst['etpdatabaseclientdir']
directory. For this reason, it is better to not touch the whole
directory but just the subdirs.
2012-03-08 22:49:38 +01:00
Fabio Erculiani 3195a23689 [entropy.security] add Security.unlocked_sync()
unlocked_sync() works just like sync() but without acquiring the
Entropy Resources Lock. This way it's possible to externally control
the mutual exclusion.
2012-03-08 21:44:13 +01:00
Fabio Erculiani b379bbc8ad [entropy.misc] FlockFile: add try_promote() method 2012-03-08 21:43:29 +01:00
Fabio Erculiani 87e984b303 [entropy.client.services] make get_document_url() fetch atomic
This avoids other instances to pull in invalid/partial data
2012-03-05 23:35:34 +01:00
Fabio Erculiani 058e448ebd [entropy.dump] do not unlink() old cache files if aging threshold is reached
Other cache consumers may still find the cache file valid, even if
for others it's old. So, don't remove the file but just return None.
This way it works more like an "overlay".
2012-03-04 18:10:22 +01:00
Fabio Erculiani bf01329f4d [entropy.services] WebService: make possible to configure cache aging
Configuring cache aging makes possible to automatically throw away
old metadata, like package votes and number of downloads.
Usually, this information is still valid, from user POV, even if
quite old. But how much old? This can be tuned via enable_cache_aging()
from outside WebService class, depending on the scenario.
2012-03-04 18:06:50 +01:00
Fabio Erculiani a48cb985c6 [entropy.cache] EntropyCacher: add cache aging support
Following entropy.dump cache aging suport, this commit exports the
same functionality to Cacher.pop()
2012-03-04 18:05:54 +01:00
Fabio Erculiani 623930ee4b [entropy.dump] add cache aging support to deserializer
If cached file is older than aging_days, it will be removed from
disk (if possible) and None shall be returned
2012-03-04 18:05:37 +01:00
Fabio Erculiani 63947241b4 [entropy.const] do not hardcode "raw_unicode_escape" 2012-03-04 17:57:57 +01:00
Fabio Erculiani 7e3c89f3d6 [entropy.dump] trivial code style update 2012-03-04 17:57:37 +01:00
Fabio Erculiani 527cbbbaa9 [entropy.dump] getobjmtime: fix possible race wrt file removal 2012-03-04 17:56:51 +01:00
Fabio Erculiani 3f45623ab5 [entropy.dump] use etpConst['conf_raw_encoding'] 2012-03-04 17:56:15 +01:00
Fabio Erculiani 21dcdcc636 [entropy.db] _migrateBaseinfoExtrainfo: make sure tables exist before running the query 2012-03-02 17:16:42 +01:00
Fabio Erculiani 81738b8d3a [entropy.client] atom_search: make possible to also search through package description 2012-03-02 08:36:06 +01:00
Fabio Erculiani de72481fbd [entropy.db] searchDescription: improve search results
Improve search results by splitting keyword into multiple words
and searching the unordered concatenation of them.
2012-03-02 08:35:05 +01:00
Fabio Erculiani 303b7ad1ea Release Entropy 1.0_rc90 2012-03-01 12:06:54 +01:00
Fabio Erculiani 3c8ec43e35 [entropy.client.interfaces.dep] SLOT intersection: alter dependency adding slot
Scenario:
dependency = dev-lang/python
SLOT intersection = enabled
available slots = 2.6, 2.7, 3.2
installed slot = 2.7

The SLOT intersection feature worked correctly but in the end
dev-lang/python was added to the unsatisfied set because there
was a dev-lang/python-2.7 update as well. This results in other
dep calculation code to match dev-lang/python against 3.2 instead.
The SLOT intersection feature has to reduce the dependency string
scope by adding a SLOT suffix (in this case: ":2.7"). This way the
correct dependency is eventually pulled in.
2012-03-01 12:01:53 +01:00
Fabio Erculiani 732482473f [entropy.dep] dep_getslot: remove tag before mangling.
Otherwise the following query will fail (at atomMatch()):
x11-drivers/nvidia-drivers:0,3.2.0-sabayon#3.2.0-sabayon
2012-03-01 11:58:47 +01:00
Fabio Erculiani 80f1e1182e [entropy.client.interfaces.dep] don't spam stdout with same progress values 2012-03-01 11:13:51 +01:00
Fabio Erculiani 095038425a [entropy.const] improve thread dump signal function
Always flush stderr on a newline and also write the number of
active threads as well.
Moreover, always install the dump function at SIGQUIT when --debug
is passed.
2012-02-29 10:21:04 +01:00
Fabio Erculiani 161ba2cfd9 [entropy.client.services.interfaces] add ClientWebService.get_images()
Don't ask me why this was missing, anyway, now we have it.
This returns a superset of get_icons(), also including normal images.
2012-02-28 21:57:18 +01:00
Fabio Erculiani b97c1dd8b7 Release Entropy 1.0_rc89 2012-02-27 23:33:12 +01:00
Fabio Erculiani 85ebb0ebac [entropy.const] rework ETP_ARCH_CONST and keywords configuration, make it work with overridden values 2012-02-27 23:18:26 +01:00
Fabio Erculiani 65506c706f Release Entropy 1.0_rc88 2012-02-27 22:56:58 +01:00
Fabio Erculiani 642da47e40 [entropy.const] add _RAW_ENCODING and fix wrong encoding detection code 2012-02-27 19:49:13 +01:00
Fabio Erculiani 18282e3764 [entropy.spm] PortagePlugin: try using raw_unicode_escape when reading .desktop files
Sometimes, upstream .desktop files are encoded using raw_unicode_escape
and we should really fallback to that encoding before giving up reading
them.
2012-02-27 19:47:16 +01:00
Fabio Erculiani a28a92c2a3 [entropy.const] do not hardcode UTF-8 string, but rather use _ENCODING 2012-02-27 19:35:15 +01:00
Fabio Erculiani c1ef9ae4b5 [entropy.client.services] make possible to retrieve documents in reverse order 2012-02-27 10:32:47 +01:00
Fabio Erculiani 550be469a1 [entropy.client.services.interfaces] drop incomplete comment 2012-02-26 22:18:37 +01:00
Fabio Erculiani 4673c622a4 [entropy.spm] improve heuristic used to detect non-kernel ebuilds
This makes possible to have external module packages inside the
sys-kernel/ category, by just adding extra checks. In particular,
we now check against ETYPE ebuild environment variable.
This should guarantee that all the kernel sources, binaries and
headers won't be tagged by Entropy.
2012-02-20 17:52:50 +01:00
Fabio Erculiani 74149c79c1 [entropy.spm] PortagePlugin: be more robust against fucked up portage world file 2012-02-19 14:53:23 +01:00
Fabio Erculiani ceeaf62dde [entropy.client] improve code that handles the Entropy Resources Lock 2012-02-19 14:43:41 +01:00
Fabio Erculiani 1c213a911b [entropy.client] add Repository.get_last_sync_time()
Everytime Repository.sync() is triggered, the last sync time
information is updated. This makes possible to figure out when
the last repository update has been run and warn user about it
if they're too old.
2012-02-17 14:14:38 +01:00