89 Commits

Author SHA1 Message Date
Mario Fetka
7072c00a77 Python 3 modernization: low-hanging fruit cleanup
- hashlib: add usedforsecurity=False to all md5/sha1/sha256/sha512 calls
  (FIPS-mode compatibility, ~34 call sites across 13 files)
- join([listcomp]): convert to generator expressions (~15 sites)
- class Foo(object): -> class Foo: across all files (Python 3 style)
- collections.OrderedDict() -> {} in db/skel.py (Python 3.7+ dicts ordered)
- open() without context manager: fix 3 bare open/close in misc.py
- Exception chaining: add 'from err' to raise statements in portage_plugin
- const_is_python3() dead code removal:
  * Simplify 7 compat functions in const.py (const_is_python3,
    const_get_stringtype, const_isunicode, const_israwstring,
    const_get_buffer, const_convert_to_unicode, const_convert_to_rawstring,
    const_isstring, const_get_int) — remove Python 2 branches
  * Remove 100+ const_is_python3() if/else branches across 27 files,
    keeping only the Python 3 path
  * Remove cPickle import in dump.py, use plain pickle with fix_imports=True
  * Remove urllib2 dead imports in misc.py, fetchers.py
  * Clean up const_is_python3 from all import lines

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-26 11:51:29 +02:00
Sławomir Nizio
e26d11b5f6 Python 3 fixes: open() and so, not file() 2019-12-04 22:01:22 +01:00
Sławomir Nizio
3e74fa847c update matter imports 2018-11-26 20:15:36 +01:00
Sławomir Nizio
6d4a3f09e6 makefiles: establish a layout for sitedir installs
The idea is that:
- entropy.* imports will work as before (so any 3rd party clients will
  work as always) - installed in "entropy" package,
- new "_entropy" package to hold a namespace for private modules (like
  ones that required adding special directories to sys.path).
  (Underscored name for a top level Python module is not very common...
  anyway, it was inspired by "_emerge.")

Layout:
site-packages/
    entropy (backwards compatible)
	const.py
	...
    kswitch (also toplevel to keep compatibility)
	...
    _entropy
	eit
	magneto
	matter
	rigo
	RigoDaemon
	solo

(Note that site-packages does not need to be actually Python's
site-packages directory but anything as it is controlled by an argument
to make. It is however intended to be the sitedir.)

Another idea for a layout would be one that mimics sources checkout, but
the layout there is somewhat scattered. (And some ugliness would be
needed to make them modules before implicit namespaces from Python 3.3.
Anyway, imports would be long and ugly.)

Now, the layout of installed Entropy is lean; installation to virtualenv
is also possible (though there would be a need to call scripts like
"python equo.py" as shebangs are not converted).

Follow up changes are needed to make it work.
2018-11-26 20:15:36 +01:00
Sławomir Nizio
252c260f06 use entropy_path_loader only for running from the checkout
After this commit alone it would not work when installed (unless module
paths are set in a special way). Next changes will introduce
installation to site-packages so no custom PYTHONPATH will be necessary.
2018-11-26 20:15:36 +01:00
Sławomir Nizio
33966f4f31 [*] allow setting PYTHON_SITEDIR for make
This way a proper separation between Python installs can be achieved.

With no PYTHON_SITEDIR, installation paths are exactly the same as
before this change.

In practise, passing nonstandard path will break Entropy but so it is
also with the currently available LIBDIR. This is a concern of a future
improvement (at least the PYTHON_SITEDIR path, not necessarily LIBDIR as
PYTHON_SITEDIR *will* be different if it's installed in Python specific
directories).
2018-10-28 21:44:54 +01:00
Sławomir Nizio
41ba9b6d63 [*] be strict about Python modules paths being loaded
For convenience (seemingly, and it really is convenient) equo and other
tools can be run from the checkout, and Entropy modules are loaded from
the checkout. Now there is a strict separation when system paths and
when paths from the checkout are used.

It makes it a bit more robust, secure and preditable at the cost of
a little more complexity.

A pleasant side effect of this change is that it is not required to
change directory to the tool (to use non-system one), as paths in the
checkout are relative to scripts.

Imports in lib/tests were not adjusted.
2018-10-27 13:03:58 +02:00
Fabio Erculiani
7e2846c491 [matter] binpms: return int on push() 2014-07-05 17:20:30 +01:00
Fabio Erculiani
fc2488e5f7 [matter] merge_list could be None, make sure it never is 2014-02-17 21:03:29 +01:00
Fabio Erculiani
8589b87465 [matter] add clear_cache() to BaseBinaryPMS to clear memory caches 2014-01-28 16:52:27 +01:00
Fabio Erculiani
97b90950fe [matter] expose more status info to pkgpost
- MATTER_PACKAGE_NAMES
- MATTER_FAILED_PACKAGES
- MATTER_BUILT_PACKAGES
- MATTER_NOT_FOUND_PACKAGES
- MATTER_NOT_INSTALLED_PACKAGES
2014-01-28 11:17:35 +01:00
Fabio Erculiani
ebf63e6060 [entropy.locks] fix shared/exclusive locking mix due to reentrancy, add tests 2013-12-08 16:15:39 +01:00
Fabio Erculiani
1917771b54 [entropy.locks] move Entropy Resources Lock code to a separate module 2013-12-08 14:37:57 +01:00
Fabio Erculiani
8c73f9b24c [matter] lazily load the Entropy Server instance object 2013-12-07 20:29:19 +01:00
Fabio Erculiani
b886d7e3fc [matter] use Server class with acquire_entropy_locks 2013-12-07 20:29:19 +01:00
Fabio Erculiani
e164b4b577 [matter] fix portdb.xmatch() when package is keyword masked
This commit fixes the use of backupenv.
2013-11-17 18:38:49 +01:00
Fabio Erculiani
7c21ee500d [matter] do not create a new settings object, but keep reusing the same one 2013-11-17 18:38:25 +01:00
Fabio Erculiani
000056afaa [matter] improve ACCEPT_KEYWORDS setup basing on the "stable" parameter
This tweak makes possible to directly skip depgraph generation if
the root node (the dependency to check, last node of the toposorted
queue) is already up-to-date (wrt to stable: yes, for example).

This is what Portage does internally (using backupenv) and may break
in future releases without notice. So, keep the settings["ACCEPT_KEYWORDS"]
setup as a guard, to ensure the safety property.
2013-11-10 19:42:44 +01:00
Fabio Erculiani
fa71378cca [matter] fix typo in variable name 2013-10-28 21:03:45 +01:00
Fabio Erculiani
7680861cbb [matter] fix post_build execution wrt collected statistics 2013-10-28 20:56:45 +01:00
Fabio Erculiani
d8a4b741a5 [matter.builder] fix needed_license_changes backtrack info handling 2013-10-28 20:51:03 +01:00
Fabio Erculiani
fc56ba5be2 [matter.builder] execute post-build operations iff there has been vdb activity 2013-10-28 16:57:36 +01:00
Fabio Erculiani
1c2b6d0153 [matter] fix function calls wrt dict expansion 2013-10-28 14:48:50 +01:00
Fabio Erculiani
eeac4a7782 [matter.builder] use __str__ or __unicode__ instead of copy.deepcopy() when possible 2013-10-28 12:49:15 +01:00
Fabio Erculiani
c2c4647c86 [matter.builder] regenerate dbapi.auxdb objects after calling close_caches() 2013-10-28 12:40:09 +01:00
Fabio Erculiani
5884c68bcd [matter] use PackageBuilder.clear_caches() 2013-10-28 12:24:46 +01:00
Fabio Erculiani
66db77935d [matter.builder] add PackageBuilder.clear_caches()
This method is now responsible of clearing all the Portage and
Binary PMS caches.
2013-10-28 12:23:58 +01:00
Fabio Erculiani
ee4452ff31 [matter.builder] use copy.deepcopy to copy more objects, avoid circular references 2013-10-28 12:23:20 +01:00
Fabio Erculiani
47ea8c70ca [matter.builder] use vardb.match without cache, reduce memory usage 2013-10-28 12:22:30 +01:00
Fabio Erculiani
3ae112caae [matter] use collections.deque instead of lists, improve memory efficiency 2013-10-28 12:20:34 +01:00
Fabio Erculiani
274adb4c3e [matter.builder] acquire a file lock before calling PackageBuilder.sync() 2013-10-27 19:35:39 +01:00
Fabio Erculiani
b5dadd905d [matter] add another example particle 2013-10-27 18:35:13 +01:00
Fabio Erculiani
aa2a6520f4 [matter.builder] expand depstring to match all the available slots
If a particle contains the depstring "www-apps/bugzilla", do not
just match the best available, which would include only the latest
slotted version but rather go through all the slotted versions
available and apply filters from there. The net result is that
we will never leave slotted packages behind.
2013-10-27 18:28:54 +01:00
Fabio Erculiani
78144d3a66 [matter] use copy.deepcopy for every portage backtrack generated object 2013-10-21 10:44:13 +02:00
Fabio Erculiani
886c441850 [matter.main] merge the use_changes dicts in a safer way 2013-10-21 10:43:43 +02:00
Fabio Erculiani
94d04cc1ef [matter] add support for needed_license_changes backtrack info 2013-10-21 10:40:56 +02:00
Fabio Erculiani
4124a3e4e9 [matter] add support for needed_p_mask_changes backtrack info 2013-10-21 10:27:45 +02:00
Fabio Erculiani
2594f3f1d2 [matter] use copy.deepcopy with Portage Package.pkg objects to avoid leaks 2013-10-21 10:18:59 +02:00
Fabio Erculiani
a079e57ccc [matter] add Portagge backtrack info "needed_unstable_keywords" support 2013-10-10 18:54:48 +02:00
Fabio Erculiani
b1ac6994fa [matter] catch OnlineMirrorError during entropysrv commit() 2013-10-08 15:55:36 +02:00
Fabio Erculiani
7eb9bfd611 [matter] fix "changes" metadata objects handling 2013-06-08 07:13:21 +02:00
Fabio Erculiani
b8f22b4956 [matter] initialize a dict for new_changes 2013-06-07 21:32:03 +02:00
Fabio Erculiani
c3517825e8 [matter] use copy.deepcopy with get_backtrack_infos() Portage data
A memory leak has been observed and may be related to keeping
Portage objects references alive.
2013-06-07 21:27:54 +02:00
Fabio Erculiani
14d131904b [antimatter] use _new_scan() when --not-installed is selected 2013-06-04 11:18:18 +02:00
Fabio Erculiani
dcbdb8d0f4 [antimatter] add --not-installed flag support 2013-06-04 11:00:25 +02:00
Fabio Erculiani
6d94e86870 [matter] add a final \n to the missing USE output 2013-05-30 00:31:00 +02:00
Fabio Erculiani
45ce99a0aa [matter] import print_generic 2013-05-30 00:27:11 +02:00
Fabio Erculiani
9a5062648a [matter] print final status report using print_generic, avoids the timestamp prefix 2013-05-30 00:24:10 +02:00
Fabio Erculiani
7022d5cccc [matter] log and report packages not built due to missing USE flags 2013-05-30 00:18:44 +02:00
Fabio Erculiani
52b329afa2 [matter] fix typo in package versions calculation for injected packages 2013-03-17 08:27:12 +00:00