1. Command to execute (diff) was getting arguments like b"path".
2. Crash when priting message.
3. In Python 3, b'x'[0] is int, not str, so a test (if) was (silently)
always true. This way, upon eit commit (at least) and with Python 3,
warning about not merged configuration files was newer printed.
(3) prevented prior discovery of (1) and (2).
Based on my checks, there are no more problems like (3), but I cannot
tell for sure.
In 0.74 -e was available, and in 0.80 it is not present.
In both versions, and with the same set of options used, -e and -v
provide the same result so -v is now used to work on both.
Changed in portage-utils, commit 951a8711a59b1a7d49125f5f5214ff1ae9e50074:
qfile: drop non-functional --exact option
Bug: https://bugs.gentoo.org/678632
Note: <tab> below can mean double <tab>.
This commit fixes cases such like the following ones:
* crash (issue !73) - e.g. KeyError: u'--mtime':
- equo security oscheck --mtime <tab>
- equo query graph --complete <tab>
(this was due to a typo in variable name)
* now prints options, previously it didn't:
- equo install -a <tab>
* crashed, or (with only variable name fixed) didn't print all options:
- equo security oscheck --mtime <tab>
(note that it worked with more special -q instead of --mtime etc.)
* didn't print all options:
- equo query list installed <tab>
(now shows also --by-user)
The following behaviour is still buggy:
* last typed word (note: no space before <tab>) disappears upon completion:
- equo security oscheck<tab>
(not a regression)
- equo install --deep<tab>
(here it's an option; regression! - previously it didn't complete but
didn't cause the word to be erased either)
These can be corrected reliably when something like ${COMP_WORDS[COMP_CWORD]}
(from complete -F) is passed to the Python side.
Without this it's not possible to distinguish between `recognized_option<tab>`
(completion of recognized_option) and `recognized_option <tab>`.
The module entropy_path_loader (used only for running from within the
checkout; otherwise not even installed) is made to provide the _entropy
namespace.
(Other ideas instead of this entropy_path_loader change would be to
reorganise files layout; drop support for running from the checkout as
is - and perhaps require virtualenvs; require sourcing a script that
sets PYTHONPATH. However, as implemented, it is not intrusive, and the
good part is that it is quite isolated, not used in normal usage after
installation. Basically, it only does sys.path + provides _entropy
namespace.)
The idea is that:
- entropy.* imports will work as before (so any 3rd party clients will
work as always) - installed in "entropy" package,
- new "_entropy" package to hold a namespace for private modules (like
ones that required adding special directories to sys.path).
(Underscored name for a top level Python module is not very common...
anyway, it was inspired by "_emerge.")
Layout:
site-packages/
entropy (backwards compatible)
const.py
...
kswitch (also toplevel to keep compatibility)
...
_entropy
eit
magneto
matter
rigo
RigoDaemon
solo
(Note that site-packages does not need to be actually Python's
site-packages directory but anything as it is controlled by an argument
to make. It is however intended to be the sitedir.)
Another idea for a layout would be one that mimics sources checkout, but
the layout there is somewhat scattered. (And some ugliness would be
needed to make them modules before implicit namespaces from Python 3.3.
Anyway, imports would be long and ugly.)
Now, the layout of installed Entropy is lean; installation to virtualenv
is also possible (though there would be a need to call scripts like
"python equo.py" as shebangs are not converted).
Follow up changes are needed to make it work.
After this commit alone it would not work when installed (unless module
paths are set in a special way). Next changes will introduce
installation to site-packages so no custom PYTHONPATH will be necessary.
pysqlite has a bug on use VACUUM with py3.6 (see https://github.com/ghaering/pysqlite/issues/109)
Hereinafter, exception related to eit push --quick --force <REPO> command:
Traceback (most recent call last):
File "/usr/lib/entropy/lib/entropy/db/sql.py", line 166, in _proxy_call
return method(*args, **kwargs)
sqlite3.OperationalError: cannot VACUUM from within a transaction
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/eit", line 17, in <module>
main()
File "/usr/lib/entropy/server/eit/main.py", line 114, in main
exit_st = func(*func_args)
File "/usr/lib/entropy/server/eit/commands/command.py", line 237, in _call_exclusive
return func(server)
File "/usr/lib/entropy/server/eit/commands/push.py", line 172, in _push
rc = self._push_repo(entropy_server, repository_id)
File "/usr/lib/entropy/server/eit/commands/push.py", line 184, in _push_repo
rc = self.__push_repo(entropy_server, repository_id)
File "/usr/lib/entropy/server/eit/commands/push.py", line 309, in __push_repo
sts = self.__sync_repo(entropy_server, repository_id)
File "/usr/lib/entropy/server/eit/commands/push.py", line 262, in __sync_repo
enable_download = False, force = self._force)
File "/usr/lib/entropy/lib/entropy/server/interfaces/mirrors.py", line 1673, in sync_repository
enable_upload, enable_download, force = force)
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 231, in update
enable_upload, enable_download, force = force).update()
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 404, in update
rc, fine_uris, broken_uris = self._sync()
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1873, in _sync
broken_uris = self._upload(uris)
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1529, in _upload
self._shrink_and_close(dbconn)
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1234, in _shrink_and_close
entropy_repository.vacuum()
File "/usr/lib/entropy/lib/entropy/db/sqlite.py", line 703, in vacuum
self._cursor().execute("vacuum")
File "/usr/lib/entropy/lib/entropy/db/sqlite.py", line 58, in execute
cur = self._proxy_call(self._cur.execute, *args, **kwargs)
File "/usr/lib/entropy/lib/entropy/db/sql.py", line 173, in _proxy_call
raise OperationalError(err)
entropy.db.exceptions.OperationalError: cannot VACUUM from within a transaction
Otherwise "entropy" module loading logic was delayed, excluding this
import (which was done early, before entropy_path_loader logic):
from entropy.locks import SimpleFileLock
and it failed when system wide PYTHONPATH is dropped.
There was a comment not to do an early import due to --no-pid-handling.
The comment was added in:
commit 59e5fd737e
Date: Tue Mar 1 13:13:18 2011 +0100
[magneto] fix lock file handling
but then the logic was reworked in:
commit 5d315c70bd
Date: Sun Feb 24 23:34:49 2013 +0100
[entropy.const] drop /var/run/entropy/entropy.lock file and directory handling
and then passing --no-pid-handling (which was unused at this point) from
mangeto file was removed in:
commit 83b8991c95
Date: Wed Feb 27 20:59:14 2013 +0000
[magneto] rewrite systray icons management, improve KDE, GNOME and MATE integration
This way a proper separation between Python installs can be achieved.
With no PYTHON_SITEDIR, installation paths are exactly the same as
before this change.
In practise, passing nonstandard path will break Entropy but so it is
also with the currently available LIBDIR. This is a concern of a future
improvement (at least the PYTHON_SITEDIR path, not necessarily LIBDIR as
PYTHON_SITEDIR *will* be different if it's installed in Python specific
directories).
This way there is no hardcoded /usr/lib/rigo, and the "executable" part
is in a well known location which can be replicated on system for
different Python implementations. (As in: it will be like /usr/bin/foo.py
which on Gentoo links to a system wrapper making it use configured
Python.)
For convenience (seemingly, and it really is convenient) equo and other
tools can be run from the checkout, and Entropy modules are loaded from
the checkout. Now there is a strict separation when system paths and
when paths from the checkout are used.
It makes it a bit more robust, secure and preditable at the cost of
a little more complexity.
A pleasant side effect of this change is that it is not required to
change directory to the tool (to use non-system one), as paths in the
checkout are relative to scripts.
Imports in lib/tests were not adjusted.
Sławomir Nizio
Ettore Di Giacinto
geaaru
Geaaru
Ettore Di Giacinto
Fabio Erculiani