pysqlite has a bug on use VACUUM with py3.6 (see https://github.com/ghaering/pysqlite/issues/109)
Hereinafter, exception related to eit push --quick --force <REPO> command:
Traceback (most recent call last):
File "/usr/lib/entropy/lib/entropy/db/sql.py", line 166, in _proxy_call
return method(*args, **kwargs)
sqlite3.OperationalError: cannot VACUUM from within a transaction
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/eit", line 17, in <module>
main()
File "/usr/lib/entropy/server/eit/main.py", line 114, in main
exit_st = func(*func_args)
File "/usr/lib/entropy/server/eit/commands/command.py", line 237, in _call_exclusive
return func(server)
File "/usr/lib/entropy/server/eit/commands/push.py", line 172, in _push
rc = self._push_repo(entropy_server, repository_id)
File "/usr/lib/entropy/server/eit/commands/push.py", line 184, in _push_repo
rc = self.__push_repo(entropy_server, repository_id)
File "/usr/lib/entropy/server/eit/commands/push.py", line 309, in __push_repo
sts = self.__sync_repo(entropy_server, repository_id)
File "/usr/lib/entropy/server/eit/commands/push.py", line 262, in __sync_repo
enable_download = False, force = self._force)
File "/usr/lib/entropy/lib/entropy/server/interfaces/mirrors.py", line 1673, in sync_repository
enable_upload, enable_download, force = force)
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 231, in update
enable_upload, enable_download, force = force).update()
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 404, in update
rc, fine_uris, broken_uris = self._sync()
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1873, in _sync
broken_uris = self._upload(uris)
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1529, in _upload
self._shrink_and_close(dbconn)
File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1234, in _shrink_and_close
entropy_repository.vacuum()
File "/usr/lib/entropy/lib/entropy/db/sqlite.py", line 703, in vacuum
self._cursor().execute("vacuum")
File "/usr/lib/entropy/lib/entropy/db/sqlite.py", line 58, in execute
cur = self._proxy_call(self._cur.execute, *args, **kwargs)
File "/usr/lib/entropy/lib/entropy/db/sql.py", line 173, in _proxy_call
raise OperationalError(err)
entropy.db.exceptions.OperationalError: cannot VACUUM from within a transaction
For convenience (seemingly, and it really is convenient) equo and other
tools can be run from the checkout, and Entropy modules are loaded from
the checkout. Now there is a strict separation when system paths and
when paths from the checkout are used.
It makes it a bit more robust, secure and preditable at the cost of
a little more complexity.
A pleasant side effect of this change is that it is not required to
change directory to the tool (to use non-system one), as paths in the
checkout are relative to scripts.
Imports in lib/tests were not adjusted.
This is a ";" separated data input (even though we may get directory
paths containing ";"... but none so far is expected to be legit.)
Split by ";" first and then deal with spaces in paths.
It looks like Portage now stores unicode paths correctly in its metadata
as opposed to what it used to be. We need to make sure that we parse those
"CONTENTS" file and content metadata in general using the correct encoding.
This will allow us to store and retrieve such metadata from the sqlite3
database correctly and also match the stored paths with the filesystem
paths exactly.
This commit may need a bit more real-life testing. Backward compat
wrt old Entropy and Portage tbz2 files should be as expected.
Unit tests attached.
Historically, Portage + Py2 were not dealing well with file paths that
were containing non-ascii chars. The situation has apparently improved
and we are able to accept paths with the correct encoding in Entropy
(separate patch will come). We need to guard against broken paths anyway,
so let's try to check which one of the paths is the one that has the
most chance to be the right one. Also, with python3 this won't be
a problem anymore...
Due to the migration to real virtual packages (away from PROVIDE= due
to EAPI=7 constraints), we failed to resolve kernel binary packages
correctly, thus packages that require them (external kernel modules.)
This patch fixes it, by making sure that we resolve new virtuals correctly.
We expect new virtuals to list kernel packages in their RDEPEND= (runtime
dependecies.)
- runs as non-root
- does not require being in entropy/portage group
- in fact, it's better (better isolation) to run as such
- thus does not modify running system
The wrapper script is ugly but very convenient.
It uses a simple rules without the need to use complex regexes. It is
not as powerful as the existing approaches so both are complementary.
For example, this:
foo/bar (.*)app-crypt/pinentry(.*)\[gtk\] \1app-crypt/pinentry-gtk2\2
expresses the intention that can be expressed simpler:
rewrite foo/bar from-dep=app-crypt/pinentry to-dep=app-crypt/pinentry-gtk2 if-dep-has-use=gtk drop-use=gtk
Previously it has a dict with complex key (with the need to pack and
unpack it in places where it was used), and data was both part of the
keys and values.
It had the small advantage of removing duplicates, but changes for them
are very low and they are harmless anyway.
In certain extreme situations, that are more likely if the install
state is old (e.g. 9 months +), Entropy may decide to remove packages
due to conflicts with the version being installed and shoot itself
in the foot, if this is the case of dev-lang/python :-).
So, this commit tries to avoid the scenario by filtering out package
removals for those that are being installed anyway as part of the
transaction (e.g. a system upgrade).
This should fix the very specific case, but increase uncertainty
and thus entropy in other cases where once a package that lists
conflicts is installed, it may not expect to find that package
installed. However, this case should be handled by the dependency
graph generator (and it currently does not), which would need to
reorder the execution of the queue to comply with conflicts.
Perhaps, someone in the future will improve that too, for now,
let's enjoy one more snowflake.
Current improvement:
- foo-1 and bar-2 are set to be installed
- foo-1 conflicts with <bar-2
- bar-1 is installed
- foo-1 wanted to remove bar-1 as part of foo-1 install transaction.
(now fixed, it won't...)
- bar-2 is then installed
Better improvement:
- foo-1 and bar-2 are set to be installed
- foo-1 conflicts with <bar-2 (but bar-2 itself can be installed
and does not conflict with foo-1)
- bar-2 is set to be installed before foo-1
- when entropy gets to foo-1 there is nothing to remove
Entropy preserved libs handling code did not check all the followed symlink
before removing them. This causes the deletion of unexpected libraries during the
removal of packages. For example, app-emulation/vmware-workstation contains
the following symlink:
(A) /opt/vmware/lib/vmware/lib/libssl.so.1.0.1/libssl.so.1.0.1 -> (B) /usr/lib64/libssl.so.1.0.1
Upon removal of vmware-workstation, A is garbage collected and deleted,
if it's a symlink, the link is followed and (B) is removed as well, without
validating the action (i.e. checking if a package is still owning (B).)
This commit fixes this. See Sabayon bug #5812.
Traceback (most recent call last):
File "/usr/lib64/python3.4/base64.py", line 519, in _input_type_check
m = memoryview(s)
TypeError: memoryview: str object does not have the buffer interface
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/bin/equo", line 15, in <module>
main()
File "/usr/lib/entropy/client/solo/main.py", line 332, in main
exit_st = func(*func_args)
File "/usr/lib/entropy/client/solo/commands/command.py", line 406, in _call_exclusive
return func(client)
File "/usr/lib/entropy/client/solo/commands/update.py", line 115, in _update
rc = self._normal_update(entropy_client)
File "/usr/lib/entropy/client/solo/commands/update.py", line 236, in _normal_update
rc = repo_intf.sync()
File "/usr/lib/entropy/lib/entropy/client/interfaces/repository.py", line 327, in sync
rc = self._run_sync()
File "/usr/lib/entropy/lib/entropy/client/interfaces/repository.py", line 146, in _run_sync
self._entropy, repo, self.force, self._gpg_feature)
File "/usr/lib/entropy/lib/entropy/client/interfaces/db.py", line 2764, in update
return updater.update()
File "/usr/lib/entropy/lib/entropy/client/interfaces/db.py", line 2029, in update
selected = self._select_database_mirror()
File "/usr/lib/entropy/lib/entropy/client/interfaces/db.py", line 428, in _select_database_mirror
https_validate_cert = https_validate_cert)
File "/usr/lib/entropy/lib/entropy/client/interfaces/db.py", line 1992, in _remote_revision
fetch_rc = fetcher.download()
File "/usr/lib/entropy/lib/entropy/fetchers.py", line 391, in download
status = downloader()
File "/usr/lib/entropy/lib/entropy/fetchers.py", line 600, in _urllib_download
self.__http_basic_user, self.__http_basic_pwd)).replace('\n', '')
File "/usr/lib64/python3.4/base64.py", line 548, in encodestring
return encodebytes(s)
File "/usr/lib64/python3.4/base64.py", line 536, in encodebytes
_input_type_check(s)
File "/usr/lib64/python3.4/base64.py", line 522, in _input_type_check
raise TypeError(msg) from err
TypeError: expected bytes-like object, not str
