[entropy.client] fix conflicts removal handling during upgrade.

In certain extreme situations, that are more likely if the install
state is old (e.g. 9 months +), Entropy may decide to remove packages
due to conflicts with the version being installed and shoot itself
in the foot, if this is the case of dev-lang/python :-).

So, this commit tries to avoid the scenario by filtering out package
removals for those that are being installed anyway as part of the
transaction (e.g. a system upgrade).

This should fix the very specific case, but increase uncertainty
and thus entropy in other cases where once a package that lists
conflicts is installed, it may not expect to find that package
installed. However, this case should be handled by the dependency
graph generator (and it currently does not), which would need to
reorder the execution of the queue to comply with conflicts.

Perhaps, someone in the future will improve that too, for now,
let's enjoy one more snowflake.

Current improvement:
- foo-1 and bar-2 are set to be installed
- foo-1 conflicts with <bar-2
- bar-1 is installed
- foo-1 wanted to remove bar-1 as part of foo-1 install transaction.
  (now fixed, it won't...)
- bar-2 is then installed

Better improvement:
- foo-1 and bar-2 are set to be installed
- foo-1 conflicts with <bar-2 (but bar-2 itself can be installed
  and does not conflict with foo-1)
- bar-2 is set to be installed before foo-1
- when entropy gets to foo-1 there is nothing to remove

client/solo/commands/install.py

@@ -606,11 +606,16 @@ Install or update packages or package files.
             # state.
             notif_acquired = notification_lock.try_acquire_shared()
 
-            for count, pkg_match in enumerate(run_queue, 1):
+            metaopts = {
+                'removeconfig': config_files,
+                # This can be used by PackageAction based classes
+                # to know what's the overall package schedule for
+                # both upgrade and install actions. This way, we
+                # can better handle conflicts.
+                'install_queue' : run_queue,
+            }
 
-                metaopts = {
-                    'removeconfig': config_files,
-                }
+            for count, pkg_match in enumerate(run_queue, 1):
 
                 if onlydeps:
                     metaopts['install_source'] = \

lib/entropy/client/interfaces/package/actions/install.py

@@ -328,6 +328,55 @@ class _PackageInstallAction(_PackageInstallRemoveAction):
                 package_ids]
 
             if not package_ids:
+                const_debug_write(
+                    __name__, "_remove_conflict_phase: no package_ids, no need to filter.")
+                return 0
+
+            # Now we have a list of packages that should be removed, but
+            # we may want to check whether a new version of such package
+            # would not cause the removal to begin with.
+            # We have two options:
+            # 1. we are already planning to install the package, and perhaps
+            # we should just let this happen.
+            # 2. we are not planning to install the package, and perhaps
+            # we should.
+            install_queue = self._opts.get('install_queue', [])
+            if not install_queue:
+                const_debug_write(
+                    __name__, "_remove_conflict_phase: empty install_queue.")
+
+            # This is O(nm) [m = number of packages to install that could get us here]
+            # But the chance to get here is low (basically m is low), the cardinality
+            # of n is within 1-2000 worst case, typical case for a 12 months old system
+            # is around 700-1000 (according to my experience). Plus, this data is
+            # super fast to retrieve (retrieveKeySlotTag is heavily cached, see
+            # entropy/db/sqlite.py.) So, as long as this is not a real problem, there is
+            # no need for further optimizations.
+            key_slot_tags = set()
+            for pkg_id, repo_id in install_queue:
+                repo = self._entropy.open_repository(repo_id)
+                key_slot_tag = repo.retrieveKeySlotTag(pkg_id)
+                if key_slot_tag:
+                    key_slot_tags.add(key_slot_tag)
+
+            filtered = set()
+            for package_id in package_ids:
+                inst_key_slot_tag = inst_repo.retrieveKeySlotTag(package_id)
+                if inst_key_slot_tag in key_slot_tags:
+                    const_debug_write(
+                        __name__,
+                        "_remove_conflict_phase: %s is being installed, skipping." % (
+                            inst_key_slot_tag,))
+                    filtered.add(package_id)
+
+            key_slot_tags.clear()  # help Python gc, FWIW.
+            package_ids = [x for x in package_ids if x not in filtered]
+
+            if not package_ids:
+                const_debug_write(
+                    __name__,
+                    "_remove_conflict_phase: no package_ids left after "
+                    "install_queue filtering.")
                 return 0
 
         # make sure to run this without locks, or deadlock happenz

rigo/RigoDaemon/app.py

@@ -2544,6 +2544,14 @@ class RigoDaemonService(dbus.service.Object):
 
         action_factory = self._entropy.PackageActionFactory()
 
+        metaopts = {
+            # This can be used by PackageAction based classes
+            # to know what's the overall package schedule for
+            # both upgrade and install actions. This way, we
+            # can better handle conflicts.
+            'install_queue' : install_queue,
+        }
+
         try:
             for pkg_match in install_queue:
 
@@ -2566,7 +2574,7 @@ class RigoDaemonService(dbus.service.Object):
                 try:
                     pkg = action_factory.get(
                         action_factory.INSTALL_ACTION,
-                        pkg_match)
+                        pkg_match, opts=metaopts)
 
                     msg = "++ %s" % (purple(_("Application Install")),)
                     self._entropy.output(msg, count=(count, total),