Commit Graph

711 Commits

Author SHA1 Message Date
Fabio Erculiani
831237b7fc [entropy.server] insert the package SHA1 checksum into the file name
This commit partially reverts commit d6b7a21314.
Package revision is no longer incremented across package moves.
This has two advantages:
  - much less traffic generated on the mirror infrastructure
  - less updates for sabayon-limbo users when packages are moved to main

Bumping the revision was required by sabayon-weekly, which had the problem
that some package files were replaced during normal activity on
sabayonlinux.org and sabayon-limbo on the mirror. This generated checksum
errors, thus adding the checksum in the package file name ensures that
Entropy Server will never overwrite package file names unless the checksum
also matches.
Having a SHA1 checksum in the file name is also good for security, and
we may even want to create a SHA1 from the GPG signature in future.
2013-03-20 15:21:31 +00:00
Fabio Erculiani
f0fad25ca5 Release Entropy 185 2013-03-19 15:27:33 +00:00
Fabio Erculiani
e601bc1144 [entropy.client.dep] fix or dependencies selection when none is installed 2013-03-19 15:27:00 +00:00
Fabio Erculiani
af55cef3be [entropy.db.sql] wrap the whole tuple() and frozenset() generation 2013-03-19 15:02:23 +00:00
Fabio Erculiani
bc9e6e2146 [entropy.db.sql] wrap _cur2tuple and _cur2frozenset functions using SQLCursorWrapper 2013-03-19 14:57:03 +00:00
Fabio Erculiani
f0f3883a86 Release Entropy 184 2013-03-17 08:27:56 +00:00
Fabio Erculiani
bf766fe571 Release Entropy 183 2013-03-13 12:25:46 +00:00
Fabio Erculiani
c360738b95 [kswitch] fix install path, resolve bug 4073 2013-03-13 12:03:39 +00:00
Fabio Erculiani
b36a8669d4 [conf] drop the "remote web services server url" part from repository "repo =" param
This has been deprecated for long time.
2013-03-12 21:10:17 +00:00
Fabio Erculiani
d3fab6937c Release Entropy 182 2013-03-12 14:13:56 +00:00
Fabio Erculiani
f02283523d [entropy.db,entropy.spm] rewrite SPM package metadata update code
This commit moves all the SPM package metadata update logic, including
pkgmove and slotmove to entropy.spm plugins.
This makes possible to handle the event more reliably. In particular,
Entropy now controls what Portage does, basing on the repository treeupdates
information.
The scenario in where the Portage tree was not updated and the called
"emaint --fix moveinst" did not do what it was supposed to do is now
handled correctly.
2013-03-12 13:55:01 +00:00
Fabio Erculiani
6804e79b37 [entropy.client.interfaces.repository] drop is_root(), this is always running as root 2013-03-12 13:53:12 +00:00
Fabio Erculiani
9b81edfd18 [entropy.core] RepositoryConfigParser: always catch KeyError when raised 2013-03-10 22:38:14 +00:00
Fabio Erculiani
ddf1f9ce3c [entropy.core] RepositoryConfigParser: respect repository definition order 2013-03-10 22:37:49 +00:00
Fabio Erculiani
bc63d82b97 [entropy.server] add support for server-side repositories.conf.d/ repository config files
Entropy Server now supports repositories defined inside
    /etc/entropy/repositories.conf.d/ files, written using the
    syntax detailed below. This improves the ability to enable, disable,
    add and remove repositories programmatically. Furthermore, it
    makes possible to extend the supported parameters without breaking
    backward compatibility.

    In order to differentiate Entropy Client repository definitions between
    Entropy Server ones, each repository section must start with "[server=".

    This is an example of the syntax (with a complete listing
    of the supported arguments):

    [server=sabayon-limbo]
    desc = Sabayon Linux Official Testing Repository
    repo = ssh://username@full.host:~username/sabayon-limbo
    enabled = <true/false>

    [server=sabayon-limbo]
    desc = This statement will be ignored.
    repo-only = ssh://username@repo.host:~username/sabayon-limbo
    pkg-only = ssh://username@pkg.host:~username/sabayon-limbo

    [server=sabayon-base]
    desc = This is the base repository.
    repo-only = ssh://username@repo.host:~username/sabayon-base
    pkg-only = ssh://username@pkg.host:~username/sabayon-base
    base = <true/false>

    As you can see, multiple statements for the same repository
    are allowed. However, only the first desc = statement will be
    considered, while there can be as many {pkg,repo}* = as you want.

    The repository order is important, but this is guaranteed by the
    fact that configuration files are parsed in lexical order.

    Statements description:
    - "desc": stands for description, the repository name description.
    - "repo": the push & pull URI, for both packages and repository database.
    - "repo-only": same as repo, but only for the repository database
                   push & pull.
    - "pkg-only": same as repo, but only for the packages push & pull.
             The supported protocols are those supported by entropy.fetchers.
    - "enabled": if set, its value can be either "true" or "false". The default
                 value is "true". It indicates if a repository is configured
                 but currently disabled or enabled. Please take into account
                 that config files in /etc/entropy/repositories.conf.d/ starting
                 with "_" are considered to contain disabled repositories. This
                 is just provided for convienence.
    - "base": if set, its value can be either "true" or "false". The default
              value is "false". If no repository has the flag set, the first
              listed repository will be the base one. Only the first repository
              with "base = true" will be considered. The base repository is the
              repository that is considered base for all the others
              (the main one).
2013-03-10 22:12:27 +00:00
Fabio Erculiani
81baa1d58e [entropy.core] BaseConfigParser: store section parse order (and properly rename variables) 2013-03-10 18:25:56 +00:00
Fabio Erculiani
6e96b38a7e [entropy.core] properly pass encoding= to RepositoryConfigParser 2013-03-10 00:08:47 +00:00
Fabio Erculiani
05f446d51a [entropy.client] add .ini like repositories.conf.d/ files support
This is backward compatible with the previous repository
    implementation, in the sense that old repository syntax
    is just ignored. However, a mix of old and new statements
    may result in an undefined behaviour.

    This is an example of the new syntax (with a complete listing
    of the supported arguments):

    [sabayon-limbo]
    desc = Sabayon Linux Official Testing Repository
    repo = http://pkg.sabayon.org,sabayon.org
    pkg = http://pkg.sabayon.org
    pkg = http://dl.sabayon.org/entropy
    enabled = <true/false>

    [sabayon-limbo]
    desc = This statement will be ignored.
    repo = This url will be ignored.
    pkg = http://some.more.mirror.org/entropy
    pkg = http://some.more.mirror.net/entropy

    As you can see, multiple statements for the same repository
    are allowed. However, only the first desc = and repo =
    statements will be considered, while there can be as many pkg =
    as you want.

    Statements description:
    - "desc": stands for description, the repository name description.
    - "repo": the repository database URL string, plus other parameters
              as supported in the previous configuration file syntax:
              <db url prefix>[,<remote web services server url][#<compression>].
    - "pkg": the repository packages URL string. This must be a valid URL.
             The supported protocols are those supported by entropy.fetchers.
    - "enabled": if set, its value can be either "true" or "false". The default
                 value is "true". It indicates if a repository is configured
                 but currently disabled or enabled. Please take into account
                 that config files in /etc/entropy/repositories.conf.d/ starting
                 with "_" are considered to contain disabled repositories. This
                 is just provided for convienence.
2013-03-09 23:51:58 +00:00
Fabio Erculiani
1b18a831f1 Release Entropy 181 2013-03-08 13:42:32 +01:00
Fabio Erculiani
b44c89ad19 [kernel-switcher] turn the kernel switcher library into an OO interface 2013-03-06 15:51:35 +00:00
Fabio Erculiani
d05426e618 [kernel-switcher] move kswitch to entropy lib, kernel-switcher back to equo 2013-03-06 15:28:56 +00:00
Fabio Erculiani
502d9c8b9e [entropy.client.package] add support for package.splitdebug.mask, see bug 3707 2013-03-05 15:52:53 +00:00
Fabio Erculiani
5cb3939718 [entropy.client] drop package.satisfied support, see bug #3879
It never worked as users expected and can lead to misinterpretation.
It will be reimplemented in future through something similar to package.provided
2013-03-05 15:09:44 +00:00
Fabio Erculiani
84b0b92c4e [entropy.client.dep] rewrite simple or dependencies resolution
Selected matches was not respected by the simple or dependencies
resolution code. This commit addresses the problem and fixes the
virtual/glu dependendency handling.
2013-03-05 13:20:28 +00:00
Fabio Erculiani
7573e0e1d5 Release Entropy 180 2013-03-03 01:08:10 +00:00
Fabio Erculiani
b905b98525 [entropy.server] _calculate_sync_queues: ignore remote package file if a local .weak file exists 2013-03-03 01:06:31 +00:00
Fabio Erculiani
873ae5267b [entropy.server] _calculate_sync_queues: ignore local package file if .weak file exists 2013-03-03 01:03:52 +00:00
Fabio Erculiani
46577fe93f [entropy.server] _calculate_local_package_files: properly handle .weak files 2013-03-03 00:54:01 +00:00
Fabio Erculiani
3a49a8cc47 [entropy.server] _calculate_local_package_files: take into consideration .weak files 2013-03-03 00:39:54 +00:00
Fabio Erculiani
55110bf230 [entropy.server] _show_sync_queues: inform about package download as well 2013-03-03 00:27:22 +00:00
Fabio Erculiani
552698d2d6 [entropy.server] tidy_mirrors: fix weak file detection logic 2013-03-03 00:18:51 +00:00
Fabio Erculiani
9f0bdb959c [entropy.server] tidy_mirrors: do not show packages as weak if they've already marked 2013-03-03 00:12:06 +00:00
Fabio Erculiani
be758fae7b [entropy.server] tidy_mirrors: do not show packages as expired if they've already marked 2013-03-03 00:03:40 +00:00
Fabio Erculiani
843c9ad6f8 [entropy.server] s/extra_package_rel/package_rel/ in tidy_mirrors() 2013-03-02 23:54:46 +00:00
Fabio Erculiani
5801abf88d [entropy.server] fix typos in tidy_mirrors() 2013-03-02 23:50:08 +00:00
Fabio Erculiani
ee38930096 Release Entropy 179 2013-03-02 18:58:16 +00:00
Fabio Erculiani
7bef53d99c [entropy.server] add "weak package files" support
Weak package files is a feature that makes possible to save a lot of
local space if enabled. Basically, all the package files no longer in
the repository are locally removed before their expiration. This reduces
local data redundancy with no real reliability impact.
2013-03-02 18:54:12 +00:00
Fabio Erculiani
17d19458a7 [entropy.client.dep] use set.clear and dict.clear instead of del 2013-02-28 13:48:30 +00:00
Fabio Erculiani
d996460b30 [entropy.client.dep,solo] add support for --onlydeps, as per bug #3817
equo {install,source,download} now support --onlydeps.
Using --onlydeps, only dependencies of the selected packages are installed,
unless the selected packages are dependencies themselves.
2013-02-28 13:47:50 +00:00
Fabio Erculiani
5d315c70bd [entropy.const] drop /var/run/entropy/entropy.lock file and directory handling
This is no longer used by Entropy, which does use a more fine grained locking
instead.
2013-02-24 23:34:49 +01:00
Fabio Erculiani
7771d641e2 [entropy.client] extend Client.calculate_updates() to return critical updates info
calculate_updates() now returns a dictionary containing the same pieces of
information available before. In addition, it now also returns if critical
updates were found and returned by the method itself.
This is particularly useful for enforcing relaxed deps calculation in case
of critical updates (which is what happens now).

This commit breaks the Entropy Client API.
2013-02-24 13:30:28 +00:00
Fabio Erculiani
1fa5b9acf3 Release Entropy 178 2013-02-21 00:11:14 +01:00
Fabio Erculiani
ac243067f3 [entropy.qa] fix the usage of get_deep_dependency_list 2013-02-21 00:09:47 +01:00
Fabio Erculiani
09adfa92e6 Release Entropy 177 2013-02-20 18:56:39 +01:00
Fabio Erculiani
b3aad3da83 Revert "[entropy.qa] test_missing_dependencies: only consider direct dependencies"
This reverts commit a0037bfe76.
It turned out that only considering direct dependencies is a no-go.
Also see:
http://www.flameeyes.eu/autotools-mythbuster/pkgconfig/dependencies.html
2013-02-20 18:44:02 +01:00
Fabio Erculiani
65e5fcdb68 Release Entropy 176 2013-02-20 13:36:08 +01:00
Fabio Erculiani
dc4f23ef01 [entropy.spm] PortagePlugin: fix xblast CONTENTS mangling when splitdebug=disabled
This is the exception generated by reading CONTENTS expecting valid UTF-8 encoded
text (while it currently is raw_unicode_escape):

stacktrace: Traceback (most recent call last):
  File "/usr/bin/equo", line 15, in <module>
    main()
  File "/usr/lib/entropy/client/solo/main.py", line 333, in main
    exit_st = func(*func_args)
  File "/usr/lib/entropy/client/solo/commands/command.py", line 335, in _call_locked
    return func(client)
  File "/usr/lib/entropy/client/solo/commands/install.py", line 183, in _install
    relaxed, multifetch, packages)
  File "/usr/lib/entropy/client/solo/commands/install.py", line 609, in _install_action
    exit_st = pkg.run(xterm_header=xterm_header)
  File "/usr/lib/entropy/lib/entropy/client/interfaces/package.py", line 4266, in run
    rc = self._stepper(xterm_header)
  File "/usr/lib/entropy/lib/entropy/client/interfaces/package.py", line 4251, in _stepper
    rc = steps_data.get(step)()
  File "/usr/lib/entropy/lib/entropy/client/interfaces/package.py", line 4153, in do_install_spm
    self.pkgmeta['installed_package_id'])
  File "/usr/lib/entropy/lib/entropy/client/interfaces/package.py", line 2262, in _spm_install_package
    spm_uid = Spm.add_installed_package(self.pkgmeta)
  File "/usr/lib/entropy/lib/entropy/spm/plugins/interfaces/portage_plugin/__init__.py", line 3078, in add_installed_package
    splitdebug_dirs)
  File "/usr/lib/entropy/lib/entropy/spm/plugins/interfaces/portage_plugin/__init__.py", line 2859, in __splitdebug_update_contents_file
    line = cont_f.readline()
  File "/usr/lib64/python2.7/codecs.py", line 675, in readline
    return self.reader.readline(size)
  File "/usr/lib64/python2.7/codecs.py", line 530, in readline
    data = self.read(readsize, firstline=True)
  File "/usr/lib64/python2.7/codecs.py", line 477, in read
    newchars, decodedbytes = self.decode(data, self.errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xee in position 30: invalid continuation byte
2013-02-20 13:05:36 +01:00
Fabio Erculiani
9eb56dbc14 [entropy.const] drop several etpConst constants, replace with more OO vars
etpConst['clientdbid'] is kept for backward compatibility, but will be removed soon
While etpConst['serverdbid'] and etpConst['clientserverrepoid'] are gone.
2013-02-20 00:56:52 +00:00
Fabio Erculiani
3695655e66 [entorpy.client] fix Logger.log() usage in entropy.client.interfaces.package 2013-02-17 19:04:20 +00:00
Fabio Erculiani
e66ec77a67 [entropy.qa] warn_missing_dependencies: stop using get_deep_dependency_list, too slow and unwanted 2013-02-10 19:01:35 +00:00