[createrepo] using custom image for eit usage

README.md

@@ -8,8 +8,6 @@ You can install it in sabayon with:
 sudo equo i sabayon-devkit
 `
 
-# Building package in a clean environment
-
 ## Prerequisites
 
 * docker installed in the machine (`sudo equo i docker`), and the daemon started (`sudo systemctl start docker`)
@@ -17,7 +15,7 @@ sudo equo i sabayon-devkit
 
 Packages can be built in a clean environment with docker by running `sabayon-buildpackages`.
 
-*`sabayon-buildpackages` is just a script wrapper around the `builder` perl script which is placed inside a docker container.*
+*`sabayon-buildpackages` and ``sabayon-createrepo are just a script wrapper around the development docker container.*
 
 ## Define your workspace
 
@@ -43,56 +41,62 @@ If you want, you can define the workspace directory with the variable environmen
 
 `SAB_WORKSPACE=/whatever sabayon-createrepo`
 
-## Build your packages
-
-`sabayon-buildpackages` accepts the same arguments as the builder:
-
-
-    sabayon-buildpackages app-text/tree
-    sabayon-buildpackages plasma-meta --layman kde
-    sabayon-buildpackages app-foo/foobar --equo foo-misc/foobar --layman foo --layman bar foo
-
-
-* --layman foobar -- tells the script to add the "foobar" overlay from layman 
-* --equo foo-misc/foobar -- tells the script to install "foo-misc/foobar" before compiling
-* The arguments are the packages that you want compile, they can be also in the complete form *e.g. =foo-bar/misc-1.2*
-
 
 ## Folder structure
 
-This is the folder structure that will be automatically created:
+This is the folder structure of your workspace by default, but you can tweak part of it to your tastes with Environment variables:
 
 `
 
-    myproject/ 
+    myproject/
     myproject/portage_artifacts/
     myproject/entropy_artifacts/
     myproject/local_overlay/
     myproject/specs/
 `
 
-* myproject/portage\_artifacts/ -- will contain the portage artifacts, they will be consumed in the next steps
-* myproject/entropy\_artifacts/ -- will contain the entropy output, our Sabayon repository for the **.tbz2** packages that where in **myproject/portage_artifacts/**
+* myproject/portage\_artifacts/ -- Created when `sabayon-buildpackages` is started. It contains the portage artifacts, they will be consumed in the next steps
+* myproject/entropy\_artifacts/ -- Created when `sabayon-createrepo` is started. It contains the entropy repository files.
 * myproject/local_overlay/ -- is the location of your personal overlay (if necessary)
-* myproject/specs -- contain custom files for uses, envs, masks, unmasks and keywords for package compilation options
+* myproject/specs -- Create it to customize the building process. It can contain custom files for make.conf, uses, envs, masks, unmasks and keywords for package compilation options
 
-the `specs` folder is structured like this:
+the `specs` folder is structured like this and it's merely optional.
 
-- custom.unmask: will contain your custom unmasks
-- custom.mask: will contain your custom masks
-- custom.use: will contain your custom use flags
-- custom.env: will contain your custom env specifications
-- custom.keywords: will contain your custom keywords
+as long as you create those files they are used:
+
+- custom.unmask: that's the place for custom unmasks
+- custom.mask:  contain your custom masks
+- custom.use:  contain your custom use flags
+- custom.env:  contain your custom env specifications
+- custom.keywords: contain your custom keywords
+- make.conf:  it will replace the make.conf on the container with yours.
 
 you can override the Architecture folder in which files are placed specifying in the *SAB_ARCH* environment variable. Default is "intel" (can be *armarch* as for now)
 
+**Note: the portage_artifacts can also contain tbz2 files generated with other methods, if you already have your desired packages already compiled, you can just use `sabayon-createrepo`**
+
+# Building package in a clean environment
+
+`sabayon-buildpackages` accepts the same arguments as the builder:
+
+
+    sabayon-buildpackages app-text/tree
+    sabayon-buildpackages plasma-meta --layman kde
+    DOCKER_PULL_IMAGE=1 sabayon-buildpackages app-foo/foobar --equo foo-misc/foobar --layman foo --layman bar foo
+
+
+* --layman foobar -- tells the script to add the "foobar" overlay from layman
+* --equo foo-misc/foobar -- tells the script to install "foo-misc/foobar" before compiling
+
+ Environment variables:
+- DOCKER_PULL_IMAGE -- tells the script to update the docker image before compiling, enable it with 1, disable with 0
+- OUTPUT_DIR -- optional, default to "portage_artifacts" in your current working directory, it is the path where emerge generated tbz2 are stored (absolute path)
+- LOCAL_OVERLAY -- optional, you can specify the path to your local overlay (absolute path)
+ The arguments are the packages that you want compile, they can be also in the complete form *e.g. =foo-bar/misc-1.2*
+
+
 # Create Sabayon repository from \*.tbz2 in a clean environment
 
-## Prerequisites
-
-* docker installed in the machine (`sudo equo i docker`), and the daemon started (`sudo systemctl start docker`)
-* if you don't want to run that as root, the user where are you running the script must be in the docker group (`sudo gpasswd -a $USER docker`)
-
 You can create Sabayon repositories from packages built with emerge in a clean environment with docker by running `sabayon-createrepo`.
 
 The script will use a docker container to inject packages from  *portage_artifacts/*  in your project folder, the output will be available in *entropy_artifacts/*.
@@ -102,9 +106,11 @@ Example:
     sabayon-createrepo
 
     REPOSITORY_NAME=mytest REPOSITORY_DESCRIPTION="My Wonderful Repository" sabayon-createrepo
-    
-* REPOSITORY_NAME -- is your repository name id
-* REPOSITORY_DESCRIPTION -- is your repository description
+
+* REPOSITORY_NAME -- optional, is your repository name id
+* REPOSITORY_DESCRIPTION -- optional, is your repository description
+* PORTAGE_ARTIFACTS -- optional if you use the tools in the same dir, you can specify where portage artifacts (\*.tbz2 files) are (absolute path required)
+* OUTPUT_DIR -- optional, you can specify where the entropy repository will be stored
 
 You can also put your .tbz2 file externally built inside `entropy_artifacts/` in your workspace folder (you can create it if not already present)  and run `sabayon-createrepo` to generate a repository from them.
 

builder

@@ -16,6 +16,8 @@ my $equo_install_version = $ENV{EQUO_INSTALL_VERSION} // 0;
 my $equo_split_install   = $ENV{EQUO_SPLIT_INSTALL}   // 0;
 my $artifacts_folder     = $ENV{ARTIFACTS_DIR};
 
+my $make_conf = $ENV{MAKE_CONF};
+
 my @overlays;
 
 GetOptions( 'layman|overlay:s{,}' => \@overlays, 'equo|install:s{,}' => \@equo_install  );
@@ -163,6 +165,9 @@ qx{ls /usr/portage/licenses -1 | xargs -0 > /etc/entropy/packages/license.accept
   ;    #HAHA
 system("equo repo mirrorsort sabayonlinux.org;equo up && equo u")
   if $use_equo;    # Better don't be behind
+
+system("cp -rf $make_conf /etc/portage/make.conf") if $make_conf;
+
 qx|echo 'ACCEPT_LICENSE="*"' >> /etc/portage/make.conf|;    #just plain evil
 
 my @packages = @ARGV;
@@ -170,12 +175,12 @@ my @packages = @ARGV;
 if ($use_equo) {
     my @packages_deps;
     foreach my $p (@packages) {
-      say "[$p] Getting the package dependencies which aren't already installed on the system.. ''";
+      say "[$p] Getting the package dependencies which aren't already installed on the system.. ";
         push( @packages_deps, calculate_missing( $p , 2) )
           if $equo_install_atoms;
         push( @packages_deps, package_deps( $p, 2, 0 ) )
           if $equo_install_version;
-      say "[$p] Done ''"
+      say "[$p] Done"
     }
     @packages_deps = grep { defined() and length() } @packages_deps;   #cleaning
     say "", "[install] Installing missing dependencies with equo", @packages_deps, "";

sabayon-buildpackages

@@ -3,6 +3,11 @@ set -e
 SAB_WORKSPACE="${SAB_WORKSPACE:-$PWD}"
 DOCKER_IMAGE="${DOCKER_IMAGE:-sabayon/builder-amd64}"
 SAB_ARCH="${SAB_ARCH:-intel}"
+DOCKER_PULL_IMAGE="${DOCKER_PULL_IMAGE:-0}"
+MAKE_CONF="${MAKE_CONF:-$SAB_WORKSPACE/specs/make.conf}"
+OUTPUT_DIR="${OUTPUT_DIR:-$SAB_WORKSPACE/portage_artifacts/}"
+LOCAL_OVERLAY="${LOCAL_OVERLAY:-$SAB_WORKSPACE/local_overlay}"
+
 . /sbin/sabayondevkit-functions.sh
 
 if [ $# -eq 0 ]
@@ -11,21 +16,42 @@ if [ $# -eq 0 ]
    exit 1
 fi
 
+docker_env=(-e "LC_ALL=en_US.UTF-8")
+docker_volumes=( -v "$OUTPUT_DIR:/usr/portage/packages" )
+
+[ -z "$BUILDER_PROFILE" ] || docker_env+=(-e "BUILDER_PROFILE=$BUILDER_PROFILE")
+[ -z "$BUILDER_JOBS" ] || docker_env+=(-e "BUILDER_JOBS=$BUILDER_JOBS")
+[ -z "$USE_EQUO" ] || docker_env+=(-e "USE_EQUO=$USE_EQUO")
+[ -z "$PRESERVED_REBUILD" ] || docker_env+=(-e "PRESERVED_REBUILD=$PRESERVED_REBUILD")
+[ -z "$EQUO_INSTALL_ATOMS" ] || docker_env+=(-e "EQUO_INSTALL_ATOMS=$EQUO_INSTALL_ATOMS")
+[ -z "$EMERGE_DEFAULTS_ARGS" ] || docker_env+=(-e "EMERGE_DEFAULTS_ARGS=$EMERGE_DEFAULTS_ARGS")
+[ -z "$EQUO_INSTALL_VERSION" ] || docker_env+=(-e "EQUO_INSTALL_VERSION=$EQUO_INSTALL_VERSION")
+[ -z "$EQUO_SPLIT_INSTALL" ] || docker_env+=(-e "EQUO_SPLIT_INSTALL=$EQUO_SPLIT_INSTALL")
+[ -z "$ARTIFACTS_DIR" ] || docker_env+=(-e "ARTIFACTS_DIR=$ARTIFACTS_DIR")
+[ -d "$LOCAL_OVERLAY" ] && docker_volumes+=(-v "$LOCAL_OVERLAY:/usr/local/portage")
+
+if [ -e $MAKE_CONF ]; then
+  echo "You specified a make.conf. I hope you know what are you doing"
+  docker_volumes+=(-v "$MAKE_CONF:/etc/portage/make.conf.custom") # file make.conf.custom is in the docker container
+  docker_env+=(-e "MAKE_CONF=/etc/portage/make.conf.custom") # tell to the builder where is our custom make.conf
+fi
+
 check_docker_requirements
 
-[ -d "$SAB_WORKSPACE"/specs ] || mkdir "$SAB_WORKSPACE"/specs/
-[ -e "$SAB_WORKSPACE"/specs/custom.mask ] || touch "$SAB_WORKSPACE"/specs/custom.mask
-[ -e "$SAB_WORKSPACE"/specs/custom.unmask ] || touch "$SAB_WORKSPACE"/specs/custom.unmask
-[ -e "$SAB_WORKSPACE"/specs/custom.use ] || touch "$SAB_WORKSPACE"/specs/custom.use
-[ -e "$SAB_WORKSPACE"/specs/custom.env ] || touch "$SAB_WORKSPACE"/specs/custom.env
-[ -e "$SAB_WORKSPACE"/specs/custom.keywords ] || touch "$SAB_WORKSPACE"/specs/custom.keywords
+[ $DOCKER_PULL_IMAGE -eq 1 ] && docker pull $DOCKER_IMAGE
 
-docker run --rm \
-	-v "$SAB_WORKSPACE"/local_overlay:/usr/local/portage \
-	-v "$SAB_WORKSPACE"/portage_artifacts:/usr/portage/packages \
-	-v "$SAB_WORKSPACE"/specs/custom.unmask:/opt/sabayon-build/conf/"$SAB_ARCH"/portage/package.unmask/custom.unmask \
-	-v "$SAB_WORKSPACE"/specs/custom.mask:/opt/sabayon-build/conf/"$SAB_ARCH"/portage/package.mask/custom.mask \
-	-v "$SAB_WORKSPACE"/specs/custom.use:/opt/sabayon-build/conf/"$SAB_ARCH"/portage/package.use/custom.use \
-	-v "$SAB_WORKSPACE"/specs/custom.env:/opt/sabayon-build/conf/"$SAB_ARCH"/portage/package.env/custom.env \
-	-v "$SAB_WORKSPACE"/specs/custom.keywords:/opt/sabayon-build/conf/"$SAB_ARCH"/portage/package.keywords/custom.keywords \
-	-ti $DOCKER_IMAGE $@
+if [ -d "$SAB_WORKSPACE"/specs ]; then
+
+  [ -e "$SAB_WORKSPACE"/specs/custom.mask ] && docker_volumes+=(-v "$SAB_WORKSPACE/specs/custom.mask:/opt/sabayon-build/conf/$SAB_ARCH/portage/package.mask/99-custom.mask")
+  [ -e "$SAB_WORKSPACE"/specs/custom.unmask ] && docker_volumes+=(-v "$SAB_WORKSPACE/specs/custom.unmask:/opt/sabayon-build/conf/$SAB_ARCH/portage/package.unmask/99-custom.unmask")
+  [ -e "$SAB_WORKSPACE"/specs/custom.use ] && docker_volumes+=(-v "$SAB_WORKSPACE/specs/custom.use:/opt/sabayon-build/conf/$SAB_ARCH/portage/package.use/99-custom.use")
+  [ -e "$SAB_WORKSPACE"/specs/custom.env ] && docker_volumes+=(-v "$SAB_WORKSPACE/specs/custom.env:/opt/sabayon-build/conf/$SAB_ARCH/portage/package.env/99-custom.env")
+  [ -e "$SAB_WORKSPACE"/specs/custom.keywords ] && docker_volumes+=(-v "$SAB_WORKSPACE/specs/custom.keywords:/opt/sabayon-build/conf/$SAB_ARCH/portage/package.keywords/99-custom.keywords")
+
+fi
+
+echo "Spawning the package builder container for '$@'."
+echo ""
+echo "docker run --rm ${docker_env[@]} ${docker_volumes[@]} -ti $DOCKER_IMAGE $@"
+
+docker run --rm "${docker_env[@]}" "${docker_volumes[@]}"	-ti $DOCKER_IMAGE $@

sabayon-createrepo

@@ -7,9 +7,31 @@ entropysrv=$(mktemp)
 createrepo=$(mktemp)
 REPOSITORY_NAME="${REPOSITORY_NAME:-default}"
 REPOSITORY_DESCRIPTION="${REPOSITORY_DESCRIPTION:-My Sabayon repository}"
-DOCKER_IMAGE="${DOCKER_IMAGE:-sabayon/builder-amd64}"
+DOCKER_IMAGE="${DOCKER_IMAGE:-sabayon/eit-amd64}"
+PORTAGE_ARTIFACTS="${PORTAGE_ARTIFACTS:-$SAB_WORKSPACE/portage_artifacts}"
+OUTPUT_DIR="${OUTPUT_DIR:-$SAB_WORKSPACE/entropy_artifacts}"
+
+args=("$@")
 EDITOR=cat
 
+docker_env=(-e "EDITOR=$EDITOR"
+            -e "REPOSITORY=$REPOSITORY_NAME"
+            -e "LC_ALL=en_US.UTF-8")
+
+docker_volumes=(
+            -v "$OUTPUT_DIR:/sabayon/artifacts"
+            -v "$createrepo:/sabayon/bin/create_repo.sh"
+            -v "$entropysrv:/etc/entropy/server.conf"
+)
+
+if [ $# -eq 0 ]
+then
+ docker_volumes+=(-v "$PORTAGE_ARTIFACTS:/usr/portage/packages" )
+ else
+ echo "Packages directory: ${args[0]}"
+ docker_volumes+=(-v "${args[0]}:/usr/portage/packages" )
+fi
+
 check_docker_requirements
 
 echo "Repository: $REPOSITORY_NAME"
@@ -22,11 +44,9 @@ cat >$createrepo <<EOF
 set -e
 built_pkgs=\$(find /usr/portage/packages -name "*.tbz2" | xargs)
 repo="\${REPOSITORY:-default}"
-
-equo i entropy-server
 mkdir -p /sabayon/artifacts
 
-[[ -z "\${built_pkgs}" ]] && echo "ouch no pkgs found" && exit 2
+[[ -z "\${built_pkgs}" ]] && echo "ERROR: no tbz2s found" && exit 2
 
 if [ -d "/sabayon/artifacts/standard" ]; then
   eit unlock \$repo || true
@@ -68,11 +88,27 @@ default-repository = $REPOSITORY_NAME
 repository=$REPOSITORY_NAME|$REPOSITORY_DESCRIPTION|file:///sabayon/artifacts
 EOF
 
-docker run --rm --entrypoint /bin/bash -e REPOSITORY=$REPOSITORY_NAME -e EDITOR=cat -e LC_ALL=en_US.UTF-8 \
-		-v "$SAB_WORKSPACE"/entropy_artifacts:/sabayon/artifacts \
-		-v $createrepo:/sabayon/bin/create_repo.sh \
-		-v $entropysrv:/etc/entropy/server.conf \
-		-v "$SAB_WORKSPACE"/portage_artifacts:/usr/portage/packages \
-		-ti $DOCKER_IMAGE /sabayon/bin/create_repo.sh || true
+echo "docker run --rm --entrypoint /bin/bash ${docker_env[@]} ${docker_volumes[@]} -ti $DOCKER_IMAGE /sabayon/bin/create_repo.sh"
+
+docker run --rm --entrypoint /bin/bash "${docker_env[@]}" "${docker_volumes[@]}" -ti $DOCKER_IMAGE /sabayon/bin/create_repo.sh || true
+
+if [ -d "$SAB_WORKSPACE/entropy_artifacts/standard" ]; then
+  echo "The Sabayon repository files are in $SAB_WORKSPACE/entropy_artifacts"
+  echo "Now you can upload its content where you want"
+
+  echo ""
+  echo "Here it is the repository file how will look like "
+  echo "(if you plan to upload it to a webserver, modify the URI accordingly)"
+  echo "
+  [$REPOSITORY_NAME]
+  desc = $REPOSITORY_DESCRIPTION
+  repo=file://$SAB_WORKSPACE/entropy_artifacts#bz2
+  enabled = true
+  pkg = file://$SAB_WORKSPACE/entropy_artifacts
+  "
+else
+  echo "Something failed :("
+fi
+
 rm -rf $createrepo
 rm -rf $entropysrv

sabayondevkit-functions.sh

@@ -4,7 +4,7 @@ check_docker_requirements(){
   if [ "$(id -u)" != "0" ]; then
     groups | grep -q docker || echo "--> If you are not running the script as root, your user should be in the docker group to use it. (sudo gpasswd -a $USER docker)"
   fi
-  ps aux | grep -q '[d]ocker' || echo "--> Be sure to have the docker daemon running (sudo systemctl start docker)"
+  ps aux | grep -q '[d]ocker' || echo "--> Be sure to have the docker daemon running (sudo systemctl start docker) of configure it to run on boot (sudo systemctl enable docker). Trying to start it anyway" && sudo systemctl start docker
 }
 
 build() {