Aptly deb repository

Sergey Chulanov included in Aptly Apt Nginx Gpg Openssl Squid Apt-Mirror

2024-11-20 1093 words 6 minutes

Contents

Tested on: Ubuntu 22.04

Install Aptly

Official instruction here:

        
echo "deb [signed-by=/etc/apt/keyrings/aptly.asc] http://repo.aptly.info/ squeeze main" | sudo tee /etc/apt/sources.list.d/aptly.list
sudo mkdir -p /etc/apt/keyrings; sudo chmod 755 /etc/apt/keyrings
sudo wget -O /etc/apt/keyrings/aptly.asc https://www.aptly.info/pubkey.txt
sudo apt update && sudo apt install aptly

Create aptly config:

        
        
        
    
cat <<EOF | sudo tee /etc/aptly.conf
{
  "rootDir": "/opt/aptly",
  "downloadConcurrency": 4,
  "downloadSpeedLimit": 0,
  "architectures": ["amd64"],
  "dependencyFollowSuggests": false,
  "dependencyFollowRecommends": false,
  "dependencyFollowAllVariants": false,
  "dependencyFollowSource": false,
  "dependencyVerboseResolve": false,
  "gpgDisableSign": false,
  "gpgDisableVerify": false,
  "gpgProvider": "gpg",
  "downloadSourcePackages": true,
  "skipLegacyPool": true,
  "ppaDistributorID": "ubuntu",
  "ppaCodename": "",
  "FileSystemPublishEndpoints": {
    "on-premises-repos": {
      "rootDir": "/var/www/aptly",
      "linkMethod": "symlink",
      "verifyMethod": "md5"
    }
  },
  "enableMetricsEndpoint": false
}
EOF

Prepare repos

Create 3 repos: k8s, ceph and shared for ubuntu jammy:

        
sudo aptly repo create -comment="k8s repo" -component="main" -distribution="jammy" kubernetes30
sudo aptly repo create -comment="ceph repo" -component="main" -distribution="jammy" debian-squid
sudo aptly repo create -comment="shared repo" -component="main" -distribution="jammy" shared

Get packages:

        
        
        
    
# Get non-installed package
sudo apt install --download-only nginx

# Get package if it is already installed
sudo apt install --reinstall --download-only nmap curl

# Get package to current directory without dependencies
sudo apt-get download curl nmap

# Get all depends of package
sudo apt install apt-rdepends
package_to_install="nginx"
for pkg in `apt-rdepends ${package_to_install} | grep -v "^ " | sed 's/debconf-2.0/debconf/g'`; do sudo apt-get install --reinstall --download-only --yes "$pkg"; done

# Show downloaded packages
ls /var/cache/apt/archives/

Add packages:

sudo aptly repo add shared curl*.deb nmap*.deb
sudo aptly repo add shared /var/cache/apt/archives/*.deb

List of repos:

sudo aptly repo list
sudo aptly repo show -with-packages shared

Before publishing prepare gpg-key:

        
sudo apt-get install -y rng-tools
sudo rngd -r /dev/urandom
sudo gpg --default-new-key-algo rsa8192 --gen-key --keyring pubring
sudo gpg --list-keys

Publish repos:

bzip2 should be installed before publishing: sudo apt install bzip2

        
sudo aptly publish repo shared filesystem:on-premises-repos:shared
# where `aptly publish repo $1 filesystem:$2:$3`
# $1 repo name, created by aptly
# $2 folder name in a path filesystem (from config: /var/www/aptly)
# $3 subfolder to keep this repo

dists - keeps metadata, pool - keeps deb packages

ls /var/www/aptly/shared/
dists  pool

Show published repos:

sudo aptly publish list

To remove package from repo:

sudo aptly package show nginx
sudo aptly repo remove shared nginx-core

If any package was added or deleted from repo, need update repo:

sudo aptly publish update jammy filesystem:on-premises-repos:shared

In case drop the publication of repo:

sudo aptly publish drop jammy filesystem:on-premises-repos:shared

And, finally, drop the repo:

sudo aptly repo drop shared

Publish using nginx

Install and prepare nginx:

sudo apt install nginx

Configure nginx like that:

        
        
        
    
grep -Ev "^#|^$|#" /etc/nginx/sites-enabled/default
server {
	listen 80 default_server;
	listen [::]:80 default_server;
	root /var/www/aptly;
	index index.html index.htm index.nginx-debian.html;
	server_name proxy.test.lan;
	location / {
		autoindex on;
	}
}

Check and reload nginx:

sudo nginx -t
sudo nginx -s reload

And copy pgp-key:

sudo gpg --export --armor | sudo tee /var/www/aptly/shared/on-premises-repos.asc > /dev/null

Add repo on another host and install packages

Comment ubuntu repos:

sudo sed -i 's/^deb/#> deb/g' /etc/apt/sources.list

Add gpg-key

        
sudo mkdir -pv /etc/apt/keyrings
sudo chmod 755 /etc/apt/keyrings
curl http://proxy.test.lan/shared/on-premises-repos.asc | gpg --dearmor | sudo tee /etc/apt/keyrings/on-premises-repos.gpg > /dev/null

Add our repository

        
cat <<EOF | sudo tee /etc/apt/sources.list.d/on-premises-repos.list
deb [arch=amd64 signed-by=/etc/apt/keyrings/on-premises-repos.gpg]  http://proxy.test.lan/shared/ jammy main
EOF

Try to install nmap:

        
        
        
    
sudo apt install nmap
	Reading package lists... Done
	Building dependency tree... Done
	Reading state information... Done
	Some packages could not be installed. This may mean that you have
	requested an impossible situation or if you are using the unstable
	distribution that some required packages have not yet been created
	or been moved out of Incoming.
	The following information may help to resolve the situation:
	
	The following packages have unmet dependencies:
	 nmap : Depends: nmap-common (= 7.91+dfsg1+really7.80+dfsg1-2ubuntu0.1) but it is not installable
	        Depends: liblinear4 (>= 2.01+dfsg) but it is not installable
	        Depends: liblua5.3-0 but it is not installable
	        Depends: lua-lpeg (>= 1.0.2) but it is not installable
	E: Unable to correct problems, you have held broken packages.

Let’s fix it adding dependencies and install nmap:

        
        
        
    
# On aptly host
sudo rm -rf /var/cache/apt/archives/*.deb
sudo apt install apt-rdepends
package_to_install="nmap"
for pkg in `apt-rdepends ${package_to_install} | grep -v "^ " | sed 's/debconf-2.0/debconf/g'`; do sudo apt-get install --reinstall --download-only "$pkg"; done
sudo aptly repo add shared /var/cache/apt/archives/*.deb
sudo aptly publish update jammy filesystem:on-premises-repos:shared

# On client host
sudo apt update -y
sudo apt install nmap

We need to use tls with nginx to be able download pgp-key safely

Let’s upgrade http to https using tls. Generate CA, keys and certificates:

        
        
        
    
DOMAIN=test.lan
SUBJ="/CN=Local trust issuer"

openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -key ca.key \
  -sha256 -days 1024 -subj "$SUBJ" -out ca.crt

openssl req -new -newkey rsa:4096 -sha256 -nodes \
  -keyout "proxy.$DOMAIN.key" -subj "/CN=$DOMAIN" -out "proxy.$DOMAIN.csr"

cat <<- EOF | tee ./additional.info
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @dns_names

[dns_names]
DNS.1 = proxy.$DOMAIN
EOF

openssl x509 -req -in "proxy.$DOMAIN.csr" -CA ca.crt \
  -CAkey ca.key -CAcreateserial -out "proxy.$DOMAIN.crt" \
  -days 365 -sha256 -extfile ./additional.info

Put open and private keys into nginx directory

sudo cp proxy.test.lan.crt proxy.test.lan.key /etc/nginx/

Put ca.crt into /var/www/aptly/shared:

sudo cp ca.crt /var/www/aptly/shared

Improve nginx config to handle tls-connections

        
        
        
    
cat <<EOF | sudo tee /etc/nginx/sites-enabled/default
server {
  listen 80;
  server_name proxy.test.lan;

  location / {
    return 301 https://proxy.test.lan$request_uri;
  }
}

server {
	listen 443 ssl default_server;
	root /var/www/aptly;
	index index.html index.htm index.nginx-debian.html;
	server_name proxy.test.lan;

    ssl_certificate		proxy.test.lan.crt;
    ssl_certificate_key	proxy.test.lan.key;
    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers         HIGH:!aNULL:!MD5;

	location / {
		autoindex on;
	}
}
EOF

sudo nginx -t
sudo nginx -s reload

Modify our repository on client

        
cat <<EOF | sudo tee /etc/apt/sources.list.d/on-premises-repos.list
deb [arch=amd64 signed-by=/etc/apt/keyrings/on-premises-repos.gpg]  https://proxy.test.lan/shared/ jammy main
EOF

And get ca.crt on client

        
curl -LO https://proxy.test.lan/shared/ca.crt -k
sudo mv ca.crt /usr/local/share/ca-certificates
sudo update-ca-certificates

Bonus: Clone repo with apt-mirror

Clone repo:

sudo apt-get install apt-mirror

Let’s take ceph repository jammy (Ubuntu:22.04)

        
        
        
    
cat <<EOF | sudo tee /etc/apt/mirror.list
set base_path    /media/debian-squid
set nthreads     20
set _tilde 0
deb https://download.ceph.com/debian-squid/ jammy main
EOF

Start mirror the repo

        
sudo apt-mirror
sudo du -hsx /media/debian-squid
sudo ls /media/debian-squid/mirror/download.ceph.com/debian-squid/

Add to the repo debian-squid and publish

sudo aptly repo add debian-squid /media/debian-squid/mirror/download.ceph.com/debian-squid/pool/main/c/ceph/*.deb
sudo aptly publish repo debian-squid filesystem:on-premises-repos:debian-squid

Add repo on a client and enjoy!

        
cat <<EOF | sudo tee /etc/apt/sources.list.d/debian-squid.list
deb [arch=amd64 signed-by=/etc/apt/keyrings/on-premises-repos.gpg]  https://proxy.test.lan/debian-squid/ jammy main
EOF

Proxy for apt

Let’s run squid inside docker container:

        
sudo docker run -d --name squid-container -e TZ=UTC -p 3128:3128 ubuntu/squid:5.2-22.04_beta
docker cp squid-container:/etc/squid/squid.conf .
docker rm -f squid-container
# add line `dns_nameservers 1.1.1.1` into squid.conf
sudo docker run -d --name squid-container -v $(pwd)/squid.conf:/etc/squid/squid.conf -e TZ=UTC -p 3128:3128 ubuntu/squid:5.2-22.04_beta

Add proxy for all and ignoring proxy for local repos:

        
        
        
    
cat <<EOF | sudo tee /etc/apt/apt.conf
Acquire::http::Proxy "http://proxy.test.lan:3128";
Acquire::https::Proxy "http://proxy.test.lan:3128";
Acquire::ftp::Proxy "http://proxy.test.lan:3128";

Acquire::http::Proxy {
    proxy.test.lan DIRECT;
};
Acquire::https::Proxy {
    proxy.test.lan DIRECT;
};
EOF