Docs/Mirrors - Revision history

Simone: update hardware description

2026-01-28T17:35:27Z

update hardware description

← Older revision		Revision as of 17:35, 28 January 2026
Line 4:		Line 4:

	=== Hardware ===		=== Hardware ===
	Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant)		Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant), with 2x E5-2623 v3s, 128GB of RAM, and 12x 12TB hard drives.

	Currently iDrac is non-functional, this should be investigated.

	=== Operating System ===		=== Operating System ===
	Mirrors runs FreeBSD.		Mirrors runs FreeBSD.

	It uses ZFS as the filesystem for the root pool and primary pool that's used for the distribution mirrors (the pool named <code>~~lug~~</code>)		It uses ZFS as the filesystem for the root pool and primary pool that's used for the distribution mirrors (the pool named <code>tank</code>)

	It used to use salt, but it broke when upgrading from FreeBSD 12-14. Currently, all maintenance is done by hand (this is a good thing)		It used to use salt, but it broke when upgrading from FreeBSD 12-14. Currently, all maintenance is done by hand (this is a good thing)
Line 181:		Line 179:
	\|06:03 PM		\|06:03 PM
	\|-		\|-
	\|EPEL~~'''***'''~~		\|EPEL
	\|00:30 AM		\|00:30 AM
	\|06:30 AM		\|06:30 AM
Line 199:		Line 197:
	\|06:14 PM		\|06:14 PM
	\|-		\|-
	\|RPM Fusion~~'''***'''~~		\|RPM Fusion
	\|00:15 AM		\|00:15 AM
	\|06:15 AM		\|06:15 AM

D2wn: /* cron */

2025-10-10T22:18:26Z

cron

← Older revision		Revision as of 22:18, 10 October 2025
Line 122:		Line 122:
	\|rsync		\|rsync
	\|}		\|}



Line 128:		Line 129:
	It's just a repo that was <code>git pull</code>'d, and was not natively installed on the system via pkg.		It's just a repo that was <code>git pull</code>'d, and was not natively installed on the system via pkg.

	~~When~~ we rebuild mirrors, we should either install it via pkg (if available) or <code>git pull</code> it anew, and place it in <code>/opt/archvsync/</code>		If we rebuild mirrors, we should either install it via pkg (if available) or <code>git pull</code> it anew, and place it in <code>/opt/archvsync/</code>

	===== cron =====		===== cron =====
	<code>cron</code> calls <code>ftpsync</code> to run at certain times.		<code>cron</code> calls <code>ftpsync</code> to run at certain times.

	This is what determines that, ~~say,~~ ~~arch~~ is synced with upstream every ~15 minutes, while Debian is only synced four times a day~~, for example~~.		This is what determines that Arch Linux is synced with upstream every ~15 minutes, while Debian is only synced four times a day.

	This should be set according to ~~the~~ distribution's ~~official docs~~ on ~~mirrors~~.		This should be set according to each distribution's rules on mirror servers.

	Most distros want tier 1's to sync 4 times a day, and ~~set~~ the exact hours/minutes ~~slightly~~ randomly so ~~they~~ don't get every downstream ~~server~~ hammering requests all at once.		Most distros want tier 1's to sync 4 times a day, and want the exact hours/minutes set randomly so the Tier0's don't get every downstream Tier1 hammering requests all at once.

	Currently, this is the schedule Mirrors uses (all times in EST):		Currently, this is the schedule Mirrors uses (all times in EST):
Line 236:		Line 237:

	It has a hardcoded <code>if</code> block pointing to each distro's dataset path, I'm almost certain could just be replaced with <code>root /lug</code> in the <code>server</code> block.		It has a hardcoded <code>if</code> block pointing to each distro's dataset path, I'm almost certain could just be replaced with <code>root /lug</code> in the <code>server</code> block.



⚫	salt.tar.gz contains all the configuration for salt, and the config files it uses to overwrite the config files located in the standard location (as well as the template files it uses to 'build' configs for services like rsyncd and archvsync when a new distro is added to the primary salt config)


	=== Salt ===		=== Salt ===
	Salt used to administer these services, but it's half-broken at the moment and ''should not be reinstalled'' (in my opinion).		Salt used to administer these services, but it's half-broken at the moment and ''should not be reinstalled'' (in my opinion).
Line 250:		Line 245:

	As such, I think the way Mirrors is setup is essentially perfect, sans salt.		As such, I think the way Mirrors is setup is essentially perfect, sans salt.


		⚫	salt.tar.gz contains all the configuration for salt, and the config files it uses to overwrite the config files located in the standard location (as well as the template files it uses to 'build' configs for services like rsyncd and archvsync when a new distro is added to the primary salt config)

D2wn: /* ftpsync */

2025-10-10T22:14:57Z

ftpsync

← Older revision		Revision as of 22:14, 10 October 2025
Line 43:		Line 43:

	===== ftpsync =====		===== ftpsync =====
	The ftpsync utility from the archvsync project is what we use to synchronize content on upstream servers to ~~Mirrors~~.		The ftpsync utility from the archvsync project is what we use to synchronize content on upstream servers to our local Mirror server.

	archvsync is a script by the Debian project to provision mirror servers, and should be all-inclusive.		archvsync is a script by the Debian project to provision mirror servers, and should be all-inclusive.
Line 51:		Line 51:
	It's nice because it prevents half-sync'd files from getting served to users, so they don't pull corrupted binaries that don't verify with the package signature.		It's nice because it prevents half-sync'd files from getting served to users, so they don't pull corrupted binaries that don't verify with the package signature.

	This is especially important for downstream ~~mirrors~~, as they do not~~(?)~~ typically check package authenticity, replying on end-user's package managers to check them.		This is especially important when acting as an upstream mirror for downstream mirror servers, as they do not typically check package authenticity, replying on end-user's package managers to check them.

	Under the hood, ftpsync can use the ftp(?) or rsync protocols, and is determined by the ftpsync config for a distribution.		Under the hood, ftpsync can use the ftp(?) or rsync protocols, and is determined by the ftpsync config for a distribution.

D2wn at 22:12, 10 October 2025

2025-10-10T22:12:55Z

← Older revision		Revision as of 22:12, 10 October 2025
Line 5:		Line 5:
	=== Hardware ===		=== Hardware ===
	Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant)		Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant)

			Currently iDrac is non-functional, this should be investigated.

	=== Operating System ===		=== Operating System ===
Line 30:		Line 32:
	You can view the nginx configuration in <code>/usr/local/etc/nginx/nginx.conf</code> to view and change settings. This file is no longer managed by salt, and can be edited by hand.		You can view the nginx configuration in <code>/usr/local/etc/nginx/nginx.conf</code> to view and change settings. This file is no longer managed by salt, and can be edited by hand.

			=== Core tasks ===
			At its core, a mirror server performs two functions:

			# Synchronizing the content from upstream mirrors to itself
			# Hosting that downloaded content for end-users and other downstream mirrors to pull from

			On our mirrors, this is accomplished with archvsync+cron to synchronize content with upstream, and vsftpd+rsyncd+nginx to handle hosting the content.

	=== Pulling from upstream ===		==== Pulling from upstream ====

	==== ftpsync ====		===== ftpsync =====
	The ftpsync utility from the archvsync project is what we use to synchronize content on upstream servers to Mirrors.		The ftpsync utility from the archvsync project is what we use to synchronize content on upstream servers to Mirrors.

Line 121:		Line 130:
	When we rebuild mirrors, we should either install it via pkg (if available) or <code>git pull</code> it anew, and place it in <code>/opt/archvsync/</code>		When we rebuild mirrors, we should either install it via pkg (if available) or <code>git pull</code> it anew, and place it in <code>/opt/archvsync/</code>

	==== cron ====		===== cron =====
	<code>cron</code> calls <code>ftpsync</code> to run at certain times.		<code>cron</code> calls <code>ftpsync</code> to run at certain times.

Line 215:		Line 224:
	Keep in mind this is when syncing starts, it may take a moment before it's fully up-to-date with upstream.		Keep in mind this is when syncing starts, it may take a moment before it's fully up-to-date with upstream.

	=== Serving to downstream ===		==== Serving to downstream ====
	<code>vsftpd</code> is the ftp daemon running on port <code>21</code>, and allows all recursive content inside <code>/lug</code> to be downloaded by anonymous users.		<code>vsftpd</code> is the ftp daemon running on port <code>21</code>, and allows all recursive content inside <code>/lug</code> to be downloaded by anonymous users.

D2wn at 22:11, 10 October 2025

2025-10-10T22:11:26Z

← Older revision		Revision as of 22:11, 10 October 2025
Line 1:		Line 1:
			The LUG Mirror server mirrors a number of different Linux distros.
	[[Docs\|<small>~/Docs</small>]]

			Currently, we mirror Debian (+ISOs), Arch Linux, CentOS (+AltArch), gentoo, Fedora (+EPEL &RPMfusion), and Ubuntu (+ISOs). The full list can be seen by going to https://mirrors.lug.mtu.edu/
⚫	Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant) ~~running FreeBSD.~~

			=== Hardware ===
⚫	It uses ZFS as the filesystem for the root pool and primary pool that's used for the distribution mirrors (the pool named <code>lug</code>)
		⚫	Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant)

			=== Operating System ===
			Mirrors runs FreeBSD.

		⚫	It uses ZFS as the filesystem for the root pool and primary pool that's used for the distribution mirrors (the pool named <code>lug</code>)

	It used to use salt, but it broke when upgrading from FreeBSD 12-14. Currently, all maintenance is done by hand (this is a good thing)		It used to use salt, but it broke when upgrading from FreeBSD 12-14. Currently, all maintenance is done by hand (this is a good thing)


	At its core, a mirror server performs two functions:		At its core, a mirror server performs two functions:
Line 17:		Line 21:
	On our mirrors, this is accomplished with archvsync+cron to synchronize content with upstream, and vsftpd+rsyncd+nginx to handle hosting the content.		On our mirrors, this is accomplished with archvsync+cron to synchronize content with upstream, and vsftpd+rsyncd+nginx to handle hosting the content.

	=== ~~Certificate maintenance:~~ ===		=== Maintenance ===

			==== Certificates ====
	Put the certificate (the 'intermediate' download option from our Certificate Authority) in <code>/usr/local/share/certs/mirrors_lug_mtu_edu_bundle.cer</code>, and the key in <code>/usr/local/share/certs/mirrors_lug_mtu_edu.key</code>		Put the certificate (the 'intermediate' download option from our Certificate Authority) in <code>/usr/local/share/certs/mirrors_lug_mtu_edu_bundle.cer</code>, and the key in <code>/usr/local/share/certs/mirrors_lug_mtu_edu.key</code>

Line 23:		Line 29:

	You can view the nginx configuration in <code>/usr/local/etc/nginx/nginx.conf</code> to view and change settings. This file is no longer managed by salt, and can be edited by hand.		You can view the nginx configuration in <code>/usr/local/etc/nginx/nginx.conf</code> to view and change settings. This file is no longer managed by salt, and can be edited by hand.


	=== Pulling from upstream ===		=== Pulling from upstream ===

D2wn at 07:42, 23 April 2025

2025-04-23T07:42:20Z

← Older revision		Revision as of 07:42, 23 April 2025
Line 1:		Line 1:
	~~<small>~~[[~~Infrastructure~~\|~/~~Infrastructure]]~~</small>		[[Docs\|<small>~/Docs</small>]]

	Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant) running FreeBSD.		Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant) running FreeBSD.

D2wn: D2wn moved page Infrastructure/Mirrors to Docs/Mirrors

2025-04-23T07:39:10Z

D2wn moved page Infrastructure/Mirrors to Docs/Mirrors

← Older revision	Revision as of 07:39, 23 April 2025
(No difference)

D2wn at 21:06, 22 April 2025

2025-04-22T21:06:28Z

← Older revision		Revision as of 21:06, 22 April 2025
Line 1:		Line 1:
	<small>[[Infrastructure\|../Infrastructure]]</small>		<small>[[Infrastructure\|~/Infrastructure]]</small>

	Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant) running FreeBSD.		Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant) running FreeBSD.

D2wn at 21:01, 22 April 2025

2025-04-22T21:01:23Z

← Older revision		Revision as of 21:01, 22 April 2025
Line 1:		Line 1:
			<small>[[Infrastructure\|../Infrastructure]]</small>

	Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant) running FreeBSD.		Mirrors is a standalone [https://www.dell.com/en-us/shop/povw/poweredge-r730xd/1000 Dell R730xd] server (3.5" drive bay variant) running FreeBSD.

D2wn: we fixed the ubuntu mirrors

2025-03-12T16:53:37Z

we fixed the ubuntu mirrors

← Older revision		Revision as of 16:53, 12 March 2025
Line 186:		Line 186:
	\|06:15 PM		\|06:15 PM
	\|-		\|-
	\|Ubuntu Releases~~'''**'''~~		\|Ubuntu Releases
	\|00:45 AM		\|00:45 AM
	\|06:45 AM		\|06:45 AM
Line 192:		Line 192:
	\|06:45 PM		\|06:45 PM
	\|-		\|-
	\|Ubuntu~~'''**'''~~		\|Ubuntu
	\|00:30 AM		\|00:30 AM
	\|06:30 AM		\|06:30 AM