Docs/Switches: Difference between revisions

From MTU LUG Wiki
Jump to navigation Jump to search
mNo edit summary
(update hostnames for both switches)
 
(18 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[Infrastructure|<small>~/Infrastructure</small>]]
[[Docs|<small>~/Docs</small>]]


'''For Layer 1 network details, see [[Docs/Cables|Docs/SFP+ Cables]].'''
Our firewall/router runs [https://www.pfsense.org/ pfSense], soon to be migrated to [https://opnsense.org/ OPNsense].


'''For Layer 3 network details, see [[Docs/OPNsense|Docs/OPNSense]].'''
All IP addressing of servers and virtual machines happens through DHCP, and can be viewed in the pfSense 'DHCP Leases' tab. (except Proxmox nodes, which don't support DHCP and require static addressing)


== VLANs ==
Otherwise, most configuration can be viewed by poking around the web interface.
{| class="wikitable"
!Network
!VLAN ID
|-
|Management
|1
|-
|LAN
|2
|-
|WAN
|640
|}


== Switches ==
== Switch Ports ==
Clyde - SFP+ switch:
Our WAN is a LAGG across two ports. The link needs '''LACP enabled''' ("Static mode" '''off''' in 1Gb Ubiquiti Switch) [https://www.reddit.com/r/Ubiquiti/comments/7xs70n/lag_dynamic_vs_static/duauolg/], and '''STP off'''.


Model: Ubiquiti EdgeSwitch 16 XG
IT configures their switches to automatically shut off ports if they detect STP advertisements.


IP: 10.10.0.6
<describe vlan config>
{| class="wikitable"
!Switch port
!Client
!Client port
!VLAN 1 (Mgmt.)
!VLAN 2 (LAN)
!VLAN 640 (WAN)
|-
|1
|N/A
|
|Excluded
|Untagged
|Tagged
|-
|2
|Leskinen
|eno1
|Excluded
|Untagged
|Tagged
|-
|3
|Shell
|eno1
|Excluded
|Untagged
|Tagged
|-
|4
|Mirrors
|bxe0
|Excluded
|Untagged
|Tagged
|-
|5
|Okabe
|
|Excluded
|Untagged
|Tagged
|-
|6
|Daru
|eno1
|Excluded
|Untagged
|Tagged
|-
|7
|Mayuri
|eno1
|Excluded
|Untagged
|Tagged
|-
|8
|Luka
|eno1
|Excluded
|Untagged
|Tagged
|-
|9
|Maho
|eno1
|Excluded
|Untagged
|Tagged
|-
|10
|Kurisu
|eno1
|Excluded
|Untagged
|Tagged
|-
|11
|ravioli
|ix1 (left SFP)
|Tagged
|Tagged
|Tagged
|-
|12
|lasagna
|
|Tagged
|Tagged
|Tagged
|-
|13
|Switch LAG
|Port 51
|Tagged
|Tagged
|Tagged
|-
|14
|Switch LAG
|Port 52
|Tagged
|Tagged
|Tagged
|-
|15
|MTU Uplink
|MTU LAG 1
|Excluded
|Excluded
|Untagged
|-
|16
|MTU Uplink
|MTU LAG 2
|Excluded
|Excluded
|Untagged
|}
Bonnie - 1gb RJ45 switch:


Model: Ubiquiti EdgeSwitch 48 Lite
== OPNsense ==


IP: 10.10.0.5
=== Firewall Rules ===
{| class="wikitable"
View the WebUI for the specific firewall rules, but some of the more basic/essential ones are:
|+
!Switch port
!Client
!Client port
!VLAN 1 (Mgmt.)
!VLAN 2 (LAN)
!VLAN 640 (WAN)
|-
|1
|
|
|Untagged
|Excluded
|Excluded
|-
|2
|
|
|Untagged
|Excluded
|Excluded
|-
|3
|
|
|Untagged
|Excluded
|Excluded
|-
|4
|
|
|Untagged
|Excluded
|Excluded
|-
|5
|
|Untagged
|Excluded
|Excluded
|-
|6
|
|
|Untagged
|Excluded
|Excluded
|-
|7
|
|
|Untagged
|Excluded
|Excluded
|-
|8
|
|
|Untagged
|Excluded
|Excluded
|-
|9
|
|
|Untagged
|Excluded
|Excluded
|-
|10
|
|
|Untagged
|Excluded
|Excluded
|-
|11
|
|
|Untagged
|Excluded
|Excluded
|-
|12
|
|
|Untagged
|Excluded
|Excluded
|-
|13
|Mayuri-idrac
|iDRAC
|Untagged
|Excluded
|Excluded
|-
|14
|
|
|Untagged
|Excluded
|Excluded
|-
|15
|mirrors-idrac
|iDRAC
|Untagged
|Excluded
|Excluded
|-
|16
|Okabe-idrac
|iDRAC
|Untagged
|Excluded
|Excluded
|-
|17
|Daru-idrac
|iDRAC
|Untagged
|Excluded
|Excluded
|-
|18
|leskinen-idrac
|iDRAC
|Untagged
|Excluded
|Excluded
|-
|19
|shell-idrac
|
|Untagged
|Excluded
|Excluded
|-
|20
|
|
|Untagged
|Excluded
|Excluded
|-
|21
|Luka-idrac
|iDRAC
|Untagged
|Excluded
|Excluded
|-
|22
|
|
|Untagged
|Excluded
|Excluded
|-
|23
|
|
|Untagged
|Excluded
|Excluded
|-
|24
|
|
|Untagged
|Excluded
|Excluded
|-
|25
|
|
|Untagged
|Excluded
|Excluded
|-
|26
|
|
|Untagged
|Excluded
|Excluded
|-
|27
|
|
|Untagged
|Excluded
|Excluded
|-
|28
|
|
|Untagged
|Excluded
|Excluded
|-
|29
|
|
|Untagged
|Excluded
|Excluded
|-
|30
|
|
|Untagged
|Excluded
|Excluded
|-
|31
|
|
|Untagged
|Excluded
|Excluded
|-
|32
|
|
|Untagged
|Excluded
|Excluded
|-
|33
|
|
|Untagged
|Excluded
|Excluded
|-
|34
|
|
|Untagged
|Excluded
|Excluded
|-
|35
|
|
|Untagged
|Excluded
|Excluded
|-
|36
|
|
|Untagged
|Excluded
|Excluded
|-
|37
|
|
|Untagged
|Excluded
|Excluded
|-
|38
|
|
|Untagged
|Excluded
|Excluded
|-
|39
|
|
|Untagged
|Excluded
|Excluded
|-
|40
|
|
|Untagged
|Excluded
|Excluded
|-
|41
|
|
|Untagged
|Excluded
|Excluded
|-
|42
|
|
|Untagged
|Excluded
|Excluded
|-
|43
|
|
|Untagged
|Excluded
|Excluded
|-
|44
|
|
|Untagged
|Excluded
|Excluded
|-
|45
|
|
|Excluded
|Untagged
|Tagged
|-
|46
|
|
|Excluded
|Untagged
|Tagged
|-
|47
|
|
|Excluded
|Untagged
|Tagged
|-
|48
|
|
|Excluded
|Untagged
|Tagged
|-
|49
|
|
|Excluded
|Excluded
|Excluded
|-
|50
|
|
|Excluded
|Excluded
|Excluded
|-
|51
|Switch to Switch
|Fiber Switch 1 LAG
|Tagged
|Tagged
|Tagged
|-
|52
|Switch to Switch
|Fiber Switch 2 LAG
|Tagged
|Tagged
|Tagged
|}


=== WAN ===
# Management cannot communicate with LAN/WAN (the internet), and LAN cannot communicate with Management.
Our WAN is a LAGG across two ports. The link needs '''LACP enabled''' ("Static mode" '''off''' in 1Gb Ubiquiti Switch) [https://www.reddit.com/r/Ubiquiti/comments/7xs70n/lag_dynamic_vs_static/duauolg/], and '''STP off'''.
## Generally, Management should be restricted from everything else. (maybe even other iDrac servers?)
## OOB services tend to be ''super'' vulnerable, there are dozens of [https://github.com/mgargiullo/cve-2018-1207 premade scripts] that instapwn iDRACs and give you a root shell by just pointing them at the IP address.
## Because of this, the iDRAC web login interface should only be accessible to anyone you're okay having root on the server.
# Wireguard
## The admin/user split is so all members can be given a wireguard config to the internal network without having to worry about them being able to trivially get root on all servers running premade-exploits like [https://github.com/mgargiullo/cve-2018-1207 these] on the iDracs.
## If someone shows up to a couple meetings they're probably fine to get an admin config; this is more for peace-of-mind to not need to worry about the configs given to people who went to one meeting once at the beginning of the semester and have never been seen again.
## Neither config should have access to WAN, just to prevent someone getting LUG in hot water if they attempt to torrent or something similarly dumb through the VPN.


IT configures their switches to automatically shut off ports if they detect STP advertisements.
=== Routing ===

==== Main networks ====
We have two main networks:
* 10.10.0.0/24 - Management (OOB Management services like [https://www.dell.com/en-us/lp/dt/open-manage-idrac Dell iDRAC] / [https://www.hpe.com/us/en/hpe-integrated-lights-out-ilo.html HP iLO])
* 10.10.1.0/24 - LAN (servers/VMs)
We may also be getting a <code>/27</code> of Tech's <code>141.219.0.0/16</code> block through IT (~28-30 usable public IP addresses).

The plan is to use reverse-NAT to map the public IPs to select internal IPs, since we won't have enough IPs for every VM (so we can't do it like IT and exclusively use publicly routable addresses).

==== VPN Networks ====
In addition, there are two main VPN networks:


Reference commands to make a Cisco switch satisfy the requirements:<syntaxhighlight lang="text">
* 10.10.10.0/24 - OpenVPN
(config-if)# spanning-tree bpdufilter enable
* 10.10.11.0/24 - Wireguard
(config-if)# spanning-tree bpduguard disable
** 10.10.11.0/25 - Wireguard admin range (access to Management+LAN, no WAN)
</syntaxhighlight>
** 10.10.11.128/25 - Wireguard user range (access to only LAN, no WAN)

Latest revision as of 21:51, 5 February 2026

~/Docs

For Layer 1 network details, see Docs/SFP+ Cables.

For Layer 3 network details, see Docs/OPNSense.

VLANs

Network VLAN ID
Management 1
LAN 2
WAN 640

Switch Ports

Clyde - SFP+ switch:

Model: Ubiquiti EdgeSwitch 16 XG

IP: 10.10.0.6

Switch port Client Client port VLAN 1 (Mgmt.) VLAN 2 (LAN) VLAN 640 (WAN)
1 N/A Excluded Untagged Tagged
2 Leskinen eno1 Excluded Untagged Tagged
3 Shell eno1 Excluded Untagged Tagged
4 Mirrors bxe0 Excluded Untagged Tagged
5 Okabe Excluded Untagged Tagged
6 Daru eno1 Excluded Untagged Tagged
7 Mayuri eno1 Excluded Untagged Tagged
8 Luka eno1 Excluded Untagged Tagged
9 Maho eno1 Excluded Untagged Tagged
10 Kurisu eno1 Excluded Untagged Tagged
11 ravioli ix1 (left SFP) Tagged Tagged Tagged
12 lasagna Tagged Tagged Tagged
13 Switch LAG Port 51 Tagged Tagged Tagged
14 Switch LAG Port 52 Tagged Tagged Tagged
15 MTU Uplink MTU LAG 1 Excluded Excluded Untagged
16 MTU Uplink MTU LAG 2 Excluded Excluded Untagged

Bonnie - 1gb RJ45 switch:

Model: Ubiquiti EdgeSwitch 48 Lite

IP: 10.10.0.5

Switch port Client Client port VLAN 1 (Mgmt.) VLAN 2 (LAN) VLAN 640 (WAN)
1 Untagged Excluded Excluded
2 Untagged Excluded Excluded
3 Untagged Excluded Excluded
4 Untagged Excluded Excluded
5 Untagged Excluded Excluded
6 Untagged Excluded Excluded
7 Untagged Excluded Excluded
8 Untagged Excluded Excluded
9 Untagged Excluded Excluded
10 Untagged Excluded Excluded
11 Untagged Excluded Excluded
12 Untagged Excluded Excluded
13 Mayuri-idrac iDRAC Untagged Excluded Excluded
14 Untagged Excluded Excluded
15 mirrors-idrac iDRAC Untagged Excluded Excluded
16 Okabe-idrac iDRAC Untagged Excluded Excluded
17 Daru-idrac iDRAC Untagged Excluded Excluded
18 leskinen-idrac iDRAC Untagged Excluded Excluded
19 shell-idrac Untagged Excluded Excluded
20 Untagged Excluded Excluded
21 Luka-idrac iDRAC Untagged Excluded Excluded
22 Untagged Excluded Excluded
23 Untagged Excluded Excluded
24 Untagged Excluded Excluded
25 Untagged Excluded Excluded
26 Untagged Excluded Excluded
27 Untagged Excluded Excluded
28 Untagged Excluded Excluded
29 Untagged Excluded Excluded
30 Untagged Excluded Excluded
31 Untagged Excluded Excluded
32 Untagged Excluded Excluded
33 Untagged Excluded Excluded
34 Untagged Excluded Excluded
35 Untagged Excluded Excluded
36 Untagged Excluded Excluded
37 Untagged Excluded Excluded
38 Untagged Excluded Excluded
39 Untagged Excluded Excluded
40 Untagged Excluded Excluded
41 Untagged Excluded Excluded
42 Untagged Excluded Excluded
43 Untagged Excluded Excluded
44 Untagged Excluded Excluded
45 Excluded Untagged Tagged
46 Excluded Untagged Tagged
47 Excluded Untagged Tagged
48 Excluded Untagged Tagged
49 Excluded Excluded Excluded
50 Excluded Excluded Excluded
51 Switch to Switch Fiber Switch 1 LAG Tagged Tagged Tagged
52 Switch to Switch Fiber Switch 2 LAG Tagged Tagged Tagged

WAN

Our WAN is a LAGG across two ports. The link needs LACP enabled ("Static mode" off in 1Gb Ubiquiti Switch) [1], and STP off.

IT configures their switches to automatically shut off ports if they detect STP advertisements.

Reference commands to make a Cisco switch satisfy the requirements:

(config-if)# spanning-tree bpdufilter enable
(config-if)# spanning-tree bpduguard disable
Retrieved from "https://lug.mtu.edu/w/index.php?title=Docs/Switches&oldid=8075"