Docs/Switches: Difference between revisions

From MTU LUG Wiki
Jump to navigation Jump to search
mNo edit summary
m (Got rid of k8s)
 
(17 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[Infrastructure|<small>~/Infrastructure</small>]]
[[Docs|<small>~/Docs</small>]]


'''For Layer 1 network details, see [[Docs/Cables]].'''
Our firewall/router runs [https://www.pfsense.org/ pfSense], soon to be migrated to [https://opnsense.org/ OPNsense].


'''For Layer 3 network details, see [[Docs/OPNsense|Docs/OPNSense]].'''
All IP addressing of servers and virtual machines happens through DHCP, and can be viewed in the pfSense 'DHCP Leases' tab. (except Proxmox nodes, which don't support DHCP and require static addressing)


== VLANs ==
Otherwise, most configuration can be viewed by poking around the web interface.
{| class="wikitable"
!Network
!VLAN ID
|-
|Management
|1
|-
|LAN
|2
|-
|WAN
|640
|}


== Switches ==
== Switch Ports ==
Clyde - Fiber switch:
{| class="wikitable"
!Switch port
!Client
!Client port
!VLAN 1 (Mgmt.)
!VLAN 2 (LAN)
!VLAN 640 (WAN)
|-
|1
|Shell
|
|Excluded
|Untagged
|Tagged
|-
|2
|Storage
|
|Excluded
|Untagged
|Tagged
|-
|3
|Mirrors
|
|Excluded
|Untagged
|Tagged
|-
|4
|Kurisu
|
|Excluded
|Untagged
|Tagged
|-
|5
|Okabe
|
|Excluded
|Untagged
|Tagged
|-
|6
|Daru
|
|Excluded
|Untagged
|Tagged
|-
|7
|Mayuri
|
|Excluded
|Untagged
|Tagged
|-
|8
|Luka
|
|Excluded
|Untagged
|Tagged
|-
|9
|Maho
|
|Excluded
|Untagged
|Tagged
|-
|10
|N/A
|
|Excluded
|Untagged
|Tagged
|-
|11
|ravioli
|ix1 (left SFP)
|Tagged
|Tagged
|Tagged
|-
|12
|lasagna
|
|Tagged
|Tagged
|Tagged
|-
|13
|Switch to Switch
|Port 51
|Tagged
|Tagged
|Tagged
|-
|14
|48 Port
|Port 52
|Tagged
|Tagged
|Tagged
|-
|15
|MTU Uplink
|MTU LAG 1
|Excluded
|Excluded
|Untagged
|-
|16
|MTU Uplink
|MTU LAG 2
|Excluded
|Excluded
|Untagged
|}
Bonnie - Ethernet switch:
{| class="wikitable"
|+
!Switch port
!Client
!Client port
!VLAN 1 (Mgmt.)
!VLAN 2 (LAN)
!VLAN 640 (WAN)
|-
|1
|
|
|Untagged
|Excluded
|Excluded
|-
|2
|
|
|Untagged
|Excluded
|Excluded
|-
|3
|
|
|Untagged
|Excluded
|Excluded
|-
|4
|
|
|Untagged
|Excluded
|Excluded
|-
|5
|
|Untagged
|Excluded
|Excluded
|-
|6
|
|
|Untagged
|Excluded
|Excluded
|-
|7
|
|
|Untagged
|Excluded
|Excluded
|-
|8
|
|
|Untagged
|Excluded
|Excluded
|-
|9
|
|
|Untagged
|Excluded
|Excluded
|-
|10
|
|
|Untagged
|Excluded
|Excluded
|-
|11
|
|
|Untagged
|Excluded
|Excluded
|-
|12
|
|
|Untagged
|Excluded
|Excluded
|-
|13
|
|
|Untagged
|Excluded
|Excluded
|-
|14
|
|
|Untagged
|Excluded
|Excluded
|-
|15
|
|
|Untagged
|Excluded
|Excluded
|-
|16
|
|
|Untagged
|Excluded
|Excluded
|-
|17
|
|
|Untagged
|Excluded
|Excluded
|-
|18
|
|
|Untagged
|Excluded
|Excluded
|-
|19
|
|
|Untagged
|Excluded
|Excluded
|-
|20
|
|
|Untagged
|Excluded
|Excluded
|-
|21
|
|
|Untagged
|Excluded
|Excluded
|-
|22
|
|
|Untagged
|Excluded
|Excluded
|-
|23
|
|
|Untagged
|Excluded
|Excluded
|-
|24
|
|
|Untagged
|Excluded
|Excluded
|-
|25
|
|
|Untagged
|Excluded
|Excluded
|-
|26
|
|
|Untagged
|Excluded
|Excluded
|-
|27
|
|
|Untagged
|Excluded
|Excluded
|-
|28
|
|
|Untagged
|Excluded
|Excluded
|-
|29
|
|
|Untagged
|Excluded
|Excluded
|-
|30
|
|
|Untagged
|Excluded
|Excluded
|-
|31
|
|
|Untagged
|Excluded
|Excluded
|-
|32
|
|
|Untagged
|Excluded
|Excluded
|-
|33
|
|
|Untagged
|Excluded
|Excluded
|-
|34
|
|
|Untagged
|Excluded
|Excluded
|-
|35
|
|
|Untagged
|Excluded
|Excluded
|-
|36
|
|
|Untagged
|Excluded
|Excluded
|-
|37
|
|
|Untagged
|Excluded
|Excluded
|-
|38
|
|
|Untagged
|Excluded
|Excluded
|-
|39
|
|
|Untagged
|Excluded
|Excluded
|-
|40
|
|
|Untagged
|Excluded
|Excluded
|-
|41
|
|
|Untagged
|Excluded
|Excluded
|-
|42
|
|
|Untagged
|Excluded
|Excluded
|-
|43
|
|
|Untagged
|Excluded
|Excluded
|-
|44
|
|
|Untagged
|Excluded
|Excluded
|-
|45
|
|
|Excluded
|Untagged
|Tagged
|-
|46
|
|
|Excluded
|Untagged
|Tagged
|-
|47
|
|
|Excluded
|Untagged
|Tagged
|-
|48
|
|
|Excluded
|Untagged
|Tagged
|-
|49
|
|
|Excluded
|Excluded
|Excluded
|-
|50
|
|
|Excluded
|Excluded
|Excluded
|-
|51
|Switch to Switch
|Fiber Switch 1 LAG
|Tagged
|Tagged
|Tagged
|-
|52
|Switch to Switch
|Fiber Switch 2 LAG
|Tagged
|Tagged
|Tagged
|}

=== WAN ===
Our WAN is a LAGG across two ports. The link needs '''LACP enabled''' ("Static mode" '''off''' in 1Gb Ubiquiti Switch) [https://www.reddit.com/r/Ubiquiti/comments/7xs70n/lag_dynamic_vs_static/duauolg/], and '''STP off'''.
Our WAN is a LAGG across two ports. The link needs '''LACP enabled''' ("Static mode" '''off''' in 1Gb Ubiquiti Switch) [https://www.reddit.com/r/Ubiquiti/comments/7xs70n/lag_dynamic_vs_static/duauolg/], and '''STP off'''.


IT configures their switches to automatically shut off ports if they detect STP advertisements.
IT configures their switches to automatically shut off ports if they detect STP advertisements.


Reference commands to make a Cisco switch satisfy the requirements:<syntaxhighlight lang="text">
<describe vlan config>
(config-if)# spanning-tree bpdufilter enable

(config-if)# spanning-tree bpduguard disable
== OPNsense ==
</syntaxhighlight>

=== Firewall Rules ===
View the WebUI for the specific firewall rules, but some of the more basic/essential ones are:

# Management cannot communicate with LAN/WAN (the internet), and LAN cannot communicate with Management.
## Generally, Management should be restricted from everything else. (maybe even other iDrac servers?)
## OOB services tend to be ''super'' vulnerable, there are dozens of [https://github.com/mgargiullo/cve-2018-1207 premade scripts] that instapwn iDRACs and give you a root shell by just pointing them at the IP address.
## Because of this, the iDRAC web login interface should only be accessible to anyone you're okay having root on the server.
# Wireguard
## The admin/user split is so all members can be given a wireguard config to the internal network without having to worry about them being able to trivially get root on all servers running premade-exploits like [https://github.com/mgargiullo/cve-2018-1207 these] on the iDracs.
## If someone shows up to a couple meetings they're probably fine to get an admin config; this is more for peace-of-mind to not need to worry about the configs given to people who went to one meeting once at the beginning of the semester and have never been seen again.
## Neither config should have access to WAN, just to prevent someone getting LUG in hot water if they attempt to torrent or something similarly dumb through the VPN.

=== Routing ===

==== Main networks ====
We have two main networks:
* 10.10.0.0/24 - Management (OOB Management services like [https://www.dell.com/en-us/lp/dt/open-manage-idrac Dell iDRAC] / [https://www.hpe.com/us/en/hpe-integrated-lights-out-ilo.html HP iLO])
* 10.10.1.0/24 - LAN (servers/VMs)
We may also be getting a <code>/27</code> of Tech's <code>141.219.0.0/16</code> block through IT (~28-30 usable public IP addresses).

The plan is to use reverse-NAT to map the public IPs to select internal IPs, since we won't have enough IPs for every VM (so we can't do it like IT and exclusively use publicly routable addresses).

==== VPN Networks ====
In addition, there are two main VPN networks:

* 10.10.10.0/24 - OpenVPN
* 10.10.11.0/24 - Wireguard
** 10.10.11.0/25 - Wireguard admin range (access to Management+LAN, no WAN)
** 10.10.11.128/25 - Wireguard user range (access to only LAN, no WAN)

Latest revision as of 20:15, 4 January 2026

~/Docs

For Layer 1 network details, see Docs/Cables.

For Layer 3 network details, see Docs/OPNSense.

VLANs

Network VLAN ID
Management 1
LAN 2
WAN 640

Switch Ports

Clyde - Fiber switch:

Switch port Client Client port VLAN 1 (Mgmt.) VLAN 2 (LAN) VLAN 640 (WAN)
1 Shell Excluded Untagged Tagged
2 Storage Excluded Untagged Tagged
3 Mirrors Excluded Untagged Tagged
4 Kurisu Excluded Untagged Tagged
5 Okabe Excluded Untagged Tagged
6 Daru Excluded Untagged Tagged
7 Mayuri Excluded Untagged Tagged
8 Luka Excluded Untagged Tagged
9 Maho Excluded Untagged Tagged
10 N/A Excluded Untagged Tagged
11 ravioli ix1 (left SFP) Tagged Tagged Tagged
12 lasagna Tagged Tagged Tagged
13 Switch to Switch Port 51 Tagged Tagged Tagged
14 48 Port Port 52 Tagged Tagged Tagged
15 MTU Uplink MTU LAG 1 Excluded Excluded Untagged
16 MTU Uplink MTU LAG 2 Excluded Excluded Untagged

Bonnie - Ethernet switch:

Switch port Client Client port VLAN 1 (Mgmt.) VLAN 2 (LAN) VLAN 640 (WAN)
1 Untagged Excluded Excluded
2 Untagged Excluded Excluded
3 Untagged Excluded Excluded
4 Untagged Excluded Excluded
5 Untagged Excluded Excluded
6 Untagged Excluded Excluded
7 Untagged Excluded Excluded
8 Untagged Excluded Excluded
9 Untagged Excluded Excluded
10 Untagged Excluded Excluded
11 Untagged Excluded Excluded
12 Untagged Excluded Excluded
13 Untagged Excluded Excluded
14 Untagged Excluded Excluded
15 Untagged Excluded Excluded
16 Untagged Excluded Excluded
17 Untagged Excluded Excluded
18 Untagged Excluded Excluded
19 Untagged Excluded Excluded
20 Untagged Excluded Excluded
21 Untagged Excluded Excluded
22 Untagged Excluded Excluded
23 Untagged Excluded Excluded
24 Untagged Excluded Excluded
25 Untagged Excluded Excluded
26 Untagged Excluded Excluded
27 Untagged Excluded Excluded
28 Untagged Excluded Excluded
29 Untagged Excluded Excluded
30 Untagged Excluded Excluded
31 Untagged Excluded Excluded
32 Untagged Excluded Excluded
33 Untagged Excluded Excluded
34 Untagged Excluded Excluded
35 Untagged Excluded Excluded
36 Untagged Excluded Excluded
37 Untagged Excluded Excluded
38 Untagged Excluded Excluded
39 Untagged Excluded Excluded
40 Untagged Excluded Excluded
41 Untagged Excluded Excluded
42 Untagged Excluded Excluded
43 Untagged Excluded Excluded
44 Untagged Excluded Excluded
45 Excluded Untagged Tagged
46 Excluded Untagged Tagged
47 Excluded Untagged Tagged
48 Excluded Untagged Tagged
49 Excluded Excluded Excluded
50 Excluded Excluded Excluded
51 Switch to Switch Fiber Switch 1 LAG Tagged Tagged Tagged
52 Switch to Switch Fiber Switch 2 LAG Tagged Tagged Tagged

WAN

Our WAN is a LAGG across two ports. The link needs LACP enabled ("Static mode" off in 1Gb Ubiquiti Switch) [1], and STP off.

IT configures their switches to automatically shut off ports if they detect STP advertisements.

Reference commands to make a Cisco switch satisfy the requirements:

(config-if)# spanning-tree bpdufilter enable
(config-if)# spanning-tree bpduguard disable
Retrieved from "https://lug.mtu.edu/w/index.php?title=Docs/Switches&oldid=8041"