MTU LUG Wiki

Docs/Switches: Difference between revisions

mNo edit summary
 
(14 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[Infrastructure|<small>../Infrastructure</small>]]
[[Docs|<small>~/Docs</small>]]


'''For Layer 1 network details, see [[Docs/Cables]].'''
Our firewall/router runs [https://www.pfsense.org/ pfSense], soon to be migrated to [https://opnsense.org/ OPNsense].


'''For Layer 3 network details, see [[Docs/OPNsense|Docs/OPNSense]].'''
All IP addressing of servers and virtual machines happens through DHCP, and can be viewed in the pfSense 'DHCP Leases' tab. (except Proxmox nodes, which don't support DHCP and require static addressing)


== VLANs ==
Otherwise, most configuration can be viewed by poking around the web interface.
{| class="wikitable"
!Network
!VLAN ID
|-
|Management
|1
|-
|LAN
|2
|-
|kubernetes
|30
|-
|WAN
|640
|}


== Switches ==
== Switch Ports ==
Fiber switch:
{| class="wikitable"
!Switch port
!Client
!Client port
!VLAN 1 (Mgmt.)
!VLAN 2 (LAN)
!VLAN 30 (???)
!VLAN 640 (WAN)
|-
|1
|Shell
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|2
|Storage
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|3
|Mirrors
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|4
|Kurisu
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|5
|Okabe
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|6
|Daru
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|7
|Mayuri
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|8
|Luka
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|9
|Watch
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|10
|N/A
|
|Excluded
|Untagged
|Excluded
|Excluded
|-
|11
|ravioli
|ix1 (left SFP)
|Tagged
|Tagged
|Excluded
|Excluded
|-
|12
|lasagna
|
|Tagged
|Tagged
|Excluded
|Excluded
|-
|13
|48 Port
|Port 45
|Tagged
|Tagged
|Excluded
|Excluded
|-
|14
|48 Port
|Port 46
|Tagged
|Tagged
|Excluded
|Excluded
|-
|15
|48 Port
|Port 47
|Tagged
|Tagged
|Excluded
|Excluded
|-
|16
|48 Port
|Port 48
|Tagged
|Tagged
|Excluded
|Excluded
|}
Ethernet switch:
{| class="wikitable"
|+
!Switch port
!Client
!Client port
!VLAN 1 (Mgmt.)
!VLAN 2 (LAN)
!VLAN 30 (???)
!VLAN 640 (WAN)
|-
|1
|
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|2
|
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|3
|
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|4
|
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|5
|Lasagna
|bge0
|Excluded
|Excluded
|Excluded
|Untagged
|-
|6
|Mirrors
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|7
|Shell
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|8
|
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|9
|Ravioli?
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|10
|
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|11
|
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|12
|
|
|Excluded
|Excluded
|Excluded
|Untagged
|-
|13
|Lasagna
|igb3
|Untagged
|Tagged
|Tagged
|Excluded
|-
|14
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|15
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|16
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|17
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|18
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|19
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|20
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|21
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|22
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|23
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|24
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|25
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|26
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|27
|Shell
|
|Excluded
|Untagged
|Tagged
|Excluded
|-
|28
|Storage
|
|Excluded
|Untagged
|Tagged
|Excluded
|-
|29
|Kurisu
|
|Excluded
|Untagged
|Tagged
|Tagged
|-
|30
|Okabe
|
|Excluded
|Untagged
|Tagged
|Tagged
|-
|31
|Daru
|
|Excluded
|Untagged
|Tagged
|Tagged
|-
|32
|Luka
|
|Excluded
|Untagged
|Tagged
|Tagged
|-
|33
|Mayuri
|
|Excluded
|Untagged
|Tagged
|Tagged
|-
|34
|
|
|Excluded
|Untagged
|Tagged
|Excluded
|-
|35
|
|
|Excluded
|Untagged
|Tagged
|Tagged
|-
|36
|
|
|Excluded
|Untagged
|Tagged
|Tagged
|-
|37
|
|
|Excluded
|Untagged
|Tagged
|Tagged
|-
|38
|
|
|Excluded
|Untagged
|Tagged
|Tagged
|-
|39
|
|
|Excluded
|Untagged
|Tagged
|Excluded
|-
|40
|
|
|Excluded
|Untagged
|Tagged
|Excluded
|-
|41
|
|
|Excluded
|Untagged
|Tagged
|Excluded
|-
|42
|
|
|Excluded
|Untagged
|Tagged
|Excluded
|-
|43
|
|
|Excluded
|Untagged
|Tagged
|Excluded
|-
|44
|
|
|Excluded
|Untagged
|Tagged
|Excluded
|-
|45
|12 port
|Port 13
|Untagged
|Tagged
|Tagged
|Excluded
|-
|46
|12 port
|Port 14
|Untagged
|Tagged
|Tagged
|Excluded
|-
|47
|12 port
|Port 15
|Untagged
|Tagged
|Tagged
|Excluded
|-
|48
|12 port
|Port 16
|Untagged
|Tagged
|Tagged
|Excluded
|-
|49
|
|
|Excluded
|Excluded
|Excluded
|Excluded
|-
|50
|
|
|Untagged
|Excluded
|Excluded
|Excluded
|-
|51
|MTU UP 1
|MTU
|Excluded
|Excluded
|Excluded
|Tagged
|-
|52
|MTU UP 2
|MTU
|Excluded
|Excluded
|Excluded
|Tagged
|}

=== WAN ===
Our WAN is a LAGG across two ports. The link needs '''LACP enabled''' ("Static mode" '''off''' in 1Gb Ubiquiti Switch) [https://www.reddit.com/r/Ubiquiti/comments/7xs70n/lag_dynamic_vs_static/duauolg/], and '''STP off'''.
Our WAN is a LAGG across two ports. The link needs '''LACP enabled''' ("Static mode" '''off''' in 1Gb Ubiquiti Switch) [https://www.reddit.com/r/Ubiquiti/comments/7xs70n/lag_dynamic_vs_static/duauolg/], and '''STP off'''.


IT configures their switches to automatically shut off ports if they detect STP advertisements.
IT configures their switches to automatically shut off ports if they detect STP advertisements.


Reference commands to make a Cisco switch satisfy the requirements:<syntaxhighlight lang="text">
<describe vlan config>
(config-if)# spanning-tree bpdufilter enable

(config-if)# spanning-tree bpduguard disable
== OPNsense ==
</syntaxhighlight>

=== Firewall Rules ===
View the WebUI for the specific firewall rules, but some of the more basic/essential ones are:

# Management cannot communicate with LAN/WAN (the internet), and LAN cannot communicate with Management.
## Generally, Management should be restricted from everything else. (maybe even other iDrac servers?)
## OOB services tend to be ''super'' vulnerable, there are dozens of [https://github.com/mgargiullo/cve-2018-1207 premade scripts] that instapwn iDRACs and give you a root shell by just pointing them at the IP address.
## Because of this, the iDRAC web login interface should only be accessible to anyone you're okay having root on the server.
# Wireguard
## The admin/user split is so all members can be given a wireguard config to the internal network without having to worry about them being able to trivially get root on all servers running premade-exploits like [https://github.com/mgargiullo/cve-2018-1207 these] on the iDracs.
## If someone shows up to a couple meetings they're probably fine to get an admin config; this is more for peace-of-mind to not need to worry about the configs given to people who went to one meeting once at the beginning of the semester and have never been seen again.
## Neither config should have access to WAN, just to prevent someone getting LUG in hot water if they attempt to torrent or something similarly dumb through the VPN.

=== Routing ===

==== Main networks ====
We have two main networks:
* 10.10.0.0/24 - Management (OOB Management services like [https://www.dell.com/en-us/lp/dt/open-manage-idrac Dell iDRAC] / [https://www.hpe.com/us/en/hpe-integrated-lights-out-ilo.html HP iLO])
* 10.10.1.0/24 - LAN (servers/VMs)
We may also be getting a <code>/27</code> of Tech's <code>141.219.0.0/16</code> block through IT (~28-30 usable public IP addresses).

The plan is to use reverse-NAT to map the public IPs to select internal IPs, since we won't have enough IPs for every VM (so we can't do it like IT and exclusively use publicly routable addresses).

==== VPN Networks ====
In addition, there are two main VPN networks:

* 10.10.10.0/24 - OpenVPN
* 10.10.11.0/24 - Wireguard
** 10.10.11.0/25 - Wireguard admin range (access to Management+LAN, no WAN)
** 10.10.11.128/25 - Wireguard user range (access to only LAN, no WAN)

Latest revision as of 21:41, 29 September 2025

~/Docs

For Layer 1 network details, see Docs/Cables.

For Layer 3 network details, see Docs/OPNSense.

VLANs

Network VLAN ID
Management 1
LAN 2
kubernetes 30
WAN 640

Switch Ports

Fiber switch:

Switch port Client Client port VLAN 1 (Mgmt.) VLAN 2 (LAN) VLAN 30 (???) VLAN 640 (WAN)
1 Shell Excluded Untagged Excluded Excluded
2 Storage Excluded Untagged Excluded Excluded
3 Mirrors Excluded Untagged Excluded Excluded
4 Kurisu Excluded Untagged Excluded Excluded
5 Okabe Excluded Untagged Excluded Excluded
6 Daru Excluded Untagged Excluded Excluded
7 Mayuri Excluded Untagged Excluded Excluded
8 Luka Excluded Untagged Excluded Excluded
9 Watch Excluded Untagged Excluded Excluded
10 N/A Excluded Untagged Excluded Excluded
11 ravioli ix1 (left SFP) Tagged Tagged Excluded Excluded
12 lasagna Tagged Tagged Excluded Excluded
13 48 Port Port 45 Tagged Tagged Excluded Excluded
14 48 Port Port 46 Tagged Tagged Excluded Excluded
15 48 Port Port 47 Tagged Tagged Excluded Excluded
16 48 Port Port 48 Tagged Tagged Excluded Excluded

Ethernet switch:

Switch port Client Client port VLAN 1 (Mgmt.) VLAN 2 (LAN) VLAN 30 (???) VLAN 640 (WAN)
1 Excluded Excluded Excluded Untagged
2 Excluded Excluded Excluded Untagged
3 Excluded Excluded Excluded Untagged
4 Excluded Excluded Excluded Untagged
5 Lasagna bge0 Excluded Excluded Excluded Untagged
6 Mirrors Excluded Excluded Excluded Untagged
7 Shell Excluded Excluded Excluded Untagged
8 Excluded Excluded Excluded Untagged
9 Ravioli? Excluded Excluded Excluded Untagged
10 Excluded Excluded Excluded Untagged
11 Excluded Excluded Excluded Untagged
12 Excluded Excluded Excluded Untagged
13 Lasagna igb3 Untagged Tagged Tagged Excluded
14 Untagged Excluded Excluded Excluded
15 Untagged Excluded Excluded Excluded
16 Untagged Excluded Excluded Excluded
17 Untagged Excluded Excluded Excluded
18 Untagged Excluded Excluded Excluded
19 Untagged Excluded Excluded Excluded
20 Untagged Excluded Excluded Excluded
21 Untagged Excluded Excluded Excluded
22 Untagged Excluded Excluded Excluded
23 Untagged Excluded Excluded Excluded
24 Untagged Excluded Excluded Excluded
25 Untagged Excluded Excluded Excluded
26 Untagged Excluded Excluded Excluded
27 Shell Excluded Untagged Tagged Excluded
28 Storage Excluded Untagged Tagged Excluded
29 Kurisu Excluded Untagged Tagged Tagged
30 Okabe Excluded Untagged Tagged Tagged
31 Daru Excluded Untagged Tagged Tagged
32 Luka Excluded Untagged Tagged Tagged
33 Mayuri Excluded Untagged Tagged Tagged
34 Excluded Untagged Tagged Excluded
35 Excluded Untagged Tagged Tagged
36 Excluded Untagged Tagged Tagged
37 Excluded Untagged Tagged Tagged
38 Excluded Untagged Tagged Tagged
39 Excluded Untagged Tagged Excluded
40 Excluded Untagged Tagged Excluded
41 Excluded Untagged Tagged Excluded
42 Excluded Untagged Tagged Excluded
43 Excluded Untagged Tagged Excluded
44 Excluded Untagged Tagged Excluded
45 12 port Port 13 Untagged Tagged Tagged Excluded
46 12 port Port 14 Untagged Tagged Tagged Excluded
47 12 port Port 15 Untagged Tagged Tagged Excluded
48 12 port Port 16 Untagged Tagged Tagged Excluded
49 Excluded Excluded Excluded Excluded
50 Untagged Excluded Excluded Excluded
51 MTU UP 1 MTU Excluded Excluded Excluded Tagged
52 MTU UP 2 MTU Excluded Excluded Excluded Tagged

WAN

Our WAN is a LAGG across two ports. The link needs LACP enabled ("Static mode" off in 1Gb Ubiquiti Switch) [1], and STP off.

IT configures their switches to automatically shut off ports if they detect STP advertisements.

Reference commands to make a Cisco switch satisfy the requirements:

(config-if)# spanning-tree bpdufilter enable
(config-if)# spanning-tree bpduguard disable
Retrieved from "https://lug.mtu.edu/w/index.php?title=Docs/Switches&oldid=7953"