« Bank of America strikes againOrac is looking strangely bare, with Zen taking over. »

VLANs & FreeBSD 9.0/Jails

11/25/12

  07:55:00 pm, by The Dreamer   , 1327 words  
Categories: Software, Computer, Networking, Operating Systems, FreeBSD, CFEngine

VLANs & FreeBSD 9.0/Jails

In my online searches, most of the examples I found had environments that allowed asymetric routing or had non-routed vlans and used NAT/proxy or some other mechanism to cross boundaries. So, if we were to bind to a vlan inside the 'global' and wanted to route....pretty much the only way would be to use policy based routing.

For IPF, an example is:

Code

pass out quick on bce0 route-to vlan1101:10.33.1.1 from 10.33.1.0/24 to any keep state
pass in quick on vlan1101 reply-to vlan1101:10.33.1.1 proto tcp from x.y.z.128/26 to 10.33.1.3/32 port = 22 keep state

It isn't clear on how a host in vlan1101 can talk directly to another host in vlan1101, etc. Was going to investigate that later if I had gotten it working right. Basically the problem I found, is that data coming back from the a jail in the vlan was delayed and chunky. But, only when I was going in via a pass in statement. If I was in the jail and connected out, and then streamed a large file….no delays or chunking. (IE: ssh to the jail and cat /usr/ports/UPDATING - slow. while ssh to jailserver and cat /usr/ports/UPDATING is fast, as is ssh to jailserver, ezjail-admin console to jail and cat /usr/ports/UPDATING is fast. Also ssh to jailserver, ezjail-admin console to jail and ssh to my desktop and cat /usr/ports/UPDATING is fast.)

I tried using pf, same problem. I tried ipfw, but that would require IPFIREWALL_FORWARD option in the kernel. Later I came across setfib(1) & jails. But, GENERIC kernel only has 1 ROUTETABLE. I read that current has it set as 2 and that 10 will have it set to 4.

It appears 9.1-STABLE still has it as 1.

So, I built a custom JAILKERN, and set some options…and ROUTETABLE fixed the problem, so I removed some of the options and rebuilt it again (kept IPFILTER as compiled in instead of a loadable module)

/usr/src/sys/amd64/conf/JAILKERN

Code

include GENERIC
ident   JAILKERN
 
options ROUTETABLE=16
options IPFILTER
options IPFILTER_LOG
options IPFILTER_DEFAULT_BLOCK
options IPSTEALTH

There are also other options set in make.conf on jail1 that may or may not be helpful. Namely to build an extra optimized kernel.

Build and install kernel

% cd /usr/src
% make buildkernel KERNCONF=JAILKERN
% make installkernel KERNCONF=JAILKERN


reboot

To make use of these changes:

create/edit /etc/rc.local

Shell

#!/bin/sh
#
#fib 0 is for native
 
#fib for vlan1101
setfib 1 route add default 10.33.1.1
setfib 1 route add -host 10.33.1.3 127.0.0.1
setfib 1 route add -host 10.33.1.4 127.0.0.1
setfib 1 route add -host 10.33.1.5 127.0.0.1
#
exit 0


* edit jail definition in ''/usr/local/etc/ezjail''

changing jail_<normalized_jail_name>_fib="" to jail_<normalized_jail_name>_fib="#", where # is the desired fib

Now the jail is started using the specified routing table instead of the default one.

Not sure if rules like “pass in on vlan1150 from …. ” are needed of if “pass in from …” will apply to all. Though it did catch me that lo0 is a real interface and subject to ipf again like it used to be on Solaris before it got virtualized and exempted.

'route add -host 10.133.1.3 127.0.0.1', etc. lines. are to add routing to allow jails to talk to each other. These are present in the default fib 0, but aren't in any of the other fibs. For the example here, I only added the hosts in the same vlan, but if you want to talk to other jails on the jailserver, they'll need to be added as well. Though leaving them out can be good as well. And IPFILTER does act on loopback on FreeBSD....

Pages: · 2

No feedback yet

Now instead of subjecting some poor random forum to a long rambling thought, I will try to consolidate those things into this blog where they can be more easily ignored profess to be collected thoughts from my mind.

Latest Poopli Updaters -- http://lkc.me/poop

bloglovin

There are 20 years 5 months 23 hours and 7 seconds until the end of time.
And, it has been 4 years 7 months 27 days 15 hours 2 minutes and 49 seconds since The Doctor saved us all from the end of the World!

Search

August 2017
Mon Tue Wed Thu Fri Sat Sun
 << <   > >>
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      
Google

Linkblog

  XML Feeds

Who's Online?

  • Guest Users: 0
This seal is issued to lawrencechen.net by StopTheHacker Inc.
powered by b2evolution

hosted by
Green Web Hosting! This site hosted by DreamHost.

monitored by
Monitored by eXternalTest
SiteUptime Web Site Monitoring Service
website uptime