Scribd
Upload
52 views

Network Protocol Development

The document discusses using virtualization for network protocol and kernel development. It describes how virtual environments allow creating multiple virtual machines on a single physical computer, enabling testing of different network configurations without needing additional hardware. Key benefits noted include the ability to easily create, suspend, and destroy virtual machines, freeing up physical resources and workspace compared to using real machines.

Uploaded by

James Johnson
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
52 views

Network Protocol Development

The document discusses using virtualization for network protocol and kernel development. It describes how virtual environments allow creating multiple virtual machines on a single physical computer, enabling testing of different network configurations without needing additional hardware. Key benefits noted include the ability to easily create, suspend, and destroy virtual machines, freeing up physical resources and workspace compared to using real machines.

Uploaded by

James Johnson
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 34

Network Protocol and Kernel Development in Virtual Environments

George V. Neville-Neil [email protected]

BSD Can 2006

[email protected]

www.neville-neil.com

Disclaimer
I do not now, nor have I ever worked for any of
these companies I have not been paid or otherwise compensated to make these statements All the software and hardware was paid for by me

2
BSD Can 2006 [email protected] www.neville-neil.com

Motivation
Kernel and Protocol development requires at
least two machines
- Client and server - Build machine and target

Loopback testing is useful only in early


stages

Real protocol testing requires more than two


machines
- Two hosts and a router - Somewhat arbitrary networks

Simulators do not fully exercise all the


possible kernel interactions
- Rarely show heisenbugs
BSD Can 2006 [email protected]

3
www.neville-neil.com

Motivation Cont
Real machines take up space
generate heat use power are difficult to freely interconnect make a mess of your workspace

4
BSD Can 2006 [email protected] www.neville-neil.com

A Simple Example

5
BSD Can 2006 [email protected] www.neville-neil.com

Complex Example

6
BSD Can 2006 [email protected] www.neville-neil.com

Lab Equipment (Physical)



2..N Machines KVM Switch or Terminal Server Hubs Network Cabling Electrical Cabling

7
BSD Can 2006 [email protected] www.neville-neil.com

Lab Equipment (Virtual)



One very fast computer Lots of memory Lots of disk One power cable One network cable VNC

8
BSD Can 2006 [email protected] www.neville-neil.com

Some Terminology
Host Machine
- The real metal on which the virtualization software runs

Host Operating System Guest


- A virtual machine running within the virtualization software

Guest OS Clone
- Guest that is a copy of another guest

Team
- A collection of guests
9
BSD Can 2006 [email protected] www.neville-neil.com

Setting up your laboratory


Machines
What kind of machines will you run? Memory Sizing Disk Sizing What to install

Connections
- Networks - Console - Serial Lines

10
BSD Can 2006 [email protected] www.neville-neil.com

10

Machines in my lab
Nightly
A machine on which nightly builds are done NFS Server Several source trees cvsup and p4 Editors Test tools

Test Targets
- Cloned machines that can be thrown away and trivially rebuilt - NFS Client - New kernels and code are installed from NFS mounted file systems - Minimal tools required - No need for source trees
BSD Can 2006 [email protected]

11

www.neville-neil.com

11

Memory Sizing
Console only FreeBSD runs fine in 128M of
RAM
- Building the world - Building kernels - Using as a router or network device

With X-Windows probably 256M of RAM If youre running a more memory intensive

workload then you may need more memory

12
BSD Can 2006 [email protected] www.neville-neil.com

12

What I install
The Nightly box will be the root of your
cloning tree

Full sources, binaries, and doc but no games sudo cvsup, p4 (optional) Emacs Ethereal Python TAHI Test Tools (IPv6/IPSec) Your tool set will vary
13
[email protected] www.neville-neil.com

BSD Can 2006

13

Virtual Disks
Two disks on the main machine Main Disk
- /, swap and /usr for the regular install - Currently 6G but could be 4G

CVS Disk
- /cvs for source trees - 8G

Target machines only get the main disk

14
BSD Can 2006 [email protected] www.neville-neil.com

14

Virtual Disk Types


Use sparse disks if you can Pre-allocate your disks only if you need the
speed

SCSI disks are faster than IDE, even if theyre


virtual

If youre going to back up your machines to


CD or DVD then go with split disks
- These are 2G files which make up a disk

Disk space isnt really infinite


- really
trust me
15
BSD Can 2006 [email protected] www.neville-neil.com

15

What does Nightly look like now?


nightly ? df -h Filesystem Size on /dev/da0s1a 989M /dev/da0s1d 4.4G /dev/da1s1d 7.7G nightly 1.8G 439M 457M 453M 55M Used 192M 2.3G 3.2G Avail Capacity 719M 1.7G 4.0G 21% 58% 45% Mounted / /usr /cvs

? du -sh * FreeBSD-CVS /* Full CVS Mirror */ FreeBSD.5 /* Checked out trees */ FreeBSD.nightly FreeBSD.stable FreeBSD.gnn /* doc and www only */

16
BSD Can 2006 [email protected] www.neville-neil.com

16

Networks
Nightly only has one network interface All targets have at least two but normally
three interfaces

Main (lnc0)
- Connected to the lab network - Best to use DHCP

Test Lan 1 (lnc1)


- Connected to a network with little or no other traffic - Statically assigned address

Test Lan 2 (lnc2)


- In testing router configurations you need a second test network - Statically assigned address
BSD Can 2006 [email protected]

17

www.neville-neil.com

17

Serial Lines
Debugging a kernel panic with printf()s is
tedious thing

Virtual serial lines work just like the real


- Only they dont require you to remember how to make a NULL modem cable

I use two lines


- Console - Debug

18
BSD Can 2006 [email protected] www.neville-neil.com

18

Other Devices
CD/DVD ROM
- Boot and install from an ISO file on disk

USB Sound

19
BSD Can 2006 [email protected] www.neville-neil.com

19

Building Your Lab



Install Software Create a single guest to use as Nightly Install the requisite software into Nightly Get your nightly build scripts running Leave Nightly running if you have the CPU horsepower

20
BSD Can 2006 [email protected] www.neville-neil.com

20

Clones
Take a guest and make a copy of it Linked clones only store new data when it is
created
- A version of copy on write

Linked clones depend on the root machine


not going away
- Never ever delete Nightly if its the root of your tree

Full clones are independent copies


- Take up more space than linked clones - Run faster than linked clones, but by how much I dont know

21
BSD Can 2006 [email protected] www.neville-neil.com

21

Teams
A team is 2 or more guests treated as a
group

Teams can have their own, private, networks

22
BSD Can 2006 [email protected] www.neville-neil.com

22

Suspension
Never turn off machines again Suspending a guest to disk releases memory

and other resources but does not require a reboot Does not depend on the OS to be able to suspend Some things are not allowed with suspended machines
- Updating configuration - Cloning

23
BSD Can 2006 [email protected] www.neville-neil.com

23

Development Process
Clone Nightly as many times as necessary
- Need to shut down Nightly first

Make one machine the master Check out code and modify on the master
- I usually call this devbox

DO NOT INSTALL THE KERNEL ON THE MASTER Mount build directory on the device under Test, crash, burn Use ddb, gdb, what have you
BSD Can 2006 [email protected]

test (DUT) Install the kernel on the DUT

24
www.neville-neil.com

24

Development Process Cont


Suspend and resume teams and guests as you
need

Eventually check in code Destroy guests you dont need Archive guests you consider important

25
BSD Can 2006 [email protected] www.neville-neil.com

25

VMWare
VMWare Workstation
Officially runs on Linux and Windows Teams Team networks Linked and independent clones

Virtual Hardware
IDE and/or SCSI Hard Disks IDE CD/DVD ROM Lance Ethernet (lnc driver) Serial and Parallel cables Sound USB

VMWare Server Beta


- Does not have teams or team networks
BSD Can 2006 [email protected]

26
www.neville-neil.com

26

Parallels

Intel Mac OS/X as well as Linux and Windows No teams No linked clones Currently in Beta (Beta 5) Very occasionally panics my machine Devices
IDE Hard Disks and CD/DVD Drives Floppies Reatek 8209 Network Interface (ed0) Serial and Parallel Cables Sound USB
27
BSD Can 2006 [email protected] www.neville-neil.com

27

Open Source Alternatives


Xen
- Requires the guest OS to be ported to the platform

Qemu
- Doesnt support acceleration on the Mac yet - Supports several different architectures
ARM PowerPC SPARC MIPS

- Does not require kernel customization - No teams or cloning - Interesting possibility for the future

28
BSD Can 2006 [email protected] www.neville-neil.com

28

Hardware Choices
How effective your virtual machines depends
on how fast your host is

Disks
- Use the highest bandwidth and lowest seek time you can afford

Memory
- The more the better - 2 G is a good number for a network of 4 machines

CPU
- Clock rate is not the most important factor - On chip L1 and L2 Cache is the most important factor - If the chip support Virtualization thats even better
29
BSD Can 2006 [email protected] www.neville-neil.com

29

Chuo

HP DL-360 G4 1U Rack Mount 4 Gigs of RAM 2x Xeon 3.4GHz 2x 73G U320 Fast/Wide SCSI Disks (RAID 0)
- Single biggest boost to performance

Host OS is Linux RH EL 4 Easily runs 4 guests Builds a kernel in

30
BSD Can 2006 [email protected] www.neville-neil.com

30

Minion

Intel Based Mac Book Pro Mac OS 10.4.6 Dual Core 2.0 GHz CPU 2 G of RAM
- You really really dont want to swap

120G SATA Hard Disk Parallels Beta 5 Builds a kernel in about 15 minutes

31
BSD Can 2006 [email protected] www.neville-neil.com

31

Other uses and benefits


Never install Windows on a real machine
again

Virus and security problem testing Keep old versions of systems just as they are Share systems with other people
- Keep them on a server - Move them on a fast network - Mail a DVD with a machine on it

32
BSD Can 2006 [email protected] www.neville-neil.com

32

A Quick Demo

33
BSD Can 2006 [email protected] www.neville-neil.com

33

Questions?

34
BSD Can 2006 [email protected] www.neville-neil.com

34

You might also like