
Testing mTCP
Michael Brutman, May 2011



I test the library in a few different ways.  Here are some ideas you
might find useful.



Code to help testing

  Stack overflow checking

    While developing new code it is a good idea to turn on the
    stack overflow checking in the compiler options.  This is
    easy to do and can give you some early warning if you are
    overflowing the stack.


  Consistency checking

    Consistency checking is a great technique for ensuring that your
    data structures are being manipulated in the way you designed
    them to be.  But good consistency checking does add a lot of
    run-time overhead.  Consistency checking is generally compiled
    in during testing, but turned off when the code is complete
    and tested.  Controlling it with compiler flags allows you to
    effectively add or remove the extra code with just a small
    makefile change.

    In general this code does not do consistency checking on the
    data structures that it uses.  I have added consistency checking
    in the past where it was needed, and will probably take some
    time to clean that code up and make the consistency checking
    code more comprehensive.


  Error injection

    This is another technique that I have used in the past and will
    probably spend more time enhancing.

    Error injection in the code is best done at the packet driver
    interface level.  That code allows you to 'conveniently' drop
    packets before the higher level protocols see it, allowing you
    to simulated lost packets.

    Besides randomly dropping packets I have sometimes used magic
    keystrokes, timers, and other methods to enable or disable the
    error injection code.  What you use depends on the nature of
    the error you are trying to create.

    As always, be careful to ensure that any error injection code
    that you use is disabled when you give code to end users.  At
    best your error injection code will cause performance problems
    for the rest of the stack.  At worst it will give the illusion
    of bad software.




Testing environments


  DOSBox

    DOSBox is a DOS emulator that runs under Windows and Linux.  DOSBox
    does not normally support networking, but there is a patched version
    known as the "H-A-L 9000 Megabuild" that includes NE2000 emulation.
    This is enough to make mTCP functional within DOSBox.

    The H-A-L 9000 Megabuild can be found at:

      http://home.arcor.de/h-a-l-9000/

    DOSBox is not pure DOS, and there are some differences.  The most
    noticeable one that I have found is the lack of proper Ctrl-Break
    handling.  While not fatal, it does make it hard to break out of
    some of the mTCP programs that scan for and react to Ctrl-Break.

    DOSBox is *really* handy because you can copy your executables
    into it without doing anything extraordinary.  It can be setup
    to use part of your host filesystem.


  VirtualBox and VMWare

    I use these two when I need something that is running real DOS.
    Once again, they are not perfect but they are pretty close.


  Real hardware

    Nothing beats the sights and sounds of running on real hardware.
    I test on a variety of machines including:

      IBM PC, XT, AT, PCjr, Convertible, PS/2 Model 25, and L40SX
      Generic 80386-40 clone
      Generic Pentium 133 clone


  No matter what hardware you use, be sure to put your code through
  a variety of tests.  Try to break it.  Do things that you don't
  expect people to do, but are possible to do.  Do long running tests
  to ensure that your code is stable.  Test against a wide variety
  of other machines to make sure that you don't have protocol
  problems.



Performance testing

  I have done a pretty good job performance testing the base TCP/IP code.
  If you are nervous about performance look at the SPDTEST program.

  SPDTEST basically tests the raw performance of the TCP/IP socket code.
  It does it by allowing you to blast data to or from the mTCP machine.
  The data only goes through TCP/IP processing; it is not read from or
  written to disk, generated, or checked in any way except to do the
  normal TCP/IP protocol checks and required checksumming. This ensures
  that all of the CPU time is spent on the data transfer and allows you
  to figure out what the upper bounds of your performance should be.
  The other machine you are testing with needs to be fast enough to
  ensure that the mTCP machine is not being slowed down by it.




Look at the logs!

  Once in a while turn on the mTCP tracing, setting it to record
  warnings to a file.  This does not add a lot of overhead and it will
  let you see warnings generated by the library.  Take the warnings
  seriously - they often point to problems in the way the code
  is using the base library.



