\documentstyle[12pt,twoside,final,times,sfheads]{rccldoc}

\begin{document}
\tolerance=1000
\pagenumbering{roman}
\title{RCCL/RCI on VxWorks}
\author{John Lloyd}
\date{March 1992}
\maketitle

\begin{abstract}
This document describes how to install and use RCCL/RCI on
VxWorks, the real-time O/S distributed by Wind River systems.
\par\vfill
\copyrightNotice
\end{abstract}

\parskip = 2ptplus3pt
\tableofcontents
\newpage
\pagenumbering{arabic}

\chapter{Introduction}

This document describes the VxWorks implementation of RCCL/RCI.

VxWorks is a real-time environment distributed by Wind River systems.
Applications written using VxWorks are developed on a host system
(usually a Sun workstation) and then downloaded into the real-time
``box'' through an ethernet or backplane connection. VxWorks provides
a few services similar or identical to those available under UNIX
(in particular, a large portion of {\tt libc} is provided), plus
additional services designed for real-time control applications.

\section{Caveats}

One of the original motivations for implementing RCCL/RCI was to 
make robot control available within the UNIX environment. It should
be understood that VxWorks does not really provide a UNIX environment,
although it does provide a few of same programming interfaces. This is
described in greater detail below.

The current version of RCCL was compiled for VxWorks 4.0., and is
not completely source compatible with other versions of VxWorks.

Much of the information described in this document is likely
to change, since the RCCL/RCI VxWorks implementation is quite new
and still in a state of flux.

\chapter{Installing RCCL/RCI on VxWorks}

RCCL/RCI VxWorks support is compiled on a system {\it in parallel}
with a standard RCCL/RCI UNIX configuration. 

\section{Editing the {\tt site.conf} File}

To configure your RCCL system for VxWorks support, you need to edit
the system configuration file {\tt site.conf} in {\filefont \$RCCL/conf}. 
In particular, within this file, you need to specify
\begin{verbatim}
        #define VxWorksSupport YES
\end{verbatim}
This declaration is close to the top of the file. At the bottom
of the file are other definitions, all beginning with the prefix
string {\tt Vxw}, relating to VxWorks. Some of these can probably be left as
provided, although you probably will want to specify {\tt VxwCcInclude},
which lists all the necessary {\tt -I<includeDir>} options
for the compiler (including the one for the 
VxWorks include directory), and
{\tt VxwCcOptions}, which gives other
options used in VxWorks compilation.
If you are using a compiler other than {\tt cc}, then you will probably
want to specify {\tt VxwCcCmd} as well.

\section{Running {\tt make}}

After the necessary information has been specified in {\tt site.conf}, you
follow the normal procedure for compiling the system: go to the root
directory, and enter the command
\begin{verbatim}
        make World
\end{verbatim}
A standard RCCL system will now be built, along with the
VxWorks binaries.

\section{Directory and Library Structure for the VxWorks Configuration}

All VxWorks binaries are created in directories named {\tt VXW}, which are
sub-directories to the main system directories. All the object
modules in each {\tt VXW} directory are then linked into a single object 
module in {\tt lib/VXW}, and all of {\it these} object modules are
then linked into one large module named {\filefont lib/vxwRCCL.o}. The
linking is done using the {\tt -r} flag so that other loads can be performed
with the same module.

For instance, the directory {\tt puma} contains the source code
for routines specific to PUMA robots. The {\tt .o} files for the
UNIX configuration are created in the {\tt puma} directory, while
the {\tt .o} files for VxWorks are created in {\tt puma/VXW}. The
latter {\tt .o} files are then linked into a single module named
{\tt lib/VXW/vxwPuma.o}, which is in turn linked into the
all inclusive module {\filefont lib/vxwRCCL.o}.

All of this linking is done automatically by the {\tt make World} command. 

Within each of the source directories, the following other
{\tt make} commands may be useful: 
\begin{verbatim}
        make vxlibs
\end{verbatim}
Builds (in the {\tt VXW} sub-directory) all the {\it library} modules that for
the UNIX configuration would be built by {\tt make libs}, and then
combines them into a single load module in {\tt lib/VXW}.

\begin{verbatim}
        make vxbins
\end{verbatim}
Builds (in the {\tt VXW} sud-directory) all the {\it program} modules that for
the UNIX configuration would be built by {\tt make bins}, and then
combines them into the single load module {\tt lib/VXW}.

\begin{verbatim}
        make vxw
\end{verbatim}
Does all the work associated with {\tt make vxlibs} and {\tt make vxbins},
and updates the master object file {\filefont lib/vxwRCCL.o} as well.

Each of these commands may also be run from the {\tt VXW} sub-directory.
If run from the {\tt \$RCCL} root directory, then the appropriate updates
are made in all of the system directories.

\section{Adjustment to the VxWorks Environment}

\subsection{The CPU startup file}

After VxWorks is booted on a particular CPU, commands 
are read in from a {\it startup} file for that particular
CPU. On our system, this file
is located in {\filefont /local/vw/config/<cpu>/startup.cmd}.

Initializations required for RCCL/RCI can be done with commands
placed in this file. Most important is loading in the libraries
and programs, which are all contained in the file
{\filefont \$RCCL/lib/vxwRCCL.o}. On our system, this is done 
with the command sequence
\begin{verbatim}
        cd "/home/rccl/lib"
        ld  <  vxwRCCL.o
\end{verbatim}

After the library has been loaded, the function
{\tt rciSetenvRCCL()} must be called to set 
the name of the RCCL root directory (which is {\filefont /home/rccl}
in the above example).
Another function, {\tt rciSetenvHOME()},
may also be used to set a name for the user's ``home'' directory.
This is used when searching for files which on UNIX systems
are sought under {\tt \$HOME}. Each function takes a single string 
as an argument. For example,
\begin{verbatim}
        rciSetenvRCCL("/home/rccl")
\end{verbatim}
sets the RCCL home directory to {\filefont /home/rccl}.

It may also be necessary to call {\tt nfsAuthUnixSet()} to
authorize the access of particular files in the 
RCCL or user's file areas.

\subsection{Clock rates}

RCI uses the auxiliary system clock, accessed through
the {\tt sysAuxClk()} routines. Typically, RCI will try to use this
clock at rates around 100 or 200 Hz. We have observed, for VxWorks
4.0 on the Cyclone 68020 CPU, that if the {\it system} clock
(which is accessed using the {\tt sysClk()} routines) 
is running
at a rate that is {\it not} a multiple of this number, then there is
a tendency for the system to hang at some random time shortly after
RCCL/RCI programs begin execution. It is presumed that this hang occurs
within the interrupt handler for the clock device, and that 
it involves some sort of race
condition with the clock timer hardware and the driving software.
The system clock is
often set to run at a base rate of 60 Hz. When the system clock
was set to run at a base rate of 100 Hz. the problem went away.

The base rate for the system clock can be set by calling {\tt sysClkRateSet()}
in the CPU startup file.

It is very possible that this problem will not appear
on your VxWorks system.

\section{Robot Configuration and Device Interrupts}

RCCL/RCI applications read in various types of information from
configuration files located in {\filefont \$RCCL/conf}, and possibly elsewhere.
The is true for both VxWorks and UNIX implementations. There
is a difference, however, with regard to the {\tt robots} file.

\subsection{Interrupt information and the {\tt robots.vxw} file}

On UNIX systems, the {\tt robots} file specifies the name of 
an RCI driver, the address of the device to be used by that driver,
and the name of the UNIX device which is to handle device interrupts.
The RCI driver does not take care of handling interrupts,
since the UNIX kernel must assume that role.

On VxWorks systems, the system reads the file {\tt robots.vxw} instead;
this permits concurrent information to be provided for both UNIX
and VxWorks implementations.

Also, interrupt handling is done not by a device configured into the
kernel, but by extra routines in the RCI driver. Information
for these routines is supplied
from additional fields in the {\tt robots.vxw} file. In addition
to the {\tt device} and {\tt addr} fields, the following
other fields must be set:

\refitem{{\tt interrupt} --}
Instead of giving the name of a system device, as on UNIX, this field
gives the name of an RCI driver for handling the interrupt.
Normally, this will be the same as the communication driver specified
by the {\tt device} field.

\refitem{{\tt intraddr} --}
Specifies an address associated with the interrupt device. Again, this will
often be the same as the address given by the {\tt address} field
(although not necessarily; the control registers which control interrupts
could be in a different place from the control registers used to do
communication).

\refitem{{\tt vector} --}
The VME vector number for the interrupt device.

\refitem{{\tt ipl} --}
The VME hardware priority level for the interrupt device.

A sample entry for a robot called {\tt Zeus} could look like:
\begin{verbatim}
        robot=Zeus device=pio address=0xfd80
                   interrupt=pio intraddr=0xfd80 ipl=4 vector=24
\end{verbatim}

Additional information about the RCI interrupt driver is given in section
\ref{secIntrdev}.

\chapter{Compiling RCCL Programs for VxWorks}

\section{Differences Between UNIX and VxWorks}

These comments apply to VxWorks 4.0.

It should be understood that VxWorks is {\bf not} UNIX in many
important ways. First, it is not really a multi-processing system in that
it is unable to provide an independent execution environment (i.e., 
a virtual machine) to different running processes. All tasks share
the same memory space, and the only way to reinitialize the memory
space to a known state is to reboot the system. Effectively, program 
execution on UNIX corresponds to reboot-and-run on VxWorks.

This problem stems mainly from the absence of memory management support in the
system. A related problem is that no memory is protected, and so it is very
easy for application code to crash the system {\bf badly} (so as to require
a hardware reset).

\section{Program Compilation}

An RCCL program is compiled with the same {\tt cc} command you use to compile
other VxWorks applications. No use is made of the {\tt rcc} command.

At the beginning of the file where {\tt main} is defined, you 
should redefine {\tt main} to the ``name'' of your program. This
can be done with a line of the form
\begin{verbatim}
        #define main <progname>_
\end{verbatim}
Note the trailing underscore after the assigned name. {\tt <progname>}
can then be used to execute the program using the {\it run} command,
which will be described shortly.

You will probably want to compile using a command similar to
\begin{verbatim}
        cc -c -I$RCCL/h <file>.c
\end{verbatim}
Additional include directories may be necessary for VxWorks
definitions. If your compiler does not support prototypes, define {\tt
-DNO\_PROTOTYPES}; if it doesn't understand {\tt const}, define
{\tt -Dconst=}, and if it doesn't understand {\tt void}, define
{\tt -Dvoid=int}.

After compilation, the modules are simply loaded into
the VxWorks CPU. 
You do not need to link the application with the VxWorks library, and
in fact you shouldn't, since this would cause the program to become
enormous and possibly too large to load into the CPU alongside the
existing RCCL library.

\section{Include File Stubs}

VxWorks 4.0 provides routines and data structures similar 
to those defined in the UNIX include files {\tt <stdio.h>}, 
{\tt <signal.h>}, etc., but the names of the appropriate include
files are different. 

In the interest of preserving source compatibility between modules
for both the VxWorks and UNIX versions of RCCL, a set of UNIX
``look alike'' include files has been provided in the directory
{\filefont \$RCCL/h/vxWorks}. These have the same names as {\tt stdio},
{\tt signal}, etc., but reference the relevant VxWorks files internally.
Only a few of the more common files are provided, and there may
be some inconsistencies that we haven't noticed.

To access these include files, put {\tt -I\$RCCL/h/vxWorks}
in the compilation command.

\section{The {\tt run} Command}

\subsection{Command usage}

RCCL/RCI applications are often presented as {\it UNIX commands}
which can be controlled by arguments supplied on the command line.
Within the application program, these arguments are accessed by reading the
variables {\tt argc} and {\tt argv}.

VxWorks 4.0 does not provide an {\tt argc} and {\tt argv} command line 
interface. RCCL/RCI emulates the UNIX command line with the command
{\tt run()}, which takes a single string argument describing
the command line as it would appear in a UNIX environment.
{\tt run()} takes the first word in this string, adds a trailing {\tt \_}
to it, and uses this to look up the name of the routine to be executed.
This routine is then called with
{\tt argc} and {\tt argv} built from the remaining words in the 
command string.

For example, the UNIX command
\begin{verbatim}
        free puma560 -sim
\end{verbatim}
would be called under VxWorks by executing
\begin{verbatim}
        run "free puma560 -sim"
\end{verbatim}

To make this work, RCCL/RCI redefines {\tt main} in the source file
for the program {\tt free} as follows:
\begin{verbatim}
        #define main free_
\end{verbatim}

User application programs should redefine {\tt main} in the same way,
so that they can also be referenced by the {\tt run} command.

String arguments can be specified by surrounding
them with single quotes ({\tt ''}). For instance, the expression
\begin{verbatim}
        run "prog 'a string' 'another string'"
\end{verbatim}
would call the function {\tt prog\_()} with {\tt argc} set to {\tt 2} and
{\tt argv} containing pointers to the strings {\tt "a string"} and
{\tt "another string"}. 

{\tt run} spawns a separate task to execute the desired function, but
suspends the shell until the task exits.

\subsection{Reseting the system state}

The {\tt run} command is necessary for executing RCCL/RCI applications
because it tries to reset the state of the RCCL/RCI library to that
which would normally be encountered when a new program begins on UNIX:
i.e., it frees memory, closes file descriptors, and resets the state
of the signal handlers.

In VxWorks 4.0, most resources (such as memory and file descriptors)
are allocated globally, and must be explicitly deallocated by application
code. This can often be difficult, because the exit path for an application
can be rather unpredictable (for instance, if a task crashes or is killed,
it may be {\it unable} to release resources that it has already allocated).
The UNIX operating system kernel takes care of releasing such resources
automatically when a process exits, regardless of how it exits. The {\tt run}
command attempts to do somewhat the same thing, but it does rely on help from
the application code to tell it what to do.

The following two routines allow application code to interact with the
run command:
\begin{verbatim}
        requestFlagInit (ip)
           int *ip;

        requestExitFxn (fxn, arg)
           int (*fxn)();
           int arg;
\end{verbatim}

{\tt requestFlagInit()} specifies the address of a integer
that should be cleared (i.e., set to {\tt 0}) the {\it next time} {\tt run} is
called. Application code can use this as follows:
\begin{verbatim}
        int initflag;

        routine()
        {
           if (initflag == 0)
            { /* do initializations ... */
              initflag = 1;
              requestFlagInit (&initflag);
            }
        }
\end{verbatim}
The use of an init flag is common in many packages, but
generally relies on the fact
that memory is normally zeroed at the beginning of a program. The call
to {\tt requestFlagInit()} in the above example ensures that {\tt
initflag} is cleared when {\tt routine()} is called under a new 
invocation of the {\tt run} command.

{\tt requestExitFxn()} requests that a specific function be called when the
task spawned by the {\tt run} command exits. The command can be used
as follows:
\begin{verbatim}
        initfxn()
        { 
           /* do cleanup stuff */
        }

        routine()
        {
           if (initflag == 0)
            { initflag = 1;
              requestFlagInit (&initflag);
              requestFxnInit (initFxn);
            }
        }
\end{verbatim}
When the {\tt run} task exits, {\tt initFxn} will be called
to do whatever cleanup may be necessary.

\subsection{Interrupts from the console}

The {\tt run} command operates using a separate task, making it
easier to set the process priority and stack size, and to
run diagnostics on the function if it crashes. 

While the {\tt run} task is in operation, the shell task is 
suspended. Aborting the shell during this time will also
cause to {\tt run} task to be aborted. This allows
RCCL/RCI applications to be aborted quickly without resorting to 
use of the shell ``task delete'' command.

\chapter{Things to Know}

\section{Exceptions}

When an exception (such as a divide by zero or memory access fault)
occurs on VxWorks, a message is usually printed and the offending
task is put into a suspended state where it can be examined using
various debugging primitives.

The planning level is implemented by the VxWorks task {\tt runCmd},
and the control level is implemented by the task {\tt rciCtrl}.
An exception at either level suspends the corresponding task.

When the planning level task is actually deleted, then the {\tt rciCtrl}
task is deleted as well, {\it unless} that task has been previously
suspended.

\section{Process Priorities}

The priority and stack size used for creating the {\tt run} task
can be set using the commands
\begin{verbatim}
        setRunTaskPri(pri)

        setRunTaskStk(stk)
\end{verbatim}
Each of these commands takes an integer
argument and returns the old value. An argument value less than {\tt 0}
causes the system default value to be set.

Likewise, the priority and stack size for the RCI control task
can be set using the commands
\begin{verbatim}
        setCtrlTaskPri(pri)

        setCtrlTaskStk(stk)
\end{verbatim}

\section{The RCI Interrupt Driver}
\label{secIntrdev}

The RCI interface drivers defined in the {\tt drivers} directory are
required to provide, under VxWorks, two extra routines for
attaching and handling interrupts. 

If the user is writing their own interface driver, they
must provide these routines.

The routine names are {\tt xxxIntrAttach()} and {\tt xxxIntrEnable()}, where
{\tt xxx} is the name of the driver package. Each routine is 
called with the following arguments:

\begin{verbatim}
        #include <IOdevice.h>

        xxxIntrAttach (idev, enable)
        IO_INTERRUPT *idev;
        int enable;
        {
                /* enable or disable interrupts */
        }

        xxxIntrHandler (idev)
        IO_INTERRUPT *idev;
        {
                /* handle the interrupt when it occurs */
        }
\end{verbatim}
The {\tt IO\_INTERRUPT} device is defined in the file
{\tt IOdevice.h}, and its fields provide the following information:
the register address for the device
(in the {\tt address} field); the VME hardware interrupt level 
(in the {\tt ipl} field); and the VME vector number (in the {\tt vector}
field). Information for these fields is specified by the entries
{\tt intraddr}, {\tt vector}, and {\tt ipl} in the {\tt robots.vxw} file.
The register address is usually a byte offset relative to one of
the address spaces provided by {\tt rciIOpage} or {\tt rciBus}.

{\tt xxxIntrAttach()} is called 
by the RCI system from the task level to
either enable or disable interrupts.
If {\tt enable} is true, the routine is expected to
set the device hardware so as to enable interrupts.
It may also have to set the vector and/or ipl values,
if the device requires these to be configured in software.
If {\tt enable} is false, the routine is expected to
disable interrupts.

{\tt xxxIntrHandler()} is called
by the RCI system from interrupt level to 
handle the interrupt when it arrives.
Often, this routine doesn't need to do anything; it is only
necessary to do something when the device hardware requires
software action to complete or clear the interrupt.

Note that these routines do {\bf not} take a pointer to the
{\tt IO\_DEVICE} structure, as do the other routines in the driver module.
They cannot assume the existence of the {\tt IO\_DEVICE} used
by the other routines.

\begin{quote}
{\bf NOTE:} The {\tt xxxProbe()} routine defined in the same
driver module may be called {\it after} the
{\tt xxxIntrAttach()} routine. Care should be taken to ensure
that either the probe routine does not change the
interrupt settings, {\it or} that interrupts are properly reenabled
by the {\tt xxxIoctl()} routine with the command
{\tt IOC\_SIGINTER\_ENABLE}.
\end{quote}

\section{Limitations}

\subsection{Signals}

The 4.0 implementation of VxWorks does not fully support UNIX signals, in that
signals {\bf will not preempt} a task which is already blocked. This is
rather unfortunate, since many RCCL applications rely on signals as
a means of sending information from the control task to the planning task.

One effect of this is the following: suppose that the planning task is
blocked waiting for input from the keyboard, and the control task
aborts. Although the control task will shut down properly, the planning task
will not be notified of this until the signal
{\tt SIGHUP} (used internally for
abort notification) can be delivered, which
will not happen until something is entered at the keyboard.

This also affects the {\tt rcclTeach()} routine (and
its wrapper program {\tt teachdemo}), in that
hitting the {\tt <record>} button on the teach pendant will not cause
the planning level to actually record the point until
the user enters something at the keyboard.

One possible way around this problem would be to not use signals at all,
but instead get the same effect by using a ``signal handling''
task controlled by semaphores.
For the moment, however, we are waiting to see
if the signal mechanism in VxWorks is fixed, as was promised
for future releases.

\subsection{The {\tt .JLS} file}

VxWorks RCCL/RCI applications read the {\tt JLS} structure for a particular
robot by directly reading and processing the ascii file {\filefont
\$RCCL/conf/<robotName>.jls}, instead of the binary {\tt .JLS} file.
This is to avoid possible binary compatibility problems between
UNIX and VxWorks.

\end{document}
