LENS

Programmer's Guide: The Structures

The following tables explain all of the C structures and important global variables used in Lens.

RootRec

 Holds the arrays of all networks and all example sets. The user can access this through the root field in any network. It is defined in network.h.
int numNetworks The number of networks in the simulator.
Network *net An array of pointers to the networks, sorted in creation order.
int numExampleSets The number of example sets loaded.
ExampleSet *set An array of pointers to the example sets, sorted in creation order.
int numGraphs This is not the actual number of graphs but the number of entries allocated in the graph array. Not all of the entries will necessarily be filled in.
Graph *graph An array of pointers to the graphs. The number of a graph, reported when it is created is the index of the graph in this array.

Network

 Defined in network.h.
char *name The name of the network.
int num The index of this network in the root net array.
mask type A bitmask containing the type of the network, which may have the CONTINUOUS or BOLTZMANN and/or the FROZEN bit on.
int numGroups The number of groups in the network.
Group *group An array of pointers to group structures. The order of groups in the array is the order in which they are traversed during updating. The index of each group should be the same as its num field.
int numUnits The number of units in the network. This is just for reference; it may not actually be used.
int numInputs The number of input units in the network.
Unit *input An array of pointers to each of the input units. The order of the units is determined by the order of the input groups. This aids in loading externalInput values from example events.
int numOutputs The number of output units in the network.
Unit *output An array of pointers to each of the output units. The order of the units is determined by the order of the output groups. This aids in loading target values from example events.
int numLinks The total number of links in the network. This is used for verifying compatibility when a weight file is loaded, and is nice for reference.
NetExt ext A pointer to the network extension structure. This is defined in extension.h and allocated in extension.c. Programmers should try to put any extensions to the network structure in there.
RootRec root A pointer to the single root structure that contains the numbers and arrays of networks and example sets.
ExampleSet trainingSet A pointer to the network's training set.
ExampleSet testingSet A pointer to the network's testing set.
Example currentExample A pointer to the most recent example on which the network was run. It is used primarily for the graphical displays.
int timeIntervals The maximum number of time intervals the network can have per example. These are to be thought of as units of "real" time.
int ticksPerInterval The number of ticks in each time interval. This is just the number of times a continuous network is updated per unit of real time.
int maxTicks Equal to timeIntervals x ticksPerInterval. This is the total number of ticks that could occur per example.
int historyLength This is the length of the history arrays in each unit. The maximum useful value is maxTicks, which is also the default. This can only be changed with the setTime command. If this is less than the maximum value, the history arrays will act as circular buffers.
int backpropTicks When training a simple recurrent network, this is the number of ticks error is propagated back through time during the backpropagation phase. It defaults to 1. historyLength should be at least this large for it to work properly.
int totalUpdates The total number of weight updates that have occurred since the network was reset.
real error The sum of the error over all output units on all ticks on all examples in the training or testing batch.
real weightCost The sum of the squared link weights. This is only calculated during a weight update if a report is to be generated.
real outputCost The sum of the cost ascribed to the output of each unit with a "unit output cost function". This is reset at the start of training or testing and acrues whenever an example is processed. It is not scaled by the outputCostStrength.
real gradientLinearity The cosine of the angle between the previous weight step vector and the new weight derivative vector. This is only calculated during a weight update if a report is to be generated.
int exampleHistoryStart This is the index into the unit history arrays that corresponds to the first tick of the current example. This is necessary because the arrays are circular.
int ticksOnExample The total number of ticks that were used on the most recent example.
int currentTick As the network goes through forward and backward passes on an example, this keeps track of the current tick being processed.
flag inGracePeriod TRUE if the network is still in the grace period of the current event.
int *eventHistory An array of length maxTicks, although only the first ticksOnExample entries are meaningful. For each tick on the previous example, it stores the event that was taking place on that tick.
flag *resetHistory An array of length historyLength. It stores a boolean value that indicates whether a new example began on that location in the circular history buffer. This is used when backpropagating SRNs across multiple ticks.
int numUpdates The number of weight updates to be done on the next or current round of training.
int batchSize The number of examples over which link error derivatives are accumulated before updating the weights. A value of 0 will make the batch the entire training set.
int reportInterval The number of weight updates between each printed report during training.
real criterion Training will stop if the total error on a batch falls below this value.
real trainGroupCrit Each group can have a groupCriterionReached function that determines whether the group's outputs are good enough for the current example. The standard function returns true during training when the difference between the activation and target for each unit of the group is below the trainGroupCrit. Other functions could use this value in different ways.
real testGroupCrit This is similar to the trainGroupCrit but applies during testing.
flag groupCritRequired If this is TRUE, an example will stop (no more events will occur) if an event ends and the group criterion was not reached. This is used primarily for continuous networks. The default is FALSE.
int minCritBatches The overall error criterion or the group criterion must be met for this many consecutive batches for training to stop. A value of 0 will prevent the criteria from stopping training. The default is 1.
flag pseudoExampleFreq If TRUE, network error and error derivatives will be scaled by the current example's frequency to simulate the effect of sampling examples based on their frequency.
mask algorithm This is a bit mask which indicated which weight update algorithm is currently being used. The algorithm can be changed on the fly during training.
real learningRate Scales the magnitude of link weight changes.
real rateIncrement Used in the delta-bar-delta weight update algorithm. This is added to a link's learning rate if its weight updates are proceeding in a consistent direction. This only exists in the ADVANCED version.
real rateDecrement Used in the delta-bar-delta weight update algorithm. This is multiplied by a link's learning rate if its weight updates are not proceeding in a consistent direction. This only exists in the ADVANCED version.
real momentum The carryover of previous weight changes to the new step.
real adaptiveGainRate This is the learning rate used in updating the gain field of units in ADAPTIVE_GAIN groups.
real weightDecay Controls the reduction of weight magnitudes following weight updates. A value of 0.0 means no decay. 0.1 is a 10% reduction on each update.
real outputCostStrength The factor used to scale the derivative of the unit cost with respect to a unit's output. This adjusts how strong of an effect the unit cost has relative to the weight decay and the main error derivatives. It does not scale the outputCost itself.
real outputCostPeak For bounded units (with an output between 0 and 1), this is the output value that is most costly (having a cost of 1.0). This is continental divide of the bounded unit cost functions.
real targetRadius The distance that targets will migrate towards outputs, thus reducing the error. If this is non-zero, the target is effectively a range, rather than a point.
real zeroErrorRadius The distance from targets within which output units will incur no error.
real dt Affects how quickly the inputs or activations of integrating units in continuous networks will change. It defaults to 1/ticksPerInterval, but may be adjusted to alter the "time constant" of the network.
real gain This is the multiplicative inverse of the temperature used in logistic transfer functions. The gain is stored, rather than the temperature, to avoid divisions. The default is 1.0.
real ternaryShift This is used for ternary units, whose transfer function is the sum of two sigmoids so the unit has stable points at 0, 0.5, and 1.0. This determines the distance between the two sigmoids and thus the width of the middle plateau. The default is 5.0.
real clampStrength This parameter is used in the SOFT_CLAMP input function and the WEAK_CLAMP output function. Generally it ranges from 0 to 1. The default is 0.5. It is also used at the beginning of the negative phase in training Boltzmann machines, when the activations of non-clamped units are shifted towards the initOutput. This governs the fraction of the way the values are shifted, as in a WEAK_CLAMP.
real initOutput This is the default initial unit output for groups that have an initOutput of NaN. However, most groups will automatically determine their own initOutput so ordinarily this won't be used. It defaults to 0.5.
real initInput This is the default initial unit input for groups that have an initInput of NaN. However, most groups will automatically determine their own initInput so ordinarily this won't be used. It defaults to 0.0.
real initGain This is the temperature at which the settling process in a Boltzmann machine will begin. Note that this is an actual temperature and not a gain. The default is 1.0.
real finalGain This is the value toward which the temperature will be annealed during the settling process in a Boltzmann machine. Note that this is an actual temperature and not a gain. The default is 1.0.
real annealTime This is the time constant of the exponential decay from the initGain towards the finalGain during settling in a Boltzmann machine. The temperature will fall halfway towards the finalGain after this many ticks. The default is 1.0.
real randMean The default mean for weight randomizations.
real randRange The default range (on either side of the mean) for weight randomizations.
real noiseRange The default range (or standard deviation for Gaussian noise) of any noise functions computed in the network.
flag autoPlot Indicates whether the network is in "autoPlot" mode, which causes the Unit Viewer plot to be re-arranged whenever the network architecture changes.
int plotCols The desired width of each row in the Unit Viewer.
int plotRows The number of rows actually used by the Unit Viewer plot so far.
int unitCellSize The size of the cells in the Unit Viewer, in pixels.
int unitCellSpacing The number of pixels between cells in the Unit Viewer.
int unitUpdates An integer specifying the event on which the Unit Viewer is updated.
int unitDisplayValue The value (Activations, Inputs, etc.) being displayed in the Unit Viewer. The possible values are defined in enum unitValues in display.h.
Unit unitDisplayUnit If the Unit Viewer is displaying link weights, this is the unit that was activated with a right mouse click.
Unit unitLocked If the active unit has been locked by left-clicking in the Unit Viewer, this is a pointer to that unit.
int unitDisplaySet This is a code for the example set displayed in the Unit Viewer. Possible values are TRAINING, TESTING, or NOSET.
real unitTemp The temperature used in calculating the colors of the cells in the Unit Viewer.
double unitLogTemp The log of the temperature used in calculating the colors of the cells in the Unit Viewer. This is linked to the Tcl variable used in the slider.
int unitPalette The palette used in the Unit Viewer.
real unitExampleProc The procedure run when an example is clicked on the in the Unit Viewer. 0 is for testing and 1 is for training.
int linkCellSize The size of the cells in the Link Viewer, in pixels.
int linkCellSpacing The number of pixels between cells in the Link Viewer.
int linkUpdates An integer specifying the event on which the Link Viewer is updated.
int linkDisplayValue The value being displayed in the Link Viewer. The possible values are LINKWEIGHTS, LINKDERIVS, and LINKDELTAS.
real linkTemp The temperature used in calculating the colors of the cells in the Link Viewer.
double linkLogTemp The log of the temperature used in calculating the colors of the cells in the Link Viewer. This is linked to the Tcl variable used in the slider.
int linkPalette The palette used in the Link Viewer.
Unit linkLockedUnit If the active link has been locked by left-clicking in the Link Viewer, this is a pointer to the unit receiving that link.
int linkLockedNum If the active link has been locked by left-clicking in the Link Viewer, this is the index of that link in the incoming array of the receiving unit.
Tcl_Obj *preEpochProc This command is run before each batch of examples is processed during training. It could be used to adjust the learning rate of other parameters automatically at certain intervals.
Tcl_Obj *postEpochProc This is run after link derivatives have been gathered on a batch of examples during training but before the weight update.
Tcl_Obj *postUpdateProc This is run after a weight update, either during normal training or when the updateWeights command is used.
Tcl_Obj *preExampleProc This is run before each example during training or testing.
Tcl_Obj *postExampleProc This is run after each example during training or testing.
Tcl_Obj *preExampleBackProc In a network that does the backward pass in one sweep for the whole example (such as a CONTINUOUS or SRBPTT net), this is run after the forward sweep and before the backward one.
Tcl_Obj *preEventProc This is run before each example event during training or testing. Note that this is not run every tick, only when a new event begins.
Tcl_Obj *postEventProc This is run after each example event during training or testing. Note that this is not run every tick, only when an event ends.
Tcl_Obj *preTickProc This is run before each tick.
Tcl_Obj *postTickProc This is run after each tick.
Tcl_Obj *preTickBackProc In a network that does a forward and backward pass on each tick, such as a standard net or SRN, this is run between the forward and backward pass.
Tcl_Obj *sigUSR1Proc This procedure will be run when a USR1 signal is received. The initial value can be set in .lensrc but defaults to "saveWeights quicksave.wt.gz -3".
Tcl_Obj *sigUSR2Proc This procedure will be run when a USR2 signal is received. It defaults to "saveParameters quicksave.prm.gz".
char *outputFileName If the network is writing outputs and targets to an output file, this is the name of the file.
Tcl_Channel outputFile If the network is writing outputs and targets to an output file, this is the channel used for writing.
flag binaryOutputFile If the network is writing outputs and targets to an output file, this is TRUE if the outputs are in binary format.
flag (*netTrain)
(mask alg)		 The function used to train the network on a series of batches. The default is standardNetTrain. alg is a type mask specifying the weight update algorithm to be used. The algorithm must have been registered with registerAlgorithm.
flag (*netTrainBatch)
(flag *allCorrect)		 The function used to gather link error derivatives on a batch of examples. The default is standardNetTrainBatch. allCorrect is used to return a boolean indicating whether the groupCriterionReached was true for all group's in the network. This is always false if no groups have a criterion function.
flag (*netTrainExample)
(Example E,
flag (*tickProc)(Event),
flag *correct)		 The function used to perform the forward and backward passes in the network on a single example. The default is standardNetRunExample. E is the example to be processed. tickProc is a function that will be called to update the activations (and possibly backpropagate error) on each tick. correct is used to return a boolean indicating whether the groupCriterionReached was true for all groups on all events in the example.
flag (*netTrainTick)
(Event V)		 The function used to perform one forward pass and possibly one backpropagation pass during a single tick of training. The default is netForwardBackward.
flag (*netTestBatch)
(int numExamples,
flag resetError)		 The function used to test the network on a bunch of examples. The default is standardNetTestBatch. If numExamples is 0, the testing set will be reset to the first example and the entire set used. If resetError is true, the network's error field will be reset to 0.0 before starting.
flag (*netTestExample)
(Example E,
flag (*tickProc)(Event),
flag *correct)		 The function used to perform the forward pass in the network on a single example during testing. The default is standardNetRunExample.
flag (*netTestTick)
(Event V)		 Similar to netTrainTick but used when testing rather than training. The default is netForward.
flag (*netRunExampleNum)
(int num, ExampleSet S)		 The function called when an example is selected in the Unit Viewer. Basically all that happens is that netRunExample is called on the example. The default is standardNetRunExampleNum.
flag (*netRunExample)
(Example E,
flag (*tickProc)(Event),
flag *correct)		 Similar to netTrainExample but this is used when an example is selected in the Unit Viewer. This is generally set to the same value as either netTrainExample or netTestExample. The default is standardNetRunExample.
flag (*netRunTick)
(Event V)		 Similar to netTrainTick but used when an example is selected in the Unit Viewer rather than when training. The default is netForward.
flag (*resetNet)
(flag randomize)		 The function used when resetNet is called. The default is standardResetNet.
flag (*saveWeights)
(char *fileName,
flag binary)		 The function used to save the network's weight to a file. The default is standardSaveWeights. binary indicates whether weights should be saved in binary format.
flag (*loadWeights)
(char *fileName)		 The function used to load the network's weights from a file. The default is standardLoadWeights.
flag (*writeExample)
(Example E)		 The function used to write the information about an example to the outputFile, if one has been opened. The default is standardWriteExample.

Group

 Defined in network.h.
char *name The name of the group.
int num The index of the group in the network's group array.
mask type Holds the fixed group type, such as INPUT or OUTPUT, as well as a number of miscellaneous types for a group.
mask inputType A conjuction of the types of all of the input functions in the group.
GroupProc inputProcs A linked list of the functions that are used to compute the input of each unit in the group.
mask outputType A conjuction of the types of all of the output functions in the group.
GroupProc outputProcs A linked list of the functions that are used to compute the output of each unit in the group.
mask costType A conjuction of the types of all of the cost functions in the group.
GroupProc costProcs A linked list of the functions that are used to compute the error and output error derivative of each unit in the group. This includes both error functions like CROSS_ENTROPY and unit output cost functions like COSINE_COST.
Network net A pointer to the network which owns this group. It may only be used in object.c for traversing the objects.
int numUnits The number of units in the group.
Unit unit An array of unit structures. The units are arranged in a contiguous block to reduce the number of pointer dereferences necessary to traverse all links projecting from the group to a particular unit.
real *output This array is used to cache the outputs of the units in the group. This reduces the amount of memory accessed in the inner loops, improving performance on large networks.
real *outputDeriv This array is used to cache the outputDerivs of the units in the group. In continuous networks, this is used to accumulate the instant output derivatives.
int numIncoming The number of links coming into the group. This is mostly for reference.
int numOutgoing The number of links going out of the group. This is mostly for reference.
GroupExt ext A pointer to the group extension structure. This is defined in extension.h and allocated in extension.c. Programmers should try to put any extensions to the group structure in there.
real trainGroupCrit If this is not NaN, it will override the network's value. The group's criterionReached function uses this during training to determine when the event or example has been processed correctly.
real testGroupCrit If this is not NaN, it will override the network's value. The group's criterionReached function uses this during testing to determine when the event or example has been processed correctly.
real learningRate The learning rate of links into this group. If not NaN, the default, this will override the network's value but can be overridden by values in the block structures of units in the group.
real momentum The momentum acting on links into this group. If not NaN, the default, this will override the network's value but can be overridden by values in the block structures of units in the group.
real weightDecay The weightDecay of links into this group. If not NaN, the default, this will override the network's value but can be overridden by values in the block structures of units in the group.
real targetRadius The distance that targets will migrate towards outputs, thus reducing the error. If this is non-zero, the target is effectively a range, rather than a point. If this is NaN, the default, the network's value will be used.
real zeroErrorRadius The distance from targets within which output units will incur no error. If this is NaN, the default, the network's value will be used.
real outputCostScale This scales the output cost assigned to units in the group, allowing the cost of some groups to be more influential than that of other groups. The default is 1.0.
real outputCostPeak If not NaN, this overrides the network's value.
real outputCost The outputCost incurred by this group on the most recent batch.
real errorScale Scales the error and error derivatives produced at this group, which would typically have to be an output group. This allows some output groups can be made more important than others.
real error The error incurred by this group on the most recent batch.
real polaritySum The sum of the polarity of all the units in the group summed over all of the times that "polarity update" was called on the group.
int polarityNum The number of observations that have gone into the polaritySum (used to compute the average polarity).
real dtScale Scales the rate at which units will adapt if this is an input or output integrating group in a continuous network. The default is 1.0, which means that the network's dt will be used as is. Higher values make the units in the group respond quicker and lower values make the units more sluggish. Note that each unit also has its own dtScale.
real gain The multiplicative inverse of the temperature used in logistic transfer functions in the group. The gain is stored to avoid divisions. If set, this will override the network's value.
real ternaryShift Overrides the network's value if not NaN.
real clampStrength Overrides the network's value if not NaN.
real minOutput The minimum output that should be produced by a unit in this group. It is computed automatically based on the group's type whenever the type changes but can then be overridden. It is used by the OUT_CROP function and affects how the outputs are displayed in the Unit Viewer.
real maxOutput The maximum output that should be produced by a unit in this group. It is computed automatically based on the group's type whenever the type changes but can then be overridden. It is used by the OUT_CROP function and affects how the outputs are displayed in the Unit Viewer.
real initOutput The value to which unit outputs are set when the group is reset at the start of an example. This is also computed automatically.
real initInput The value to which unit outputs are set when the group is reset at the start of an example. Input integrators are also reset to this value before integration begins. This is not computed automatically and defaults to 0.0.
real randMean The mean of the distribution used when randomizing the weights of links into this group. If not NaN, the default, this will override the network's value but can be overridden by values in the block structures of units in the group.
real randRange The range of the distribution used when randomizing the weights of links into this group. If not NaN, the default, this will override the network's value but can be overridden by values in the block structures of units in the group.
real noiseRange Overrides the network's value if not NaN.
real (*noiseProc)(real value,
real range)		 The command used to generate noise in this group. It will be one of multGaussianNoise, addGaussianNoise, multUniformNoise, or addUniformNoise.
flag showIncoming A boolean indicating whether incoming links to the group are being displayed in the Link Viewer.
flag showOutgoing A boolean indicating whether outgoing links from the group are being displayed in the Link Viewer.
int inPosition The horizontal position of this group on the Link Viewer.
int outPosition The vertical position of this group on the Link Viewer.
int numColumns Some algorithms treat the group as a two-dimensional array. For example, this is done in Kohonen nets. This parameter sets the number of columns (the width) of this matrix. The number of rows will depend on the number of units. If non-zero, this determines the width of the group when auto-plotted. The default is 0.
real neighborhood This is used in Kohonen nets to determine the radius of influence around the winning unit. It could be used for similar purposes in other algorithms.
int periodicBoundary This is used for networks, such as Kohonen nets, that treat the group as a two-dimensional layer with some distance metric between units. If this is true, units on opposite edges of the rectangle are considered adjacent. The default is FALSE.
flag (*groupCriterionReached)
(Group G)		 This function is called on every tick during training or testing. It indicates whether the group's activations are "good enough". The standard function tests whether the difference between each unit's output and target is less than or equal to the groupCriterion.

GroupProc

 These form a doubly linked list of procedures that are used to calculate the inputs, outputs, or error for a group. Defined in network.h.
mask type The type of the procedure.
int class The type class of the procedure. This is either GROUP_INPUT, GROUP_OUTPUT, or GROUP_COST.
Group group The group which owns this GroupProc.
void (*forwardProc)
(Group G, GroupProc P)		 The procedure run on the group in the forward pass.
void (*backwardProc)
(Group G, GroupProc P)		 The procedure run on the group in the backward pass. This typically propagates error backward to the previous procedure.
real *groupHistoryData Information these procedures need to store for each tick.
real *unitData Information these procedures need to store for each unit.
real **unitHistoryData Information these procedures need to store for each tick for each unit.
void *otherData This could be used for anything but its main current use is to hold a pointer to the source group for ELMAN_CLAMP procedures.
GroupProc next The next group in the linked list. This is NULL for the last GroupProc.
GroupProc prev The previous group in the linked list. In the first GroupProc, this points to the last one.

Unit

 Defined in network.h.
char *name The name of the unit.
int num The index of the unit in the group's unit array.
mask type The only unit type bits so far are LESIONED and FROZEN.
Group group A pointer to the unit's parent group.
int numBlocks The number of blocks of links from contiguous sending units reaching this unit.
Block block An array of block structures describing the blocks of incoming links.
int numIncoming The number of links feeding the unit. These links are actually stored in the unit.
Link incoming An array of link structures containing weights and derivatives of all of the incoming links. The block structures determine which sending units the links come from.
Link2 incoming2 This contains all of the link fields other than weight and deriv. This is kept separate because it is used less often and thus won't pollute the cache.
int numOutgoing The number of links leaving the unit. The links are stored in the receiving units.
UnitExt ext A pointer to the unit extension structure. This is defined in extension.h and allocated in extension.c. Programmers should try to put any extensions to the unit structure in there.
real input The total input to the unit.
real output The unit's output on the current tick, also called its activation.
real target The unit's target on the current tick if it is an output unit.
real externalInput The input from the example event. This is used by HARD_CLAMP, SOFT_CLAMP, and WEAK_CLAMP.
real adjustedTarget This is used in act.c when there is a targetRadius or zeroErrorRadius around the target.
real inputDeriv The instant derivative of the error w.r.t. the unit's input.
real outputDeriv The derivative of the error w.r.t. the unit's output. Includes both instantaneous derivs resulting from cost functions and backpropagated derivs.
real *inputHistory An array of the unit's inputs on each tick of the current example.
real *outputHistory An array of the unit's outputs on each tick of the current example.
real *targetHistory An array of the unit's targets on each tick of the current example.
real *outputDerivHistory An array of the unit's instantaneous outputDerivs on each tick of the current example. This only exists for groups with cost functions. The instantaneous outputDerivs include the error and output cost derivs resulting from the output on a single tick, not including any backpropagated derivatives.
real gain The gain of this unit's output function, if it has ADAPTIVE_GAIN.
real gainDeriv The derivative of the gain of this unit's output function, if it has ADAPTIVE_GAIN.
real dtScale This is like the group-level dtScale except that it applies to just this unit in determining the unit's time constant for input or output integrating units. The actual dt for the unit is the product of the network's dt, the group's dtScale, and the unit's dtScale.
int plotRow The vertical position of the unit on the Unit Viewer.
int plotCol The horizontal position of the unit on the Unit Viewer.

Block

 Defined in network.h.
mask type The link type of the links in this block. Types are created by the user with addLinkType or created automatically when a projection is formed and no type is specified. Link types are actually integers and not really bit masks. The special type ALL_LINKS (which is equal to 0) will match any type.
int numUnits The number of contiguous sending units in this block.
Unit unit A pointer to the first of the contiguous sending units.
int groupUnits The number of units in sending group. This is used in act.c because the output and outputDeriv arrays of the group are contiguous.
real *output A pointer into the output array of the group to which the sending units belong. The first value is the output of the first unit.
real learningRate The amount used to scale the derivatives when updating the weights of links in this block. If NaN, it will fall back to the group's and then to the network's value.
real momentum The momentum of weight updates of links in this block. If NaN, it will fall back to the group's and then to the network's value.
real weightDecay The amount by which the weights of links in this block are scaled after they are updated. If NaN, it will fall back to the group's and then to the network's value.
real randMean The mean of weight randomizations in this block. If NaN, it will fall back to the group's and then to the network's value.
real randRange The range of weight randomizations in this block. If NaN, it will fall back to the group's and then to the network's value.
real min If not NaN, this is the minimum weight that a link in this group can have. Values less than this are set to this after a weight update.
real max If not NaN, this is the maximum weight that a link in this group can have. Values greater than this are set to this after a weight update.
BlockExt ext A pointer to the block extension structure. This is defined in extension.h and allocated in extension.c. Programmers should try to put any extensions to the block structure in there and will need to define a copy function which will be used when blocks are split.

Link

 Defined in network.h.
real weight The link weight.
real deriv The accumulating derivative of the error w.r.t. the link weight.

Link2

 Defined in network.h. A companion to the Link.
real lastWeightDelta The last change made to the weight. This is used for momentum and for keeping statistics in steepest descent.
real lastValue This only exists in ADVANCED mode. It is used in delta-bar-delta and could be used in other algorithms that need a fourth link field.

ExampleSet

 Defined in example.h.
char *name The name of the example set.
int num The index of the set in the root set array.
mask mode The mode for selecting the next example. Possible values are defined in type.h.
int numExamples The number of examples in the set.
int numEvents The total number of events in all examples in the set. This is just for reference.
ExSetExt ext A pointer to the example set extension structure. This is defined in extension.h and allocated in extension.c. Programmers should try to put any extensions to the example set structure in there.
Example *example An array that stores pointers to each example in the order in which the examples are arranged in the linked list. The linked list is kept to avoid reallocations while reading the set from a file and the example array is compiled once the set is complete and is used from then on.
Example *permuted An array of pointers to all examples used in the PERMUTED example set mode.
int currentExampleNum The index to the current example in either the ordered or permuted array. If in PERMUTED mode, the currentExampleNum will not necessarily correspond to currentExample->num.
Example currentExample A pointer to the current or most recent example.
Example firstExample The first example in the linked list of examples.
Example lastExample The last example in the linked list of examples.
struct example pipeExample A single example structure used to store the current example in PIPE mode.
ParseRec pipeParser This contains the data necessary to read the example file when running in PIPE mode.
flag pipeLoop If this is TRUE, which is the default, example sets reading from a pipe will re-open the pipe when they reach the end.
int pipeExampleNum If in PIPE mode, the number of examples that have been read since the pipe was last opened.
Tcl_Obj *proc Procedure that is run as soon as it is read when this set was loaded from a file. Typically this would define procedures used by example or event commands.
Tcl_Obj *chooseExample Procedure that is called in CUSTOM example selection mode. It should return the index of the next example.
real maxTime The default maxTime for any event that doesn't have one specified.
real minTime The default minTime for any event that doesn't have one specified.
real graceTime The default graceTime for any event that doesn't have one specified.
real defaultInput The default defaultInput for any event that doesn't have one specified.
real activeInput The default activeInput for any event that doesn't have one specified.
real defaultTarget The default defaultTarget for any event that doesn't have one specified.
real activeTarget The default activeTarget for any event that doesn't have one specified.
short numGroupNames The number of groups referenced by name in this example set.
short maxGroupNames The number of slots currently allocated for group names in this example set.
char **groupName An array of group names used in direct group references in the example set. Keeping them in one place eliminates the need to allocate identical strings in each event.
flag (*loadEvent)
(Event V);		 Loads the inputs and targets for an event from an example in the set into the network. The default is standardLoadEvent.
flag (*loadExample)
(Example E)		 Starts loading an example into the network, including loading the first event. The default is standardLoadExample.
flag (*nextExample)
(ExampleSet S)		 Chooses the next example from the set and starts loading it into the network. The default is standardLoadNextExample.

Example

 Defined in example.h.
char *name The name of the example.
int num The index of the example in the set's ordered array.
int numEvents The number of events in the example.
Event event An array of events in the example.
ExampleSet set The set containing this example.
Example next A pointer to the next example in the linked list.
ExampleExt ext A pointer to the example extension structure. This is defined in extension.h and allocated in extension.c. Programmers should try to put any extensions to the example structure in there.
real frequency Used when selecting examples probabilistically or with pseudo-frequencies. The frequency is a value requested by the user. The default is 1.0. When in PROBABILISTIC mode, the absolute value doesn't matter only the value relative to that of other examples. When using pseudo-example-frequencies, this value directly scales the error and output-unit output derivatives on this example.
real probability When in PROBABILISTIC mode, this is the probability that this example or one before it is selected.
Tcl_Obj *proc The command run before the first event begins.

Event

 Defined in example.h.
Range input This is a linked list of range specifications which define the externalInputs of INPUT units.
flag sharedInputs If this is TRUE, the event is sharing inputs that are owned by an earlier event. This information is needed when the event is freed or the example set is saved.
Range target This is a linked list of range specifications which define the targets of OUTPUT units.
flag sharedTargets If this is TRUE, the event is sharing targets that are owned by an earlier event. This information is needed when the event is freed or the example set is saved.
real maxTime The maximum number of ticks the event will last. If NaN, the example set's maxTime will be used.
real minTime The minimum number of ticks the event will last. If NaN, the example set's minTime will be used.
real graceTime The number of ticks between turning on the inputs for this event and turning on the targets. Targets will be NaN before that. If graceTime is NaN, the example set's graceTime will be used.
real defaultInput The input value used for inputs that don't appear in a lot if sparse inputs are being used.
real activeInput The input value used for the first sparse lot if a value is not otherwise specified. This is only referred to when the example is read in.
real defaultTarget The target value used for outputs that don't appear in a lot if sparse targets are being used.
real activeTarget The target value used for the first sparse lot if a value is not otherwise specified. This is only referred to when the example is read in.
Tcl_Obj *proc The command that is run just after the event is loaded into the network.
Example example The example to which the event belongs.
EventExt ext A pointer to the event extension structure. This is defined in extension.h and allocated in extension.c. Programmers should try to put any extensions to the event structure in there.

Range

 Defined in example.h. This describes a set of inputs or targets. These can either be in a dense or a sparse encoding.
char *groupName If this is NULL, the unit indices in this range refer to the whole network. Otherwise they are only with respect to this group.
int numUnits The number of units described by this range in either the val or unit array.
int firstUnit Used only for dense encodings. This is the index of the first unit for which a value is specified in the val array.
real *val Used only for dense encodings. These are the values (either externalInput or target) for the consecutive range of numUnits units starting with firstUnit.
real value Used only for sparse encodings. This is the value (either externalInput or target) that will be assigned to each unit listed in the unit array.
int *unit Used only for sparse encodings. An array of the indices of the units which will receive the value.
Range next The next range in the linked list.

Graph

 Defined in graph.h. This maintains the state needed for a graph.
int num The graph number.
mask updateOn The event on which the graph should update itself.
int updateEvery How many of these events must occur before the graph actually updates itself.
int x A count of the number of events the graph has seen, which determines the x value of points plotted.
int maxX The maximum x value in any trace (this could be larger than x if a trace is non-active).
int updatesSinceRedraw The number of updates that have occurred since the last time the graph was redrawn from scratch. The graph is completely redrawn every 100 updates for better efficiency.
int width The width, in pixels, of the graph canvas.
int height The height, in pixels, of the graph canvas.
int cols The number of columns, or x values, that can fit in the graph window.
real max The y value to appear at the top of the graph.
real min The y value to appear at the bottom of the graph.
flag fixMax Whether the max is fixed or based on maxVal.
flag fixMin Whether the min is fixed or based on minVal.
real maxVal The maximum y value that actually occurs in any trace in the graph.
real minVal The minimum y value that actually occurs in any trace in the graph.
real scaleX A value used in plotting an (x,y) pair in graph coordinates.
real scaleY A value used in plotting an (x,y) pair in graph coordinates.
flag clearOnReset If true, the graph will be cleared when the network is reset.
flag storeOnReset If true (and clearOnReset is false), the graph will be stored when the network is reset.
flag hidden Whether the graph is hidden or visible.
flag needsRedraw True if a redraw has been requested for this graph.
flag needsPropRefresh True if the graph properties window needs to be refreshed.
flag tracesChanged True if a trace has been added or deleted and the traces needs to be redrawn in the graph properties window.
flag propertiesUp True if the graph properties window is open.
int numTraces The amount of space that has been allocated for traces in the trace array. Not all of the spots are necessarily filled.
Trace *trace An array of pointers to the traces in the graph.

Trace

 Defined in graph.h. This maintains the state needed for a single trace in a graph.
int num The number of the trace within its graph.
Graph graph The trace's parent graph.
char *label The label of the trace, only really used in exporting.
char *object The string containing either the object name or the procedure used to get values for this trace.
char *lastObject The most recent object for which this trace has been used. When this differs from the object field, it means the object has been changed and the value pointer or proc must be updated.
real *value If an object name, this points directly to said object for fast updating.
Tcl_Obj *proc If a procedure, this is the procedure whose execution provides the next value.
int numVals The number of points actually used in this trace.
int maxVals The number of points allocated for this trace.
point *val The actual data, an array of (x,y) pairs.
char *color The name of the color used for this trace, usually in hex.
flag active Whether the trace is active, meaning that it will get a new value on an update.
flag transient Whether the trace is transient, meaning that it will be destroyed on a clear.
flag visible Whether the trace is visible and exportable.
int export A value used when exporting tabular data.

Point

 Defined in graph.h. This holds a single data point in a graph trace.
int x The x coordinate of the point.
real y The y value of the point.

Algorithm

 Defined in train.h. Keeps track of weight update algorithms registered with registerAlgorithm().
mask code The bitmask identifying the algorithm. Masks are defined in type.h and new masks must be chosen by the user.
char *shortName The short name of the algorithm, such as "momentum". This becomes the name of the shell command for running the algorithm or the name of the algorithm passed to train.
char *longName The long name of the algorithm, such as "Momentum Descent". Used on the graphical displays and for messages.
void (*updateWeights)
(flag doStats)		 The command for performing weight updates with this algorithm. If doStats is true, the function should calculate the network's weightCost and gradientLinearity as it updates the weights.
Algorithm next The next algorithm in the AlgorithmTable.

ObjInfo

 Defined in object.h. Used to build a description of one of the other structures, such as the network, a group, example set or a basic type like an int, to allow the structure to be traversed and viewed using getObject or the Object Viewer. This is created with the newObject() function.
char *name The name of this type of object, such as "Network" or "Group".
int size The number of bytes in this type of object.
int maxDepth When traversing the object tree, this is the maximum depth (number of steps to children or array elements), below the network, at which this type of object will be expanded. This is because the object tree is actually circular in places and it would not be reasonable to follow the loops indefinitely. This wil be -1 for primitive types (non-structures).
void *(getName)
(void *object, char *name)		 A function to return the name of this type of object given a pointer to the object. The name will be written into the name string. For most structures, this will be the name field. For primitive types, such as an int, the type will be printed as a string.
void (*setValue)
(void *object, char *value)		 A function for setting the value of a primitive type. This will not be defined for structures..
void (*setStringValue)
(void *object, char *value)		 A function for setting the value of a primitive type given a string representation of the value. This will not be defined for structures.
MemInfo members The head of a linked list of member structures describing the fields if the object is a structure.
MemInfo *tail A pointer to the next pointer of the last member in the list to allow new members to be added to the end.

MemInfo

 Defined in object.h. Used to describe a field of a C structure to allow the structure to be traversed and viewed using getObject or the Object Viewer. This is created with the addMember() function.
char *name The name of the field. Typically this will be the same as the field name used in C, although it would not have to be.
int type This can be OBJ, OBJP, OBJA, OBJPA, OBJAA, or OBJPAA, which specifies whether this field is an object (a primitive type or an embedded structure), a pointer to an object, an array of objects, an array of pointers to objects, or a two dimensional array of objects or pointers to objects.
flag writable Whether the user can change this field from the shell.
int offset The number of bytes between the start of the parent object and the location of this field.
int (*rows)(void *) If the field is an array, this function returns an integer giving the first dimension of the array.
int (*cols)(void *) If the field is a 2D array, this function returns an integer giving the second dimension of the array.
ObjInfo info This is the ObjInfo structure which defines the type of object that this field is. If this field is an array of objects or pointers to objects, ObjInfo corresponds to the objects in the array.
MemInfo next The next field in the linked list.

String

 Defined in util.h. This holds a string that will resize itself dynamically. These are used occasionally in Lens, usually for parsing text files.
int maxChars The number of bytes allocated in s.
int numChars The number of characters, not counting the '\0', used in s.
char *s The actual string.

ParseRec

 Defined in util.h. This is used to make parsing example files easier.
Tcl_Channel channel A pointer to the actual file being read.
char *fileName The name of the file being read.
char cookie[sizeof(int)] This stores the characters that might have been in the cookie so the parser can recover from having tried to read the cookie.
int cookiePos The position within the cookie array that is currently being read.
flag binary Whether the file is in binayr format.
int line The number of the most recent line read from the file. For binary files, this is the number of the example being parsed.
String buf A dynamic string that holds the portion of the most recently read line that has yet to be completely parsed.
char *s A pointer to the character within buf->s up to which the text has been parsed.

C Macros

 These are the #define macros and constants that might be most useful to a programmer. Numerous other definitions are found in system.h, type.h, act.h, and defaults.h.
TRUE Should be used rather than 1 when a boolean value is needed.
FALSE Should be used rather than 0 when a boolean value is needed.
TCL_OK The boolean value to be returned when a function completes successfully. This is equivalent to FALSE.
TCL_ERROR The boolean value to be returned when a function incurred an error. This is equivalent to TRUE.
ADVANCED This will be defined by the Makefile if Lens is to be compiled in "advanced" mode, meaning that the links contain the extra lastValue field.
LITTLE_END This is defined if the machine uses little-endian byte ordering, which is not the network standard ordering.
HTONL(x) Converts a four-byte number (integer or real) from host byte-order to network (big endian) byte order. The new value is returned.
NTOHL(x) Converts a four-byte number (integer or real) from network (big endian) byte-order to host byte order. The new value is returned.
OFFSET((void *) object, field f) This returns the number of bytes between the start of a structure and a particular field in the object. object is a pointer to a structure and f is the name of the field in the object.
SAME(x,y) This should be use to compare two real numbers when there is some chance that they both are NaN. Ordinarily (NaN == NaN) will evaluate to FALSE. SAME(NaN, NaN), however, will be TRUE.
MIN(x,y) The minimum of two numbers, integer or real. The function imin() or rmin() should be used, rather than the macro MIN() if the arguments are complex expressions.
MAX(x,y) The maximum of two numbers, integer or real. The function imax() or rmax() should be used, rather than the macro MAX() if the arguments are complex expressions.
NO_VALUE The string used to indicate that a real number should be NaN. This defaults to "-".
CROSS_ENTROPY_ERROR(y, d) If y is an output and d is a target, this computes the cross entropy error.
CROSS_ENTROPY_DERIV(y, d) If y is an output and d is a target, this computes the derivative of the cross entropy error.
DIVERGENCE_ERROR(y, d) If y is an output and d is a target, this computes the divergence error.
DIVERGENCE_DERIV(y, d) If y is an output and d is a target, this computes the derivative of the divergence error.
FOR_EACH_LINK_FORW
(U, proc)		 Loops over all the incoming links to Unit U and computes the proc for each one. This is usually used for input functions. The proc can make use of the L_WGT and other macros listed below.
FOR_EACH_LINK_FAST_FORW
(U, proc, unrollproc)		 This is like FOR_EACH_LINK_FORW but will unroll the loop 10 times when possible for greater speed. proc is used when running on a single link and unrollproc is used when running on ten consecutive links. You should avoid using this unless you really know what you're doing.
FOR_EACH_LINK_BACK
(U, proc)		 Loops over all the incoming links to Unit U and runs the proc for each one to backpropagate error derivates. The proc can make use of the L_WGT, L_DRV, and other macros listed below.
FOR_EACH_LINK_FAST_BACK
(U, proc, unrollproc)		 This is the unrolled version of FOR_EACH_LINK_BACK. Again, avoid it unless you really know what you're doing.
L_WGT When using one of the above macros, this is the preferred method for referencing the link weight.
L_WGT_(i) When using a _FAST_ macro, this references the i_th link weight, where i ranges from 0 to 9.
L_DRV The derivative of the link.
L_DRV_(i) When using a _FAST_ macro, this is the derivative of the i_th weight.
V_OUT The output of the sending unit.
V_OUT_(i) The output of the i_th sending unit for _FAST_ macros.
V_DRV The output derivative of the sending unit.
V_DRV_(i) The output derivative of the i_th sending unit for _FAST_ macros.
UPDATE(Group G) Computes a groups input, output, and cost by calling computeInput(), computeOutput(), and computeCost().
BACKPROP(Group G) Backpropagates a group's outputDeriv to its inputDeriv and then to the links and units projecting to the group by calling computeOutputBack() and computeInputBack().
IN_BLOCK(Unit U, Block B) Evaluates to TRUE if the unit produces an outgoing link in the given block of links.
IN_GROUP(Block B, Group G) Evaluates to TRUE if the units producing the outgoing links that form the given block are in group G.
FOR_EACH_GROUP(proc) Runs the proc code for each group in the network. When it does so, the local variable G will be a pointer to the group and g will be its index.
FOR_EACH_GROUP_BACK(proc) Similar to the previous macro but loops over the groups in opposite their normal updating order.
FOR_EACH_GROUP_IN_RANGE
(first, last, proc)		 Loops over the groups with indices from first to last, inclusive.
FOR_EACH_GROUP_IN_RANGE_BACK
(last, first, proc)		 Loops over the groups with indices from last to first, inclusive.
FOR_EACH_GROUP_IN_LIST
(_list, _proc)		 Loops over the groups whose names are listed in _list and executes _proc on each. * matches all groups.
FOR_EACH_UNIT(G, proc) Runs the proc code for each unit in the group. When it does so, the local variable U will be the current unit and sU should not be messed with. This will not run the proc on LESIONED units.
FOR_EACH_UNIT2(G, proc) This is like FOR_EACH_UNIT, but it makes available the u variable containing the index of the current unit.
FOR_EVERY_UNIT(G, proc) The same as above but it will operate on all units, including LESIONED ones.
FOR_EACH_UNIT_IN_LIST(_list, _proc) If _list is a <unit-list>, this will run _proc on every unit in the list. The unit will be U.
FOR_EACH_BLOCK(U, proc) This will run proc for each block in the unit. You can use the block pointer, B, or the block index u.
FOR_EACH_LINK(U, proc) This loops over the incoming array of the unit. You can use this to do something to the weights or derivatives of the links.
FOR_EACH_GRAPH(proc) Runs the _proc code for each graph. When it does so, the local variable G will be a pointer to the graph.
FOR_EACH_GRAPH_IN_LIST(_list, _proc) Runs the _proc code for each graph whose number is listed in the string _list.
FOR_EACH_TRACE(G, proc) Runs the _proc code for each trace in graph G. When it does so, the local variable T will be a pointer to the trace.
FOR_EACH_TRACE_IN_LIST(G, _list, _proc) Runs the _proc code for each trace in G whose number is listed in the string _list.
FOR_EACH_STRING_IN_LIST(_list, _proc) Runs the _proc code for each field in the string _list. The String S (not a char *, but a String) will hold the string when _proc is called.
HISTORY_INDEX(tick) The history arrays in the units are circular. Given the tick number of the current example, this gives the index into the array where data is stored for that tick.
SET_HISTORY(Unit U,
real *array, int index, real value)		 This should be used to set any field in one of a unit's history arrays. It checks to be sure the array exists and then treats it as a circular array.
GET_HISTORY(Unit U,
real *array, int index)		 This should be used to get the value of any field in one of a unit's history arrays. It checks to be sure the array exists and then treats it as a circular array. It evaluates to the value of the requested element, or NaN if the element doesn't exist.
STORE_TO_HISTORY
(G, value, array, tick)		 This stores the values in the value field of a unit to one of its history arrays.
STORE_TO_OTHER_HISTORY
(G, value, H, array, tick)		 This stores the values in the value field of the units in group G to a history array in the units of group H. The two groups should have the same number of units.
RESTORE_FROM_HISTORY
(G, array, tick, value)		 This copies values from a history array to the value field for each unit in G.
RESTORE_FROM_OTHER_HISTORY
(G, array, tick, H, value)		 This copies values from a history array in group G to the value field of the units in group H. The two groups should have the same number of units.
SET_VALUES
(G, value, val)		 This sets field value of each unit in G to val.
COPY_VALUES
(F, valueF, T, valueT)		 This copies values from field valueF in the units of group F to field valueT in the units of group T.
COPY_HISTORY
(F, arrayF, tickF, T, arrayT, tickT)		 This copies values from one tick in a history array of group F to one tick in a history array of group T.
CLEAR_HISTORY
(G, tick)		 This sets the output and target histories of group G for the specified tick to NaN.
STORE_HISTORY
(G, tick)		 This stores the outputs to the outputHistory and the targets to the targetHistory for the specified tick.
RUN_PROC
(proc)		 This is a safe way to run one of the network's procedures, such as the preEpochProc. If the proc is not set, nothing will happen.
UPDATE_WEIGHTS
(proc)		 This is used in defining weight update functions. It takes care of all of the traversal and calculating stats for the report. The proc you specify just does the weight update on a single link.

Global C Variables

 This represents nearly all of the global variables shared between modules in Lens. Other global variables of type ObjInfo containing the description of object structures are not listed here, nor are those only used within a single module.
extern RootRec Root A pointer to the single root structure that contains the arrays of all networks and all example sets. Each network has a copy of this pointer so users can access all networks and example sets.
extern Network Net By far the most-used global variable, this is a pointer to the current network.
extern char **LinkTypeName An array of names for the user-defined link types.
extern Algorithm AlgorithmTable A linked list of registered weight-update algorithms.
extern flag UnitUp Whether the Unit Viewer is visible.
extern flag LinkUp Whether the Link Viewer is visible.
extern int ViewTick The tick that is being viewed in the Unit Viewer.
extern int ParallelState Indicates whether the Lens process is acting as a client, a server, or not involved in parallel training.
extern int ClientsWaiting The number of clients waiting at a barrier for the network to release them.
extern Tcl_Interp *Interp A pointer to the Tcl interpreter running in the Lens process. This is often needed for creating return values or making other Tcl library calls.
extern char Buffer[ ] A buffer used to temporarily store commands to be evaluated, return values, or just for use as a generic temporary storage.
extern flag Gui Indicates whether the Main Window is viewable and, thus, Lens is running in graphical mode.
extern flag Batch Indicates whether Lens is running in batch mode, and thus none of the graphical displays will ever be viewable.
extern flag Console Indicates whether the Lens console has been opened.
extern int Verbosity This controls how verbose messages are in Lens. 1 is the default. 0 will turn off status messages that aren't errors. 2 may be useful at some point, but there hasn't been much call for it yet.
extern char *RootDir The main Lens directory.

Global Tcl Variables

 These are global variables used in the Tcl code. In addition, there will be a global variable for each object that is set with setObject whose name is the name of the object preceded with a period. There are a number of other global Tcl variables not listed here.
.ADVANCED Whether Lens has been compiled in "advanced" mode (allowing more complicated learning algorithms.
.BATCH Whether Lens is running in (graphics disabled) batch mode.
.RootDir The .../Lens directory from which Lens is running.
.ManPage The web page containing the Lens manual.
.Version The current version number.
.Commands A list of all of the registered Lens commands.
_Synopsis An array containing a short explanation of each Lens command.
_Aliases An array containing the code to be substituted for aliased commands.
auto_path The directory path used to search for commands to load automatically.
env An array containing the values of all of the shell environment variables.
.unitUp A boolean indicating whether the Unit Viewer is displayed.
.unitUpdates A number indicating the rate at which the Unit Viewer is updated.
.linkUp A boolean indicating whether the Link Viewer is displayed.
.linkUpdates A number indicating the rate at which the Link Viewer is updated.
.borderColor The color of the raised ridge around window frames.
.entryNormalColor The background of a normal entry box.
.entryPendingColor The color of an entry containing a value that hasn't been committed.

Douglas Rohde
Last modified: Thu Nov 16 13:35:39 EST 2000