Miscellanea: Version Notes

2.64

• The BIAS_CLAMP output function now sets the units' output to the initOutput, rather than 1.0. The default initOutput for BIAS groups is still 1.0, but this allows you to change the bias value.
• Fixed a bug that occurred when a lot of group names are used in an example file (thanks to Richard Thompson).
• Added protections to prevent changing to an example set while it is loading (also thanks to Richard Thompson).

2.63

• Added the UNLESIONED projection type to the connectGroups command.
• Added the -r flag to the test command to allow it to return the results instead of printing them (added parameters to the reportError and netTestBatch commands).
• Added adaptive gains for TANH units.
• Worked around some compile bugs affecting binary files.
• Added copy*Extension() functions to extension.c and extension.h for customized copying of extension structures.
• Fixed a potential divide by zero error when the history length is set to 0 (thanks to Mike Squillace).
• Made the linksChanged() function part of the display module.

2.62

• Fixed a bug in printLinkValues (thanks to Gautam Vallabha).
• Added the AVOID_NAN_TEST flag to allow the tests of NaN to be skipped on machines that don't behave.

2.6

• Lens should now compile more smoothly under Cygwin on Windows.
• The interface for the graphs has been completely rewritten to allow you to create and configure graphs and traces using script commands, rather than just the graphical interface. Graphs and their traces are now objects that can be accessed with the Object Viewer or setObject commands. You can also hide graphs so they update much faster if they are not needed immediately. The graph, trace, and exportGraph commands have been added and the updateGraphs command was removed.
• The active unit or active link can now be locked in the Unit and Link Viewers, by clicking on it with the left mouse button, to watch the value on that unit or link change without having to keep the mouse over that square.
• Clicking on a square in the Unit or Link Viewer with the middle mouse button opens a menu that allows you to print the value, open its object in the Object Viewer, or create a graph that will track the value.
• The polarity command has been added to measure the degree to which the units in a group are adopting binary versus intermediate output values.
• The resetDerivs command has been added to allow you to reset the link and unit error derivatives without updating the weights. This is useful for customized training procedures.
• The OUT_WINNER group output type has been added. This implements a winner-take-all filter that shuts off all but the most active unit.
• A bug has been fixed in link lesioning.
• Errors in the initial script should no longer leave Lens in a strange directory.

2.5

• Lens can now be compiled to run in Unix emulation mode under Mac OS X.
• Changed the unit initialization so its gain, when using ADAPTIVE_GAIN, is copied from the group's gain, or the network's gain if the group's is not set.
• Added the ability for most of the commands that write to files to append to an existing file.
• Added support for bzip2 compression.

2.4

• Upgraded to the latest stable release of Tcl/Tk, 8.3.4.
• The test command used to return its results as a return value. Now it prints its results to the shell. It also saves the results in an array for easy access.
• Most of the hard limits such as that on the number of group names specified in an example file have been removed.
• Error for fully recurrent networks is now computed differently. It used to be scaled by the network's dt and by the group's dtScale. This was to keep the error relatively constant if the number of ticksPerInterval was changed. The problem is that if the user changes the time constant of the network or a group, it will affect the error, which isn't right. The logical thing to do is to scale the error by 1/ticksPerInterval, rather than the dt. Ordinarily, this will be the same, since dt defaults to 1/ticksPerInterval. But if you change dt or dtScale, it will now no longer affect the error.
• A dtScale parameter has been created for each unit. This allows units in a single group in a fully recurrent network have varying time constants. The actual dt for a unit is the product of the network's dt, the group's dtScale, and the unit's dtScale.
• The MAX_CRIT criterion has been changed so that it properly ignores output units with no target.
• The -append flag has been added to openNetOutputFile command to allow the information to be appended to a file.
• Errors in loading example and weight files are now reported more consistently.
• If a group is specified in an example event and that group does not exist in the network, it no longer causes an error. This allows example sets that specify input or targets only used in some networks, but it also makes it more likely that errors in coding the example set will go undetected.
• The Unit Viewer now has larger entries for reporting link and unit values. It no longer clears those values when the mouse is removed from a unit.
• Wheel mice can now be used to activate vertical scrollbars.
• Problems with the graphical display colors on some systems have been fixed.
• Error messages resulting from bad group types are now much smaller.

2.3.3

• Fixed a problem when copying with moveExamples.
• Fixed a bug that caused the link structure to occasionally be built in random order, which interfered with loading old weight files.
• Fixed a bug that caused the backward direction of bidirectional projections to have the same link type as the forward direction, which isn't right.

2.3.2

• The network now has a writeExample() function pointer which is used to writing data to the activation output file. Thus, you can write your own C function (to replace standardWriteExample) in order to customize the format of the output files.
• I moved the test for no examples out of nextExample and into the individual loading procedures to be more flexible. Example choosing modes like PIPE don't care if there are no examples loaded, but most modes will care. This only affects people who have written their own example choosing mode.
• safeRealloc() has been made somewhat more robust. It seemed to have problems on Windows.
• Fixed a bug that caused bad values to be generated by the randGaussian function if you frequently change the mean and range of the distribution.
• A bug that prevented reading from zipped files in Windows, which appeared in 2.3.1, was fixed.

2.3.1

• A bug in the use of link weight maximum bound was fixed.
• The connectGroupToUnits command now permits a group list, rather than just a single group.
• The connectUnits command now permits a series of unit lists, with each unit in a list connecting to each unit in the next list. It also accepts the "-bidirectional" flag.

2.3

• Lens can now be compiled as a library and embedded in other programs.
• The new grapher was added.
• An updateGraphs command has been added.
• The format of the activation output files was changed and the WRITE_OUTPUTS group type added to allow the activations of non-output groups to be recorded.
• RESET_ON_EXAMPLE now defaults to true for all groups. Before, it was only true for continuous networks, but that was too confusing.
• The drawPlot command has been renamed drawUnits.
• The randGroupWeights and randUnitWeights commands are now subsumed by the randWeights command.
• The setBlockValues, setGroupBlockValues, and setUnitBlockValues commands have been subsumed by the setLinkValues command.
• The freezeNet, freezeGroups, and freezeUnits commands have been subsumed by the freezeWeights command.
• The thawNet, thawGroups, and thawUnits commands have been subsumed by the thawWeights command.
• The netPath, groupPath, unitPath, and setPath commands have been subsumed by a single path command.
• A GAUSSIAN group output type has been added. This is used for radial basis units. The gain is the inverse of the width of the Gaussian. A higher gain means a sharper Gaussian. Like LOGISTIC, GAUSSIAN output can be used along with ADAPTIVE_GAIN to have trainable, individual gains for each unit.
• The min and max fields have been added to the block structures. These can be used to bound the weight values to a certain range. For example, you might force the bias links to be negative. They should be set with the setLinkValues command.
• Parameters for the Unit and Link viewers are now specific to the network.
• Using the "-mode PIPE" flag in loadExamples automatically puts the example set in PIPE mode now.
• Training with 0 batch size when reading from a pipe generates an error.
• Boltzmann network output groups will now get an error function by default. The error is calculated at the end of training for testing purposes but doesn't affect the training of the network.
• Renamed the TARG_COPY group type to TARGET_COPY.

2.2

• The Unit and Link viewers now support Hinton diagrams. These may be useful for printing or on greyscale displays. The unit viewer also now lets you view just outputs or targets, in addition to the default combination of outputs and targets.
• The groupCriterion parameter in the network and groups has been replaced by trainGroupCrit and testGroupCrit. This provides separate group criteria when training and testing, because it is useful to be able to use the group criterion function only when testing or only when training.
• A new group criterion function, MAX_CRIT, has been added. This is true if the output unit with the largest target is also the most active output unit. This is most useful for networks doing a one-bit-on task.
• An initNetworkExtension() function has been added to the extension module. You will need to add that to your extension.c.
• The trainEpochProc has been changed to preEpochProc and a number of other handles have been provided so Tcl commands can be executed before and after examples, events, ticks, and so on.
• The copyConnect command and the IN_COPY, OUT_COPY, and TARG_COPY group types have been added. These allow one group to set its inputs, outputs, or targets to the inputs, outputs, targets, or derivatives of another group. This lets you do a lot of interesting things, like having groups that share inputs or targets or having the output of one group become the target of another group.
• Fixed a major bug that affected networks with lesioned units. The unit outputs weren't being cached properly so outputs and derivatives were attributed to the wrong units.
• control.c was using Tcl variables, such as i, that did not start with a "." and could thus write over user variables.
• There is now a sysext.h file in which general customizations can be placed to get Lens to compile properly on your machine. This is included by the system.h file, which you should avoid editing directly.
• Groups now have an outputCost field which accumulates unit costs just as the error field accumulates error. By giving a group a unit cost function, such as COSINE_COST, and monitoring this value during testing, you can determine how binary the units in a group tend to be. If logistic units do not have values close to 0 or 1, it may indicate that the group is too small.
• Fixed a bug that caused problems when "-;" appeared in an example file. The "-" is often used to signify a NaN target.
• If you set a group's numColumns field to a positive number, the group will be plotted with that many columns when the autoPlot command is run. This gives you a bit more control over how the network appears in the unit viewer, short of specifying the layout by hand using plotRow and friends.
• In the console, you can now use the middle mouse button to paste the current selection into the middle of a line.
• You can now use the arrow keys and PageUp/PageDown to scroll the Object Viewer. Pressing the spacebar in the Unit Viewer will step through the examples (equivalent to the down arrow).
• You can now add fields to your extension structures that use uppercase names, although I don't recommend it as the Lens code by convention reserves uppercase for global variables.

2.1.2

• Added an error field to each group which keeps track of the total output error due to that group. This is useful when you have multiple output groups. Otherwise, it will be the same value as the network error. Only error due to immediate targets is counted, not backpropagated error.
• The format of the output of the test command has been changed. If there is more than one output group, it will now give the results for individual groups as well.
• Fixed a major problem with the error derivative procedures. They now will properly ignore units with NaN targets. Previously they were using the most recent non-NaN targets.
• setBlockValues has been made more general. It now works on reals, ints, flags, and masks and on fields in block extensions. Note that extension.c has changed and, if you have your own version, you will need to add a final NULL argument to any calls to newObject().
• Changed the default Windows fonts.

2.1.1

• All files and pipes are now accessed with Tcl channels rather than FILE *'s. This makes it more platform independent and solves several problems, including the problem of child processes in a pipe not dying when they should.
• Fixed some bugs in object.c that caused problems when registering custom extension fields.
• Fixed a bug in the time command.
• Fixed a bug that prevented block extensions from being created.
• Changed the Makefile a bit and added ALPHA customizations.

2.1.0

• The version numbering scheme has changed. The first number is for major rewrites. The second is for significant changes, such as adding new commands or network types. The third is for minor releases involving changes that won't affect most users.
• Simple recurrent backprop through time (SRBPTT) networks have been added. These use sequential unit updates but backpropagate error through all ticks in the example in a single pass.
• Parallel training was rewritten (again). It now works in Windows and between Unix and Windows.
• You are now allowed to run in batch mode under Windows, but there are some problems that seem to cause it to freeze now and then. So use this at your own risk.
• You can now access networks, groups, and units directly by name using setObject and getObject. Instead of "getObject [groupPath hidden].unit(5).output" you can now do "getObject hidden.unit(5).output" or just "getObject hidden:5.output"
• The resetOnExample group parameter was replaced with the RESET_ON_EXAMPLE group type flag. This defaults to FALSE for non-continuous networks and TRUE for continuous.
• The syntax for specifying group types has changed. Now, +TYPE adds the group type, -TYPE removes it, and just TYPE toggles it. To create a group with no bias inputs, -BIASED should be used instead of UNBIASED.
• The updateWeights command was added to permit customized training without using the train command.
• A "Wait" button was added to the Main Window. This executes the wait command which causes Lens to go idle until another command is invoked or the wait is terminated. This is useful for pausing training.
• A task depth counter was also added to the Main Window. This indicates the number of tasks (such as training, loading examples, and waiting) currently on the stack.
• Calling test on an example-set in PIPE mode with no maximum number of examples specified will run until the pipe gives out. You don't want to do this with pipeLoop on or with an infinite example generator.
• The COSINE cost function was added. This computes 1.0 minus the cosine of the angle between the output and target vectors. This is most useful for evaluation but can also be used for training as well as evaluation. However, training can be tricky because there is only pressure for the angle of the output vector to be correct, not the absolute values of the outputs. You could use SOFT_MAX outputs or a unit cost function (such as LOGISTIC_COST) on the output units to encourage them to be binary if that is desired.
• The verbosity command was added. This adjusts the amount of stuff that gets printed out.
• Added the alarm command, which makes a few beeps. You might use this in a script to tell you when training is done..
  Programmers:
  
• The incoming array for each unit has been split into incoming and incoming2. This speeds it up because more of the critical link values can fit in the cache.
• The sigmoid was replaced with a fast table lookup with linear interpolation. It is accurate to within 2e-7 for all inputs.
• There is now a version of the inner FOR_EACH_LINK loops, FOR_EACH_LINK_FAST, which allows for hand unrolling of the loops, which is about 20% faster. Currently it is only used for dot product inputs.
• The FOR_EACH_UNIT macro no longer provides the unit index variable, u, but FOR_EACH_UNIT2 does. This speeds up the more common FOR_EACH_UNIT.

2.01

• Lens now runs under Windows! Just about everything works except parallel training, which currently uses UNIX file-descriptors.
• It has also been upgraded to Tcl/Tk 8.3.1. This uses shared libraries and the distribution nows looks a bit different.
• You can now put an extension on the version number by putting something like this:
  char *VersionExt = "dr"; in your extension.c file. It will probably not compile if you don't.
• The -DARRAY_TRAVERSAL compile flag can now be used to cause the inner loops to use array index traversal instead of pointer-stepping. This may be faster with some compilers but tends to be slower with gcc.
• You can now encode inputs and targets that share the same values with a single command in the example file. Instead of "I:" or "T:" you use "B:".
• The assumeIntegrate flag was added to the network structure. If this is true, which is the default for CONTINUOUS networks, when a group's type is set and no integration function is specified, OUT_INTEGR will be assumed. Otherwise, no integrator will be assumed unless specified.
• You can now write your own procedures to select the order of examples during training or testing. Set the example set mode to CUSTOM and then set the example set's chooseExample field to a procedure that returns the index of the next example.
• More information was added to the Unit and Link viewers.
• The viewConsole command was added to allow the console window to be recovered after it has been closed.
• The prompts are now settable in the lensrc.default file. The default secondary prompt is now nothing to allow multi-line commands to be cut and pasted more easily.
• Fixed some problems with the console. You can now use \ to break a line.
• Fixed a bug in parsing long example set lines.
• Fixed some bugs in lesionLinks and added the -discard flag.
• Lens is now much faster at building networks and loading example files.
• Changed a million other things.

2.00

• Fixed a bug that caused error derivatives to build up in source groups in SRNs.
• Fixed a bug parsing certain NaN dashes in example files.

2.0b4

• You can now run multiple commands and load multiple scripts on the command-line.
• If a group is given the ADAPTIVE_GAIN type along with the LOGISTIC output function, it will train the units' gain parameters along with the weights. Gain is updated using steepest descent with the learning rate given by the network's adaptiveGainRate parameter.
• Example loading is now faster because .initExample and .initEvent are only called if they exist.
• Files and pipes are now closed in a safer manner, eliminating some potential errors.
• It no longer causes an error if example loading is terminated part-way through or if you choosing not to replace an existing group if a new group is added with the same name.

2.0b3

• Upgraded probabilistic example selection to use binary search.
• Weights can now be loaded to and from frozen links, thawed links, or all links.
• Fixed a bug in standardTrainExample.

2.0b1

• Added the ability to register network types, group types, projection types, and example selection types to make customization easier.
• Created row and column functions for describing objects to replace the old offsets.
• Made type.h a customizable function.
• Moved the autoNet command to the addNet command.

2.0b0

• The biggest change in this release is that the group types have been completely re-done and act.c was almost entirely re-written. Now, the input and output functions of each group are actually a pipeline of small procedures. For example, the time-integration procedure is now separate from the basic input or output computing procedures so you could integrate anything.
• The command usages have been completely revised to allow them to be more consistent and flexible. Now they tend to use options, such as "-foo 5". Procedures operating on networks, groups, and units can now take a list of objects or * to indicate all objects. For example, this allows connectGroups to build complex sets of projections with a single command. The change in usage may require you to revise many of you scripts. Commands that may be problematic in particular are addNet, addGroup, and loadExamples.
• Several commands have been eliminated because they are no longer needed. These include addSoftClampGroup, elmanReset, deleteAllNets, listExampleSets, and setHistoryLength.
• Holding and releasing of groups has been eliminated for the time-being because it was more trouble than it was worth.
• The example file format has been extended to allow a more flexible use of dense and sparse encodings. Now the number of input and output units need not be specified in the set header (which can be eliminated entirely) and units can be specified relative to a named group or across the whole network.
• Lens is now much less susceptible to faults due to multiple tasks running simultaneously. Extended tasks, such as training or loading example files are recorded in a stack and can be selectively canceled. New commands will not be invoked if they could potentially cause problems.
• The user interface is now much more responsive while Lens is busy with ongoing tasks. The displays are updated every 100 msec to eliminate any noticeable lag.
• The trainParallel command now returns only when training is done. This eliminates the need to call wait after starting to train.
• Lens now uses HOSTTYPE sub-directories in the Obj, Bin, and TclTk directories. This allows it to be compiled on multiple architectures without duplicating the directories.

1.15

• Added the clamped flag to units and altered the meanings of NaN inputs to be "not clamped" rather than "remain clamped to the same value".
• INPUT groups can now also have HIDDEN or OUTPUT type. If so, they will receive bias inputs and a unit combin function can be used for unclamped units. A transfer function can be specified for any INPUT group, which will override the standard transfer function that is similar to LINEAR.
• Continuous networks now operate a little differently. On the first tick of an event, the new inputs will only reach the outputs of the INPUT groups, not to the outputs of the next groups, as it was before.
• INTEGR_INPUT and INTEGR_OUTPUT were turned into combin and transfer functions. They can be used with arbitrary transfer or combin functions, respectively. Or they could be used together. Some problems with their implementation were fixed.
• A LOG_DOT_PRODUCT combin function has been added. This combines a dot product combin with a logistic transfer. It is intended to be used for the unclamped units in INPUT groups when clamped units are in the range [0,1].
• A LIN_INTEGR_OUTPUT transfer function was created which is a linear transfer that integrates the values. The standard INTEGR_OUTPUT uses a logistic transfer. LIN_INTEGR_OUTPUT is functionally equivalent to an INPUT_INTEGR with a linear transfer, but it can be used with INPUT groups which have hard-clamped inputs.
• SMOOTH_CLAMP has been supplanted by using an INPUT group with a LOG_DOT_PRODUCT combin (if units may be unclamped) and a LIN_INTEGR_OUTPUT transfer function.
• The format for the weight files has changed to include the totalUpdates field from the network. When a weight file is loaded into the entire network, the network's totalUpdates is set to that value. If the weight file is loaded into only the non-frozen links, this value will not be set.
• Fixed some problems with the !! and ^ history recall commands in the console.
• Fixed a bug with deleting INPUT or OUTPUT groups.
• Fixed some minor problems with the console.
• Fixed a precision problem with the measurement of elapsed time.

1.14

• Added the console, which is a TclTk-based shell. This has nicer command-line editing and history recall than the standard shell and also provides command and file-name completion (with the Tab key). It must be used when running under NT. This can be activated with the -console or -c command-line argument.
• Weights can now be saved or loaded selectively to or from non-frozen links. This can be used to save and load sub-networks. Load weights no longer resets the network. Also, to save weights in text format you must use the "-text" flag rather than "0".
• The Link Viewer now draws frozen links with a yellow border.
• sigUSR1Proc and sigUSR2Proc are network procedures that will be executed whenever Lens receives a USR1 or USR2 signal. The signals can be sent using "kill -USR1 <process-id>". These can be used to do emergency things while Lens is still running, even if it is in batch mode and is non-responsive. The default procedures will save the weights and save the parameters, respectively, in preset files. You can change the default procedures by editing the .lensrc file or you can change them on the fly with setObject.
• Added the backpropTicks parameter which can be used with simple recurrent networks to extend the backpropagation backward in time and potentially across examples.
• Added the SMOOTH_CLAMP transfer type, which can be given to an INPUT group in a continuous network to have the activation of that group gradually ramp up to the input value.
• Added the repeat command which makes writing loops a little easier.
• Added the groupCritRequired parameter, which determines whether the group criterion must be met at the end of an event to go on to the next event.
• Some of the error measures were changed so they are less likely to result in infinite or NaN values. Now the error is bounded to 1e8.
• DELTA was replaced by SMALL_VALUE and its value was decreased to 1e-8.
• Added some more information to the training reports.
• Changed the way the SET_HISTORY and GET_HISTORY macros are used, which may interest programmers.
• train and test now return an error code if interrupted with a signal so that scripts that train or test in a loop repeatedly will stop once interrupted.

1.13

• Improved performance for all but the smallest networks by creating arrays to cache copies of unit outputs and outputDerivs.
• It is now possible to save weight deltas and link learning rates (lastValues) in the weight files.
• The saveParameters command now allows you to save parameters from the network into a script file. Sourcing the script file will reset the parameter values.
• The source command is now able to read from pipes, Tcl channels, and compressed files, allowing you to compress parameter files.
• Lens can now be compiled to use double-precision floating point. This is faster on Alphas but slower on most other machines. Binary input and output still uses 4-byte floats for compatibility with single-precision versions. The -DDOUBLE_REAL compile flag will enable double precision. If you ever decide to switch from floats to doubles, be sure to "make clean" first so everything recompiles.
• Added the changeGroupType command which allows you to selectively turn on and off group types.
• Added the minCritBatches parameter to the network, which sets the number of consecutive batches on which the network must achieve the error or group criteria before training will stop, even for Hebbian-type networks that have no targets or error.
• Criteria are now considered to be met only if the performance is better than and not equal to the criterion. Therefore, if criterion is set to 0.0, it will never cause training to stop.
• Fixed some integer and pointer casting ambiguities that cause warnings on the Alpha.
• Fixed some bugs with softClamp groups.
• Removed a number of tests that prevent things being done to lesioned units but waste time for normal networks. Any simulations that use lesioned units should be checked to make sure they are working properly.
• Restructured system.h.

1.12

• Added Kohonen networks (actually just a special transfer function).
• unitCriterion has been changed to groupCriterion.
• Added the ability to write different group criterion functions or to have none at all, which is now the default for all but continuous networks.
• Added the trainEpochProc handle that lets you specify a script command to be run before every batch is processed when training. This makes it easier to change parameters or safe the network on the fly.
• MACHINE_VIM has been replaced with MACHINE_LINUX in the Makefile and system.h.
• Fixed a bunch of small bugs and starting making things Windows compatible.

1.11

• Temporarily removed quickProp and added "Doug's Momentum". This is similar to standard momentum descent except that the magnitude of the weight change vector (prior to the addition of the momentum term) is normalized so it will not be greater than the learning rate. This prevents overly large weight steps when the error surface is steep and allows higher learning rates.
• Added ternary units, which are similar to logistic units but with three stable states at -1, 0, and 1.
• Fixed a bug with loading binary example files in PIPE mode.
• Fixed a bug that prevented mean and range from being set with connectGroups.
• Fixed the handling of command line arguments a bit.
• Fixed a bug when clearing graphs that have fixed max or min.

1.10

• Continuous networks now operate for an extra time tick. The first tick occurs before the first event begins and just sets the unit outputs to their initial value. These networks were not working quite right before.
• The group type is now stored in two 32-bit masks instead of one to allow more group types. The change may have introduced bugs.
• Unit cost functions have been added, which can be used to encourage hidden representations to the binary.
• The group types DOTPROD, SOFTMAX, CROSSENTROPY, and SUMSQUARE have been changed to DOT_PROD, SOFT_MAX, CROSS_ENTROPY, and SUM_SQUARE to be consistent with other type names.
• The learning rate scale in delta-bar-delta is now correctly being set to 1.0 when the network is initialized.
• Added unitCostStrength and gain to the Main Window.

1.09

• Fixed a bug by which unit output derivatives were not being reset for input-integrating hidden units, causing problems in some continuous networks.
• Put bounds on derivatives when using cross entropy or divergence error functions or tanh transfer functions. This should reduce the appearance of NaNs, although any training parameters that cause the network to test these bounds may be too aggressive.
• Changed the name of the resetOnWrap parameter to resetOnExample and changed the default value to TRUE.
• Made softClamp groups use the second group's gain when computing the inverse sigmoid.
• Fixed a bug reading piped example files.
• Fixed the way command-line arguments are processed so errors are reported correctly.
• The link viewer no longer incorrectly reports the weight rather than the delta when a link was clicked in "Link Deltas" mode.

1.08

• Fixed several bugs with Boltzmann machines, including one that prevented the bias weights from updating correctly and some problems with mixed INPUT and OUTPUT groups.
• Added the lastInput field to the unit and made it so that NaN inputs in either the dense or sparse representation will cause the input to remain what it was.
• The number of lines argument to plotRow is now optional, with a default of 1.
• Creating a ONE_TO_ONE projection between groups that are not the same size no longer causes an error, it just connects as many units as it can.

1.07

• Rewrote the example file format to allow more compact and flexible files. It is not backward compatible with the old format.
• Allowed looping piped example files.
• Changed the Unit Viewer to count events starting with 0.
• Added the default.h file which stores the default values for various network and example set parameters.
• Improved the way it guesses default example file names and the directory and file name when saving or loading weights, example sets, or output files.
• Removed the silent argument to the checkCompatibility function for example sets.

1.06

• Renamed the ordered array in each example set to example.
• Renamed inverseTemp to gain.
• Fixed a number of undesirable "features" in deterministic Boltzmann machines.
• Added procedures such as .initNetwork to the lensrc.default file to allow users to set their own default values for fields in any major structure.
• Added the -BI projection type flag to connectGroups to allow the creation of both forward and backward projections at once.
• Fixed a tiny bug in the Unit Viewer.

1.05

• Implemented Deterministic Boltzmann Machines.
• Changed the way the Unit Viewer traverses time and added automatic playback of an example.
• Changed the network's example training, testing, and running function pointers somewhat.
• Made it possible to perform either the training or testing procedures when selecting an example in the Unit Viewer or when calling doExample.
• Made the history length, which determines the size of the arrays stored in each unit, adjustable. The setTime command was modified and the setHistoryLength command was added to allow this.
• Added a temperature to each logistic function. This is adjusted by setting the inverseTemp field in the network or group objects.
• Changed Delta-Bar-Delta so the link's learning rates scale the default learning rate rather than replace it.
• Added rand and randInt commands.
• Fixed the Object Viewer geometry.
• Fixed a potential bug with the history command.

1.04

• Fixed a bug with target Elman layers.
• Changed type.h slightly.
• Changed cleanupGroupTypes().

1.03

• Added the Bin directory to hold different versions and made lens and alens symbolic links to the latest version.
• Fixed a few minor bugs with addNet and the Unit Viewer.
• Changed the Network's fields from "inputs" and "outputs" to "input" and "output".

1.02

• Replaced linked lists of networks and example sets with arrays and created the Root structure to holds the arrays.
• Parameters on the main display are now registered, rather than hard-coded, allowing the user to customize them.
• Added a block extension record.
• Fixed how command-line arguments are processed.
• Restored missing help files.
• Added spaces to the Object Viewer and rearranged the object fields.

1.01

• Replaced the setRandValues, setGroupRandValues, and setUnitRandValues commands with setBlockValues, setGroupBlockValues, and setUnitBlockValues.
• Put learningRate, momentum, weightDecay, randMean, and randRange parameters in the group and block structures to allow the network defaults to be overridden.
• Created targetRadius and changed the definition of zeroErrorRadius. Put copies in the group structure to allow the network defaults to be overridden.
• Added targetRadius and unitCriterion to the Main Window.
• The event array in the example structure was changed from an array of pointers to an array of structures.
• eventMaxTime, eventMinTime, and eventGraceTime were moved from the network to the example set and renamed. These can now be specified in the example file and have become part of the binary example file, making old binary example files incompatible.

1.00

• The first official release appeared on June 9, 1998.