Special Topics: Basic Group Types

Fixed Types

The fixed type is the primary type of the units in a group. The main effect of the fixed type is that it determines the default input, output, and cost functions of the group. The process of determining default functions can be fairly complicated as it depends on network type and other group types. So if you don't want to figure it all out, you may want to just check the types of your groups with the groupType command to make sure types are assigned as you expect and then explicitly specify them as necessary.

Note that there is no HIDDEN type. A hidden group simply has no fixed type. The default input type for a hidden group is DOT_PRODUCT. The default output type is LOGISTIC. However, if the network is a BOLTZMANN net, the default input type is IN_BOLTZ and the default output type is OUT_BOLTZ. Hidden groups will automatically receive an incoming link from the bias unit. This can be prevented with the -BIASED type, as explained below.

BIAS

A bias group is one that always maintains an output of 1.0 (or whatever the initOutput is set to). Ordinarily, a bias group will be created automatically with the network and there is probably no need for more than one, so you may never need to use this type. BIAS cannot be combined with any other fixed type. A BIAS group by default is itself unbiased, has no input type, and has a BIAS_CLAMP output type. The BIAS_CLAMP function simply sets the output to the group's initOutput.

INPUT

The main implication of the INPUT type is that externalInputs will be assigned to units in the group when an example event is loaded. Any group which is expecting external inputs should be of type INPUT. If the externalInput is NaN, the input group should act like any other group, computing the inputs based on incoming connections and computing the output with ordinary output functions.

However, it is usually the case that input groups will always have external inputs. Therefore, to save processing time there is no default input type for an INPUT group and there are no bias inputs created by default. If no basic output type is specified and no clamping output type is specified and there is not a SOFT_CLAMP input type, a HARD_CLAMP output function will be prepended to the output types of an INPUT group. If a SOFT_CLAMP input type is specified, the default output type is LOGISTIC.

OUTPUT

The main implication of the OUTPUT type is that targets will be assigned to units in the group when an example event is loaded. These groups are treated the same as hidden groups. However, OUTPUT groups will also be assigned a default error function. Groups with KOHONEN or OUT_BOLTZ output functions will have no default error procedure. LINEAR groups will have SUM_SQUARED error. SOFT_MAX groups will have DIVERGENCE error. All other groups will have CROSS_ENTROPY as their default error function. OUTPUT groups in CONTINUOUS networks have a STANDARD_CRIT criterion function by default.

The OUTPUT type may be combined with the INPUT or ELMAN types (although the latter doesn't really make much sense). If a group has multiple types, the default functions will be assigned as if it were just an OUTPUT group.

ELMAN

The ELMAN type can be used to identify context groups in simple recurrent networks. Like BIAS and INPUT groups, ELMAN groups have no default bias inputs and no default input function. The default output function is ELMAN_CLAMP. Before an ELMAN_CLAMPED group is functional, you must use elmanConnect to assign a source group to it.

Group Criterion Functions

STANDARD_CRIT

This criterion is true if the absolute difference between the output and target of each unit is less than the groupCriterion. Units with NaN targets are ignored.

MAX_CRIT

This criterion is true if the unit with the largest target is also the most active unit and the difference between the output and target is less than the groupCriterion. Units with NaN targets are ignored.

History Parameters

USE_INPUT_HIST

Forces the inputHistory to be written when the unit inputs are computed.

USE_OUTPUT_HIST

Forces the outputHistory to be written when the unit outputs are computed.

USE_TARGET_HIST

Forces the targetHistory to be allocated and written when the cost is computed.

USE_OUT_DERIV_HIST

Forces the outputDerivHistory to be allocated and written when the unit output derivatives are computed.

Noise Types

DERIV_NOISE

This injects noise on the derivatives of incoming links to the group just before each weight update. There is currently no way to specify exactly which type of links are to be affected.

EXT_INPUT_NOISE

This injects noise to the external inputs when they are loaded from the example event. This differs from INPUT_NOISE in that input noise changes every tick but this remains constant across the event.

Other Types

BIASED

If this is on, the units in the group will receive incoming links from the bias unit when they are created. This only has an effect when the group is created. This defaults to off for BIAS, INPUT, and ELMAN groups and to on for HIDDEN and OUTPUT groups.

RESET_ON_EXAMPLE

If this is on, the outputs of units in the group will be reset to the initOutput at the start of each example and any integrating information reset. This defaults to on for CONTINUOUS networks and to off for other networks.

ADAPTIVE_GAIN

If this is on, units using a LOGISTIC, TANH, or GAUSSIAN output function will train their gain parameter along with the incoming weights. The adaptiveGainRate is the learning rate for the gain changes. Steepest descent is always used. When the network is reset, unit gains will be set to the group or network default.

WRITE_OUTPUTS

When the unit activations and targets are written to the activation output file, which is opened with openNetOutputFile command, only groups with this type will have their values written. This type is added by default to OUTPUT groups, because those are the groups whose activations one is usually interested in.

LESIONED

This is turned on if any unit in the group is lesioned. This makes it slower to traverse the units in the group because it must check each unit and avoid the lesioned ones. The user can't set this group type.

FROZEN

This is turned on if the group is frozen and thus no weight updates will be performed on incoming links to the group. The user can't set this group type.