Building networks

The model

A network model is defined as follows: A GeNNModel must be created with a default precision (see ref floatPrecision) and a name:

model = GeNNModel("float", "YourModelName")

By default the model will use a hardware-accelerated code-generation backend if it is available. However, this can be overriden using the backend keyword argument. For example, the single-threaded CPU backend could be manually selected with:

model = GeNNModel("float", "YourModelName",
                  backend="single_threaded_cpu")

When running models on a GPU, smaller models may not fully occupy the device. In some scenarios such as gradient-based training and parameter sweeping, this can be overcome by running multiple copies of the same model at the same time (batching in Machine Learning speak). Batching can be enabled on a GeNN model with:

model.batch_size = 512

Parameters and sparse connectivity are shared across all batches. Whether state variables are duplicated or shared is controlled by the VarAccess or CustomUpdateVarAccess enumeration associated with each variable. Please see TODO for more details.

Additionally, any preferences exposed by the backend can be configured here. For example, the CUDA backend allows you to select which CUDA device to use via the manual_device_id:

model = GeNNModel("float", "YourModelName",
                  backend="cuda", manual_device_id=0)

Populations

Populations formalise the concept of groups of neurons or synapses that are functionally related or a practical grouping, e.g. a brain region in a neuroscience model or a layer in a machine learning context.

Parameters

Parameters are initialised to constant numeric values which are homogeneous across an entire population:

para = {"m": 0.0529324, ...}

They are very efficient to access from models as their values are either hard-coded into the backend code or, on the GPU, delivered via high-performance constant cache. However, they can only be used if all members of the population have the exact same parameter value. Parameters are always read-only but can be made dynamic so they can be changed from the host during the simulation by calling pygenn.NeuronGroup.set_param_dynamic() on a NeuronGroup, i.e.

pop.set_param_dynamic("tau")

where pop is a neuron group returned by GeNNModel.add_neuron_population() or synapse group returned by GeNNModel.add_synapse_population() and “tau” is one of the population’s declared parameters.

Warning

Derived parameters will not change if the parameters they rely on are made dynamic and changed at runtime.

Extra Global Parameters

When building more complex models, it is sometimes useful to be able to access arbitarily sized arrays. In GeNN, these are called Extra Global Parameters (EGPs) and they need to be manually allocated and initialised before simulating the model. For example, the built-in neuron_models.SpikeSourceArray() model has a spikeTimes EGP which is used to provide an array of spike times for the spike source to emit. Given two numpy arrays: spike_ids containing the ids of which neurons spike and spike_times containing the time at which each spike occurs, a neuron_models.SpikeSourceArray() model can be configured as follows:

# Calculate start and end index of each neuron's spikes in sorted array
end_spike = np.cumsum(np.bincount(spike_ids, minlength=100))
start_spike = np.concatenate(([0], end_spike[0:-1]))

# Sort events first by neuron id and then
# by time and use to order spike times
spike_times = spike_times[np.lexsort((spike_times, spike_ids))]

model = GeNNModel("float", "spike_source_array_example")

ssa = model.add_neuron_population("SSA", 100, "SpikeSourceArray", {},
                                  {"startSpike": start_spike, "endSpike": end_spike})
ssa.extra_global_params["spikeTimes"].set_init_values(spike_times)

Variables

Variables contain values that are individual to the members of a population and can change over time. They can be initialised in many ways. The initialisation is configured through a Python dictionary that is then passed to GeNNModel.add_neuron_population() or GeNNModel.add_synapse_population() which create the populations.

To initialise variables one can use the backend, e.g. GPU, to fill them with a constant value:

ini = {"m": 0.0529324, ...}

or copy a sequence of values from Python:

ini = {"m": np.arange(400.0), ...}

or use a variable initialisation snippet returned by the following function:

pygenn.init_var(snippet, params={})

Initialises a variable initialisation snippet with parameter values

Parameters:

  • snippet (InitVarSnippetBase | str) – variable init snippet, either as a string referencing a built-in snippet (see init_var_snippets) or an instance of InitVarSnippetBase (for example returned by create_var_init_snippet())

  • params (Dict[str, int | float]) – parameter values for the variable init snippet (see Parameters)

For example, the built-in model “Normal” could be used to initialise a variable by sampling from the normal distribution with a mean of 0 and a standard deviation of 1:

init = init_var("Normal", {"mean": 0.0, "sd": 1.0})

The resulting initialisation snippet can then be used in the dictionary in the usual way:

ini = {"m": init, ...}

Variables references

As well as variables and parameters, various types of models have variable references which are used to reference variables associated with other populations. For example, postsynaptic update models can reference variables in the postsynaptic neuron model and custom updates are ‘attached’ to other populations based on their variable references.

Variable reference can be created to various types of per-neuron variable using:

pygenn.create_var_ref(*args, **kwargs)

Overloaded function.

  1. create_var_ref(arg0: GeNN::NeuronGroup, arg1: str) -> GeNN::Models::VarReference

Creates a reference to a neuron group variable.

  1. create_var_ref(arg0: GeNN::CurrentSource, arg1: str) -> GeNN::Models::VarReference

Creates a reference to a current source variable.

  1. create_var_ref(arg0: GeNN::CustomUpdate, arg1: str) -> GeNN::Models::VarReference

Creates a reference to a custom update variable.

References can also be created to various types of per-neuron variable owned by synapse groups using:

pygenn.create_psm_var_ref(arg0: GeNN::SynapseGroup, arg1: str) → GeNN::Models::VarReference

Creates a reference to a postsynaptic model variable.

pygenn.create_wu_pre_var_ref(arg0: GeNN::SynapseGroup, arg1: str) → GeNN::Models::VarReference

Creates a reference to a weight update model presynaptic variable.

pygenn.create_wu_post_var_ref(arg0: GeNN::SynapseGroup, arg1: str) → GeNN::Models::VarReference

Creates a reference to a weight update model postsynapticvariable.

Finally, references can be made to various internal variables. This is particularly useful when implementing custom updates to reset model state mid-simulation. Synapse group’s postsynaptic output is summed into a variable where the dynamics defined by the postsynaptic model are applied. These variables can be reset in custom updates via references created with:

pygenn.create_out_post_var_ref(arg0: GeNN::SynapseGroup) → GeNN::Models::VarReference

Creates a reference to a synapse group’s postsynaptic output buffer

When models use dendritic delays, Synapse group’s postsynaptic outputs are buffered and can persist for SynapseGroup.max_dendritic_delay_timesteps timesteps. These variables can be reset in custom updates via references created with:

pygenn.create_den_delay_var_ref(arg0: GeNN::SynapseGroup) → GeNN::Models::VarReference

Creates a reference to a synapse group’s dendritic delay buffer

Weight update or neuron models can use GeNN’s built in functionality to track the times of previous spikes (see Weight update models). These variables can be reset in custom updates via references created with:

pygenn.create_spike_time_var_ref(arg0: GeNN::NeuronGroup) → GeNN::Models::VarReference

Creates a reference to a neuron group’s spike times

pygenn.create_prev_spike_time_var_ref(arg0: GeNN::NeuronGroup) → GeNN::Models::VarReference

Creates a reference to a neuron group’s previous spike times

To match their initialisation, these times should be reset to a large negative number e.g. np.finfo(np.float32).min. While references to all these types of per-neuron variable can be used interchangably in the same custom update, as long as all referenced variables have the same delays and belong to populations of the same size, per-synapse weight update model variables must be referenced with slightly different syntax:

pygenn.create_wu_var_ref(*args, **kwargs)

Overloaded function.

  1. create_wu_var_ref(sg: GeNN::SynapseGroup, var_name: str, transpose_sg: GeNN::SynapseGroup = None, transpose_var_name: str = ‘’) -> GeNN::Models::WUVarReference

Creates a reference to a weight update model variable.

  1. create_wu_var_ref(arg0: GeNN::CustomUpdateWU, arg1: str) -> GeNN::Models::WUVarReference

Creates a reference to a custom weight update variable.

  1. create_wu_var_ref(arg0: GeNN::CustomConnectivityUpdate, arg1: str) -> GeNN::Models::WUVarReference

Creates a reference to a custom connectivity update update variable.

These ‘weight update variable references’ also have the additional feature that they can be used to define a link to a ‘transpose’ variable:

wu_transpose_var_ref = {"R": create_wu_var_ref(sg, "g", back_sg, "g")}

where back_sg is another SynapseGroup with tranposed dimensions to sg i.e. its postsynaptic population has the same number of neurons as sg’s presynaptic population and vice-versa.

After the update has run, any updates made to the ‘forward’ variable will also be applied to the tranpose variable

Note

Transposing is currently only possible on variables belonging to synapse groups with SynapseMatrixType.DENSE connectivity

Variable locations

Once you have defined how your variables are going to be initialised you need to configure where they will be allocated. By default memory is allocated for variables on both the GPU and the host. However, the following alternative ‘variable locations’ are available:

class pygenn.VarLocation(self: pygenn._genn.VarLocation, value: int)

Supported combination of VarLocationAttribute

Members:

DEVICE : Variable is only located on device. This can be used to save host memory.

HOST_DEVICE : Variable is located on both host and device. This is the default.

HOST_DEVICE_ZERO_COPY : Variable is shared between host and device using zero copy memory.

This can improve performance if data is frequently copied between host and device but, on non cache-coherent architectures e.g. Jetson, can also reduce access speed.

Note

‘Zero copy’ memory is only supported on newer embedded systems such as the Jetson TX1 where there is no physical seperation between GPU and host memory and thus the same physical memory can be shared between them.

Extra global parameter references

When building models with complex Custom updates and Custom Connectivity updates, it is often useful to share data stored in extra global parameters between different groups. Similar to variable references, such links are made using extra global parameter references. These can be created using:

pygenn.create_egp_ref(*args, **kwargs)

Overloaded function.

  1. create_egp_ref(arg0: GeNN::NeuronGroup, arg1: str) -> GeNN::Models::EGPReference

Creates a reference to a neuron group extra global parameter.

  1. create_egp_ref(arg0: GeNN::CurrentSource, arg1: str) -> GeNN::Models::EGPReference

Creates a reference to a current source extra global parameter.

  1. create_egp_ref(arg0: GeNN::CustomUpdate, arg1: str) -> GeNN::Models::EGPReference

Creates a reference to a custom update extra global parameter.

  1. create_egp_ref(arg0: GeNN::CustomUpdateWU, arg1: str) -> GeNN::Models::EGPReference

Creates a reference to a custom weight update extra global parameter.

  1. create_egp_ref(arg0: GeNN::CustomConnectivityUpdate, arg1: str) -> GeNN::Models::EGPReference

Creates a reference to a custom connectivity update extra global parameter.

pygenn.create_psm_egp_ref(arg0: GeNN::SynapseGroup, arg1: str) → GeNN::Models::EGPReference

Creates a reference to a postsynaptic model extra global parameter.

pygenn.create_wu_egp_ref(arg0: GeNN::SynapseGroup, arg1: str) → GeNN::Models::EGPReference

Creates a reference to a weight update model extra global parameter.

Neuron populations

Neuron populations contain a number of neurons with the same model and are added using:

GeNNModel.add_neuron_population(pop_name, num_neurons, neuron, params={}, vars={})

Add a neuron population to the GeNN model

Parameters:

  • pop_name (str) – unique name

  • num_neurons (int) – number of neurons

  • neuron (NeuronModelBase | str) – neuron model either as a string referencing a built-in model (see neuron_models) or an instance of NeuronModelBase (for example returned by create_neuron_model())

  • params (Dict[str, int | float]) – parameter values for the neuron model (see Parameters)

  • vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial variable values or initialisers for the neuron model (see Variables)

Return type:

NeuronGroup

For example, a population of 10 neurons using the built-in Izhikevich model and the standard set of ‘tonic spiking’ parameters could be added to a model as follows:

pop = model.add_neuron_population("pop", 10, "Izhikevich",
                                  {"a": 0.02, "b": 0.2, "c": -65.0, "d": 6.0},
                                  {"V": -65.0, "U": -20.0})

Synapse populations

Synapse populations connect two neuron populations via synapses. Their behaviour is described by a weight update model and a postsynaptic model. The weight update model defines what kind of dynamics (if any) occurs at each synapse and what output they deliver to postsynaptic (and presynaptic) neurons. Weight update models are typically initialised using:

pygenn.init_weight_update(snippet, params={}, vars={}, pre_vars={}, post_vars={}, pre_var_refs={}, post_var_refs={})

Initialises a weight update model with parameter values, variable initialisers and variable references.

Parameters:

  • snippet – weight update model either as a string referencing a built-in model (see weight_update_models) or an instance of WeightUpdateModelBase (for example returned by create_weight_update_model())

  • params (Dict[str, int | float]) – parameter values (see Parameters)

  • vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial synaptic variable values or initialisers (see Variables)

  • pre_vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial presynaptic variable values or initialisers (see Variables)

  • post_vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial postsynaptic variable values or initialisers (see Variables)

  • pre_var_refs (Dict[str, VarReference]) – references to presynaptic neuron variables, typically created using create_var_ref() (see Variables references)

  • post_var_refs (Dict[str, VarReference]) – references to postsynaptic neuron variables, typically created using create_var_ref() (see Variables references)

For example, the built-in static pulse model with constant weights could be initialised as follows:

weight_init = init_weight_update("StaticPulseConstantWeight", {"g": 1.0})

Postsynaptic models define how synaptic input translates into an input current (or other type of input for models that are not current based) and are typically initialised using:

pygenn.init_postsynaptic(snippet, params={}, vars={}, var_refs={})

Initialises a postsynaptic model with parameter values, variable initialisers and variable references

Parameters:

  • snippet (PostsynapticModelBase | str) – postsynaptic model either as a string referencing a built-in model (see postsynaptic_models) or an instance of PostsynapticModelBase (for example returned by create_postsynaptic_model())

  • params (Dict[str, int | float]) – parameter values for the postsynaptic model (see Parameters)

  • vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial synaptic variable values or initialisers for the postsynaptic model (see Variables)

  • var_refs (Dict[str, VarInit | int | float | ndarray | Sequence]) – references to postsynaptic neuron variables, typically created using create_var_ref() (see Variables references)

For example, the built-in conductance model with exponential current shaping could be initialised as follows:

postsynaptic_init = init_postsynaptic("ExpCond", {"tau": 1.0, "E": -80.0},
                                      var_refs={"V": create_var_ref(pop1, "V")})

where pop1 is a reference to the postsynaptic neuron population (as returned by GeNNModel.add_neuron_population())

GeNN provides a number of different data structures for implementing synaptic connectivity:

class pygenn.SynapseMatrixType(self: pygenn._genn.SynapseMatrixType, value: int)

Members:

DENSE : Synaptic matrix is dense and synaptic state variables are stored individually in memory.

DENSE_PROCEDURALG : Synaptic matrix is dense and all synaptic state variables must either be constant or generated on the fly using their variable initialisation snippets.

BITMASK : Connectivity is stored as a bitmask.

For moderately sparse (>3%) connectivity, this uses the least memory. However, connectivity of this sort cannot have any accompanying state variables. Which algorithm is used for propagating spikes through BITMASK connectivity can be hinted via SynapseGroup::ParallelismHint.

SPARSE : Connectivity is stored using a compressed sparse row data structure and synaptic state variables are stored individually in memory.

This is the most efficient choice for very sparse unstructured connectivity or if synaptic state variables are required.

PROCEDURAL : Sparse synaptic connectivity is generated on the fly using a sparse connectivity initialisation snippet and all state variables must be either constant or generated on the fly using variable initialisation snippets.

Synaptic connectivity of this sort requires very little memory allowing extremely large models to be simulated on a single GPU.

PROCEDURAL_KERNELG : Sparse synaptic connectivity is generated on the fly using a sparse connectivity initialisation snippet and state variables are stored in a shared kernel.

TOEPLITZ : Sparse structured connectivity is generated on the fly a Toeplitz connectivity initialisation snippet and state variables are stored in a shared kernel.

This is the most efficient choice for convolution-like connectivity

pygenn.SynapseMatrixType.DENSE and pygenn.SynapseMatrixType.DENSE_PROCEDURAL connectivity can be initialised on the GPU by simply using the variable initialisation snippets described in Variables to initialise the weight update model variables. pygenn.SynapseMatrixType.SPARSE, pygenn.SynapseMatrixType.BITMASK and pygenn.SynapseMatrixType.PROCEDURAL synaptic connectivity can be initialised on the GPU using:

pygenn.init_sparse_connectivity(snippet, params={})

Initialises a sparse connectivity initialisation snippet with parameter values

Parameters:

  • snippet (InitSparseConnectivitySnippetBase | str) – sparse connectivity init snippet, either as a string referencing a built-in snippet (see init_sparse_connectivity_snippets) or an instance of InitSparseConnectivitySnippetBase (for example returned by create_sparse_connect_init_snippet())

  • params (Dict[str, int | float]) – parameter values for the sparse connectivity init snippet (see Parameters)

For example, the built-in “FixedProbability” snippet could be used to generate connectivity where each pair of pre and postsynaptic neurons is connected with a probability of 0.1:

init = init_sparse_connectivity("FixedProbability", {"prob": 0.1})

pygenn.SynapseMatrixType.TOEPLITZ can be initialised using:

pygenn.init_toeplitz_connectivity(init_toeplitz_connect_snippet, params={})

Initialises a toeplitz connectivity initialisation snippet with parameter values

Parameters:

For example, the built-in “Conv2D” snippet could be used to generate 2D convolutional connectivity with a \(3 \times 3\) kernel, a \(64 \times 64 \times 1\) input and a \(62 \times 62 \times 1\) output:

params = {"conv_kh": 3, "conv_kw": 3,
          "conv_ih": 64, "conv_iw": 64, "conv_ic": 1,
          "conv_oh": 62, "conv_ow": 62, "conv_oc": 1}

init = init_toeplitz_connectivity("Conv2D", params))

Note

This should be used to connect a presynaptic neuron population with \(64 \times 64 \times 1 = 4096\) neurons to a postsynaptic neuron population with \(62 \times 62 \times 1 = 3844\) neurons.

Finally, with these components in place, a synapse population can be added to the model:

GeNNModel.add_synapse_population(pop_name, matrix_type, source, target, weight_update_init, postsynaptic_init, connectivity_init=None)

Add a synapse population to the GeNN model

Parameters:
Return type:

SynapseGroup

For example, a neuron population src_pop could be connected to another called target_pop using sparse connectivity, static synapses and exponential shaped current inputs as follows:

pop = model.add_synapse_population("Syn", "SPARSE",
                                   src_pop, target_pop,
                                   init_weight_update("StaticPulseConstantWeight", {"g": 1.0}),
                                   init_postsynaptic("ExpCurr", {"tau": 5.0}),
                                   init_sparse_connectivity("FixedProbability", {"prob": 0.1}))

Current sources

Current sources are added to a model using:

GeNNModel.add_current_source(cs_name, current_source_model, pop, params={}, vars={}, var_refs={})

Add a current source to the GeNN model

Parameters:

  • cs_name (str) – unique name

  • current_source_model (CurrentSourceModelBase | str) – current source model either as a string referencing a built-in model (see current_source_models) or an instance of CurrentSourceModelBase (for example returned by create_current_source_model())

  • pop (NeuronGroup) – neuron population to inject current into

  • params (Dict[str, int | float]) – parameter values for the current source model (see Parameters)

  • vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial variable values or initialisers for the current source model (see Variables)

  • var_refs (Dict[str, VarReference]) – variables references to neuron variables in pop, typically created using create_var_ref() (see Variables references)

Return type:

CurrentSource

For example, a current source to inject a Gaussian noise current can be added to a model as follows:

cs = model.add_current_source("noise", "GaussianNoise", pop,
                              {"mean": 0.0, "sd": 1.0})

where pop is a reference to a neuron population (as returned by GeNNModel.add_neuron_population())

Custom updates

The neuron groups, synapse groups and current sources described in previous sections are all updated automatically every timestep. However, in many types of model, there are also processes that would benefit from GPU acceleration but only need to be triggered occasionally. For example, such updates could be used in a classifier to reset the state of neurons after a stimulus has been presented or in a model which uses gradient-based learning to optimize network weights based on gradients accumulated over several timesteps.

Custom updates allows such updates to be described as models, similar to the neuron and synapse models described in the preceding sections. The custom update system also provides functionality for efficiently calculating the tranpose of variables associated with synapse groups (current only with SynapseMatrixType.DENSE connectivity). Custom updates are added to a model using:

GeNNModel.add_custom_update(cu_name, group_name, custom_update_model, params={}, vars={}, var_refs={}, egp_refs={})

Add a custom update to the GeNN model

Parameters:

  • cu_name (str) – unique name

  • group_name (str) – name of the ‘custom update group’ to include this update in. All custom updates in the same group are executed simultaneously.

  • custom_update_model (CustomUpdateModelBase | str) – custom update model either as a string referencing a built-in model (see custom_update_models) or an instance of CustomUpdateModelBase (for example returned by create_custom_update_model())

  • params (Dict[str, int | float]) – parameter values for the custom update model (see Parameters)

  • vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial variable values or initialisers for the custom update model (see Variables)

  • var_refs (Dict[str, VarReference] | Dict[str, WUVarReference]) – references to variables in other populations to access from this update, typically created using either create_var_ref() or create_wu_var_ref() (see Variables references).

  • egp_refs (Dict[str, EGPReference]) – references to extra global parameters in other populations to access from this update, typically created using create_egp_ref() (see Extra global parameter references).

For example, a custom update to calculate transpose weights could be added to a model as follows:

cu = model.add_custom_update("tranpose_pop", "transpose", "Transpose",
                             var_refs={"variable": create_wu_var_ref(fwd_sg, "g",
                                                                     back_sg, "g")})

where fwd_sg and back_sg are references to synapse populations (as returned by GeNNModel.add_synapse_population()). This update could then subsequently be triggered using the name of it’s update group with:

model.custom_update("transpose")

Custom connectivity updates

Like custom update, custom connectivity updates are triggered manually by the user but, rather than updating model variables, they update model connectivity (current only with SynapseMatrixType.SPARSE connectivity). Custom connectivity updates are added to a model using:

GeNNModel.add_custom_connectivity_update(cu_name, group_name, syn_group, custom_conn_update_model, params={}, vars={}, pre_vars={}, post_vars={}, var_refs={}, pre_var_refs={}, post_var_refs={}, egp_refs={})

Add a custom connectivity update to the GeNN model

Parameters:

  • cu_name (str) – unique name

  • group_name (str) – name of the ‘custom update group’ to include this update in. All custom updates in the same group are executed simultaneously.

  • syn_group (SynapseGroup) – Synapse group to attach custom connectivity update to

  • custom_conn_update_model (CustomConnectivityUpdateModelBase | str) – custom connectivity update model either as a string referencing a built-in model (see custom_connectivity_update_models) or an instance of CustomConnectivityUpdateModelBaseUpdateModelBase (for example returned by create_custom_connectivity_update_model())

  • params (Dict[str, int | float]) – parameter values for the custom connectivity model (see Parameters)

  • vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial synaptic variable values or initialisers (see Variables)

  • pre_vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial presynaptic variable values or initialisers (see Variables)

  • post_vars (Dict[str, VarInit | int | float | ndarray | Sequence]) – initial postsynaptic variable values or initialisers (see Variables)

  • var_refs (Dict[str, WUVarReference]) – references to synaptic variables, typically created using create_wu_var_ref() (see Variables references)

  • pre_var_refs (Dict[str, VarReference]) – references to presynaptic variables, typically created using create_var_ref() (see Variables references)

  • post_var_refs (Dict[str, VarReference]) – references to postsynaptic variables, typically created using create_var_ref() (see Variables references)

  • egp_refs (Dict[str, EGPReference]) – references to extra global parameters in other populations to access from this update, typically created using create_egp_ref() (see Extra global parameter references).