Imitation learning models¶

Networks¶

Distributed Network¶

Adapted from https://github.com/MarcoVernaUSI/distributed/blob/Refactored/network.py

class networks.distributed_network.DistributedNet(input_size)¶

Simplified “distributed network”, that takes as input an array containing the response values of the sensors – which can be either prox_values, prox_comm or all_sensors – and the output is an array of 1 float that represents the speed of the wheels, which is assumed to be the same both right and left.

The architecture of the network is straightforward: there are three linear layers each of size (inpuz_size, 10), (10, 10) and (10, 1) respectively, where input_size is the sum of the shape of the sensing – 7 or 14. The Tanh non-linear activation function is applied to the first and the second layer.

Parameters: input_size – dimension of the sensing vector

controller(thymio) → Callable[[Sequence[Sensing]], Tuple[Sequence[Control], Sequence[Communication]]]¶

Parameters: thymio – unused
Return f: controller function

forward(xs, _)¶

Parameters

xs – input sensing batch
_ – mask of the batch

Return output

speed

Communication Network¶

Adapted from https://github.com/MarcoVernaUSI/distributed/blob/Refactored/com_network.py

class networks.communication_network.CommunicationNet(input_size, device, sync: networks.communication_network.Sync = <Sync.sequential: 2>, module: torch.nn.modules.module.Module = <class 'networks.communication_network.SingleNet'>, input_fn=<function input_from>)¶

High-level module that handle the sensing of the agents.

Parameters

input_size – dimension of the sensing vector (can be 7 or 14)
device – device used (cpu or gpu)
sync – kind of synchronisation
module – SingleNet
input_fn – input function

Variables

self.tmp_indices – communication indices
self.distribution – distribution used for the initialisation of the communication

controller(thymio, sync: networks.communication_network.Sync = <Sync.sync: 3>) → Callable[[Sequence[Sensing]], Tuple[Sequence[Control], Sequence[Communication]]]¶

Parameters

thymio – number of agents in the simulation
sync – kind of synchronisation

Return f

controller function

forward(batch, batch_size)¶

Parameters

batch – batch elements
batch_size – batch mask

Return robots_control

network output

step(xs, comm, sync: networks.communication_network.Sync, sim=False, i=None)¶

Danger

Sync of types ‘random’, ‘sequential’ and ‘random_sequential’ are actually not working in simulation.

Parameters

xs – input sensing of a certain timestep (shape: 1 x N x sensing)
comm – communication vector (shape: 1 X N)
sync – kind of synchronisation
sim – boolean true if step executed in simulation
i – index of the thymio in the row

Return control

speed of the agent

class networks.communication_network.CommunicationNetNoSensing(input_size, device, sync: networks.communication_network.Sync = <Sync.sequential: 2>, module: torch.nn.modules.module.Module = <class 'networks.communication_network.SingleNetNoSensing'>, input_fn=<function input_from_no_sensing>)¶

High-level module that handle the sensing of the agents.

Parameters

input_size – dimension of the sensing vector (can be 7 or 14)
device – device used (cpu or gpu)
sync – kind of synchronisation
module – SingleNetNoSensing
input_fn – input function

Variables

self.tmp_indices – communication indices
self.distribution – distribution used for the initialisation of the communication

class networks.communication_network.SingleNet(input_size)¶

Low-level module that works on the sensing and the communication received by a single agent (in a certain timestep), producing as output the control and the communication to transmit.

Parameters: input_size – dimension of the sensing vector

forward(in_put)¶

Parameters: in_put – input of the network, vector containing the sensing and the messages received by the robot (can be multidimensional, that means a row for each robot)
Return output: output of the network containing the control and the message to communicate (shape: 1 x 2)

class networks.communication_network.SingleNetNoSensing(_)¶

Low-level module that works on the communication received by a single agent (in a certain timestep), producing as output the communication to transmit and the probability of a certain colour.

Parameters: _ – dimension of the sensing vector

forward(in_put)¶

Parameters: in_put – input of the network, vector containing the messages received by the robot (can be multidimensional, that means a row for each robot)
Return output: output of the network containing the message to communicate and the probability of a certain colour (shape: 1 x 2)

class networks.communication_network.Sync(value)¶: Class used to define the synchronisation approach of the communication updates, that can be random, sequential, synchronous and random_sequential

networks.communication_network.init_comm(thymio: int, distribution)¶

Initialise the communication vector (for the initial timestep of each sequence).

Parameters

thymio – number of agents
distribution – distribution used for the initialisation

Return out

communication vector

networks.communication_network.input_from(ss, comm, i, sim=False)¶

Parse the sensing and the communication and prepare it to be the input on the single net.

Parameters

ss – sensing
comm – communication vector
i – index of the thymio in the row
sim – boolean True if executed in simulation, False otherwise

Return in_put

parsed input containing the sensing values and the communication received

networks.communication_network.input_from_no_sensing(_, comm, i)¶

Prepare the communication vector to be the input on the single net.

Parameters

comm – communication vector
i – index of the thymio in the row

Return in_put

parsed input containing the communication received

Metrics¶

Adapted from https://github.com/GiorgiaAuroraAdorni/learning-relative-interactions-through-imitation

class networks.metrics.NetMetrics(t: tqdm.std.tqdm, metrics_path)¶

This class is supposed to create a dataframe that collects, updates and saves to file the metrics of a model.

Parameters

t – tqdm
metrics_path – file where to save the metrics

finalize()¶

Returns

update(train_loss, valid_loss)¶

Parameters

train_loss – training loss
valid_loss – validation loss

class networks.metrics.StreamingMean¶

Compute the (possibly weighted) mean of a sequence of values in streaming fashion.

This class stores the current mean and current sum of the weights and updates them when a new data point comes in.

This should have better stability than summing all samples and dividing at the end, since here the partial mean is always kept at the same scale as the samples.

property mean¶

mean

Type: return self._mean

reset()¶

update(sample, weight=1.0)¶

Parameters

sample – sample
weight – weight

Training¶

network_training_distribute.network_train(indices, file_losses, runs_dir, model_dir, model, communication, net_input, save_net)¶

Split the dataset also defining input and output, using the indices. Generate tensors. Create the neural network and optimizer, and set the device. Train the model and save it.

Parameters

indices – sample indices
file_losses – file where to save the metrics
runs_dir – directory containing the simulation runs
model_dir – directory containing the network data
model – network
communication – states if the communication is used by the network
net_input – input of the network (between: prox_values, prox_comm and all_sensors)
save_net – states if save or not the network

network_training_distribute.train_net(epochs: int, train_dataset: torch.utils.data.dataset.TensorDataset, valid_dataset: torch.utils.data.dataset.TensorDataset, test_dataset: torch.utils.data.dataset.TensorDataset, net: torch.nn.modules.module.Module, metrics_path: AnyStr, device, batch_size: int = 100, learning_rate: float = 0.01, criterion=MSELoss()) → networks.metrics.NetMetrics ¶

Parameters

epochs – number of epochs
train_dataset – training dataset
valid_dataset – validation dataset
test_dataset – testing dataset
net – model
metrics_path – file where to save the metrics
device – device used (cpu or cuda)
batch_size – size of the batch (default: 100)
learning_rate – learning rate (default: 0.01)
criterion – loss function (default: Mean Squared Error)

Return training_loss, validation_loss, testing_loss

output losses of the datasets

network_training_distribute.validate_net(net, device, valid_minibatch, criterion=MSELoss())¶

Parameters

net – model
device – device used (cpu or cuda)
valid_minibatch – validation batch
criterion – loss function

Return validation_loss.mean

resulting loss of the validation set

network_training_colour.network_train(indices, file_losses, runs_dir, model_dir, model, communication, net_input, save_net)¶

Split the dataset also defining input and output, using the indices. Generate tensors. Create the neural network and optimizer, and set the device. Train the model and save it.

Parameters

indices – sample indices
file_losses – file where to save the metrics
runs_dir – directory containing the simulation runs
model_dir – directory containing the network data
model – network
communication – states if the communication is used by the network
net_input – input of the network (between: prox_values, prox_comm and all_sensors)
save_net – states if save or not the network

network_training_colour.train_net(epochs: int, train_dataset: torch.utils.data.dataset.TensorDataset, valid_dataset: torch.utils.data.dataset.TensorDataset, test_dataset: torch.utils.data.dataset.TensorDataset, net: torch.nn.modules.module.Module, metrics_path: AnyStr, device, batch_size: int = 100, learning_rate: float = 0.01, criterion=BCELoss()) → networks.metrics.NetMetrics ¶

Parameters

epochs – number of epochs
train_dataset – training dataset
valid_dataset – validation dataset
test_dataset – testing dataset
net – model
metrics_path – file where to save the metrics
device – device used (cpu or cuda)
batch_size – size of the batch (default: 100)
learning_rate – learning rate (default: 0.01)
criterion – loss function (default: Mean Squared Error)

Return training_loss, validation_loss, testing_loss

output losses of the datasets

network_training_colour.validate_net(net, device, valid_minibatch, criterion=MSELoss())¶

Parameters

net – model
device – device used (cpu or cuda)
valid_minibatch – validation batch
criterion – loss function

Return validation_loss.mean

resulting loss of the validation set

Evaluation¶

network_evaluation.controller_plots(model_dir, ds, ds_eval, groundtruth, prediction, communication)¶

Parameters

model_dir – directory containing the trained model
ds – name of the dataset
ds_eval – name of the dataset for the evaluation (usually the manual one)
groundtruth – evidence
prediction – output control
communication – states if the communication is used by the network

network_evaluation.evaluate_controller(model_dir, ds, ds_eval, groundtruth, sensing, net_input, communication, goal, controllers)¶

Parameters

model_dir – directory containing the trained model
ds – name of the dataset
ds_eval – name of the dataset for the evaluation (usually the manual one)
groundtruth – evidence
sensing – used to obtain the prediction
net_input – input of the network (between: prox_values, prox_comm and all_sensors)
communication – states if the communication is used by the network
goal – task to perform (in this case distribute)
controllers – reference to the controller class

network_evaluation.evaluate_net(model_img, model, net, net_input, net_title, sensing, index, x_label, goal, communication, controllers)¶

Parameters

model_img – directory for the output image of the model
model – nae of the model
net – model used
net_input – input of the network (between: prox_values, prox_comm and all_sensors)
net_title – network title
sensing – input sensing
index – 1D vector
x_label – label of the x axis
goal – task to perform (in this case distribute)
communication – states if the communication is used by the network
controllers – reference to the controller class

network_evaluation.generate_sensing()¶

Return x, s: x is the configuration, s is the zeros array

network_evaluation.network_evaluation(indices, file_losses, runs_dir, model_dir, model, model_img, ds, ds_eval, communication, net_input, avg_gap=None, task='Task1')¶

Parameters

indices – sample indices
file_losses – file where to save the metrics
runs_dir – directory containing the simulation runs
model_dir – directory containing the network data
model – network
model_img – directory for the output image of the model
ds – name of the dataset
ds_eval – name of the dataset for the evaluation (usually the manual one)
communication – states if the communication is used by the network
net_input – input of the network (between: prox_values, prox_comm and all_sensors)
avg_gap – average distance among the robots
task – task to be performed

network_evaluation.network_plots(model_img, dataset, model, net_input, prediction, training_loss, validation_loss, x_train, y_valid, communication, goal)¶

Parameters

model_img – directory for the output image of the model
dataset – name of the dataset
model – netowork
net_input – input of the network (between: prox_values, prox_comm and all_sensors)
prediction – output control of the model
training_loss – error on the train set
validation_loss – error on the validation set
x_train – train input
y_valid – validation output
communication – states if the communication is used by the network

← Controllers Examples →