Concepts

Engine

The essence of the framework is the class ignite.engine.Engine, an abstraction that loops a given number of times over provided data, executes a processing function and returns a result:

while epoch < max_epochs:
    # run once on data
    for batch in data:
        output = process_function(batch)

Thus, a model trainer is simply an engine that loops multiple times over the training dataset and updates model parameters. Similarly, model evaluation can be done with an engine that runs a single time over the validation dataset and computes metrics. For example, model trainer for a supervised task:

def update_model(trainer, batch):
    model.train()
    optimizer.zero_grad()
    x, y = prepare_batch(batch)
    y_pred = model(x)
    loss = loss_fn(y_pred, y)
    loss.backward()
    optimizer.step()
    return loss.item()

trainer = Engine(update_model)
trainer.run(data, max_epochs=100)

Events and Handlers

To improve the ignite.engine.Engine’s flexibility, an event system is introduced that facilitates interaction on each step of the run:

• engine is started/completed

• epoch is started/completed

• batch iteration is started/completed

Complete list of events can be found at ignite.engine.Events.

Thus, user can execute a custom code as an event handler. Let us consider in more detail what happens when ignite.engine.Engine.run() is called:

fire_event(Events.STARTED)
while epoch < max_epochs:
    fire_event(Events.EPOCH_STARTED)
    # run once on data
    for batch in data:
        fire_event(Events.ITERATION_STARTED)

        output = process_function(batch)

        fire_event(Events.ITERATION_COMPLETED)
    fire_event(Events.EPOCH_COMPLETED)
fire_event(Events.COMPLETED)

At first engine is started event is fired and all this event handlers are executed (we will see in the next paragraph how to add event handlers). Next, while loop is started and epoch is started event occurs, etc. Every time an event is “fired”, attached handlers are executed.

Attaching an event handler is simple using method ignite.engine.Engine.add_event_handler() or ignite.engine.Engine.on() decorator:

trainer = Engine(update_model)

trainer.add_event_handler(Events.STARTED, lambda engine: print("Start training"))
# or
@trainer.on(Events.STARTED)
def on_training_started(engine):
    print("Another message of start training")

# attach handler with args, kwargs
mydata = [1, 2, 3, 4]

def on_training_ended(engine, data):
    print("Training is ended. mydata={}".format(data))

trainer.add_event_handler(Events.COMPLETED, on_training_ended, mydata)

Note

User can also register custom events with ignite.engine.Engine.register_events(), attach handlers and fire custom events calling ignite.engine.Engine.fire_event() in any handler or process_function.

See the source code of ignite.contrib.engines.create_supervised_tbptt_trainer for an example of usage of custom events.

State

A state is introduced in ignite.engine.Engine to store the output of the process_function, current epoch, iteration and other helpful information. For example, in case of supervised trainer, we can log computed loss value, completed iterations and epochs:

trainer = Engine(update_model)

def on_iteration_completed(engine):
    iteration = engine.state.iteration
    epoch = engine.state.epoch
    loss = engine.state.output
    print("Epoch: {}, Iteration: {}, Loss: {}".format(epoch, iteration, loss))

trainer.add_event_handler(Events.ITERATION_COMPLETED, on_iteration_completed)

Note

A good practice is to use ignite.engine.State also as a storage of user data created in update or handler functions. For example, we would like to save new_attribute in the state:

def user_handler_function(engine):
    engine.state.new_attribute = 12345