An approach which seems in general much more robust is the use of an agent database. Here, we mean that each traveler/agent keeps a memory of options that he/she tried out, and some measure of the performance of each option. This approach is similar to classifier systems, genetic algorithms, or reinforcement learning, with the difference that the number of agents, typically several millions, is much higher in large scale transportation simulations than in typical applications of the mentioned areas.
The simulation would start with each agent having one or more options, which all have preliminary scores. Each iteration would consist of the following steps:
Each agent would chose an option according to the scores, for example taking the option with the best score.
The simulation would be carried out.
Each agent would note the new score of the option that it just carried out.
In addition, it is necessary to inject new options into the system. For example, in each iteration one could give new options to a fraction of the agents, and then ``force'' those agents to immediately try them out. If these options lead to bad scores, the agents will rarely or never try them again.
Although such an approach is easy to state in principle, it is difficult to implement in practice because of performance limitations. Using a relational database such as MySQL is possible but slow with several millions of agents. Also, although a relational database provides support such as indexing and sorting, it's emphasis is on consistent and secure operation, not on computational speed. This is a subject of active research.
With respect to our practical examples, the easiest solution is to not worry about the routing choice, but remember starting times and performance only. That is, after a simulation run one would parse the events file, and for each agent note the starting time and the corresponding trip time. That information would be merged together with pre-existing information into some agent data base.
(One could for example do a flat file of agent performance for each iteration; the departure time choice module would then read all these files.)
For each agent that does departure time choice, the experienced trip times would be used as a base. For departure times outside the experienced interval, free speed travel times could be used. For departure times in between experienced travel times, some kind of interpolation (e.g. linear) could be used.
Note that agent memory needs to age, otherwise agents may remember information that is no longer relevant. One possibility would be to only read the agent experience from the last 10 iterations.
This would again be a cooperation between the systems integration specialists and the route/acts gen specialists.