next up previous contents
Next: Smart agents and non-predictability Up: Learning and feedback Previous: Relation to game theory   Contents

Relation to machine learning

There is also a connection of our simulations to machine learning. This connection becomes clear if we consider each agent as a learning machine - in consequence, all knowledge from machine learning (which typically considers a single agent in an environment) could be applied to our agents. In other word, each agent could be programmed as a learning machine, using the best of methods available from machine learning. This leads to several issues:

In how far are machine learning methods applicable under the constraints that we face? In particular, we need to have of the order of $10^7$ learning agents, and we have a non-stationary environment (since also the other agents learn).31.1

On the other hand, very little of what we have considered concerns states being dependent on each other, i.e. the situation faced in reinforcement learning that the expected pay-off has both immediate and long-term contributions. This is however a simplification in transportation that does not truly apply. For example, path finding could also be considered as a state-dependent operation; and weekly activity lists where leisure, shopping, going to the doctor has to be distributed across several days leads to similar issues.

In how far does the result resemble human learning? In other words, how far different is human learning and machine learning for the questions we are interested in?

Does our system have anything to do with distributed machine learning? That is, can the whole transportation system be considered as a large multi-agent learning system? In contrast to typical approaches in artificial intelligence, there is no obvious goal that the transportation system attempts to optimize.

In other words: How large is the difference between distributed learning systems for solving a given task, and distributed learning systems as models for human society?

The last aspect also becomes apparent when comparing the concept of a Nash Equilibrium with the concept of a System Optimum (SO). Whereas the first assumes that every agent opimizes its own utility, the latter assumes that some system-wide quantity is optimized. For example, one could optimize the sum of all travel times rather than having each individual agent optimizing its travel time. The results are in general not the same; the NE solutions lead to larger travel times.

[[additional section: Gibbs sampling (Markov chain monte carlo)?]]

[[with-day section-??]]


next up previous contents
Next: Smart agents and non-predictability Up: Learning and feedback Previous: Relation to game theory   Contents
2004-02-02