Since it is our duty to choose which future we want to create, it is equally important to picture the world we dream of beyond the existing applications of current research. From E-learning to Permaculture or Circular economy, this page tries to embrace the potential of sequential learning for shaping our future societies.
I am generally a theory guy. But here, I want to talk a little about applications, in order to push the applied reinforcement learning community to applications that are not only scientifically exciting, but also have a great potential for the world. I won’t talk much of applications of sequential learning to the optimization of webpages designs to an incoming person, nor to the design of optimal ad placement strategies, that, though very relevant to many big companies, is a little gray from an ethical point of view. The same tools can be used to develop applications with a much higher societal value, and also give rise to a huge avenue of exciting scientific challenges.
Let us list some of these applications of Sequential Decision Making, perhaps from the least expected ones:
- Computational permafarming: Given a farm, with plants that are strongly interacting and sharing resources, the goal is to decide which action to perform (planting/moving which plant) in order to maximize the resistance of the system to attacks from the weather, insects or diseases while minimizing the external resources added to the system. Handling the strong dependency between the plants is a beautiful challenge. This directly falls under the scope of computational sustainability.
- Robust hydraulic systems: Say you want to monitor the level of rivers, in order to avoid flooding or that certain pollution reach critical sites. An incoming rain, storm, or pollution is seen as an attack, and the goal is to control the system in a robust way. Such a formalism may also apply to the management of electrical systems, when we further constraint the communications between control nodes.
- E-Learning: The goal is to recommend a series of exercises to incoming students in order maximize their learning level. Since each learner improves at each time steps, the learning progress of a student is a non-stationary reward signal. Handling such a signal for recommender systems is a great challenge.
- Job recommendations: Here the goal is to match job announcements to people looking for jobs. The task is fairly close to news article recommendation, and involves some natural language processing. Obviously the potential impact on unemployment is exciting.
- Urban recommendations: Consider you display the activity of a set of town councils to a web platform and record a feedback from citizens. Based on this data, you now want to help a council make better decisions, by recommending projects that may work well and alerting about possibly bad projects. Here one challenge is about long term decisions.
- Sustainable Economy: Consider a graph of economic agents exchanging resources. Here you want to price the additional number of resources needed at the source nodes of the graph, in order to increase the production at a certain node, and then recommend to a user which producer to favor when they both output the same resource. Handling the network activity of many agents is especialy challenging, and crucial for the development of closed-loop economy.