The first annual International Workshop on Self-Supervised Learning was held on 27th & 28th of February, 2020. Leading researchers in the field presented papers covering the state of the art in the field of self-supervised learning (SSL). This expanding field of research aims to create systems that can learn without human intervention or hand-built knowledge datasets.
As machine learning becomes increasingly connected with our daily lives, the limiting factor of our AI technology is revealing itself to be its dependence on people. In other words, our machine learning algorithms still need too much assistance from software designers and data labelling workers. In most deep learning systems, humans give the algorithm desired labels, ontologies, or categories in a subject area, so it can develop a basic knowledge of that area. For example, our current object recognition software requires thousands of hours of human image annotations in order to identify street signs or dog breeds. Even with immense amounts of data, these object recognition algorithms can be easily tricked by something so small as rotating the object 90 degrees! Human designers must decide what is important and what represents good examples of the objects being recognized. This becomes completely unviable for many complex real-world applications, especially when there are interactions with the complexity of human behavior.
With SSL, it is the system and not the human programmer who chooses what and how to learn. The agent is able to collect data in a simulation where it can experiment, build its own knowledge representations, update old concepts, or build new ones. Since this is all done within a simulated environment which can run thousands of times faster than the real world, the learning process can theoretically
Initially, an SSL system will explore in order to develop a simple understanding of the mechanics of its world. It is self-motivated to create knowledge about its relationship to its environment, and it begins to develop new behaviors based on its environment. Low level behaviors such as object recognition and movement can eventually be improved upon, and the system can develop complex behaviors such as learning tool-use without human intervention. Having the SSL system decide what is important to learn takes a massive burden off of developers as they no longer need to invest time and money for humans to create and curate datasets.
SSL-based agents are more likely than other agents to enable general-purpose intelligence in the future, due to their ability to learn from their own mistakes and apply themselves to a wide variety of domains without the assistance of human designers. The world needs a clear path to general purpose intelligence, for without it, there will be only incremental improvements in the capacity of AI systems to solve our biggest challenges.