VIVI wants to grow intelligence without a pretrained language model

The main risk with VIVI is not that the project is trying to build "intelligence like a living creature's". The risk is different: without a vast store of ready-made knowledge the system may end up not more intelligent but simply far less useful at the start.

The project proposes abandoning the familiar scheme. Today's AI usually acquires knowledge from large corpora before it starts working and is then fine-tuned. VIVI wants intelligence to appear gradually — through accumulating experience, memory, self-organisation of knowledge, the formation of concepts and of an internal model of the world.

There is an important idea in that. A pretrained model answers quickly because a large part of the work has already been done in advance. But its knowledge and its ways of generalising are formed in advance too. VIVI treats intelligence as a process of development rather than a set of ready weights: the system should not only extract an answer from statistics but build its own picture of the world.

For an applied user this is not yet a new replacement for ChatGPT but a direction for experiment. The approach could potentially yield more independent learning and memory, but the price of the idea is a slow start and the need to solve on your own the problems that ordinary models have already closed off through pretraining.

The limitations of the description are substantial: it does not disclose VIVI's architecture, its training method, experimental results, metrics or examples of the system's behaviour. So it cannot be claimed that cognitive intelligence has already appeared. So far only the intent is confirmed — to grow it without a vast pretrained language model.

Which matters more to you: a system that knows a great deal straight away, or one that learns more slowly but forms a memory of its own? Source: All articles / Machine Learning / Habr