Original): Okazykin
Originally published in the direction of artificial intelligence.
Seeing the future
When I saw the conditions for the first time causal, stable, linearAND Inconceivable timeI thought: Here is another wall of jargon. But after working on a few examples, I realized that each property answers a very human question about how the system behaves:
- Can the system see the future?
- Will it blow up if I give him normal input data?
- Is it consistent with the simple principles of mathematics we expect?
- Will he keep the same tomorrow as today?
These are all these properties. But let's reduce by a second: what we mean by system?
System ENcaspusules stood
The system is any machine (code, filter, model, pipeline), which in time adopts the sequence of inputs and produces sequence of exit. Formally, it is the T operator who mats the input sequence X to the output sequence y:
Where:
- X is an input sequence,
- y is the output sequence,
- T Is the entry of a system operator mapping to the output.
- N is an index of discrete time.
System contains behavior. It can be without memory (only uses current entry), has a short memory (uses several latest input data) or long/complex memory (RNN, attention in transformers that look back or forward). At the base of the system is only the rule of transforming inputs. In artificial intelligence and machine learning, we meet the systems everywhere: a neural network that makes an image and displays a label, a speech model that converts the sound to text, and even a recommendation engine that faces your previous clicks to a set of new suggestions.
System taking clear steps
Now, when we are talking about a discreet system, we are talking about systems where input and output signals are defined in different steps – think sequences, not continuous waves. In practice, it is a digital world in which we live: Audio recording thousands of points per second, pixels in the image net or tokens sequences transferred to the language model. Each of them is a discreet signal, and the algorithm that processes it is a system.
Why should we depend on whether the system is linear, free, stable or unchanged time? Because they tell us if the system is Well brought up in fact. For example:
- AND causal The system is like a good AI assistant in real time-it is not based on information from the future.
- AND stable The system is safe – it does not produce exploding outputs when you give it normally limited input data.
- AND linear The system is predictable – scaling or combining input data behaves as you can expect.
- AND Inconceivable time The system is consistent – if it worked yesterday, it will work in the same way tomorrow.
When we see this in this way, these concepts stop feeling abstract and illegal. These are only basic mental health controls, how the system processes data. In the rest of this post we will break them in a simple language and go through two specific examples step by step.
“Big 4” without fog
Here's how we can think about the properties of the “big four” without mathematical fog.
Causal → no time machines.
The causal system is one that does not cheat, looking to the future – only looks at the present and past entrance data. If I want to rewrite speech in real time, the system can only rely on the words spoken so far. He cannot stop, wait for the next sentence, and then return to the “magical” improvement in past transcription. In mathematical terms, this means going out in time N It depends only on the input values at time or earlier. Exit in time N He should never rely on data values that have not yet happened.
Example: The weather forecast that uses today and past data is causal. The forecast that somehow uses tomorrow's data is not.
Stable → no explosions.
The system is stable if reasonable input data do not create wild, infinite outputs. Formally, if the entrance is limited (the maximum has never passed), the exit also remains limited.
Stability is to maintain control. Imagine that normal limited input data to the system is given – something like an audio signal that never exceeds a certain volume. If the system reacts, producing infinite spikes or values rising without associated, it is unstable. In practice, this appears in machine learning during training divergence or exploded gradients. However, a stable system guarantees that limited inputs always produce limited outputs.
Example: the speaker that reproduces your louder voice is stable. A broken amplifier, which makes the tiny noise explode to the sounding of a shocking ear, is not.
Linear → Mathematics game Fair.
If you double the input data, you should double your output. If you add two inputs, the output should be the sum of two individual outputs. It's linearity.
The linearity concerns justice and predictability. If the system is linear, the scaling of the entrance at two should scale the output by two. If you add two entries together, the output should be the output sum that can be obtained from each input individually. Many simple filters and transformations are linear, but most of the interesting models that we build today – with non -linear activations, such as Relu, sigmoid or cosine – no. Non -linearity is powerful, but also hinders analysis.
Example: two fans blowing to you → the elevator adds up (linear). Two toasters forming toast → does not produce one gigantic toast (not linear).
Failure to apply time → consistent rules.
If you move the input signal in time, the output should move in the same way. The rules do not change with the clock.
The inexhaustrity of time concerns consistency. If you move the input data in time, the output should move exactly the same way. A system that behaves differently depending on when you serve it, the signal is variable in time, not an unchanged time. In the world of mL, the NEURONE weaves rely on this property in space, not time: if you slightly move the image, the functions also move, which makes them solid for translation.
Example: The coffee machine gives the same coffee, regardless of whether you press the “infusion” at 8 am or 20:00 (unchanging times). There is no bar with a happy-hour price (different outputs at different times).
To sum up – it's about answering human questions
These four properties-hyphen, stability, linearity and timeless time-to-be-extinguished to look intimidating on paper, but they come down to questions about health about the behavior of systems. The causal system does not go into the future. The stable system does not explode when you give it reasonable input data. The linear system respects the rules of scaling and adding. And the system unchanged time behaves the same way as tomorrow.
In mathematics, these are fields that you can check to understand the system. In machine learning, these are mental health controls that help us reason whether the model or algorithm will work in practice. Seeing in this way, jargon falls off, and what is left is a handful of very human questions about whether the machine behaves as it should.
More information on mathematics for AI and ML, check it out list:
Mathematics for AI/ML
Display the list5 stories
Published via AI
