Randomness and determinism, is it possible to quantify these notions?

PETRU CARDEI

Computer Engineering

INMA

6, Ion Ionescu de la Brad Blvd., Sector 1, Bucharest, 013813

ROMANIA

Abstract: - The article presents results obtained in attempts to quantify randomness characteristics for real

statistical tests, the method proposed in this article determines a better resolution for the classification of strings

and, in addition, it can designate them as belonging to a class of randomness similar to that of some known

strings, such as finite substrings of prime numbers, pseudorandom strings generated by common programs,

trigonometric strings, etc.

The attempt to quantify the randomness of real numerical strings, the results of which are presented in this article,

is a first step in characterizing the randomness of experimental numerical strings, this being the final goal of the

investigations.

Key-Words: - Randomness, Determinism, Quantification, Relative Entropy, random sequences, random strings

Received: March 29, 2022. Revised: October 26, 2022. Accepted: November 21, 2022. Published: December 31, 2022.

or deterministic. Also, a problem very close to the

one mentioned is to decide if, between two signals

designated to be random, one of them can be

characterized by a "higher random intensity". In other

words, is it possible to quantify the notion of

randomness? This is the main problem of our

research. The proposed method is tested on examples

of strings or sequences recognized as random through

the "vague" characterization of this notion, common

in current scientific expositions.

using the theory of random functions and their

statistical dynamics, possibly optimizations with

random functions. Secondly, such a quantification of

the degree of uncertainty (randomness, [12]) is a

desideratum of knowledge and offers an argument for

the theoretical space in which we fit the analysis of

physical, social, biological or other phenomena.

In [5] it is shown that deterministic signals are a

special category of frequency stationary signals and

relatively constant amplitude over a long period of

deterministic signals are those whose evolution

cannot be anticipated with certainty, such as vocal,

video, seismic signals, etc. The unpredictability of

the signals is positively correlated with the amount of

energy transported. For example, the signal received

during the transmission of news to a radio station is

listened to with interest, due to its novelty. In the case

of non-deterministic signals, so that the information

can be received, the one who transmits it and the one

who receives it uses the same language (code,

deterministic signal has specific characteristics,

namely media, dispersion, global media, global

dispersion, histogram, spectral power density, etc.

The signal can have a certain degree of predictability

of its evolution over time. Depending on certain

characteristics of it, the non-deterministic signal can

be:

• Stationary - media and dispersion do not depend on

time, but are constant,

impossibility of the random true, the effort is directed

to study the random degrees". Also, the same author

shows that it can be proved that there is an infinite

hierarchy (in terms of quality or power) of the forms

of random.

According to [13], „a randomness test (or test for

randomness), in data evaluation, is a test used to

analyse the distribution of a set of data to see if it can

be described as random (pattern less). In stochastic

modelling, as in some computer simulations, the

hoped-for randomness of potential input data can be

verified, by a formal test for randomness, to show that

and theoretical question. Tests for randomness can be

used to determine whether a data set has a

recognisable pattern, which would indicate that the

process that generated it is significantly non-random.

For the most part, statistical analysis has, in practice,

been much more concerned with finding regularities

in data as opposed to testing for randomness. Many

"random number generators" in use today are defined

by algorithms, and so are actually pseudo-random

number generators. The sequences they produce are

[14] though it was shown to have an effective key

size far smaller than its actual size [15] and to

perform poorly on a chi-squared test [16]. The use of

an ill-conceived random number generator can put

the validity of an experiment in doubt by violating

used statistical testing techniques such as NIST

standards, Yongge Wang showed that NIST

standards are not sufficient. Furthermore, Yongge

Wang [17] designed statistical–distance–based and

law–of–the–iterated–logarithm–based testing

techniques. Using this technique, Yongge Wang and

Tony Nicol [18] detected the weakness in commonly

used pseudorandom generators such as the well-

known Debian version of the OpenSSL

Linear-feedback shift registers, Generalized

Fibonacci generators, Cryptographic generators,

Quadratic congruential generators, Cellular

automaton generators, Pseudo-random binary

sequences. These different generators have varying

degrees of success in passing the accepted test suites.

There are many practical measures of randomness for

a binary sequence. These include measures based on

statistical tests, transforms, and complexity or a

mixture of these. A well-known and widely used

collection of tests was the Diehard Battery of Tests,

introduced by Marsaglia; this was extended to the

measures of randomness. T. Beth and Z-D. Dai

purported to show that Kolmogorov complexity and

linear complexity are practically the same, [20],

although Y. Wang later showed that their claims are

incorrect, [21]. Nevertheless, Wang also

demonstrated that for Martin-Löf random sequences,

the Kolmogorov complexity is essentially the same

as linear complexity.

The need to quantify an important characteristic of

phenomena in the field of physics or in the field of

tries to determine how likely certain outcomes are if

some aspects of the system are not exactly known.

Many problems in the natural sciences and

engineering are also rife with sources of uncertainty.

Computer experiments by computer simulations are

uncertainty quantification, [26], [27], and [28].

concrete problems.

Uncertainty is sometimes classified into two

categories, [31], and [32]. Aleatoric uncertainty is

also known as stochastic uncertainty and is

representative of unknowns that differ each time we

run the same experiment, [33]. Also from [33],

epistemic uncertainty is also known as systematic

uncertainty, and is due to things one could in

principle know but does not in practice. This may be

separately. The quantification for the aleatoric

uncertainties can be relatively straightforward, where

traditional (frequentist) probability is the most basic

form. Techniques such as the Monte Carlo method

are frequently used. A probability distribution can be

represented by its moments (in the Gaussian case, the

mean and covariance suffice, although, in general,

even knowledge of all moments to arbitrarily high

order still does not specify the distribution function

uniquely), or more recently, by techniques such as

uncertainty, which is the quantification of the

uncertainties at the outputs of the system, resulting

from uncertainties and, the reverse quantification of

the uncertainty, which involves calibration

parameters or simply calibration.

A direct approach to the random or deterministic

character for numerical or alphanumeric strings is

found on the web page [22]. In 4.4 we used the

program from [22] to compare its decision with the

the string or lead it to the vicinity of a series with

known behaviour, with whose randomness or

determinism, the analysed string can be assimilated.

Considering the impressive theoretical and applied

developments that addressed the notions of

randomness, uncertainty and others from the same

family of words, the only novelty in this attempt is

the possible introduction of entropy as a measure of

randomness or uncertainty, for now for a narrow

category of some mathematical objects ( real

numerical strings, very common in experimental and

theoretical-empirical techniques). In relation to other

[0, 1], where the relative entropy varies, can be

divided by those who perform the analysis. In

addition, they have at their disposal special classes of

random strings with which to make a comparison: the

string of prime numbers, pseudo-random generated

by various programs, and original strings, with the

desired length. As I showed above, in the end, the

distribution of a string analysed in random classes

(sets), provides an orientation of the mathematical

model of the phenomenon that generates the analysed

the sum of the entropies of these sources;

e5) entropy will be maximum if all messages are

equally likely.

The greater the entropy, the greater the uncertainty of

the message transmitted by the random variable.

According to [4], randomness is the characteristic of

an uncertain event, which depends on future

conditions, themselves uncertain. The same source

with reduced uncertainty and randomness. In other

words, a random variable is "more random", the

higher the entropy values.

If you work with finite strings of numbers, the

probabilities involved in calculating the relative

entropy are calculated starting from the histograms of

the values of the strings analysed. For the calculation

of the number of intervals of the string histograms,

we adopted some rules used in many works, [8-11].

Numerical studies have shown that relative entropy

depends on the number of classes considered when

constructing the diagrams. I used, for some reference

order to give an image of how these numerical

analyses and the relative entropy can be used in the

decision of the random or deterministic character of

some numerical strings, we averaged the relative

entropy values obtained from the six results

calculated using the specified formulas for the

number of classes of histograms.

In order to understand the behaviour of the relative

entropy in describing the intensity of the randomness

of the numerical data strings, some examples of

from various sources are given in this chapter. Due to

the large number of formulas proposed for

calculating the number of classes of histograms, we

calculated the relative entropy as an average of six of

the most common formulas for the number of classes

of histograms, which are found in table 1 from [11].

The strings for which the calculations were

performed are listed in table 1.

(CV) and the amplitude of the standard deviation

sequences (1, 2, 3), fig. 1, strings generated with

generation programs for pseudorandom strings (4, 5,

6), [7], fig. 2, and sinusoidal strings (7, 8, 9), defined

by formulas below the table, and finite substrings of

prime numbers.

Also, classic random strings obtained from the string

formula, 1979 and two control formulas of the

behaviour of entropy relative, having the number of

classes equal to the number of elements of the string,

respectively with its half.

The ranking of the strings according to the value of

the relative entropy is given in table 2. The indexation

of the strings has been preserved as in table 1, and the

observations with lowercase written after table 1

remain valid for table 2.

O1) The highest values of the relative entropy in

table 1 correspond to the finished sequences of

prime numbers, and the relative entropy

increases with the length of the sequences of

prime numbers (it is easier to see in table 2);

O2) Values close to the entropy relative to those

records are characterized by values of relative

entropy over 85% but at a remarkable distance to

the pseudo-random strings obtained using [7];

O4) Between the strings given by sinusoidal formulas

it is observed that the sinusoid with a single

component, composed with the floor function, (the

set of values of this function has only three elements),

sinusoidal sequence that comes from a pure sinusoid

has a value maximum value of entropy, even higher

than any of the strings from experimental

the collection of curves examined in this study.

O6) The relative entropy produces a strict order

relationship on the set of the strings in table 1, which

can be used for the purpose of ranking the intensity

of the random character or of the uncertainty of the

random strings described by such strings, as follows:

O6.1) It can be appreciated, at a first evaluation, that

random variables or strings with relative entropy

greater than 50% can be considered as suspects of

and strings whose relative entropy is strictly higher

than 66%, can be considered suspicious of intense

randomness. Strings whose relative entropy is

between 33% and 66% can be considered as having

an undecidable character.

O6.3) Another criterion for appreciating the intensity

of the random or deterministic character of a string of

numerical data can be obtained by comparison with

standard strings whose relative entropy is known and

does not vary much with the number of classes of

histograms used for the calculation of it. Thus, for

example, the strings of pseudo-random numbers can

by a high degree of determinism. The sinusoidal

sequences can be located in the area of random

sequences, in the undecidable or deterministic area,

according to the values of amplitudes, frequencies, or

the way of generation (composition with the floor

function, for example, or with generators of pseudo-

random numbers).

number of the histogram’s classes

The variation of entropy relative to the number of

classes of histograms poses difficult problems for

analysis. First of all, we wonder if the number of

classes of histograms increases, it can be reached in

the situation that the random string changes the

characteristic, from random to determinist, or vice

versa (intuitive, the last option is unlikely)? The

denominator of the fraction that defines the relative

entropy (4), tends to infinity with the number of

probability. The existence of some classes of the

histogram with zero elements implies belonging to

the sum that defines the relative entropy of some

meaningless terms, but which, at the limit, tend to

zero. Therefore, intuitively, starting with the number

analysed string. As a result of this reasoning, I sought

to stop the process of growing the number of classes

of the histograms to a number near . If 󰇝󰇞,

it is the random string, the number of samples in each

  (they can be equal, if the

string is constant), then, assuming a number of the

histogram, with equal intervals, we obtain the size or

entropy curve - the number of classes of the

with few values, for example, or are concentrated on

narrow ranges of numbers, the interpolation can be

done with rational power type curves or with

discontinuous curves and for this reason, the

proposed algorithms must be assisted by the human

operator in thing, for now.

number is two, and the 1000th is 7919, the minimum

distance between two of the elements of this string is

1 (the distance between 2 and 3). The criterion of the

number of classes where each class contain only one

element explained in the relations (5) and (6), leads

to a number of 7917 classes. According to the method

presented in 3.1, the histogram, probabilities, and

finally, entropy, relative entropy and other

characteristics of the string of the first 1000 prime

numbers are calculated.

amplitude of the standard deviation above the

average. In fig. 3 and 4, it can be observed that the

entropy increases monotonically asymptotically,

while the relative entropy decreases monotonically

with an unclear asymptotic trend. For this reason, the

selection criterion of the "optimal" number of classes

(intervals) necessary for a "good enough" assessment

of the relative entropy was based on the variation of

the entropy and not on the relative entropy with the

number of classes of the histogram. Thus, the

study, we chose:

value of the entropy, 󰇛󰇜 . The stopping

value is a bit exaggerated (below 0.126% of the

maximum entropy value), but this numerical study is

only an example. For the histogram with 131 classes,

4.3 A procedure for choosing the number of

classes of the histograms used to estimate the

relative entropy based on the interpolation of

the dependence curve of entropy by the

number of classes of the histogram

This appendix gives the results of the relative entropy

evaluation for the strings in table 1, obtained using a

criterion for determining the number of classes of the

the selection algorithm of the number of histogram

classes given in E4.1. A curve of dependence

between entropy and the number of classes of the

histogram is given in fig. 6. This curve corresponds

to the experimental data (tensometry records) from

table 1, at position 1.

By putting the condition that the minimum number of

classes of the histogram to be the smallest abscissa,

for which the slope has less than 1o, on the curve in

fig. 5, are obtained the next results   

E4.1, with the same constant (the tangent of 1-degree

angle), leads to the solution:   

the number of classes of histogram the criterion of the

average of three consecutive differences of entropy.

If the selection is made using the simple successive

differences between the elements of the entropy

string, the solution obtained is the next: 

taken as an example produce the same ranking of

randomness, [12]. Only the characteristic values

differ, but not significantly, see Table 5.

ranking of the strings in table 2, the changes are

small, namely, the string of five sinusoids is inserted

between the first two strings of experimental origin,

increasing in relative entropy value. From the point

of view of the category, there are no transitions from